Quantcast
Channel: Desktop topics
Viewing all articles
Browse latest Browse all 213989

Folder Data Source and Pivot

$
0
0

Hi,

 

I have used the folder data source. In the latest version of PBI Desktop it automates most of the steps. My code is below. I have an issue where I end up with duplicate rows and the pivot fails for the duplicate rows. I would like to add the line number of each file to see if this helps with the duplicate issue. I have tried adding a index column just before the pivot but that give me a martix like solution and the data is on multiple rows. The index is from 1 to the end which is the end of all files. I want to add the row number of each file and the row number will reset to 1 when the next file is processed.

 

let
    Source = Folder.Files("C:\Users\Data"),
    #"Invoke Custom Function1" = Table.AddColumn(Source, "Transform Binary from Data", each #"Transform Binary from Data"([Content])),
    #"Renamed Columns1" = Table.RenameColumns(#"Invoke Custom Function1", {"Name", "Source.Name"}),
    #"Removed Other Columns1" = Table.SelectColumns(#"Renamed Columns1", {"Source.Name", "Transform Binary from Data"}),
    #"Expanded Table Column1" = Table.ExpandTableColumn(#"Removed Other Columns1", "Transform Binary from Data", Table.ColumnNames(#"Removed Other Columns1"[#"Transform Binary from Data"]{0})),
    #"Changed Type" = Table.TransformColumnTypes(#"Expanded Table Column1",{{"Source.Name", type text}, {"Column1", type text}}),
    #"Added Conditional Column" = Table.AddColumn(#"Changed Type", "Custom", each if Text.StartsWith([Column1], " | +- 00000000") then "DataRaw" else if Text.Contains([Column1], "UTC") then "DateRaw" else if Text.StartsWith([Column1], " Header:") then "MachineRaw" else null ),
    #"Filtered Rows" = Table.SelectRows(#"Added Conditional Column", each ([Custom] <> null)),
    #"Reordered Columns" = Table.ReorderColumns(#"Filtered Rows",{"Source.Name", "Custom", "Column1"}),
    #"Renamed Columns" = Table.RenameColumns(#"Reordered Columns",{{"Custom", "Category"}}),
    #"Changed Type1" = Table.TransformColumnTypes(#"Renamed Columns",{{"Category", type text}}),
    #"Added Index" = Table.AddIndexColumn(#"Changed Type1", "Index", 1, 1),
    #"Pivoted Column" = Table.Pivot(#"Added Index", List.Distinct(#"Added Index"[Category]), "Category", "Column1"),
    DataRaw = #"Pivoted Column"{0}[DataRaw]
in
    DataRaw


Viewing all articles
Browse latest Browse all 213989

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>