Metadata injection: abstracting PDI
When we have a collection of files to read with PDI and they all have the same format development is easy: list all files, run a transformation for each one, Text file input, etc. But if different files have different formats and there's no way to tell a priori what the formats may be things get tricky. In this talk we'll see an example of a collection of heterogeneous CSV files being parsed by a single PDI transformation with the Metadata Injection step. Sample files and PDI transformations will be available on github
YOU MAY ALSO LIKE:
Thanks to our sponsors
Metadata injection: abstracting PDI
Pedro Alves
Pentaho BI Specialist, author CDF, CDE, CDA, CCC (...), founder of WebDetails.pt and member of Mozilla Metrics team
When we have a collection of files to read with PDI and they all have the same format development is easy: list all files, run a transformation for each one, Text file input, etc. But if different files have different formats and there's no way to tell a priori what the formats may be things get tricky. In this talk we'll see an example of a collection of heterogeneous CSV files being parsed by a single PDI transformation with the Metadata Injection step. Sample files and PDI transformations will be available on github
YOU MAY ALSO LIKE:
Thanks to our sponsors
Metadata injection: abstracting PDI
Pedro Alves
Pentaho BI Specialist, author CDF, CDE, CDA, CCC (...), founder of WebDetails.pt and member of Mozilla Metrics team