Question posted in
Some documentation to get you started can be found here.

How to extract and route only specified columns from a CSV files and drop all other columns – Apache

I want to extract few fields along with its value from a CSV file and drop/delete all other fields in the file. Please help. I think we can use RoutText processor.Please tell me how to write the regular expression for the routing only specified fields and dropping everything else. Thanks

Example- from he snapshot attached I only want to route ‘Firstname,Lastname and Siblings’ fields along wit hits value(each record/row). Delete the remaining columns like ‘State, Age, Apt no,Country,Gender’.

Please tell me what is the correct processor for this and what configuration properties to use in order to achieve this. Thanks

Snapshot of comma separated CSV file

Attaching snapshot for reference.

Tags:

2

Answers


  1. 0

    I think using regular expression is not the best solution: Here is how I should do:

    1. First you need to explore the csv:

      $handle = fopen("test.csv", "r")

    2. Map through the data

      $data = fgetcsv($handle, 1000, ",")

    3. Create New Header and Array from existed $data with wanted fields

    4. Put new data into new csv.

      $fp = fopen('file.csv', 'w');

foreach ($data as $fields) {
    fputcsv($fp, $fields);
}

fclose($fp);
    Login or Signup to reply.
  2. 0

    You can use ConvertRecord for this. Provide the full schema to the CSVReader, and provide the schema with only the fields you want to the CSVRecordSetWriter. If you don’t know the input schema (but you know it includes at least the fields you want to send along), you can have the reader Use String Fields From Header, that will create an input schema (using the header line) and assume all fields are strings. However the output schema would have the selected fields along with their types, and ConvertRecord will handle the “deletion” of the other fields, as well as any conversion from String to the desired data type for each of the selected fields.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search