Question posted in
Some documentation to get you started can be found here.

How to convert a column in a csv file using Nifi? – Apache

I Have a raw csv file as below 

   SVEIS , SVESID,SVETIME
   Quneter,53553,01122003010005
   renchure,37151,12092008011005

now I need to change the data from day month year to year day month as below 

   SVEIS , SVESID,SVETIME
  Quneter,53553,2003-01-12 01:00:05
  renchure,37151,2008-12-09 01:10:05

I need to change this using NIFI

I have used replace text processor but I am getting error like unable to process 10 of 10 records in nifi

Tags:

3

Answers


  1. 0

    We can start from a simple expression using quantifiers and capturing groups:

    (.+,)([0-9]{2})([0-9]{2})([0-9]{4})([0-9]{2})([0-9]{2})([0-9]{2})

    and replace it with:

    $1$4-$3-$2 $5:$6:$7

    where $1-$7 are our groups with new desired format.

    DEMO

    RegEx Circuit

    jex.im visualizes regular expressions:

    enter image description here

    Demo

    const regex = /(.+,)([0-9]{2})([0-9]{2})([0-9]{4})([0-9]{2})([0-9]{2})([0-9]{2})/gm;
const str = `Quneter,53553,01122003010005
renchure,37151,12092008011005`;
const subst = `$1$4-$3-$2 $5:$6:$7`;

// The substituted value will be contained in the result variable
const result = str.replace(regex, subst);

console.log(result);
    Login or Signup to reply.
  2. 0

    If awk can be used, this is possible to solve like this:

    awk -F, '{
    f1=substr($3,1,2);
    f2=substr($3,3,2);
    f3=substr($3,5,2);
    f4=substr($3,7,2);
    f5=substr($3,9,2);
    f6=substr($3,11,2);
    f7=substr($3,13,2)}
    NR<2 {print $1,$2,$3;next}
    {print $1,$2,f3 f4"-"f1"-"f2,f5":"f6":"f7}' file

   SVEIS   SVESID SVETIME
   Quneter 53553 2003-01-12 01:00:05
   renchure 37151 2008-12-09 01:10:05
    Login or Signup to reply.
  3. 0

    While both of the existing answers explain how to accomplish this task using String parsing via regular expressions, those solutions will be fairly complicated to change if the incoming data changes in any way. The question is for Apache NiFi, and the solution is to treat the CSV data as records and use the UpdateRecord processor. This processor allows you to use RecordPath syntax to specify the field (column) you’re interested in and replace it with a new value, which can be static or determined via Expression Language. You can use the #toDate and #format methods to convert the string from the incoming format to the desired one (or simply insert the delimiters directly).

    There are good articles about using the RecordPath mechanics as well.

    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search