Javascript Question posted in
A very good W3school tutorial can be found here.

Javascript – Read from nth row in csv typescript

I am trying to read a csv file from 4th row, in my csv file first 3 rows contains some meta data in column1. But the actual number of columns in csv is 6.

eg:

metadata1
metadata2
metadata3
head01,head02,head03,head04,head05,head06
data11,data12,data13,data14,data15,data16
data21,data22,data23,data24,data25,data26

Code I am trying is:

fs.createReadStream('/path/file.csv').pipe(
        parse({ from_line: 4, fromLine: 4 }, function(data, err) {
        console.log(data);
    })
);

I am getting data as undefined. Also If I am removing the args of parse, I am getting error Error: Invalid Record Length: expect 1, got 6 on line 4. Please guide!

Tags:

2

Answers


  1. 0

    The issue you’re encountering stems from the fact that the CSV parser is expecting a consistent number of columns across all rows. Since the first three rows contain metadata (likely only one column), the parser is confused when it suddenly encounters six columns on the fourth row. So what you need to do it to skip first three rows and start from the fourth row. Parser expects same amount of columns and that will happen after the 4th row.

    const fs = require('fs');
const parse = require('csv-parse');

fs.createReadStream('/path/file.csv').pipe(
    parse({
        from_line: 4, // Start parsing from the 4th line
        relax_column_count: true, // Allow variable column counts for metadata rows
    }, function(err, data) {
        if (err) {
            console.error(err);
            return;
        }
        console.log(data);
    })
);

    Let me know if this helps. I would love to help more if you need further assistance.

    Login or Signup to reply.
  2. 0

    This issue is because parse function expects the number of columns to be consistent throughout the file. The metadata lines have only 1 column, while the data lines have 6 columns, which is causing the parser to expect only 1 column when reading the 4th line.

    You need to skip the first three lines before passing the stream to the parser.

    Here’s the corrected code.

    fs.createReadStream('/path/file.csv')
  .pipe(parse({ from_line: 4 }))
  .on('data', (data) => {
    console.log(data);
  })
  .on('error', (err) => {
    console.error(err.message);
  });
    Login or Signup to reply.
Please signup or login to give your own answer.

Back To Top
Search