RSS

Monthly Archives: June 2012

Reading a CSV File line by line

Ever needed to read a CSV file? Typically you can accomplish this in Java(and similar languages) by creating a instance of CSVStreamReader and using the readAll() method to read in the file.Though this may read the full file it causes a heavier IO load on the file system.

Here is a method to read a CSV file line by line that is less IO intensive.

function readCsvLine( reader : CSVStreamReader ) {
     var line : Array = reader.readNext();
     if ( line != null ) {
        for ( var i : Number = 0; i < line.length; i++ )  {
            if ( line[ i ].search( /^”\d+”$/ ) == 0 ) {
                line[ i ] = line[ i ].substr( 1, line[ i ].length-2 );
            }
     }
  }
  return line;
}
Now you can test your reader Implementation by using the follow below. NOTE: this is a hybrid Java and the syntax may be a little different if your using standard J2SE/J2EE etc..
while ( ( fieldValues = readCsvLine(reader)) != null) {
     //process line data here
}
Since we are using the readNext() method to read each line rather than using the read() the IO reads are processed in short bursts during each iteration rather than one lone continuous IO stream waiting for all the data to read. In situation where you are dealing with large amount of data this is a very safe way to ensure IO does not bottleneck.
Advertisements
 
Leave a comment

Posted by on June 9, 2012 in html, javascript, web