Accessing data parsed by readline & fs in node.js outside of the callback function

This question is different from the one linked as already having an answer. It is specifically this piece of code adapted from node.js documentation regarding the use of fs and readfile and looking for an end of file flag, which I’ve learned is readfile.close method. Thanks for the answer.

I wrote a small utility locally to try to convert a text file of key:value pairs with a blank line separating programs into a JSON file to use in a React project.

I got the foundation of the readline function directly from node.js documentation. I’m using node 6.9.0 on my mac

here is the full script:

const readline = require('readline');
const fs = require('fs');

const rl = readline.createInterface({
  input: fs.createReadStream('usat-ncaa-programs.txt')
});

var newPairs=["test"];
rl.on('line',
  function (line) {
  if (line===null){
    newPairs.push("}], [ {")
  } else if (line) {
    var keyValue = line.match(/^(.*):(.*)/)
    var newKeyValuePair =  "'" + keyValue[1].trim() + "':  '" + keyValue[2].trim() + "'"
    newPairs.push(newKeyValuePair)
    //console.log (newKeyValuePair)
  }

})

console.log(newPairs)

The input file looks like this (there are about 12 programs), i’ve only included 2 1/2 so you can see the format:

University: Arizona State University
Division: I
University Home Page: http://www.asu.edu/
Recruiting Link: https://questionnaire.acsathletics.com/Questionnaire/Questionnaire.aspx?&SPSID=1061112&SPID=177408&DB_LANG=C&DB_OEM_ID=30300&q=2015&s=159130&o=143
Team Homepage: http://www.thesundevils.com/index.aspx?path=triathlon
Head Coach: Cliff English
w: 480.965.0546
e: [e-mail]
bg-color: #990033
color: #FFB310

University: Belmont Abby College
Division: II
University Home Page: http://belmontabbeycollege.edu/
Recruiting Link: https://coach.scoutforce.com/p/414f3219dd
Team Homepage: http://abbeyathletics.com/sports/wtri/index
Head Coach: Nick Radkewich
w: 704.461.5010
e: [e-mail]
Twitter: https://twitter.com/AbbeyTri
bg-color: #FFFDD0
color: #DC143C

University:Black Hills State University 
Division: II
University Home Page: http://www.bhsu.edu/
...

My problem is that while I can read the text file line by line and parse some information that looks like a JSON file, I am unable to access that data outside of the callback function.

I don’t know how to save this data into a new file, or even just output the object to my console for a cut & paste and manually edit.

In teh above script the logged output of the variable newPairs is [“test”] rather than the line by line parsing that I’m trying to acccomplish.

If I place the console.log inside the callback, I get information logged with every iteration of the file read. I only want to work with the data when the file is done.

I did not find an EOF or similar flag in the node documentation for either fs or readline.

Additionally, if there is an easier way to get the data I am inputting into JSON format, I’d love to hear. Thanks in advance.

Answer

You have to understand that the callback function is executed asynchronously. This means that console.log(newPairs) gets executed before your callback, therefore it only results in “test”.

You should listen to Readline’s close event, like so:

rl.on('close', function() {
  console.log(newPairs);
});

As the documentation states:

The ‘close’ event is emitted when one of the following occur:

• The rl.close() method is called and the readline.Interface instance has relinquished control over the input and output streams;
• The input stream receives its ‘end’ event; The input stream receives -D to signal end-of-transmission (EOT);
• The input stream receives -C to signal SIGINT and there is no SIGINT event listener registered on the readline.Interface instance.
• The listener function is called without passing any arguments.

The readline.Interface instance should be considered to be “finished” once the ‘close’ event is emitted.

So this would be the ‘EOF’ you’re looking for 🙂