Create big json object js

Tags: , , ,



I am using Nodejs to Create a JSON file from a really large JSON object (1GB). In order to avoid memory problems, I’m using createWriteStream :

var writeStream = fs.createWriteStream('./output/outPut.json')

After using Builder(custom function ) it will return a big object JSON.

the final step is to create this file :

 writeStream.write(JSON.stringify(search_index), (err) => {
             if (err) throw err
             console.log('File  Ready... ')
                    })

but unfortunately, JSON.stringify cannot be used with such heavy Object

JSON.stringify throws RangeError: Invalid string length for huge objects

Is there a solution to this issue, please?

Answer

A write stream is of no use if you only write one large string. You cannot use the native JSON.stringify here. There are however quite a few libraries available that will implement JSON serialisation into a stream (and even more for deserialising from a stream), use one of them.



Source: stackoverflow