Can someone explaint Difference between createReadStream
and readable
in node.js? By my observes they are similar, so what is under the hood difference, and when should each be used?
for example
const s3 = new AWS.S3({ accessKeyId: AWSConfig.AWSConfig.ACCESS_KEY_ID, secretAccessKey: AWSConfig.AWSConfig.SECRET_ACCESS_KEY, }); const params = { Bucket: AWSConfig.AWSConfig.bucket, Key: "somebucketName/1620072325205", }; const file = await s3.getObject(params).promise(); const fileSize = file.ContentLength / (1024 * 1024); const read = new Readable({ read(fileSize) { this.push(file.Body); this.push(null); }, }); read.pipe(res);
this is similar to
const s3 = new AWS.S3({ accessKeyId: AWSConfig.AWSConfig.ACCESS_KEY_ID, secretAccessKey: AWSConfig.AWSConfig.SECRET_ACCESS_KEY, }); const params = { Bucket: AWSConfig.AWSConfig.bucket, Key: "somebucketName/1620072325205", }; const file = await s3.getObject(params).createReadStream(); file.pipe(res)
Advertisement
Answer
In a NodeJS, you can create a readable stream in a few ways:
SOLUTION 1
You can do it with fs
module. The function fs.createReadStream()
allows you to open up a readable stream and all you have to do is pass the path of the file to start streaming in.
const fs = require('fs'); const readable_stream = fs.createReadStream('file_path');
SOLUTION 2
If you don’t want to create file, you can create an in-memory stream and do something with it (for example, upload it somewhere). You can do this with stream
module. You can import Readable
from stream
module and you can create a readable stream. When creating an object, you can also implement read()
method which is used to read the data out of the internal buffer. If no data available to be read, null
is returned. The optional size
argument specifies a specific number of bytes to read. If the size
argument is not specified, all of the data contained in the internal buffer will be returned.
const Readable = require('stream').Readable; const readable_stream = new Readable({ read(size) { // ... } });
SOLUTION 3
When you are fetching something over the network, that can be fetched like stream (for example you are fetching a PDF document from some API).
const axios = require('axios'); const readable_stream = await axios({ method: 'get', url: "pdf_resource_url", responseType: 'stream' }).data;
SOLUTION 4
Third party packages can support creating of streams as a feature. That is a way with aws-sdk
package from your example.
SUMMARIZE AND CONCLUSION
You can create a readable
stream in a few ways. Since you are already using aws-sdk
package, I would say that you should go with using their createReadStream()
, instead of importing stream
module and creating readable
stream with it.