Store large amount of data in Javascript

Im receiving some file chunks in bytes format from my server and im collecting them into one variable in my frontend to download it later. And I can’t change my server conception (receiving a file sliced into chunks).

My problem is that if the file is heavy (from 500MB), my variable length is starting to be very very big and im having an error :

RangeError: Invalid string length

This is because my variable has reach the limit of character (536 800 000).

Here’s how im adding my data to my variable :

this.socket.on('new_file', (data: string) => {
  this.receivedFile += data;
}

My download part :

public download(): void {
    const byteCharacters = this.receivedFile;
    const byteArrays = [];
    const sliceSize=512

    for (let offset = 0; offset < byteCharacters.length; offset += 
     sliceSize) {
      const slice = byteCharacters.slice(offset, offset + sliceSize);

      const byteNumbers = new Array(slice.length);
      for (let i = 0; i < slice.length; i++) {
        byteNumbers[i] = slice.charCodeAt(i);
      }

      const byteArray = new Uint8Array(byteNumbers);
      byteArrays.push(byteArray);
    }

    const blob = new Blob(byteArrays, {type: this.fileInfos.type});

    saveAs(blob, this.fileInfos.name);
  }

Which approach can I do ? Or does it exist some variable type in Javascript to accept more char ? Thanks

Answer

Don’t collect the chunks into a huge string. Instead, just convert each chunk into a bytearray immediately (that you’ll need later anyway) and collect these:

this.socket.on('new_file', (data: string) => {
  const bytes = new Uint8Array(data.length);
  for (let i = 0; i < data.length; i++) {
    bytes[i] = data.charCodeAt(i);
  }
  this.byteArrays.push(bytes);
}

then

public download(): void {
  const blob = new Blob(this.byteArrays, {type: this.fileInfos.type});
  saveAs(blob, this.fileInfos.name);
}

I don’t think you need to make 512-byte-sized slices.