Skip to content
Advertisement

Using Ajax GET to download compressed JSON files

Background.

I’m tasked with debugging some PHP and JavaScript code designed to pull static, gzip‘ed JSON files from the host server, and manipulate the resulting JSON object’s parameters.

Apologies in advance for my misuse of terminology. I have some experience with software development, but very little with web/server development (and almost none with PHP/JavaScript).

Code.

To “pull” the .json.gz file from the host server, jQuery’s Ajax function is used:

function myJsonReader() {
    var myJsonFileUrl = getUrl();
    
    function doFunStuffWithJsonFile(jsonObject) {
        // Fun things happen
    }

    $.ajax({
        url: myJsonFileUrl,
        type: "GET",
        dataType: "json", 
        success: doFunStuffWithJsonFile,
        error: function(jqxhr, status, exception) {
            console.log("Exception " + exception);
             alert('Failure:', exception);
             
        }
    }); 

}

Server-side, we have an index.php file, the first few lines of which are:

<?php
  header('Accept-Encoding: gzip');
  header('Content-Encoding: gzip');
?>

<html>
<head>
... some HTML code

The Ajax GET request fails. I know the file exists, and I’m passing the absolute path. Moreover, I’m able to read the compressed file if I use dataType: "text", but that, of course, returns “garbage”:

����%^��0��jN�L/��8�?@��x���351�g��+��~���ׯ���/߿�7���^��di9�y;jY��6�$K�=��4��QTMB�^or��M�P��̡�*}��G�t��K!#�8�Z@[d�9�#����R��N�����y��Պ�������;9�T����B+��VM�#��.:�<ĩ�F�PZ��Ȕl[K̔N[� GȡĚ�.5;P6�H��jeͮ��<�e""�h�-!�>��S�E��Q�m�H�.ڌSAyc�S�MsFHF�K]�H)/ry`�6.��&��-ME���s�����GA��@�rJ�.����)��kR�Vi�6��h�K-`��������

If I check the XHR response using:

error: function(xhr, status, error) {
        console.log(xhr.responseText);
        }

I also get garbage.

The file is a valid JSON file, readable by Vim with it’s built-in gunzip capabilities. Moreover, this code was previously able to read these gzipped JSON files without issue using an Ajax call and without the use of external libraries, until some unknown change is presumed to have been made (being messy and mostly undocumented, it hasn’t been possible to track down that change).

It is not possible to simply unzip all files we wish to read with this function, nor is it possible to make use of external libraries (e.g. zlib), unfortunately.

Several sources suggest this is should be relatively straight-forward (e.g. this StackOverflow post, and this one). Sadly, it isn’t clear to me what I’m missing.

Edit.

After re-loading the page, I check the Network tab in Chrome’s Inspector. I see the Fetch/XHR request for my JSON file. The headers for which include the following:

Requested Method: GET
Content-Type: application/x-gzip
Accept: application/json, text/javascript, */*; q=0.01
Accept-Encoding: gzip, deflate
Status Code: 200 OK

Which, to my untrained eye, seems to be properly configured for accepting compressed files.

Advertisement

Answer

If your server returns this header for the AJAX call:

Content-Type: application/x-gzip

without this one:

Content-Encoding: gzip

then the browser will not uncompress the response. For it to transparently uncompress it, you need something like this instead:

Content-Type: application/json
Content-Encoding: gzip

Adding Accept-Encoding or Content-Encoding headers to your HTML page (as you did) has of course no effect. You should remove them.

The headers sent by Apache depend on the way it is configured. Your server configuration may have been changed recently.

You have 2 solutions:

  • Modify your Apache configuration so that it sends the expected headers for .json.gz files
  • Use a PHP script to retrieve your compressed JSON files

Here is an example of such a script (getJSON.php):

<?php
$name = $_GET['name'];

header('Content-Type: application/json');
header('Content-Encoding: gzip');
readfile("path/to/$name.json.gz");
?>

The URL to retrieve test.json.gz would be:

url: 'getJSON.php?name=test'

Remark: as stated in a comment, a recommended practice is to check the value of $name (e.g. allow only letters and numbers) to forbid the use of ../ characters, which would allow an attacker to access .json.gz files outside of the intended directory.

UPDATE

You said in a comment that the following .htaccess file was present in the directory that contains the JSON files:

<filesMatch ".(gz)$">
    <ifModule mod_headers.c>
        Header set Content-Encoding "gzip"
    </ifModule>
</filesMatch>

The directive should send the Content-Encoding: gzip header for .gz files. However, .htaccess files may not be processed by Apache, depending on the value of the AllowOverride directive. If AllowOverride is set to None (and AllowOverrideList is also None, which is the case by default), then .htaccess files are ignored.

Note the default value for AllowOverride:

Default: AllowOverride None (2.3.9 and later), AllowOverride All (2.3.8 and earlier)

It means that, as of Apache 2.3.9, AllowOverride is set to None by default.

You said in a comment that you upgraded to version 2.4.6 recently. If AllowOverride is not explictly set in your Apache configuration, then it explains why your .htaccess file has no effect anymore.

A solution is to add this in your Apache configuration file:

<Directory /path/to/your/json/files/>
    AllowOverride All
</Directory>
User contributions licensed under: CC BY-SA
10 People found this is helpful
Advertisement