Async Loop Not Honoring Async

I have been a bit stuck on an Async function.

What I am trying to accomplish – I am creating a batchProcessing function (batchGetSubs) which will loop through a set of files, read a ID, then make an API request, wait for a response (THE ISSUE) and then to write to a new file with the formatted data.

The issue – I have tried both Async and Await, as well as pushing promises and trying to use a Promise.all to wait for promises to be resolved, but with no success. Current behavior is that I get all of my console.logs that are in the Promise.all section before the API calls actually return all the data. I have used these articles as reference:

• Asynchronous Process inside a javascript for loop
• Promise All with Axios
• Javascript Promise push value into array (only from function or outside?)

The Code –

async function batchGetSubs(data, command) {
    console.time('batchGetSubs')
    iteration = 1;
    let dataArray = []; promises = [];

    // LOOP THROUGH FILES, THEN GET TENANTS BY SUB
    for (i = iteration; i < totalIterations; i++) {
        let msIds = await loopAndDump(iteration);

        // LOOP THROUGH TENANTIDS AND GET SUBSCRIPTIONS
        msIds.map(async item => {
            let newRecord = await getSubsByTenantId(item);
            promises.push(await newRecord);
        });
        }
        Promise.all([promises]).then(() => { // FIXME: WHY IS THIS NOT WAITING FOR ALL RESPONSES?
            console.log(p1SubscriptionArray),
            console.timeEnd('batchGetSubs')
        });
    }



async function getSubsByTenantId(msTenantId) {
    let newRecord; let subscriptionArray = [];
    let bearerToken = p1BearerToken;
    let totalSubs = 0;
    const subIdConfig = {
        method: 'get',
        url: ``,
        headers: { 'Authorization': bearerToken }
    }
    await delay();
    await axios(subIdConfig)
        .then(async res => {
            console.log('AXIOS RESPONSE', res.data);
            if (res.data.items) {
                let subItems = res.data.items;
                console.log('^^^^^^^^^^^^^^^^^^', res.data.items)
                // LOOP THROUGH AND ADD TO SUBSCRIPTION ARRAY
                subItems.map(async subscription => {
                    if (subscription.friendlyName) {
                        totalSubs++;
                        subscriptionArray.push(subscription.friendlyName);
                    }
                });
                newRecord = { "tenantId": msTenantId, "totalSubs": totalSubs, "subscriptionFriendlyName": subscriptionArray };
            } else {
                // NO SUBS
                newRecord = { "tenantId": msTenantId, "totalSubs": totalSubs, "subscriptionFriendlyName": ['NONE'] };
                let statusCode, errorMessage;
                if (error && error.response === undefined) { // GETTING STATUS -4039 or -4077 INSTEAD OF 429 WHEN CHECKING SUBS. FORCE A RESEND.
                    if (error.errno && error.errno === -4039 || error.errno && error.errno === -4077) statusCode = 429; errorMessage = error.code;
                } else {
                    statusCode = error.response.status;
                    errorMessage = error.response.statusText;
                }
                console.error('ERROR:: SUBIDCONFIG SECTION THROWING ERROR: ', statusCode, portal, errorMessage);
                // SORT NON-200 CALLS BASED ON STATUS CODE
                switch (statusCode) {
                    case 403:
                        status403.push('subs', newRecord);
                        break;
                    case 404:
                        status404.push('subs', newRecord);
                        erroring = true;
                        break;
                    case 429:
                        status429.push('subs', newRecord);
                        erroring = true;
                        break;
                    default:
                        statusOther.push('subs', newRecord)
                        erroring = true;
                        break;
                }
            }
        })
        .catch(err => {
            newRecord = { "tenantId": msTenantId, "totalSubs": totalSubs, "subscriptionFriendlyName": ['NONE'] };
            console.error('ERROR: REQUEST IN GETSUBSBYTENANTID(): ', err)
        })
        .then(res => {
            console.log('DO WE HAVE ANY INFORMATION? ', newRecord);
            p1SubscriptionArray.push(newRecord);
            resolve();
        });
}

Answer

I only checked the first function, where you had put your question:

WHY IS THIS NOT WAITING FOR ALL RESPONSES?

For several reasons:

1. The promise array is still empty when you call Promise.all. This is because you only do a push after an await, and so that push happens asynchronously (read: later).

2. Even when the promises array gets populated, it will not have promises objects, but resolved values (i.e. newRecord values)

3. Even if promises would have been array of promises, you don’t pass that array correctly to Promise.all: you wrap that array in yet another array, which then only has one entry, and that entry is not a promise, but an array.

Not related to your problem, but:

• please make it a habit to explicitly declare all your variables. You did not do this for iteration, promises, nor i.

• Only use .map when you do something with the return value. For pure iteration use .forEach or for. In this case, you can use the return value to extend the promises array.

• If you intend to call batchGetSubs and need to know when all is done, then make sure it returns a useful promise: return Promise.all()

Here is the suggested correction for that function:

async function batchGetSubs(data, command) {
    console.time('batchGetSubs')
    let iteration = 1; // Declare!
    let dataArray = []; 
    let promises = []; // declare

    // declare i
    for (let i = iteration; i < totalIterations; i++) {
        let msIds = await loopAndDump(iteration);

        // Use the return value of the map method. No need for async callback
        promises.push(...msIds.map(item => {
            // Get the promise, not the resolved value, as that will come too late:
            return getSubsByTenantId(item);
        }));
    }
    // promises is already an array; don't make it an array of arrays.
    // And: return the resulting promise: it may be useful to the caller.
    return Promise.all(promises).then(() =>  {
        console.log(p1SubscriptionArray),
        console.timeEnd('batchGetSubs')
    });
}