Why the following code takes more time locally?

I wrote the following lambda to move messages from queueA to queueB

async function reprocess_messages(fromQueue, toQueue) {
    try {
        const response1 = await sqs.send(new GetQueueUrlCommand({ QueueName: fromQueue }));
        const response2 = await sqs.send(new GetQueueUrlCommand({ QueueName: toQueue }));

        const fromQueueUrl = response1.QueueUrl;
        const toQueueUrl = response2.QueueUrl;

        let completed = false;
        while (!completed) {
            completed = await moveMessage(toQueueUrl, fromQueueUrl);
            // console.log(status);
        }
        // console.log(completed);
        return completed;

    } catch (err) {
        console.error(err);
    }
}


async function moveMessage(toQueueUrl, fromQueueUrl) {
    try {
        const receiveMessageParams = {
            MaxNumberOfMessages: 10,
            MessageAttributeNames: ["Messsages"],
            QueueUrl: fromQueueUrl,
            VisibilityTimeout: 2,
            WaitTimeSeconds: 0,
        };

        const receiveData = await sqs.send(new ReceiveMessageCommand(receiveMessageParams));
        // console.log(receiveData);

        if (!receiveData.Messages) {
            console.log("finished");
            return true;
        }

        const messages = [];
        receiveData.Messages.forEach(msg => {
            messages.push({ body: msg["Body"], receiptHandle: msg["ReceiptHandle"] });
        });

        const sendMsg = async ({ body, receiptHandle }) => {
            const sendMessageParams = {
                MessageBody: body,
                QueueUrl: toQueueUrl
            };
            await sqs.send(new SendMessageCommand(sendMessageParams));
            // console.log("Success, message sent. MessageID: ", sentData.MessageId);
            return "Success";
        };

        const deleteMsg = async ({ body, receiptHandle }) => {
            const deleteMessageParams = {
                QueueUrl: fromQueueUrl,
                ReceiptHandle: receiptHandle
            };
            await sqs.send(new DeleteMessageCommand(deleteMessageParams));
            // console.log("Message deleted", deleteData);
            return "Deleted";
        };

        const sent = await Promise.all(messages.map(sendMsg));
        // console.log(sent);
        await Promise.all(messages.map(deleteMsg));
        // console.log(deleted);
        console.log(sent.length);
        return false;

    } catch (err) {
        console.log(err);
    }
}


export const handler = async function (event, context) {
    console.log("Invoking lambda");
    const response = await reprocess_messages("queueA", "queueB");
    console.log(response);
}

With lambda config of 256 MB it takes 19691ms and with 512 MB it takes 10171ms to move 1000 messages from queueA to queueB. However, on my local system when I run reprocess_messages(qA, qB) it takes around 2 mins to move messages from queueA to queueB.

Does this mean that if I increase the memory limit to 1024 MB it will take only around 5000ms and how can I find the optimal memory limit?

Answer

It will most likely always be the case that running code on your local machine to interact with AWS services will be slower than if you were to run this code on an AWS service like Lambda. When your code is running on Lambda and interacting with AWS services in the same region, the latency is often only from the AWS network which can be drastically lower than the latency between your network and the AWS region you’re working with.

In terms of finding the optimal performance, this is trial and error. You are trying to find the sweet spot between price and performance. There are tools like compute optimizer that can assist you with this.

A useful note to bear in mind here is that as you increase Lambda memory, you can avail of further vCPU cores, it is roughly every 1,720mb more gives you another vCPU core. So while your current performance increase is tied to just a memory increase, if you were to increase to a number that gives an additional vCPU core, it may have a greater affect. Unfortunately it can be difficult to theorize the results and it is best just to trial and error the different scenarios