Skip to content

Better way to schedule cron jobs based on job orders from php script

So I wrote simple video creator script in NodeJS.

It’s running on scheduled cron job.

I have a panel written in PHP, user enter details and clicks “Submit new Video Job” Button. This new job is saving to DB with details, jobId and status=”waiting” data.

PHP API is responsible for returning 1 status at a time, checks status=”waiting” limits query to 1 then returns data with jobID when asked

Video Creation Script requests every x seconds to that API asks for new job is available.

It has 5 tasks.

available=true.

  1. Check if new job order available (With GET Request in every 20 seconds), if has new job; available=false
  2. Get details (name, picture url, etc.)
  3. Create video with details.
  4. Upload Video to FTP
  5. Post data to API to update details. And Mark that job as “done”

available=true;

These tasks are async so everytask has to be wait previous task to be done.

Right now, get or post requesting api if new job available in every 20 seconds (Time doesnt mattter) seems bad way to me.

So any way / package / system to accomplish this behavior?

Code Example:

const cron = require('node-cron');
let available=true;

var scheduler = cron.schedule(
    '*/20 * * * * *',
    () => {
        if (available) {
            makevideo();
        }
    },
    {
        scheduled: false,
        timezone: 'Europe/Istanbul',
    }
);


let makevideo = async () => {
    available = false;
    let {data} = await axios.get(
        'https://api/checkJob'
    );
    if (data == 0) {
        console.log('No Job');
        available = true;
    } else {
        let jobid = data.id;
    await createvideo();
    await sendToFTP();
    
        await axios.post('https://api/saveJob', {
            id: jobid,
            videoPath: 'somevideopath',
        });
        available = true;
    }
};
scheduler.start();

Answer

While it is possible to use the database as a queue, it is commonly known as an anti-pattern (next to using the database for logging), and as you are looking for:

So any way / package / system to accomplish this behavior?

I use the free-form of your question thanks to the placed bounty to suggest: Beanstalk.

Beanstalk is a simple, fast work queue.

Its interface is generic, but was originally designed for reducing the latency of page views in high-volume web applications by running time-consuming tasks asynchronously.

It has client libraries in the languages you mention in your question (and many more), is easy to develop with and to run in production.