Scheduler in NodeJS

Introduction

What is scheduler?

Scheduling refers to the process of triggering a job, task, or function at a predetermined time or when certain events occur. Most job schedulers are based on cron, the time-based job scheduler in Unix-like systems.

Scheduler in NodeJS

There are several job schedulers for the Node.js runtime environment. Some of these schedulers run both in the browser and Node, while others run only in Node. These schedulers each have their own unique features, so choosing the best one for your use case can be a daunting and time-consuming process.

In this article, I will take you through one of popular and simple job scheduler called Bull in Node.js.

In addition to the key features and functionalities offered by this package, I will highlight useful functions and methods to schedule repeated jobs, time based jobs, one time jobs etc.

Introduction to Bull

Bull is a Node library that implements a fast and robust queue system based on redis.

Although it is possible to implement queues directly using Redis commands, this library provides an API that takes care of all the low-level details and enriches Redis basic functionality so that more complex use-cases can be handled easily.

Although its a queue, here we are more focusing on how we can use it as a scheduler. Let’s dive into installation.

Installation

$ npm install bull --save

Or

yarn add bull

In order to work with Bull, you also need to have a Redis server running. For local development you can easily install it using docker.

Bull will by default try to connect to a Redis server running on localhost:6379

Implementation

Let’s write Hello world using Bull.

const Queue = require('bull')
const queue = new Queue('my-first-queue')

// creating job
queue.add({
  content: 'Hello World'
})

// processing job
queue.process(job => {
  console.log(job.data.content)
})

Queue

new Queue(queueName: string, url?: string, opts?: QueueOptions)

This is the Queue constructor. It creates a new Queue that is persisted in Redis. Everytime the same queue is instantiated it tries to process all the old jobs that may exist from a previous unfinished session.

The optional url argument, allows to specify a redis connection string such as for example: redis://<redis-password>@<redis-host>:6379

For example,

const queue = new Queue('my-queue', 'redis://mypassword@localhost:6379')

// OR

const queue = new Queue('my-queue', {
  redis: {
    port: 6379,
    host: 'localhost',
    password: 'mypassword'
  }
})

Full API reference for Queue constructor.

Queue#add

add(name?: string, data: object, opts?: JobOpts): Promise<Job>

The .add() method creates a new job and adds it to the queue. If the queue is empty the job will be executed directly, otherwise it will be placed in the queue and executed as soon as possible.

An optional name can be added, so that only process functions defined for that name will process the job.

For example,

queue.add(
    'email-scheduler',
    {
        emailId: '[email protected]',
        subject: 'Birthday wishes!'
        content: 'Happy birthday John!'
    },
    {
        repeat: {
            cron: '0 0 18 12 *',
        }
    }
)

The above example will schedule a job that process at 12AM on December 18th of every year.

Full API reference for Queue#add constructor.

Queue#process

process(processor: ((job, done?) => Promise<any>) | string)

It defines a processing function for the jobs in a given Queue. It is a callback which is called every time a job is placed in the queue. It is passed an instance of the job as first argument.

For example,

queue.process(job => {
  // do the processing here
  EmailService.sendEmail(job.data)
})

If the callback signature contains the second optional done argument, the callback will be passed a done callback to be called after the job has been completed.

Full API reference for Queue#process constructor.

Job Lifecycle

In order to use the full potential of Bull queues, it is important to understand the lifecycle of a job. From the moment a producer calls the add method on a queue instance, a job enters a lifecycle where it will be in different states, until its completion or failure.