Queue API: Process execution by parts

The latest version of Drupal aims to be more than just a content management system (CMS) by incorporating tools characteristic of a framework. These improvements include enhanced form-building, database queries, caching systems, library theming, and more. However, one often-overlooked feature is the Queue API. Let’s explore what it is and how it works.

What is the Queue API?

The Queue API in Drupal manages task instructions similar to a process queue in other development environments. It serves as a repository of tasks awaiting execution, organized as a task list that Drupal processes in an orderly, programmer-friendly way. Using familiar concepts of queues as data structures, this API supports multiple operations commonly seen in other programming languages, making it easy for developers to leverage queue-based processing.

Use Case for the Queue API: Sending Mass Emails

To illustrate the use of the Queue API, consider a scenario where a form sends a message to all users of a specific role. At first glance, the logic for building and sending emails might not appear too complex. However, processing can become an issue when dealing with a large user base. Here are some alternative approaches:

Alternative 1: Sending Emails with the Form Submission

In this approach, we query the database for users of a particular role, then call a function to send each an email. While straightforward, this approach falters with scale. If there are around 200 users, for example, the process may fail due to excessive time or memory usage as the server terminates it prematurely.

Alternative 2: Caching User Data

Another option is to save user data in the cache (either system or custom) and retrieve it later using a hook. Although helpful, this approach may still overload the server as the caching process can be as resource-intensive as sending the emails directly.

Alternative 3: Leveraging the Queue API

The Queue API provides a superior solution. It allows developers to save tasks in different queues, which Drupal can later process systematically. With the Queue API, instead of overwhelming the server with a bulk process, tasks are organized for subsequent execution, allowing developers to inform users that emails will be sent without burdening the system immediately.

How the Queue API Works

The Queue API operates in conjunction with cron, a scheduler that automates the execution of queued tasks. Here’s how the process works, following the mass email example:

1. Generate User Information for Emails Define a function to create a list of users and their corresponding email information.

   function mymodule_pending_users_info() {  return array(    array('subject' => 'Asunto 1', 'from' => 'mail1@example.com', 'to' => 'mail2@example.com', 'message' => '...'),    array('subject' => 'Asunto 2', 'from' => 'mail3@example.com', 'to' => 'mail4@example.com', 'message' => '...'),    // Additional items...  ); } 

2. Send Email to Each User Define the function that will send an email based on the provided user information.

   function mymodule_send_mail_to_user($user_info) {  drupal_mail(...); } 

3. Store Data in a Queue Use the Queue API to store each user’s information as a task in the queue. Here’s an example code snippet:

   $queue = DrupalQueue::get('my_queue'); foreach(mymodule_pending_users_info() as $item) {  $queue->createItem($item); } 

   This code only saves data in queues without fully utilizing the Queue API’s potential. Processing items in a single call may still burden PHP.

4. Automate Task Execution with Cron Drupal’s hook_cron_queue_info integrates the queue with cron, automating task execution without manual intervention. Here’s how to implement it:

   /** * Implements hook_cron_queue_info(). */ function mymodule_cron_queue_info() {  $queues = array();  $queues['my_cron_queue'] = array(    'worker callback' => 'mymodule_send_mail_to_user',    'time' => 60, // Time limit for each cron job  );  return $queues; } 

With this setup, Drupal handles queue operations (creation, retrieval, space management) and executes queued tasks without further user requests. Instead, tasks are processed according to the cron schedule on the server.

Final Thoughts

The Queue API is a powerful tool for Drupal developers. By queuing tasks and integrating with cron, it enables efficient handling of large processes without overloading the server. This functionality makes it ideal for repetitive or resource-heavy tasks, ensuring smooth performance and a better user experience.

Happy Drupal-ing!

We recommend you this video