Work queues

 1. Introduction

There are many cases where an asynchronous process execution context is needed and the work queue (wq) API is the most commonly used mechanism for such cases. Some of the cases as bottom half for interrupt handler, some time a task scheduled for specific time etc.,

When such an asynchronous execution context is needed, a work item describing which function to execute is put on a queue.  An independent thread serves as the asynchronous execution context.  The queue is called work queue and the thread is called worker. While there are work items on the work queue the worker executes the functions associated with the work items one after the other.  When there is no work item left on the work queue the worker becomes idle.

Note: All the work items are schedulable, run in process context and can sleep.

 How work queues are implemented based on version (3.11.0)---> (Need to update the line numbers once again after releasing 3.11)

1) Initilization of workqueue happens in the kernel/workqueue.c

_init init_workqueues(void) {
  }
early_initcall(init_workqueues);

2) Initilizes the notifiers to take care of cpu up and down

 

cpu_notifier(workqueue_cpu_up_callback, CPU_PRI_WORKQUEUE_UP);
 hotcpu_notifier(workqueue_cpu_down_callback, CPU_PRI_WORKQUEUE_DOWN);

  wq_numa_init();

 3) 

4955        /* initialize CPU pools */
4956        for_each_possible_cpu(cpu) {
...............
...............
4960                for_each_cpu_worker_pool(pool, cpu) {
4961                        BUG_ON(init_worker_pool(pool));
.......
......
4974        /* create the initial worker */
4975        for_each_online_cpu(cpu) {


Initializes 5 work queues as described below.System-wide workqueues which are always present

system_wq is the one used by schedule[_delayed]_work[_on](). Multi-CPU multi-threaded.  There are users which expect relatively short queue flush time.  Don't queue works which can run for too long.

system_highpri_wq

------- TO DO  -------

 

system_long_wq is similar to system_wq but may host long running works.  Queue flushing might take relatively long.

system_unbound_wq is unbound workqueue.  Workers are not bound to any specific CPU, not concurrency managed, and all queued works are executed immediately as long as max_active limit is not reached and resources are available.

system_freezable_wq is equivalent to system_wq except that it's freezable

All these data structures are available globally.

struct workqueue_struct *system_wq __read_mostly;
struct workqueue_struct *system_highpri_wq __read_mostly;
struct workqueue_struct *system_long_wq __read_mostly;
struct workqueue_struct *system_unbound_wq __read_mostly;
struct workqueue_struct *system_freezable_wq __read_mostly;



rescuer_thread - the rescuer thread function

Workqueue rescuer thread function.  There's one rescuer for each workqueue which has WQ_MEM_RECLAIM set. Regular work processing on a pool may block trying to create a new worker which uses GFP_KERNEL allocation which has slight chance of developing into deadlock if some works currently on the same queue need to be processed to satisfy the GFP_KERNEL allocation.  This is the problem rescuer solves. When such condition is possible, the pool summons rescuers of all workqueues which have works queued on the pool and let them process those works so that forward progress can be guaranteed. This should happen rarely.

How to use workqueues

 

Functions exported

__init_work

destroy_work_on_stack
delayed_work_timer_fn
queue_delayed_work_on
mod_delayed_work_on
drain_workqueue

flush_workqueue

flush_scheduled_work
flush_work
flush_delayed_work

cancel_delayed_work

cancel_work_sync
cancel_delayed_work_sync

execute_in_process_context

__alloc_workqueue_key
destroy_workqueue
workqueue_set_max_active
workqueue_congested
work_busy
work_on_cpu

 

Initialization macros.

 

INIT_WORK( work, func );
__INIT_DELAYED_WORK

INIT_DELAYED_WORK( work, func );
INIT_DELAYED_WORK_DEFERRABLE( work, func );

INIT_WORK_ONSTACK
__INIT_DELAYED_WORK_ONSTACK

INIT_DELAYED_WORK
INIT_DELAYED_WORK_ONSTACK

INIT_DEFERRABLE_WORK
INIT_DEFERRABLE_WORK_ONSTACK

 

Difference between TASKLETS and Work Queues

Tasklets provide a low-latency mechanism that is simple and straightforward, while work queues provide a flexible API that permits queuing of multiple work items. Each defers work from the interrupt context, but only tasklets run atomically in a run-to-complete fashion, where work queues permit handlers to sleep, if necessary. Either method is useful for work deferral, so the method selected is based on your particular needs.
 

References:

1.       Linux/Documentation/workqueue.txt

2.       Kernel/workqueue.c

3.        lxr.linux.no ---->for source code browsing