Table of Contents
https://git.opendaylight.org/gerrit/#/q/topic:JC
Job Coordinator is a framework for executing jobs in sequential/parallel based on their job-keys. One such job,to give an example, can be for MD-SAL config/operational datastore updates.
The concept of datastore jobcordinator was derived from the following pattern seen in many ODL project implementations :
This feature will support following use cases:
The proposed feature adds a new module in infrautils called “jobcoordinator”, which will have the following functionalities:
N/A
Applications can define their own worker threads for their job. A job is defined as a piece of code that can be independently executed.
Applications should define a rollback worker, which will have the code to be executed in case the main job fails permanently. In usual scenarios, this will be the code to clean up all partially completed transactions by the main worker.
Applications should carefully choose the job-key for their job worker. All jobs based on the same job-key will be executed sequentially, and all jobs on different keys will be executed parallelly depending on the available threadpool size.
Applications can enqueue their job worker to JC framework for execution.JC has a hash structure to handle the execution of the tasks sequentially/parallelly. Whenever a job is enqueued, JC creates a Job Entry for the particular job. A Job Entry is characterized by - job-key, the main worker, the rollback worker and the number of retries. This JobEntry will be added to a JobQueue, which inturn is part of a JobQueueMap.
There is a JobQueueHandler task which runs periodically, which will poll each of the JobQueues to execute the main task of the corresponding JobEntry. Within a JobQueue, execution will be synchronized.
The list of listenable futures for the transactions from the application main worker will be available to JC, and if at all the transaction fails, the main worker will be retried the ‘max-retries’ number of times which is application specified. If all the retries fail, JC will bail out and the rollback worker will be executed.
This feature is aiming at improving the scale and performance of applications by providing the cabability to execute their functions parallelly wherever it can be done.
Carbon.
JC synchronization is not currently clusterwide.
N/A
This feature doesn’t add any new karaf feature.
JobCoordinator provides the below APIs which can be used by other applications:
void enqueueJob(String key, Callable<List<ListenableFuture<Void>>> mainWorker).
void enqueueJob(String key, Callable<List<ListenableFuture<Void>>> mainWorker, RollbackCallable rollbackWorker).
void enqueueJob(String key, Callable<List<ListenableFuture<Void>>> mainWorker, int maxRetries).
void enqueueJob(String key, Callable<List<ListenableFuture<Void>>> mainWorker, RollbackCallable rollbackWorker,
int maxRetries).
key is the JobKey for synchronization, mainWorker will be the actual Job Task, maxRetries is the number of times a Job will be retried if the mainWorker results in ERROR, rollbackWorker is the Task to be executed if the Job fails with any ERROR maxRetries times.
Appropriate UTs will be added for the new code coming in once framework is in place.
This will require changes to Developer Guide.
Developer Guide can capture the new set of APIs added by JobCoordinator as mentioned in API section.