.. contents:: Table of Contents :depth: 3 =============================================== Arbitrator Reconciliation using OpenFlow bundle =============================================== `Arbitrator Reconciliation Reviews `__ This spec addresses following enhancement in openflowplugin module: Addition of new reconciliation mode in openflowplugin which will allow applications to program flow/group within reconciliation window instead of frm reads and pushes the configuration down to the openflow switch. AUTONOMOUS mode implies that Openflowplugin shall perform reconciliation autonomously as it does now without any change in the workflow - ie. Switch-connection-handling followed by flow-based/bundle-based reconciliation execution followed by publishing of switch to the Inventory Operational datastore. This will be the default mode until arbitrated mode is enabled. ARBITRATED mode implies that the default openflowplugin reconciliation will be disabled and consumer application will have to initiate and complete the reconciliation including any error-handling. In the current implementation ARBITRATED mode will only supported bundle based reconciliation. Openflowplugin will switch to arbitrator reconciliation mode based on the upgradeState provided by ServiceUtils. Problem description =================== During replay based upgrade, the inventory configuration DS will be empty and applications has to program flows/groups based on the configuration pushed by user or external orchestrator. These new configurations has to applied on the switch without datapath disruption. This can be achieved using OpenFlow bundles. Bundle is a sequence of OpenFlow requests from odl controller that switch will apply in atomic transaction. Use Cases --------- Application controlled reconciliation of OpenFlow devices after controller re/start. Proposed change --------------- Arbitrator Reconciliation using bundles support will be provided. Openflowplugin will switch to arbitrator reconciliation based on the upgradeState provided by ServiceUtils. Orchestrator can enable or disable this mode as per their deployment requirements. upgradeInProgress presents in ServiceUtils project and can be changed to true to enable arbitrator reconciliation. .. code-block:: none :caption: serviceutils-upgrade-config.xml false ArbitratorReconciliation module registers itself with reconciliation framework with priority 1. When OpenFlow switch connect event received by Openflowplugin, it notifies Reconciliation Framework(RF). FlowNode Reconciliation will be notified first by RF as it registered with higher priority. FlowNode reconciliation module is the one responsible for reconciliation of OpenFlow node. It can be done either via flow/group based or OpenFlow bundle based. When upgradeInProgress is set to true, FlowNode reconciliation will be skipped as the config datastore will be empty and return success to the RF. RF callbacks Arbitrator Reconcilition to executes its task. Arbitrator Reconcilition will do the following steps in arbitrator-reconcilition(upgradeInProgress) mode * Open OpenFlow bundle on the connected OpenFlow switch and stores the bundle id in the local cache * Send delete-all groups and delete-all flows message to the opened bundle in the OpenFlow switch NOTE: Above clean up step is needed during upgrade to clean the previous version controller states, but the real switch clean-up will only happen when controller will commit the bundle. Arbitrator Reconciliation module sends success to RF if the previous steps are successful or it sends failure. RF notifies Openflowplugin with the completion state. * Success: Openflowplugin writes the OpenFlow node information into operational inventory datastore. * Failure: OpenFlow node will be disconnected and all the above steps will be repeated on the next reconnect till the mode is in arbitrator reconciliation Consumer application listening to inventory data store will receive Node added, Port status Data Tree Change Notification(DTCN) from data store. Applications programs flows and groups into config inventory datastore and Forwarding Rules Manager(FRM) application in in Openflowplugin receives DTCN from config inventory for the flows and groups. `Arbitrator Reconciliation exposes rpc to get Active bundle id for the OpenFlow node.` FRM Flow/Group Forwarder invokes get-active-bundle rpc and gets the bundle id. GetActiveBundle will executes the following steps. * Check if bundle commit is in progress for the requested node, if yes wait on commit bundle future * Returns Active bundle id and the same will be used by FRM forwarder to push the configuration via bundle add messages. * This call will return null in case of arbitrator-reconciliation disabled and FRM will push the configuration via normal Flow/Group messages. .. code-block:: none :caption: arbitrator-reconcile.yang rpc get-active-bundle { description "Fetches the active available bundle in openflowplugin"; input { uses "inv:node-context-ref"; leaf node-id { description "Node for which the bundle active has to be fetched"; type uint64; } } output { leaf result { description "The retrieved active bundle for the node"; type "onf-ext:bundle-id"; } } } Routed RPC will be exposed for committing the bundle on a specified Openflow node. It's orchestrator responsibility to commit the bundle across connected OpenFlow node. Configurations will be pushed only via OpenFlow bundles till the commit bundle rpc is invoked. .. code-block:: none :caption: arbitrator-reconcile.yang rpc commit-active-bundle { description "Commits the active available bundle for the given node in openflowplugin"; input { uses "inv:node-context-ref"; leaf node-id { description "Node for which the commit bundle to be executed"; type uint64; } } output { leaf result { description "Success/Failure of the commit bundle for the node"; type boolean; } } } Consumer application calls commit-active-bundle rpc with OpenFlow node id * It commits the current active bundle on the OpenFlow node and stores the future till it gets completed. * When bundle commit is in progress, configuration pushed via config datastore will be blocked on the commit future. This will make sure the new configuration is not lost during the transient state. The logic during arbitrator reconciliation will clear all the existing flows and groups and programs the new configuration and if we allow the flow programming during commit bundle phase, we might loose the new configuration. * When commit bundle is done, it will return the rpc result to the orchestrator and removes the future from the cache. * Subsequent flow/group provisioning will be done via flow-mod/group-mod messages. * Orchestrator can decide further actions based on the rpc result. Once commit bundle executes on all the connected OpenFlow switch, orchestrator can disable the arbitrator reconciliation by invoking rest rpc call on ServiceUtils `http://localhost:8383/restconf/config/odl-serviceutils-upgrade:upgrade-config/`. Subsequent OpenFlow switch connect/re-connect will go through FlowNode reconciliation. Note: There is no bundle timeout logic available as of now and the same will be added in future and will be kept as configurable parameter by user. Pipeline changes ---------------- None Yang changes ------------ Below yang changes will done to enable arbitrator reconciliation. RPC will be exposed to get current active bundle id for the given openflow node. .. code-block:: none :caption: arbitrator-reconcile.yang rpc get-active-bundle { description "Fetches the active available bundle in openflowplugin"; input { uses "inv:node-context-ref"; leaf node-id { description "Node for which the bundle active has to be fetched"; type uint64; } } output { leaf result { description "The retrieved active bundle for the node"; type "onf-ext:bundle-id"; } } } RPC will be exposed for external application/user/consumer applications to commit the active bundle for OpenFlow switch. .. code-block:: none :caption: arbitrator-reconcile.yang rpc commit-active-bundle { description "Commits the active available bundle for the given node in openflowplugin"; input { uses "inv:node-context-ref"; leaf node-id { description "Node for which the commit bundle to be executed"; type uint64; } } output { leaf result { description "Success/Failure of the commit bundle for the node"; type boolean; } } } Configuration impact -------------------- None Clustering considerations ------------------------- User can fire the commit-bundle rpc call to any controller node in the cluster. This rpc will only be executed by the node that currently be owning the device. Other Infra considerations -------------------------- None Security considerations ----------------------- None Scale and Performance Impact ---------------------------- Unknown Targeted Release ---------------- Flourine Alternatives ------------ Default reconciliation will be used or application can just reconfigure all the configuration using the normal flow/group add/remove process. Usage ===== None REST API -------- .. code-block:: none :caption: http://localhost:8181/restconf/operations/arbitrator-reconcile:get-active-bundle Output: ====== { "output": {1} } .. code-block:: none :caption: http://localhost:8181/restconf/operations/arbitrator-reconcile:commit-bundle-node { "input": { "node": "/opendaylight-inventory:nodes/opendaylight-inventory:node[opendaylight-inventory:id='openflow:']", "node-id": "" } } CLI --- None. Implementation ============== Assignee(s) ----------- Primary assignee: Arunprakash D Gobinath Suganthan Muthukumaran K Work Items ---------- * Implementation of arbritrator reconcile module * Changes in FRM for flow/group programming via openflow bundle * Read reconciliation mode(upgradeInProgress) from service utils * Expose RPC to commit bundle for a given OpenFlow node Dependencies ============ No new dependencies. Testing ======= Unit Tests ---------- CSIT ---- Documentation Impact ==================== None References ========== `Bundle Extension Support `__