Architecture

contivpp.io consists of several components, each of them packed and shipped as a Docker container. Two of them deploy on Kubernetes master node only:

and the rest of them deploy on all nodes within the k8s cluster (including the master node):

The following section briefly describes the individual Contiv components, which are displayed as orange boxes on the picture below:

contivpp arch

Contiv KSR

Contiv KSR (Kubernetes State Reflector)is an agent that subscribes to k8s control plane, watches k8s resources and propagates all relevant cluster-related information into the Contiv ETCD data store. Other Contiv components do not access the k8s API directly, they subscribe to Contiv ETCD instead. For more information on KSR, read the KSR Readme.

Contiv CRD netctl

Contiv CRD handles k8s Custom Resource Definitions defined in k8s API and processes them into configuration in Contiv ETCD. Currently it covers Contiv-specific configuration of individual k8s nodes such as IP address and default gateway, etc. Apart from this functionality, it also runs periodic validation of the topology, and exports the results as another CRD entry. The contiv-netctl tool which sits in the same Docker container can be used to explore runtime state of the cluster, such us current IPAM assignments, VPP state etc., or to execute a debug CLI on any of the VPPs in the cluster.

Contiv ETCD

Contiv-VPP uses its own instance of ETCD database for storage of k8s cluster-related data reflected by KSR, which are then accessed by Contiv vSwitch Agents running on individual nodes. Apart from the data reflected by KSR, ETCD also stores persisted VPP configuration of individual vswitches (mainly used to restore the operation after restarts), as well as some more internal metadata.

Contiv vSwitch

vSwitch is the main networking component that provides the connectivity to PODs. It deploys on each node in the cluster, and consists of two main components packed into a single Docker container: VPP and Contiv VPP Agent.

VPP is the data plane software that provides the connectivity between PODs, host Linux network stack and data-plane NIC interface controlled by VPP: - PODs are connected to VPP using TAP interfaces wired between VPP and each POD network namespace, - host network stack is connected to VPP using another TAP interface connected to the main (default) network namespace, - data-plane NIC is controlled directly by VPP using DPDK. Note that this means that this interface is not visible to the host Linux network stack, and the node either needs another management interface for k8s control plane communication, or STN (Steal The NIC) deployment must be applied.

Contiv-VPP Agent is the control plane part of the vSwitch container. It is responsible for configuring the VPP according to the information gained from ETCD and requests from Contiv STN. It is based on the Ligato VPP Agent
code with extensions that are related to k8s. For communication with VPP, it uses VPP binary API messages sent via shared memory using GoVPP. For connection with Contiv STN, the agent acts as a GRPC server serving CNI requests forwarded from the Contiv CNI.

Contiv CNI

Contiv CNI is a simple binary that implements the Container Network Interface API and is being executed by Kubelet upon POD creation and deletion. The CNI binary just packs the request into a GRPC request and forwards it to the Contiv VPP Agent running on the same node, which then processes it (wires/unwires the container) and replies with a response, which is then forwarded back to Kubelet.

Contiv STN

As already mentioned, the default setup of Contiv-VPP requires 2 network interfaces per node: one controlled by VPP for data facing PODs and one controlled by the host network stack for k8s control plane communication. In case that your k8s nodes do not provide 2 network interfaces, contivpp.io can work in the single NIC setup, when the interface will be “stolen” from the host network stack just before starting the VPP and configured with the same IP address on VPP, as well as on the host-VPP interconnect TAP interface, as it had in the host before it. For more information on STN setup, read the Single NIC Setup README.

Contivpp UI

Contivpp UI is composed of two components. The first is a customized GUI enabling the user to display the k8s cluster including the Contiv-VPP system pods. It also allows access to the configuration (e.g. IPAM), k8s, contivpp and namespace and k8s services mapped to contiv vswitches. The other component is a proxy providing REST APIs to the front-end GUI and per-vswitch APIs to the contiv vswitches deployed in the cluster. The contivpp UI is deployed as a docker container and is optional both in the production and demo systems.

Contivpp Flow

contivpp arch