667377b849
This patch fixes some bugs related legacy cnf operations below: * Delete the "unsupported key check" of policies in TOSCA template, to resolve `openstack vnf create` command failure due to property validation of policies. * Change the branch condition of legacy/sol, to resolve the `openstack vnf scale` command failure due to keyError. * Add interval to each retry loops of delete_wait_legacy, to resolve the `openstack vnf delete` command failure. And this patch revises the documentation for legacy cnf operations to work properly. Closes-Bug: #1920085 Change-Id: If14b95b2f9dfafe994a5ebf4f447a2bf7d27981c
19 KiB
Experimenting containerized VNFs with Kubernetes VIM
In the past, Tacker only supports creating virtual machine based VNF using Heat. This section covers how to deploy containerized VNF using Kubernetes VIM in Tacker.
Prepare Kubernetes VIM
To use Kubernetes type of VNF, firstly user must register Kubernetes
VIM. Tacker supports Kubernetes authentication with two types: basic
authentication (username and password) or Bearer token. User can secure
the connection to Kubernetes cluster by providing SSL certificate. The
following vim-config.yaml file provides necessary
information to register a Kubernetes VIM.
auth_url: "https://192.168.11.110:6443"
username: "admin"
password: "admin"
project_name: "default"
ssl_ca_cert: None
type: "kubernetes"More details about registering Kubernetes VIM, please refer1
Sample container TOSCA templates
1. One container per VDU example
Currently, because Kubernetes does not support multiple networks such as choosing networks, connection points for applications, therefore users only deploys their applications with default networks (Pod and Service networks). In this case, user need to provide only information about VDU to create a VNF in Tacker.
The following example shows TOSCA template of containerized VNF for pure Kubernetes environment with one container per VDU.
tosca-vnfd-containerized-two-containers.yaml
tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0
description: A sample containerized VNF with two containers per VDU
metadata:
template_name: sample-tosca-vnfd
topology_template:
node_templates:
VDU1:
type: tosca.nodes.nfv.VDU.Tacker
properties:
namespace: default
mapping_ports:
- "80:80"
- "88:88"
service_type: NodePort
vnfcs:
front_end:
num_cpus: 0.5
mem_size: 512 MB
image: nginx
ports:
- "80"
rss_reader:
num_cpus: 0.5
mem_size: 512 MB
image: nickchase/rss-php-nginx:v1
ports:
- "88"
policies:
- SP1:
type: tosca.policies.tacker.Scaling
targets: [VDU1]
properties:
min_instances: 1
max_instances: 3
target_cpu_utilization_percentage: 40
default_instances: 1 # required parameter but ignored for cnf
increment: 1 # required parameter but ignored for cnfIn "vnfcs", there are 2 components: front_end and rss_reader. We model them as Containers2 inside a Pod3. To provide recover ability of these containers, we put all of containers inside a Deployment4 object, that can warrant the number of replica with auto-healing automatically.
The following table shows details about parameter of a Container. Config is translated to ConfigMap5, when user want to update the VNF (config), equivalent ConfigMap will be updated.
+-----------------------------------------------------------------------------------------------+
| vnfcs | Example | Description |
+-----------------------------------------------------------------------------------------------+
| name | front_end | Name of container |
+-----------------------------------------------------------------------------------------------+
| num_cpus | 0.5 | Number of CPUs |
+-----------------------------------------------------------------------------------------------+
| mem_size | 512 MB | Memory size |
+-----------------------------------------------------------------------------------------------+
| image | nginx | Image to launch container |
+-----------------------------------------------------------------------------------------------+
| ports | - "80" | Exposed ports in container |
+-----------------------------------------------------------------------------------------------+
| command | ['/bin/sh','echo'] | Command when container was started |
+-----------------------------------------------------------------------------------------------+
| args | ['hello'] | Args of command |
+-----------------------------------------------------------------------------------------------+
| config | param0: key1 | Set variables |
| | param1: key2 | |
+-----------------------------------------------------------------------------------------------+In Tacker, VDU is modeled as a Service6 in Kubernetes. Because Pods can be easily replaced by others, when the number of replica increased, workload should be shared between Pods. To do this task, we model VDU as Service, it acts as a Load balancer for Pods. Currently, we support some parameters as the following table.
+--------------------------------------------------------------------------------------------------------------------------------+
| VDU properties | Example | Description |
+--------------------------------------------------------------------------------------------------------------------------------+
| namespace | default | Namespace in Kubernetes where all objects are deployed |
+--------------------------------------------------------------------------------------------------------------------------------+
| mapping_ports | - "443:443" | Published ports and target ports (container ports) of Service Kubernetes |
| | - "80:8080" | |
+--------------------------------------------------------------------------------------------------------------------------------+
| labels | "app: webserver" | Labels which is set for Kubernetes objects, it is used as Selector to |
| | | Service can send requests to Pods |
+--------------------------------------------------------------------------------------------------------------------------------+
| service_type | ClusterIP | Set service type for Service object. |
| | | |
+--------------------------------------------------------------------------------------------------------------------------------+
| vnfcs | | Vnfcs are modeled by Containers and Deployment object. User can limit |
| | | resource, set image, publish container ports, set commands and variables |
+--------------------------------------------------------------------------------------------------------------------------------+User can also set scaling policy for VDU by adding the following policy. These information is translated to Horizontal Pod Autoscaler in Kubernetes. In the current scope, we just support auto-scaling with CPU utilization, more metrics will be added in the future.
policies:
- SP1:
type: tosca.policies.tacker.Scaling
targets: [VDU1]
properties:
min_instances: 1
max_instances: 3
target_cpu_utilization_percentage: 40
default_instances: 1 # required parameter but ignored for cnf
increment: 1 # required parameter but ignored for cnf2. Two containers per VDU example
Similar to the above example, in this scenario, we define 2 containers in VDU1.
tosca-vnfd-containerized.yaml
tosca_definitions_version: tosca_simple_profile_for_nfv_1_0_0
description: A sample containerized VNF with two containers per VDU
metadata:
template_name: sample-tosca-vnfd
topology_template:
node_templates:
VDU1:
type: tosca.nodes.nfv.VDU.Tacker
properties:
namespace: default
mapping_ports:
- "80:8080"
labels:
- "app: webserver"
service_type: ClusterIP
vnfcs:
web_server:
num_cpus: 0.5
mem_size: 512 MB
image: celebdor/kuryr-demo
ports:
- "8080"
config: |
param0: key1
param1: key2
policies:
- SP1:
type: tosca.policies.tacker.Scaling
targets: [VDU1]
properties:
min_instances: 1
max_instances: 3
target_cpu_utilization_percentage: 40
default_instances: 1 # required parameter but ignored for cnf
increment: 1 # required parameter but ignored for cnfViewing a containerized VNF
Create sample containerized VNF
$ openstack vnf descriptor create --vnfd-file tosca-vnfd-containerized.yaml VNFD1
Created a new vnfd:
+-----------------+-------------------------------------------------------------------------------------------------------+
| Field | Value |
+-----------------+-------------------------------------------------------------------------------------------------------+
| created_at | 2018-01-21 14:36:51.757044 |
| description | A sample containerized VNF with one container per VDU |
| id | fb4a0aa8-e410-4e73-abdc-d2808de155ef |
| name | VNFD1 |
| service_types | vnfd |
| template_source | onboarded |
| tenant_id | 2d22508be9694091bb2f03ce27911416 |
| updated_at | |
+-----------------+-------------------------------------------------------------------------------------------------------+
$ openstack vnf create --vnfd-name VNFD1 --vim-name vim-kubernetes VNF1
Created a new vnf:
+----------------+-------------------------------------------------------------------------------------------------------+
| Field | Value |
+----------------+-------------------------------------------------------------------------------------------------------+
| created_at | 2018-01-21 14:37:23.318018 |
| description | A sample containerized VNF with one container per VDU |
| error_reason | |
| id | 1faf776b-8d2b-4ee6-889d-e3b7c7310411 |
| instance_id | default,svc-vdu1-05db44 |
| mgmt_ip_address| |
| name | VNF1 |
| placement_attr | {"vim_name": "vim-kubernetes"} |
| status | PENDING_CREATE |
| tenant_id | 2d22508be9694091bb2f03ce27911416 |
| updated_at | |
| vim_id | 791830a6-45fd-468a-bd85-e07fe24e5ce3 |
| vnfd_id | fb4a0aa8-e410-4e73-abdc-d2808de155ef |
+----------------+-------------------------------------------------------------------------------------------------------+
$ openstack vnf list
+--------------------------------------+------+----------------------------+--------+--------------------------------------+--------------------------------------+
| id | name | mgmt_ip_address | status | vim_id | vnfd_id |
+--------------------------------------+------+----------------------------+--------+--------------------------------------+--------------------------------------+
| 1faf776b-8d2b-4ee6-889d-e3b7c7310411 | VNF1 | | ACTIVE | 791830a6-45fd-468a-bd85-e07fe24e5ce3 | fb4a0aa8-e410-4e73-abdc-d2808de155ef |
+--------------------------------------+------+----------------------------+--------+--------------------------------------+--------------------------------------+To test VNF is running in Kubernetes environment, we can check by running following commands
$ kubectl get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kubernetes ClusterIP 192.168.28.129 <none> 443/TCP 5h
svc-vdu1-05db44 ClusterIP 192.168.28.187 <none> 80/TCP 12m
$ kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
svc-vdu1-05db44 1 1 1 1 16m
$ kubectl get pod
NAME READY STATUS RESTARTS AGE
svc-vdu1-05db44-7dcb6b955d-wkh7d 1/1 Running 0 18m
$ kubectl get hpa
NAME REFERENCE TARGETS MINPODS MAXPODS REPLICAS AGE
svc-vdu1-05db44 Deployment/svc-vdu1-05db44 <unknown> / 40% 1 3 1 17m
$ kubectl get configmap
NAME DATA AGE
svc-vdu1-05db44 2 17mUser also can scale VNF manually, by running the following commands:
$ openstack vnf scale --scaling-policy-name SP1 --scaling-type out VNF1
$ kubectl get deployment
NAME DESIRED CURRENT UP-TO-DATE AVAILABLE AGE
svc-vdu1-651815 2 2 2 1 3h
$ kubectl get pods
NAME READY STATUS RESTARTS AGE
svc-vdu1-651815-5b894b8bfb-b6mzq 2/2 Running 0 3h
svc-vdu1-651815-5b894b8bfb-b7f2c 2/2 Running 0 40sIn the same way, user also scale in VNF with scaling-type is 'in'. The range of scaling manually is limited by 'min_instances' and 'max_instances' user provide in VNF template.
Multi-Interface for C-VNF
To use multi-interface for C-VNF, User should follow below procedure.
1. Checking kuryr.conf
After installation, user should check kuryr.conf configuration.
$ sudo cat /etc/kuryr/kuryr.conf | grep multi_vif_drivers
multi_vif_drivers = npwg_multiple_interfaces2. Adding K8s CustomResourceDefinition
To use CustomResourceDefinition, user needs to add CRD. User can make a additional network using yaml file like below. Create yaml file like below and register it.
$ cat ./crdnetwork.yamlapiVersion: apiextensions.k8s.io/v1beta1
kind: CustomResourceDefinition
metadata:
name: network-attachment-definitions.k8s.cni.cncf.io
spec:
group: k8s.cni.cncf.io
version: v1
scope: Namespaced
names:
plural: network-attachment-definitions
singular: network-attachment-definition
kind: NetworkAttachmentDefinition
shortNames:
- net-attach-def
validation:
openAPIV3Schema:
properties:
spec:
properties:
config:
type: stringRegister crdnetwork.yaml
$ kubectl create -f ~/crdnetwork.yamlGet crd list
$ kubectl get crd
NAME CREATED AT
kuryrnetpolicies.openstack.org 2019-07-31T02:23:54Z
kuryrnets.openstack.org 2019-07-31T02:23:54Z
network-attachment-definitions.k8s.cni.cncf.io 2019-07-31T02:23:55Z3. Adding neutron subnet id information to k8s CRD
To use neutron subnet in kubernetes, user should register neutron subnet to CRD. At first, user should create subnet.yaml file.
$ cat ./kuryr-subnetname1.yaml
apiVersion: "k8s.cni.cncf.io/v1"
kind: NetworkAttachmentDefinition
metadata:
name: subnetname1
annotations:
openstack.org/kuryr-config: '{"subnetId": "$subnet_id"}'After making a yaml file, user should create subnet with yaml file.
$ kubectl create -f ~/kuryr-subnetname1.yamlAfter created, user can check subnet info like below.
$ kubectl get net-attach-def
NAME AGE
k8s-multi-10 7d
k8s-multi-11 7dKnown Issues and Limitations
- Does not support Volumes in Kubernetes
- Horizontal Pod AutoScaler only support CPU utilization
- Add support Kuryr-Kubernetes for making hybrid network in the future
References
https://opendev.org/openstack/tacker/src/branch/master/doc/source/install/kubernetes_vim_installation.rst↩︎
https://kubernetes.io/docs/concepts/workloads/pods/init-containers↩︎
https://kubernetes.io/docs/concepts/workloads/pods/pod-overview↩︎
https://kubernetes.io/docs/concepts/workloads/controllers/deployment↩︎
https://kubernetes.io/docs/tasks/configure-pod-container/configure-pod-configmap↩︎
https://kubernetes.io/docs/concepts/services-networking/service↩︎