From edadd64d7bf77fcb80e5693b8b8246a19bfd0460 Mon Sep 17 00:00:00 2001 From: Litao Gao Date: Mon, 29 Nov 2021 05:13:00 -0500 Subject: [PATCH] Add documentation of Sample applications on StarlingX - Add a new top level section for Sample Application Deployments - Add doc of FlexRAN 21.07 build, deploy and run on StarlingX Story: 2009750 Task: 44207 Signed-off-by: Litao Gao Change-Id: I1ecfe52e8061275470674e2fc1285380b9b3c06c --- .../bare_metal/aio_simplex_hardware.rst | 4 +- doc/source/index.rst | 9 + doc/source/sample_apps/flexran.rst | 844 ++++++++++++++++++ doc/source/sample_apps/index.rst | 14 + 4 files changed, 870 insertions(+), 1 deletion(-) create mode 100644 doc/source/sample_apps/flexran.rst create mode 100644 doc/source/sample_apps/index.rst diff --git a/doc/source/deploy_install_guides/r6_release/bare_metal/aio_simplex_hardware.rst b/doc/source/deploy_install_guides/r6_release/bare_metal/aio_simplex_hardware.rst index 6a3ca6c42..3e214279f 100644 --- a/doc/source/deploy_install_guides/r6_release/bare_metal/aio_simplex_hardware.rst +++ b/doc/source/deploy_install_guides/r6_release/bare_metal/aio_simplex_hardware.rst @@ -1,3 +1,5 @@ +.. _aio_simplex_hardware_r6: + ===================== Hardware Requirements ===================== @@ -55,4 +57,4 @@ host types are: Prepare bare metal servers -------------------------- -.. include:: prep_servers.txt \ No newline at end of file +.. include:: prep_servers.txt diff --git a/doc/source/index.rst b/doc/source/index.rst index 91e34a92e..e378fd3a0 100755 --- a/doc/source/index.rst +++ b/doc/source/index.rst @@ -193,6 +193,15 @@ Distributed Cloud dist_cloud/index +------------------------------ +Sample Application Deployments +------------------------------ + +.. toctree:: + :maxdepth: 2 + + sample_apps/index + ----------------- Deprecated guides ----------------- diff --git a/doc/source/sample_apps/flexran.rst b/doc/source/sample_apps/flexran.rst new file mode 100644 index 000000000..58d8edf56 --- /dev/null +++ b/doc/source/sample_apps/flexran.rst @@ -0,0 +1,844 @@ +.. _deploy-flexran-on-starlingx-c4efa00b1b98: + +=========================== +Deploy FlexRAN on StarlingX +=========================== + +.. contents:: + :local: + :depth: 1 + +----- +Scope +----- + +`FlexRAN `__ +is a vRAN reference implementation for virtualized cloud-enabled radio access networks. +FlexRAN is not an open-source project. It is provided here as an example of a 5G application running on |prod|. + +In this document, based on |prod| solution, the details on how to build +FlexRAN software, how to generate containerized version of the prebuilt +FlexRAN binaries, and deploy on |prod| solution are provided. + +.. note:: + All of the content is generated based on FlexRAN 21.07, the + instructions are subject to change in future releases. + +----------------- +Intended Audience +----------------- + +The intended audience for this document are software engineers and +system architects who want to design and develop 5G systems using the +O-RAN Specifications based on FlexRAN and |prod| OpenSource Edge +computing platform. + +------------------------ +AIO Simplex Installation +------------------------ + +************************************************* +Prepare USB stick with StarlingX Installation ISO +************************************************* + +#. Get StarlingX Installation ISO from following location + + http://mirror.starlingx.cengn.ca/mirror/starlingx/master/centos/latest_build/outputs/iso/bootimage.iso + +#. Burn the image to a USB stick + + .. note:: + Be sure to use the correct USB device name when copying the image. + + .. code:: + + dd if=bootimage.iso of=/dev/sdc bs=1M + +***************************** +Follow the installation guide +***************************** + +https://docs.starlingx.io/deploy_install_guides/r6_release/bare_metal/aio_simplex.html + +Besides :ref:`aio_simplex_hardware_r6` for |prod|, additional hardware is required for FlexRAN application. + ++---------------------------+--------------------------------------------------------------------------------+ +| Minimum Requirement | All-in-one Controller Node | ++===========================+================================================================================+ +| Minimum processor class | Single-CPU Intel Xeon Cascade Lake (14 nm) or IceLake (10 nm) | ++---------------------------+--------------------------------------------------------------------------------+ +| Minimum memory | 64 GB single socket | ++---------------------------+--------------------------------------------------------------------------------+ +| Minimum network ports | OAM: 1x1GE, If only test timer mode, no other NIC required. | ++---------------------------+--------------------------------------------------------------------------------+ +| BIOS settings | - Hyper-Threading technology: enabled; | +| | - Virtualization technology: enabled; | +| | - VT for directed I/O: enabled; | +| | - CPU power and performance policy set to performance; | +| | - CPU C state control: enabled; | +| | - Plug & play BMC detection: disabled; | +| | - Uncore Frequency Scaling : Disabled; | +| | - Performance P-limit : Disabled; | +| | - Enhanced Intel SpeedStep (R) Tech : Enabled; | +| | - Intel(R) Turbo Boost Technology : Enabled | ++---------------------------+--------------------------------------------------------------------------------+ +| Accelerator Card | Mt. Bryce ACC100 (Intel eASIC chip which can be mounted on third party card) | ++---------------------------+--------------------------------------------------------------------------------+ + +The FlexRAN application on |prod| has been tested on Intel Reference Hardware platform: **Wilson City** (housing ICX-SP). + +.. note:: + Some third party platforms like SuperMicro / HPE / Dell / Quanta / + others can also be used based on customer platform requirements + with the optimization for low-latency and power savings mode by the + platform vendors. + +:ref:`aio_simplex_install_kubernetes_r6`: + +#. In addition to required |prod| configuration, ceph backend for Kubernetes PVC, isolcpus and hugepages + + .. code:: bash + + source /etc/platform/openrc + + NODE=controller-0 + OAM_IF= + + # if you use flat oam network + system host-if-modify ${NODE} $OAM_IF -c platform + system interface-network-assign ${NODE} $OAM_IF oam + + # if you use vlan oam network + VLANID= + system host-if-modify -n pltif -c platform $NODE $OAM_IF + system host-if-add ${NODE} -V $VLANID -c platform oam0 vlan pltif + system interface-network-assign ${NODE} oam0 oam + + system host-label-assign $NODE sriovdp=enabled + system host-label-assign $NODE kube-topology-mgr-policy=restricted + + # ceph backend for k8s pvc + system storage-backend-add ceph --confirmed + system host-disk-list ${NODE} | awk '/\/dev\/sdb/{print $2}' | xargs -i system host-stor-add ${NODE} {} + + # isolate cpus depends on number of the physical core + system host-cpu-modify -f application-isolated -p0 28 controller-0 + + # allocate/enable hugepages for dpdk usage + system host-memory-modify $NODE -1G 10 0 + + system host-unlock $NODE + +#. After the system has been unlocked and available for the 1st time, configure ACC100 / Mount Bryce. + + .. code:: bash + + source /etc/platform/openrc + + system host-lock $NODE + + # get the device name of the Mout Bryce, we assume it is pci_0000_85_00_0 here. + system host-device-list controller-0 + + # Modify the Mount Bryce device to enable it, specify the base driver and vf driver, and configure it for 1 VFs + + # NOTE: If this is the initial install and have not unlocked, you will get following error message. + # Cannot configure device 73b13ddf-99be-44c8-8fbe-db85eb8d99ba until host controller-0 is unlocked for the first time. + system host-device-modify controller-0 pci_0000_85_00_0 -e true --driver igb_uio --vf-driver vfio -N 1 + + system host-unlock $NODE + +----------------------------- +FlexRAN Software Prerequisite +----------------------------- + +***************************** +FlexRAN 21.07 Release Package +***************************** + +FlexRAN Software Wireless Access Solutions is available from the following page: + +https://www.intel.com/content/www/us/en/developer/topic-technology/edge-5g/tools/flexran.html + + +******************************* +FlexRAN DPDK BBDEV v21.07 Patch +******************************* + +This patch file is also available in FlexRAN Software Wireless Access +Solutions mentioned above. + +******************** +DPDK version 20.11.1 +******************** + +DPDK version 20.11.1 is available in http://static.dpdk.org/rel/dpdk-20.11.1.tar.xz + +****************** +Intel C++ Compiler +****************** + +The Intel C++ Compiler is used to compile Intel |DPDK| and L1 software. The +Intel C++ Compiler can be obtained using the following link +https://software.intel.com/en-us/system-studio/choose-download with +community license. Recommended version of Compiler: **icc (ICC) +19.0.3.206 20190206** + +----------------------------- +FlexRAN Build, Deploy and Run +----------------------------- + +Generally speaking, the building environment and execution environment +should not be the same environment. In order to facilitate the building, +deploying, and running the process on |prod|, a containerized building +environment has been prepared and verified, developers can use the +instructions to build the building docker image themselves or use +the prebuilt docker image directly. + +Using this method, developers can: + +#. Start the building soon after |prod| is ready +#. Use the scripts provided to generate a docker image with pre-built + FlexRAN binaries +#. Launch the FlexRAN Pod using the image just generated +#. Execute L1 test cases + +The following procedures provide detailed instructions for completing the stages described above. + +************************* +FlexRAN build preparation +************************* + +For details, see: + +https://www.intel.com/content/www/us/en/developer/topic-technology/edge-5g/tools/flexran.html + +You can find build instructions in the Compilation Chapter of :title:`FlexRAN 5GNR Reference Solution 21.07`. + +The following steps provide a quick-start procedure for developers. + +#. Create a |PVC| for FlexRAN build storage. + + .. note:: + The PVC size should be larger than 70G + + .. code:: bash + + cat > volume-ceph.yaml << 'EOF' + kind: PersistentVolumeClaim + apiVersion: v1 + metadata: + name: flexran-storage + spec: + accessModes: + - ReadWriteOnce + resources: + requests: + storage: 80Gi + storageClassName: general + EOF + + kubectl create -f volume-ceph.yaml + + kubectl create -f volume-ceph.yaml + persistentvolumeclaim/flexran-storage created + controller-0:~$ kubectl get pvc + NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS AGE + flexran-storage Bound pvc-43e50806-785f-440b-8ed2-85bb3c9e8f79 80Gi RWO general 9s + +#. Launch the `quick start building Pod `__ attaching to the |PVC|. + + .. note:: + This pod is assumed to be assigned enough resource to launch quickly after FlexRAN being built. + If you don't have isolated CPU, hugepage and accelerator resources configured as part of the + system used for building, just feel free to remove related content from the yaml spec file, + hugepages-1Gi and intel.com/intel_acc100_fec are not required for the building. + + .. code:: bash + + cat > flexran-buildpod.yml << 'EOF' + apiVersion: v1 + kind: Pod + metadata: + name: buildpod + annotations: + spec: + restartPolicy: Never + containers: + - name: buildpod + image: wrsnfv/flexran-builder:centos7.9 + imagePullPolicy: IfNotPresent + volumeMounts: + - name: usrsrc + mountPath: /usr/src + - mountPath: /hugepages + name: hugepage + - name: lib-modules + mountPath: /lib/modules + - name: pvc1 + mountPath: /opt + - name: docker-sock-volume + mountPath: /var/run/docker.sock + command: ["/bin/bash", "-ec", "sleep infinity"] + securityContext: + privileged: true + capabilities: + add: + ["IPC_LOCK", "SYS_ADMIN"] + resources: + requests: + memory: 32Gi + hugepages-1Gi: 10Gi + intel.com/intel_acc100_fec: '1' + limits: + memory: 32Gi + intel.com/intel_acc100_fec: '1' + hugepages-1Gi: 10Gi + volumes: + - name: usrsrc + hostPath: + path: /usr/src + - name: lib-modules + hostPath: + path: /lib/modules + - name: hugepage + emptyDir: + medium: HugePages + - name: docker-sock-volume + hostPath: + path: /var/run/docker.sock + type: Socket + - name: pvc1 + persistentVolumeClaim: + claimName: flexran-storage + EOF + + kubectl create -f flexran-buildpod.yml + +#. (Optional) Instructions of the FlexRAN building image creation. + + .. note:: + You can use following instructions to build the default image or a customized version to meet your needs. + + .. code:: bash + + mkdir dockerbuilder && cd dockerbuilder + + # prepare the artifacts used for FlexRAN prebuilt binary docker image + mkdir docker-image-building + cat > docker-image-building/readme << 'EOF' + # Instructions of docker image generation + + # Following steps are supposed to be executed inside building Pod, + # after building FlexRAN from source code + + flxr_install_dir=/opt/flexran/ + # populate flexran related env var + cd ${lxr_install_dir} + source set_env_var.sh -d + + # prepare the FlexRAN binaries + ./transport.sh + + # build the docker image + docker build -t flr-run -f Dockerfile . + + # tag and push + orgname=somename + docker tag flr-run ${orgname}/flr-run + + EOF + + cat > docker-image-building/transport.sh << 'EOF' + #!/bin/bash + + # ICCPATH=/opt/intel/system_studio_2019/compilers_and_libraries_2019.5.281/linux + + echo "Make sure source set_env_var.sh -d first.(located in FlexRAN installation directory)" + + [[ -z "$MKLROOT" ]] && { echo "MKLROOT not set, exit..."; exit 1; } + [[ -z "$DIR_WIRELESS_SDK_ROOT" ]] && { echo "DIR_WIRELESS_SDK_ROOT not set, exit..."; exit 1; } + + ICCPATH=`echo $MKLROOT | awk -F '/mkl' '{print $1}'` + FLXPATH=`echo $DIR_WIRELESS_SDK_ROOT| awk -F '/sdk' '{print $1}'` + + [[ -d stuff ]] && { echo "Directory stuff exists, move it to old."; mv -f stuff stuff.old; } + + mkdir stuff; cd stuff + + mkdir libs + cp $ICCPATH/mkl/lib/intel64_lin/libmkl_intel_lp64.so libs + cp $ICCPATH/mkl/lib/intel64_lin/libmkl_core.so libs + cp $ICCPATH/mkl/lib/intel64_lin/libmkl_intel_thread.so libs + cp $ICCPATH/mkl/lib/intel64_lin/libmkl_avx512.so libs + cp $ICCPATH/mkl/lib/intel64_lin/libmkl_avx2.so libs + cp $ICCPATH/mkl/lib/intel64_lin/libmkl_avx.so libs + cp $ICCPATH/ipp/lib/intel64/libipps.so libs + cp $ICCPATH/ipp/lib/intel64/libippe.so libs + cp $ICCPATH/ipp/lib/intel64/libippcore.so libs + cp $ICCPATH/ipp/lib/intel64_lin/libippee9.so libs + cp $ICCPATH/ipp/lib/intel64_lin/libippse9.so libs + cp $ICCPATH/compiler/lib/intel64_lin/libiomp5.so libs + cp $ICCPATH/compiler/lib/intel64_lin/libirc.so libs + cp $ICCPATH/compiler/lib/intel64_lin/libimf.so libs + cp $ICCPATH/compiler/lib/intel64_lin/libsvml.so libs + cp $ICCPATH/compiler/lib/intel64_lin/libintlc.so.5 libs + cp $ICCPATH/compiler/lib/intel64_lin/libirng.so libs + + cp $FLXPATH/libs/cpa/bin/libmmwcpadrv.so libs + cp $FLXPATH/wls_mod/libwls.so libs + + mkdir -p flexran/sdk/build-avx512-icc/ + cp -rf $FLXPATH/sdk/build-avx512-icc/source flexran/sdk/build-avx512-icc/ + cp -rf $FLXPATH/sdk/build-avx512-icc/install flexran/sdk/build-avx512-icc/ + cp -rf $FLXPATH/bin flexran/ + cp -rf $FLXPATH/set_env_var.sh flexran/ + + # testcase files + mkdir -p tests/nr5g/ + cd tests/nr5g/ + for cfg in $FLXPATH/bin/nr5g/gnb/testmac/icelake-sp/*.cfg + do + cat $cfg | grep TEST_FD > /tmp/$$.testfile + while IFS= read line + do + array=($(echo "$line" | sed 's/5GNR,/ /g')) + for i in "${array[@]}"; do + if [[ "$i" =* \.cfg ]]; then + casedir=`echo "$i"| cut -d / -f 1-3 | xargs` + caseabsdir=$FLXPATH/tests/nr5g/$casedir + [[ ! -d $casedir ]] && { mkdir -p $casedir; cp -rf $caseabsdir/* $casedir; } + fi + done + done < /tmp/$$.testfile + done + + echo "Transportation Completed." + EOF + + chmod a+x docker-image-building/transport.sh + + cat > docker-image-building/set-l1-env.sh << 'EOF' + # source this script to l1 binary location + + export WORKSPACE=/root/flexran + export isa=avx512 + + cd $WORKSPACE + source ./set_env_var.sh -i ${isa} + + MODE=$1 + [[ -z "$MODE" ]] && read -p "Enter the MODE(LTE or 5G): " MODE + + if [ $MODE = LTE ]; then + cd $WORKSPACE/bin/lte/l1/ + fi + if [ $MODE = 5G ]; then + cd $WORKSPACE/bin/nr5g/gnb/l1 + fi + EOF + + cat > docker-image-building/set-l2-env.sh << 'EOF' + # source this script to l2 binary location + + export WORKSPACE=/root/flexran + export isa=avx512 + + cd $WORKSPACE + source ./set_env_var.sh -i ${isa} + + MODE=$1 + [[ -z "$MODE" ]] && read -p "Enter the MODE(LTE or 5G): " MODE + + if [ $MODE = LTE ]; then + cd $WORKSPACE/bin/lte/testmac/ + fi + if [ $MODE = 5G ]; then + cd $WORKSPACE/bin/nr5g/gnb/testmac + fi + EOF + + cat > docker-image-building/res-setup.sh << 'EOF' + #!/bin/bash + + [[ -z "$PCIDEVICE_INTEL_COM_INTEL_ACC100_FEC" ]] && { echo "ACC100 not used, sleep..."; sleep infinity; } + + sed -i 's#.*dpdkBasebandFecMode.*# 1#' /root/flexran/bin/nr5g/gnb/l1/phycfg_timer.xml + sed -i 's#.*dpdkBasebandDevice.*# '"$PCIDEVICE_INTEL_COM_INTEL_ACC100_FEC"'#' /root/flexran/bin/nr5g/gnb/l1/phycfg_timer.xml + + echo "Resource setup Completed, sleep..." + sleep infinity + EOF + + chmod a+x docker-image-building/res-setup.sh + + cat > docker-image-building/Dockerfile << 'EOF' + FROM centos:7.9.2009 + + RUN [ -e /etc/yum.conf ] && sed -i '/tsflags=nodocs/d' /etc/yum.conf || true + + RUN yum install -y libhugetlbfs* libstdc++* numa* gcc g++ iproute \ + module-init-tools kmod pciutils python libaio libaio-devel \ + numactl-devel nettools ethtool + RUN yum clean all + + COPY stuff/libs/* /usr/lib64/ + + WORKDIR /root/ + COPY stuff/flexran ./flexran + COPY stuff/tests ./flexran/tests + COPY set-l1-env.sh ./ + COPY set-l2-env.sh ./ + COPY res-setup.sh ./ + + CMD ["/root/res-setup.sh"] + EOF + + cat > Dockerfile << 'EOF' + FROM centos:7.9.2009 + + RUN [ -e /etc/yum.conf ] && sed -i '/tsflags=nodocs/d' /etc/yum.conf || true + + RUN yum groupinstall -y 'Development Tools' + + RUN yum install -y vim gcc-c++ libhugetlbfs* libstdc++* kernel-devel numa* gcc git mlocate \ + cmake wget ncurses-devel hmaccalc zlib-devel binutils-devel elfutils-libelf-devel \ + numactl-devel libhugetlbfs-devel bc patch git patch tar zip unzip python3 sudo docker + RUN yum install -y centos-release-scl + RUN yum install -y devtoolset-8 + RUN yum clean all + + RUN pip3 install meson && \ + pip3 install ninja pyelftools + + # ENV HTTP_PROXY="" + # ENV HTTPS_PROXY="" + + WORKDIR /usr/src/ + RUN git clone https://github.com/pkgconf/pkgconf.git + WORKDIR /usr/src/pkgconf + RUN ./autogen.sh && ./configure && make && make install + + WORKDIR /usr/src/ + RUN git clone git://git.kernel.org/pub/scm/utils/rt-tests/rt-tests.git + WORKDIR /usr/src/rt-tests + RUN git checkout stable/v1.0 + RUN make all && make install + + COPY docker-image-building /root/docker-image-building + + WORKDIR /opt + + # Set default command + CMD ["/usr/bin/bash"] + EOF + + # build the docker image for FlexRAN; building environment + orgname=somename + docker build -t flexran-builder . + docker tag flexran-builder ${orgname}/flexran-builder:centos7.9 + docker login + docker push ${orgname}/flexran-builder:centos7.9 + +******************** +Build FlexRAN in Pod +******************** + +#. Use a shell inside Pod to build FlexRAN. + + .. code:: bash + + kubectl exec -it buildpod -- bash + +#. Use ``scp`` to copy the FlexRAN related files into the pod's |PVC|. + + .. code:: bash + + mkdir -p /opt/scrach && cd /opt/scrach + scp flexran-21.07.tar.gz.part00 . + scp flexran-21.07.tar.gz.part01 . + scp dpdk_21.07.patch . + scp system_studio_2019_update_5_ultimate_edition.tar.gz . + + cat flexran-21.07.tar.gz.part00 flexran-21.07.tar.gz.part01 > flexran-21.07.tar.gz + rm flexran-21.07.tar.gz.part00 + rm flexran-21.07.tar.gz.part01 + +#. Copy |DPDK| source code into the pod's |PVC|. + + .. code:: bash + + cd /opt && wget http://static.dpdk.org/rel/dpdk-20.11.1.tar.xz + tar xf dpdk-20.11.1.tar.xz + mv dpdk-stable-20.11.1 dpdk-flxr-21.07 + cd /opt/dpdk-flxr-21.07 && patch -p1 < /opt/scrach/dpdk_21.07.patch + +#. Install Intel System Studio 2019 Update 5. + + .. code:: bash + + cd /opt/scrach + + tar zxvf system_studio_2019_update_5_ultimate_edition.tar.gz + cd /opt/scrach/system_studio_2019_update_5_ultimate_edition + + # replace xxxx-xxxxxxxx with yours for ACTIVATION_SERIAL_NUMBER + sed -i -r -e 's/^ACCEPT_EULA=.*/ACCEPT_EULA=accept/' -e 's/^ACTIVATION_TYPE=.*/ACTIVATION_TYPE=serial_number/' -e 's%^#?ACTIVATION_SERIAL_NUMBER=.*%ACTIVATION_SERIAL_NUMBER=xxxx-xxxxxxxx%' silent.cfg + + cd /opt/scrach/system_studio_2019_update_5_ultimate_edition && ./install.sh -s silent.cfg + +#. Extract FlexRAN and populate the environment variables. + + .. code:: bash + + cd /opt/scrach/ && tar zxvf flexran-21.07.tar.gz && ./extract.sh + # input '/opt/flexran' for Extract destination directory + + cd /opt/flexran && source ./set_env_var.sh -d + # When following promote message shows: + # Enter Intel SystemStudio / ParallelStudio Install Directory for icc + # input: /opt/intel/system_studio_2019/ + # When following promote message shows: + # Enter DPDK Install Directory, or just enter to set default + # input: /opt/dpdk-flxr-21.07 + +#. Switch to devtoolset-8 environment. + + .. code:: bash + + scl enable devtoolset-8 bash + +#. Build FlexRAN SDK first. + + .. code:: bash + + cd /opt/flexran && ./flexran_build.sh -e -r 5gnr_sub6 -b -m sdk && ./flexran_build.sh -e -r 5gnr_mmw -b -m sdk + +#. Build |DPDK| with the FlexRAN patch. + + .. code:: bash + + cd /opt/dpdk-flxr-21.07 && meson build + cd /opt/dpdk-flxr-21.07/build && meson configure -Dflexran_sdk=/opt/flexran/sdk/build-avx512-icc/install && ninja + +#. Build the FlexRAN applications. + + .. code:: bash + + cd /opt/flexran + # Linux 5.6 deprecated ioremap_nocache, need to change to use ioremap_cache instead + sed -i 's#ioremap_nocache#ioremap_cache#g' ./libs/cpa/mmw/rec/drv/src/nr_dev.c + ./flexran_build.sh -e -r 5gnr_sub6 -b -n && ./flexran_build.sh -e -r 5gnr_mmw -b -n + +#. (Optional) Build the test applications. + + .. code:: bash + + cd /opt/flexran && ./flexran_build.sh -e -r 5gnr_sub6 -b -m 5gnr_testmac + +******************************************* +Generate Docker image with FlexRAN binaries +******************************************* + +.. note:: + Since host path ``/var/run/docker.sock`` has been mounted into the building pod, you can build the + docker image using the FlexRAN binaries from the previous step inside the building pod. + The artifacts used by :command:`docker build` have been integrated into the build image and are ready to use. + +#. Prepare the env var for the script in ``/root/docker-image-building/transport.sh``. + + .. code:: bash + + cd /opt/flexran && source ./set_env_var.sh -d + +#. Prepare binaries and scripts for docker build. + + .. code:: bash + + cd /root/docker-image-building + ./transport.sh + +#. Build docker image which will be saved in local host. + + .. code:: bash + + docker build -t flr-run -f Dockerfile . + + +********************************* +Run the FlexRAN Test cases in Pod +********************************* + +After the build and docker image generation steps above, you can launch the FlexRAN execution pod from the host. + +#. Push the docker image to a registry, for example, `dockerhub.io `__. + + .. code:: bash + + docker login + orgname=somename + docker tag flr-run ${orgname}/flr-run:v1 + docker push ${orgname}/flr-run:v1 + +#. Launch the FlexRAN Pod. + + Adjust the CPU and memory for your configuration. + Memory should be more than 32Gi for the test case pass rate. + + .. note:: + ``command`` should not be used in the spec, otherwise it will overwrite the + default container command which does accelerator |PCI| address filling for L1. + + .. code:: bash + + cat > runpod-flxr.yml << 'EOF' + apiVersion: v1 + kind: Pod + metadata: + name: runpod + annotations: + spec: + restartPolicy: Never + containers: + - name: runpod + image: somename/flr-run:v1 + imagePullPolicy: IfNotPresent + volumeMounts: + - name: usrsrc + mountPath: /usr/src + - mountPath: /hugepages + name: hugepage + - name: lib-modules + mountPath: /lib/modules + securityContext: + privileged: true + capabilities: + add: + ["IPC_LOCK", "SYS_ADMIN"] + resources: + requests: + memory: 32Gi + hugepages-1Gi: 8Gi + intel.com/intel_acc100_fec: '1' + limits: + memory: 32Gi + intel.com/intel_acc100_fec: '1' + hugepages-1Gi: 8Gi + volumes: + - name: usrsrc + hostPath: + path: /usr/src + - name: lib-modules + hostPath: + path: /lib/modules + - name: hugepage + emptyDir: + medium: HugePages + EOF + + kubectl create -f runpod-flxr.yml + +#. Execute L1. + + #. Enter the L1 directory inside Pod. + + .. code:: bash + + kubectl exec -it runpod -- bash + source set-l1-env.sh 5G + + + #. Edit L1 configuration file. + + .. note:: + phycfg_timer.xml has been modified by entry script to use the FEC accelerator: + ``1`` + ``0000:8b:00.0`` + + This configuration is scripted and ran automatically, no manual configuration needed. + You can use :command:`printenv PCIDEVICE_INTEL_COM_INTEL_ACC100_FEC` + to check dpdkBasebandDevice. + + .. code:: console + + # change default CPU binding in section of in phycfg_timer.xml + # use the first 3 assigned CPUs for the Applications threads + + + + + 2, 0, 0 + + + 3, 96, 0 + + + 4, 96, 0 + + + + 4, 96, 0 + + + 2, 99, 0 + + + + #. Run L1 application. + + .. code:: bash + + # launch L1app + ./l1.sh -e + +#. Execute testmac after L1 is up and running in another terminal. + + #. Enter the testmac directory inside Pod. + + .. code:: bash + + kubectl exec -it runpod -- bash + source set-l2-env.sh 5G + + #. Edit testmac configuration file. + + .. code:: console + + # Modify default CPU binding in section of in testmac_cfg.xml + # make sure use the CPU from the CPU whose ID is bigger than 13, this way, + # the Application Threads will not overlapping with the BBUPool CPUs. + + + + 16, 90, 0 + + + 14, 0, 0 + + + 14, 89, 0 + + + 15, 90, 0 + + + # workaround the known issue of parsing zero value in the config file + sed -i '/>0