-
Notifications
You must be signed in to change notification settings - Fork 27
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Adding centos.json to be used with packer to generate a VM with PROX
Based on a CentOS cloud image downloadable from the web, you can now use packer to create an image that has DPDK and PROX installed in the proper directories. The VM is also tuned for guest VNFs. A service is also created to deal with the proper isolated_cpu settings for tuned. The service will also make sure uio and igb_uio are loaded. The prox_user_data.sh is not needed any more and hence deleted. ctrl-c of runrapid.py will now also stop prox in all the VMs so a new test can be started. Change-Id: I16cc59878e2d4912757f42e05f98d51dff5feb89 Signed-off-by: Luc Provoost <[email protected]>
- Loading branch information
Showing
16 changed files
with
328 additions
and
84 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| { | ||
| "_Copyright": "Copyright (c) 2010-2018 Intel Corporation", | ||
| "_License": "SPDX-License-Identifier: Apache-2.0", | ||
| "builders": [ | ||
| { | ||
| "type": "openstack", | ||
| "ssh_username": "centos", | ||
| "image_name": "newrapidVM", | ||
| "source_image": "aa0f8333-d8e1-42db-8210-6b39d9ee7267", | ||
| "flavor": "7aaec028-eccc-4612-8f9b-29a1e039943a", | ||
| "networks": "97bbe96f-ee1b-4789-ace8-e922f76bea3f", | ||
| "use_floating_ip": true, | ||
| "floating_ip_pool": "1f564356-8c23-41e2-8292-7696ffd98adf", | ||
| "security_groups": "8fd498fc-5bd8-4be6-94e4-a2cc3b02d151" | ||
| } | ||
| ], | ||
| "provisioners": [ | ||
| { | ||
| "type": "file", | ||
| "source": "./check_prox_system_setup.sh", | ||
| "destination": "/home/centos/" | ||
| }, | ||
| { | ||
| "type": "file", | ||
| "source": "./check-prox-system-setup.service", | ||
| "destination": "/home/centos/" | ||
| }, | ||
| { | ||
| "type": "shell", | ||
| "script": "deploycentos.sh" | ||
| } | ||
| ] | ||
| } |
11 changes: 11 additions & 0 deletions
11
VNFs/DPPD-PROX/helper-scripts/openstackrapid/check-prox-system-setup.service
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,11 @@ | ||
| [Unit] | ||
| Description=Check PROX system setup (isolated_cores, uio) | ||
| DefaultDependencies=no | ||
| After=multi-user.target | ||
|
|
||
| [Service] | ||
| Type=oneshot | ||
| ExecStart=/usr/local/libexec/check_prox_system_setup.sh | ||
|
|
||
| [Install] | ||
| WantedBy=multi-user.target |
51 changes: 51 additions & 0 deletions
51
VNFs/DPPD-PROX/helper-scripts/openstackrapid/check_prox_system_setup.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,51 @@ | ||
| #!/usr/bin/env bash | ||
| ## | ||
| ## Copyright (c) 2010-2018 Intel Corporation | ||
| ## | ||
| ## Licensed under the Apache License, Version 2.0 (the "License"); | ||
| ## you may not use this file except in compliance with the License. | ||
| ## You may obtain a copy of the License at | ||
| ## | ||
| ## http://www.apache.org/licenses/LICENSE-2.0 | ||
| ## | ||
| ## Unless required by applicable law or agreed to in writing, software | ||
| ## distributed under the License is distributed on an "AS IS" BASIS, | ||
| ## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| ## See the License for the specific language governing permissions and | ||
| ## limitations under the License. | ||
| ## | ||
| NCPUS="$(lscpu | egrep '^CPU\(s\):' | awk '{ print $2 }')" | ||
| MAXCOREID="$((NCPUS-1))" | ||
|
|
||
| filename="/etc/tuned/realtime-virtual-guest-variables.conf" | ||
| logfile="/home/centos/prox_system_setup.log" | ||
| if [ -f "$filename" ] | ||
| then | ||
| while read -r line | ||
| do | ||
| case $line in | ||
| isolated_cores=1-$MAXCOREID*) | ||
| echo "Isolated CPU(s) OK, no reboot: $line">>$logfile | ||
| modprobe uio | ||
| insmod /root/dpdk/build/kmod/igb_uio.ko | ||
| exit 0 | ||
| ;; | ||
| isolated_cores=*) | ||
| echo "Isolated CPU(s) NOK, change the config and reboot: $line">>$logfile | ||
| sed -i "/^isolated_cores=.*/c\isolated_cores=1-$MAXCOREID" $filename | ||
| tuned-adm profile realtime-virtual-guest | ||
| reboot | ||
| exit 0 | ||
| ;; | ||
| *) | ||
| echo "$line" | ||
| ;; | ||
| esac | ||
| done < "$filename" | ||
| echo "isolated_cores=1-$MAXCOREID" >> $filename | ||
| echo "No Isolated CPU(s) defined in config, line added: $line">>$logfile | ||
| tuned-adm profile realtime-virtual-guest | ||
| reboot | ||
| else | ||
| echo "$filename not found.">>$logfile | ||
| fi |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
90 changes: 90 additions & 0 deletions
90
VNFs/DPPD-PROX/helper-scripts/openstackrapid/deploycentos.sh
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,90 @@ | ||
| #!/bin/bash | ||
|
|
||
| ## | ||
| ## Copyright (c) 2010-2018 Intel Corporation | ||
| ## | ||
| ## Licensed under the Apache License, Version 2.0 (the "License"); | ||
| ## you may not use this file except in compliance with the License. | ||
| ## You may obtain a copy of the License at | ||
| ## | ||
| ## http://www.apache.org/licenses/LICENSE-2.0 | ||
| ## | ||
| ## Unless required by applicable law or agreed to in writing, software | ||
| ## distributed under the License is distributed on an "AS IS" BASIS, | ||
| ## WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| ## See the License for the specific language governing permissions and | ||
| ## limitations under the License. | ||
| ## | ||
| sudo sh -c '(echo "export RTE_TARGET=\"build\"";echo "export RTE_SDK=\"/root/dpdk\"";echo "export AESNI_MULTI_BUFFER_LIB_PATH=\"/home/centos/intel-ipsec-mb-0.48\"";) >> /root/.bashrc' | ||
| sudo yum install deltarpm -y | ||
| sudo yum update -y | ||
| sudo yum-config-manager --add-repo http://www.nasm.us/nasm.repo | ||
| sudo yum install git wget gcc unzip libpcap-devel ncurses-devel libedit-devel lua-devel kernel-devel iperf3 pciutils numactl-devel vim tuna openssl-devel nasm -y | ||
| # Enabling root ssh access | ||
| sudo sed -i '/disable_root: 1/c\disable_root: 0' /etc/cloud/cloud.cfg | ||
| # The following line is commented since this was a workaround for a problem with the content of /etc/resolv.conf. | ||
| # That file could contain DNS information coming from the dataplane which might be wrong. A solution is to confire the correct DNS for the dataplne | ||
| # in OpenStack. DNS might be slowing down ssh access. We don't need that for our dataplane benchmarking purposes | ||
| # sudo sed -i '/#UseDNS yes/c\UseDNS no' /etc/ssh/sshd_config | ||
|
|
||
| # Mounting huge pages to be used by DPDK | ||
| sudo mkdir -p /mnt/huge | ||
| sudo umount `awk '/hugetlbfs/ { print $2 }' /proc/mounts` >/dev/null 2>&1 | ||
| sudo mount -t hugetlbfs nodev /mnt/huge/ | ||
| sudo sh -c '(echo "vm.nr_hugepages = 1024") > /etc/sysctl.conf' | ||
|
|
||
| # Downloading the Multi-buffer library | ||
| wget https://github.com/01org/intel-ipsec-mb/archive/v0.48.zip | ||
| unzip v0.48.zip | ||
| export AESNI_MULTI_BUFFER_LIB_PATH=/home/centos/intel-ipsec-mb-0.48 | ||
| cd $AESNI_MULTI_BUFFER_LIB_PATH | ||
| make -j8 | ||
| # Clone and compile DPDK | ||
| cd /home/centos/ | ||
| git clone http://dpdk.org/git/dpdk | ||
| cd dpdk | ||
| git checkout v17.11 | ||
| export RTE_TARGET=build | ||
| export RTE_SDK=/home/centos/dpdk | ||
| make config T=x86_64-native-linuxapp-gcc | ||
| # The next sed lines make sure that we can compile DPDK 17.11 with a relatively new OS. Using a newer DPDK (18.5) should also resolve this issue | ||
| sudo sed -i '/CONFIG_RTE_LIBRTE_KNI=y/c\CONFIG_RTE_LIBRTE_KNI=n' /home/centos/dpdk/build/.config | ||
| sudo sed -i '/CONFIG_RTE_LIBRTE_PMD_KNI=y/c\CONFIG_RTE_LIBRTE_PMD_KNI=n' /home/centos/dpdk/build/.config | ||
| sudo sed -i '/CONFIG_RTE_KNI_KMOD=y/c\CONFIG_RTE_KNI_KMOD=n' /home/centos/dpdk/build/.config | ||
| sudo sed -i '/CONFIG_RTE_KNI_PREEMPT_DEFAULT=y/c\CONFIG_RTE_KNI_PREEMPT_DEFAULT=n' /home/centos/dpdk/build/.config | ||
| # Compile with MB library | ||
| sudo sed -i '/CONFIG_RTE_LIBRTE_PMD_AESNI_MB=n/c\CONFIG_RTE_LIBRTE_PMD_AESNI_MB=y' /home/centos/dpdk/build/.config | ||
| make -j8 | ||
| cd /home/centos | ||
| # Copy everything to root since the scripts are assuming /root as the directory for PROX | ||
| sudo cp -r dpdk /root/ | ||
|
|
||
| # Clone and compile PROX | ||
| git clone https://git.opnfv.org/samplevnf | ||
| cp -r /home/centos/samplevnf/VNFs/DPPD-PROX /home/centos/prox | ||
| cd /home/centos/prox | ||
| make -j8 | ||
| cd /home/centos | ||
| # Copy everything to root since the scripts are assuming /root as the directory for PROX | ||
| sudo cp -r /home/centos/prox /root/ | ||
|
|
||
| # Enabling tuned with the realtime-virtual-guest profile | ||
| wget http://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/tuned-profiles-realtime-2.8.0-5.el7_4.2.noarch.rpm | ||
| wget http://linuxsoft.cern.ch/cern/centos/7/rt/x86_64/Packages/tuned-profiles-nfv-guest-2.8.0-5.el7_4.2.noarch.rpm | ||
| # Install with --nodeps. The latest CentOS cloud images come with a tuned version higher than 2.8. These 2 packages however | ||
| # do not depend on v2.8 and also work with tuned 2.9. Need to be careful in the future | ||
| sudo rpm -ivh /home/centos/tuned-profiles-realtime-2.8.0-5.el7_4.2.noarch.rpm --nodeps | ||
| sudo rpm -ivh /home/centos/tuned-profiles-nfv-guest-2.8.0-5.el7_4.2.noarch.rpm --nodeps | ||
| # Although we do no know how many cores the VM will have when begin deployed for real testing, we already put a number for the | ||
| # isolated CPUs so we can start the realtime-virtual-guest profile. If we don't, that command will fail. | ||
| # When the VM will be instantiated, the check_kernel_params service will check for the real number of cores available to this VM | ||
| # and update the realtime-virtual-guest-variables.conf accordingly. | ||
| echo "isolated_cores=1" | sudo tee -a /etc/tuned/realtime-virtual-guest-variables.conf | ||
| sudo tuned-adm profile realtime-virtual-guest | ||
|
|
||
| # Install the check_tuned_params service to make sure that the grub cmd line has the right cpus in isolcpu. The actual number of cpu's | ||
| # assigned to this VM depends on the flavor used. We don't know at this time what that will be. | ||
| sudo cp -r /home/centos/check_prox_system_setup.sh /usr/local/libexec/ | ||
| sudo cp -r /home/centos/check-prox-system-setup.service /etc/systemd/system/ | ||
| sudo systemctl daemon-reload | ||
| sudo systemctl enable check-prox-system-setup.service |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -24,3 +24,4 @@ then | |
| else | ||
| echo Assuming port is already bound to DPDK | ||
| fi | ||
| exit 0 | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -42,6 +42,5 @@ sub mode=l3 | |
| rx port=if0 | ||
| tx port=if0 | ||
| delay us=10 | ||
| probability=100 | ||
| probability=80 | ||
| local ipv4=${local_ip} | ||
|
|
||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.