Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add stage for performance tests #37

Merged
merged 1 commit into from
Oct 19, 2023
Merged

Add stage for performance tests #37

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 33 additions & 1 deletion Jenkinsfiles/hw_test_set
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ resultsDirectory = ''
configPath = ''
zipImagePath = ''
bootSuite = 'boot_test.robot'
performanceSuite = 'performance/'
batSuite = './'
deviceName = ''
usbHubSerial = ''
Expand All @@ -27,6 +28,7 @@ pipeline {
def splitted = "${JOB_NAME}".split('/')
bootJob = "${splitted[0]}/Testing/boot_test"
batJob = "${splitted[0]}/Testing/smoke_tests"
performanceJob = "${splitted[0]}/Testing/Performance_tests"
configPath = "/home/${params.label}/Jenkins-agent/workspace/${splitted[0]}/Testing/test_config.json"

// Check for which agent and which target device
Expand Down Expand Up @@ -166,7 +168,37 @@ pipeline {
} else {
catchError(stageResult: 'FAILURE', buildResult: 'FAILURE'){
error("BAT build failed")
}
}
}
}
}
}
stage('Performance tests') {
steps {
script{
// Make directory for test results
sh "mkdir -p ${params.resultsPath}/${params.buildID}/${resultsDirectory}/performance"
build = build(
job: "${performanceJob}", propagate: false,
parameters: [
[$class: 'StringParameterValue', name: 'RF_SUITE', value: "${performanceSuite}"],
[$class: 'StringParameterValue', name: 'DESCRIPTION', value: "${params.server} buildID: ${params.buildID}"],
[$class: 'StringParameterValue', name: 'DEVICE_NAME', value: "${deviceName}"],
[$class: 'StringParameterValue', name: 'BUILD_ID', value: "${params.buildID}"]
]
)
// copy report, log and plots
sh "cp ~/Jenkins-agent/workspace/${performanceJob}/Robot-Framework/test-suites/report.html ${params.resultsPath}/${params.buildID}/${resultsDirectory}/performance/report.html"
sh "cp ~/Jenkins-agent/workspace/${performanceJob}/Robot-Framework/test-suites/log.html ${params.resultsPath}/${params.buildID}/${resultsDirectory}/performance/log.html"
sh "cp ~/Jenkins-agent/workspace/${performanceJob}/Robot-Framework/test-suites/*.png ${params.resultsPath}/${params.buildID}/${resultsDirectory}/performance/"
if(build.result == "SUCCESS") {
buildResults."bat" = "SUCCESS"
echo "BUILD NUMBER: ${build.number} SUCCESSFULLY BUILD"
} else {
echo "BUILD NUMBER: ${build.number} SUCCESSFULLY BUILD"
catchError(stageResult: 'FAILURE'){
error("Error during performance tests")
}
}
}
}
Expand Down
6 changes: 6 additions & 0 deletions Robot-Framework/config/variables.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,12 @@
*** Settings ***
Library OperatingSystem


*** Variables ***

${BUILD_ID} ${EMPTY}


*** Keywords ***

Set Variables
Expand Down
18 changes: 16 additions & 2 deletions Robot-Framework/test-suites/performance/network.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,16 +21,18 @@ TCP speed test
[Tags] tcp SP-T91
Run iperf server on DUT
&{tcp_speed} Run TCP test
Save Speed Data ${TEST NAME} ${buildID} ${tcp_speed}
Save Speed Data ${TEST NAME} ${BUILD_ID} ${tcp_speed}
Log <img src="${DEVICE}_${TEST NAME}.png" alt="TCP Plot" width="1200"> HTML
[Teardown] Stop iperf server

UDP speed test
[Documentation] Measure RX and TX speed for UDP
[Tags] udp SP-T92
Run iperf server on DUT
&{udp_speed} Run UDP test
Save Speed Data ${TEST NAME} ${buildID} ${udp_speed}
Save Speed Data ${TEST NAME} ${BUILD_ID} ${udp_speed}
Log <img src="${DEVICE}_${TEST NAME}.png" alt="UDP Plot" width="1200"> HTML
[Teardown] Stop iperf server


*** Keywords ***
Expand All @@ -42,6 +44,7 @@ Common Setup

Run iperf server on DUT
[Documentation] Run iperf on DUT in server mode
Clear iptables rules
${command} Set Variable iperf -s
Execute Command nohup ${command} > output.log 2>&1 &

Expand All @@ -59,3 +62,14 @@ Run UDP test
Log ${output.stdout}
&{udp_speed} Parse iperf output ${output.stdout}
[Return] &{udp_speed}

Clear iptables rules
[Documentation] Clear IP tables rules to open ports
Execute Command iptables -F sudo=True sudo_password=${PASSWORD}

Stop iperf server
@{pid}= Find pid by name iperf
IF @{pid} != @{EMPTY}
Log to Console Close iperf server: @{pid}
Kill process @{pid}
END
13 changes: 7 additions & 6 deletions Robot-Framework/test-suites/performance/performance.robot
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ Library ../../lib/PerformanceDataProcessing.py ${DEVICE}
Suite Setup Common Setup
Suite Teardown Close All Connections


*** Test Cases ***

CPU One thread test
Expand All @@ -21,7 +22,7 @@ CPU One thread test
${output} Execute Command sysbench cpu --time=10 --threads=1 --cpu-max-prime=20000 run
Log ${output}
&{cpu_data} Parse Cpu Results ${output}
Save Cpu Data ${TEST NAME} ${buildID} ${cpu_data}
Save Cpu Data ${TEST NAME} ${BUILD_ID} ${cpu_data}
Log <img src="${DEVICE}_${TEST NAME}.png" alt="CPU Plot" width="1200"> HTML

CPU multimple threads test
Expand All @@ -32,7 +33,7 @@ CPU multimple threads test
${output} Execute Command sysbench cpu --time=10 --threads=${threads_number} --cpu-max-prime=20000 run
Log ${output}
&{cpu_data} Parse Cpu Results ${output}
Save Cpu Data ${TEST NAME} ${buildID} ${cpu_data}
Save Cpu Data ${TEST NAME} ${BUILD_ID} ${cpu_data}
Log <img src="${DEVICE}_${TEST NAME}.png" alt="CPU Plot" width="1200"> HTML

Memory Read One thread test
Expand All @@ -44,7 +45,7 @@ Memory Read One thread test
${output} Execute Command sysbench memory --time=60 --memory-oper=read --threads=1 run
Log ${output}
&{cpu_data} Parse Memory Results ${output}
Save Memory Data ${TEST NAME} ${buildID} ${cpu_data}
Save Memory Data ${TEST NAME} ${BUILD_ID} ${cpu_data}
Log <img src="${DEVICE}_${TEST NAME}.png" alt="Mem Plot" width="1200"> HTML

Memory Write One thread test
Expand All @@ -56,7 +57,7 @@ Memory Write One thread test
${output} Execute Command sysbench memory --time=60 --memory-oper=write --threads=1 run
Log ${output}
&{cpu_data} Parse Memory Results ${output}
Save Memory Data ${TEST NAME} ${buildID} ${cpu_data}
Save Memory Data ${TEST NAME} ${BUILD_ID} ${cpu_data}
Log <img src="${DEVICE}_${TEST NAME}.png" alt="Mem Plot" width="1200"> HTML

Memory Read multimple threads test
Expand All @@ -68,7 +69,7 @@ Memory Read multimple threads test
${output} Execute Command sysbench memory --time=60 --memory-oper=read --threads=${threads_number} run
Log ${output}
&{cpu_data} Parse Memory Results ${output}
Save Memory Data ${TEST NAME} ${buildID} ${cpu_data}
Save Memory Data ${TEST NAME} ${BUILD_ID} ${cpu_data}
Log <img src="${DEVICE}_${TEST NAME}.png" alt="Mem Plot" width="1200"> HTML

Memory Write multimple threads test
Expand All @@ -80,7 +81,7 @@ Memory Write multimple threads test
${output} Execute Command sysbench memory --time=60 --memory-oper=write --threads=${threads_number} run
Log ${output}
&{cpu_data} Parse Memory Results ${output}
Save Memory Data ${TEST NAME} ${buildID} ${cpu_data}
Save Memory Data ${TEST NAME} ${BUILD_ID} ${cpu_data}
Log <img src="${DEVICE}_${TEST NAME}.png" alt="Mem Plot" width="1200"> HTML


Expand Down