How to run JMeter scripts in Jenkins?

By Qautomation on June 19, 2019 • ( 3 Comments )

Jenkins is an open source automation server written in Java. Jenkins helps to automate the non-human part of the software development process, with continuous integration and facilitating technical aspects of continuous delivery.

As performance testers, we contribute to our development team by easing the execution of performance tests. The goal is to integrate testing to make it a natural part of the lifecycle of any development project. In this blog post, we will see how to integrate our open source JMeter performance tests into open source Continuous Integration tool Jenkins.

Jenkins easily enables users to integrate load testing into its workflow. By using Jenkins as part of the CI process and to trigger jobs by commits, users are taking advantage of automation and process speed-up capabilities.

Why Performance Testing is Important in Your CI Process

Adding performance tests to your CI scope will help you reduce the risk of performance degradation’s whenever you add a new feature or fix a bug on your product.

For example: if your latest commit inadvertently causes response times to go above an acceptable threshold or SLA, your CI system will automatically raise a ‘red flag’ and mark the build as ‘failed’. This will make it very easy for you to identify the cause (and the responsible person! 🙂 behind the problem.

What’s more; it’s much better to start running performance tests in the early software development stages. If you detect a major error in the pre-release phase, it will be much harder to identify and fix the issue.

Advantages of Integrating JMeter into Jenkins.

JMeter is one of the most popular open-source load testing systems. By integrating JMeter into Jenkins, users can enjoy:

Unattended test executions right after software build and deploy
Automatic build failures in case of performance degradation
Easy access to test reports that show application metric trends – all tests are in one place and available to anybody with the correct permissions
Automated routine work of test configuration, execution and baseline results analysis. Users’ hands and minds are free for more important, complex and interesting tasks.

How to Install the Jenkins and JMeter Plugin

I’m assuming that you already have Jenkins up and running somewhere.If not then please refer here to install Jenkins.

How to Install the Performance Plugins in Jenkins.

To get JMeter support on Jenkins, you need to use the Performance Plugin. Once again, it’s really easy to install this plugin. Here’s how:

Download the latest version of the plugin from the Performance Plugin page
Copy the performance.hpi file to the plugins folder of your Jenkins’ installation. If you run Jenkins from the .war file – drop the plugin to the .jenkins/plugins path under your user home folder
Restart Jenkins to pick the plugin up
If everything is fine you should see the “Publish Performance Test Result Report” option under Jenkins -> Your Project -> Configure -> “Add post-build action” dropdown

How to Run a JMeter Test in Jenkins?

1. let’s go to Jenkins home page and click on “create new jobs”, specify a name, select Freestyle project and click “OK”.

2.To create parameterized build ,Click check box “This project is parameterized” and add parameters as per your requirement like SCRIPT_PATH,THREADS , RESULT_PATH RAMPUP,DURATION etc.

3. Performed below setting :

a. Add the jmeter.save.saveservice.output_format=xml line to the user.properties file (this is located in /bin folder of your JMeter installation).

Update JMeter script by adding property to pass through the command line.

Note – You can also use CSV files with the Performance Plugin. However, this requires extra configuration whereas XML files work fine out of the box.

b.Select the “Execute windows batch command” from Add Build Step.

c.Pass the property via the -J command-line argument, such as: C:\PerformanceTesting\apache-jmeter-5.0\bin\jmeter -n -t %SCRIPT_PATH% -JThreads=%THREADS% -JRampUP=%RAMPUP% –Jduration=%DURATION% -Jjmeter.save.saveservice.output_format=xml -l %RESULT_PATH%

Note– For instance, if you define the parameter with the name “foo” and the value “bar”, you can refer to it as:

%foo% – if the Jenkins build is executed on a Windows machine
$foo – if the Jenkins build is executed on Linux, Unix, or MacOSX machine

How to Use the Performance Plugin.

The Performance Plugin displays statistics, trends and can also be used to mark builds as failed based on results.

To view JMeter reports on Jenkins, you can use the Performance plugin.

Install the Performance plugin
Configure the plugin
The Performance plugin can be added as a “Post-build Action”. When the JMeter test is finished, the plugin will:
- Collect the data
- Conditionally fail the build if the error threshold is exceeded
- Build or update the performance trend chart for the project

3.Click Save button

It will automatically detect the format of the result file CSV or JTL.

Provide the path of result file in Source Data Files

Configuration options explained:

Performance report – For JMeter you will need to upload a file in XML format.
Select mode – The choices are in the Relative Threshold and Error Threshold. The Relative Threshold compares the difference from the previous test, and if it exceeds the defined value the build will be marked as failed/unstable. The Error Threshold marks the the build as unstable or failed if the amount of errors will exceed the specified value.
Build result – If the JMeter test doesn’t generate the output jtl file(s) – the build will be failed.
Use error thresholds on single build – Define error thresholds for the current build.
Average response time threshold – Set the maximum acceptable value of the Average Response Time metric.
Use relative thresholds for build comparison – Set the percentage difference of errors. The “source” build can be either the previous build or “known good” build which is used as a baseline.
Performance per Test Case Mode – If you need to see separate graphs for each single test case on the Performance Trend chart you can trigger the behavior by this option.
Show Throughput Chart – Set whether to display “Throughput” trend chart on project dashboard or not.

Executing JMeter test in Jenkins

Follow belows steps:

Go to project in Jenkins
Click Build with Parameters
Provide the values for required build parameters .
Click on Build button

To verify the execution was successful (or to get extra information regarding any problems), check the “Console Output” tab of the project in question.

In the JMeter log file and the .jtl results file, you can find it under the Project’s “Workspace”.

On the first build with the plugin enabled, you’ll see empty charts for the “Performance Trend”.

The charts will change as you create more builds. They will display average, median and 90 percentile values along with the errors count. The charts are clickable, so you can get more detailed information on each executed sampler.

References

https://www.blazemeter.com/blog/continuous-integration-101-how-run-jmeter-jenkins