Jmeter performance testing in practise with Powershell automation

1    JMETER

In our project is Jmeter used for performance and load testing. Jmeter is simple to use with al lot of good functionalities and good reporting. Simple usage can lead to misunderstand of some things like timers, throughput timers, especially variables and parameters.
Jmeter is able to run in non gui and property files to configure parameters in test file. All of these possibilities Jmeter give  a tester unlimited usage of test cases, configuration without huge effort.
In our company we uses Jmeter to create and execute test cases included in different thread groups. All test cases are included all different thread groups ( repetition, duration stepping and ulitmate). To minimize duplicate code for test cases, they  were created in special thread group which is disabled. Test case is set controllers which are calling web services.  Simple controllers are using HTTP Requests to call  a web service and are saved  in thread group 'Modules'. Services are configured  to each server, called with different parameters, timeouts.
To avoid splitting Jmeter file in more separated files per each test case, and multiplying using of services, are all test cases include in one .jmx files. Maintenance of the big file is more lightweight than editing more files. Because all test case are included in .jmx file , file is very big more than 5 MB in my case. Execution of jmeter .jmx file uses a lot of memory when low number of threads  are used on remote desktops. To avoid memory consumption all tests in thread groups were disabled, enabled is only test in execution. Powershell is running Jmeter script wich enables specific test. Structure of test plan in picture bellow:

 

1.1    Structure of Jmeter test plan

Test Plan is separated in five parts:

• Variables
• Thread groups
• Test scenarios
• Modules
• Functions

1.2    Variables/ Parameters

Variables, parameters are used to configure Thread groups, test scenarios and modules.  Variables:

•  Hostname Variables; define hosts, like  application servers, database servers, services used in tests
•  RL - Thread - Variables; define how will behave  run time thread group. Define repetitions of the test, number of threads, etc. All these variables can be additional configured in .properties file. Default values are 0, because this means this thread group will not execute. More information about how variables are used  in headline 1.2.2 ${__machineName()}}: RL..
•  TL - Thread - Variables; define how will behave  run time thread group. This variables define duration of test, threads, etc.  All these variables can be additional configured in .properties file. Default values are 0, because this means this thread group will not execute. More information about how variables are used  in headline 1.2.3${__machineName()}}: TL.
• STG - Thread - Variables; define how will behave  stepping thread group. All these variables can be additional configured in . properties file. Default values are 0, because this means this thread group will not execute. More information about how variables are used  in headline1.2.4${__machineName()}}: STG.
• UTG - Thread - Variables; define how will behave  ultimate thread group. All these variables can be additional configured in . properties file. Default values are 0, because this means this thread group will not execute. By default are possible to set 10 different steps. More information about how variables are used  in headline 1.2.5${__machineName()}}: UTG.
• Test Case Variables; this variables are use to configure tests, numbers, etc. These variables can be configured in property file.
• Test Case Think Times; this variables defined delay (think time) after each action. Variables can be configured in property file.
• Test Case Throughput; this variables define how many times each action in test case will be executed. Values represent percentage. Saved are also throughput variables which defines Throughput timer. Variables can be configured in property file.
• Test Case Timeouts; defines timeout of each method, call of web service or web page. Variables can be configured in property file.

1.3    Thread groups

1.3.1    Setup thread group

Set up thread group is used for initialization of all services on each application server. Also are here defined some other function, which update parameters used in threads, defines and calculate threads per remote computer and assign application servers to remote computers, etc.

1.3.2    ${__machineName()}}: RL

Thread group RL is repetition thread group. All parameters are defined in headline Variables. Picture below shows how are variables used in Thread group.



Thread group include different test case switch can be executed. All test cases are disabled. Why test cases are disabled will be described later in the document.

 

1.3.3    ${__machineName()}}: TL

Thread group TL is duration thread group. All variables are defined in headline Variables. Picture below shows how are variables used in Thread group.



Thread group include different test case switch can be executed. All test cases are disabled. Why test cases are disabled will be described later in the document.

1.3.4    ${__machineName()}}: STG

Thread group STG is stepping thread group. All steps are defined in headline Variables. Picture below shows how are variables used in Thread group.

 

Thread group include different test case switch can be executed. All test cases are disabled. Why test cases are disabled will be described later in the document.

1.3.5    ${__machineName()}}: UTG

Thread group UTG is stepping thread group. All steps are defined in headline Variables. Picture below shows how are variables used in Thread group.



Thread group include different test case switch can be executed. All test cases are disabled. Why test cases are disabled will be described later in the document.

1.4    Test Scenarios

Test scenarios represent one completed scenario. Login scenario is presented in pictures below. Each test case has its own default variables ' Create REST defaults' and generated variable ' Random Variable Username'. Structure of test case is below on picture:

 

Next controller is Parametrized controller with parameters defined in variables and properties files. See picture below



Then module controller is called ' Test Plan > Modules > PP Login'. See picture below

 

1.5    Module

Module PP Login with input parameters  is call Test case. Structure of calling service PP Login:



For calling Web service is used HTTP Request which is calling web service with parameters. All data HTTP Request are parametrized, due to easy change of configuration.

 

1.6    Functions

Functions are supporting different action do reduce multiplying of same code.  Call of this functions generate samplers which are used in reports. To avoid putting samplers in reports is used ' Bean Shell Preprocessor 'but it is duplicating same code.

2    TEST CASE FOLDERS

Each test case can be executed with different valuables, this mean test case can be parametrized. To parametrize test case are used properties files. Files need to run test case are represented here:

• NameoftheTestcase - General.properties; is files it has common variables used for all test cases. in this case name of test case is ' Test_Scenario_ Base_Line_100'.
• Test_Scenario_ Base_Line_100.properties; is file where can be placed specific values only for this test case
• Test_Scenario_Base_Line_100_STG.ps1; Is powershell script which defines Jmeter on which remote host will be used, name of test case, etc... suffix in the name '_STG' defines this test case uses STG thread group for executing test case.
• Test_Scenario_ Base_Line_100_Thread.properties; in this file are saved values how Stepping thread group will behave.

Structure of files for one test case and more test scenarios:

 

3    POWERSHELL - RUN TESTS

Each test scenario has its own parameters for execution.  This parameters are:

• $global:TestScenario; name of the test

• $global:TestCaseName; name of test case scenario, with this variable powerheshell open Jmeter.jmx files enable test in thread groups RT, TL, STG, UTG. Enables 'If' so this test will be executed.

• $global:TestName; name of test case jmeter file

• $global:Remote_Hosts; remote host to run jmeter

How is powershell running Jmeter with all properties files and other properties:
cmd.exe /c jmeter.bat --nongui -JforcePerfmonFile=true  --runremote --testfile "$TestName.jmx.bkp" `
--logfile "$JMeterReportFileFolder\$($TestName)-output.jtl" `
--jmeterlogfile "$JMeterReportFileFolder\$($TestName)-jmeter.log" `
--globalproperty LogDir="$($JMeterReportFileFolder)\" `
--globalproperty "$TestCaseName\$($TestCaseName) - General.properties" `
--globalproperty "$TestCaseName\$($TestScenario).properties" `
--globalproperty "$TestCaseName\$($TestScenario)_Thread.properties" ``
--globalproperty "perf.properties" `
--jmeterproperty "remote_hosts=$($Remote_Hosts)" `
--globalproperty "ServerRun=$($Remote_Hosts)"

Powershell is very power full tool, with it IIS on application servers is restarted, performance counters were started and stopped and on the end, logs are copy on network, excel and xml file are generated from .jtl file and on the end the word document is generated.

3.1    Powershell enable test case

Jmeter .jmx file is growing nad it now big few MB. When all test cases are enabled and test is executing on remote host, jmeter is using allot of memory to run complete jmx file. To avoid this, all test cases are disabled and enabled is only one named in configuration file .ps1.

4    JVISUALM

To measure memory on each remote host , search for Java installation and run \bin\ jvisualvm.exe.
In my case is :C:\Program Files\Java\jdk1.7.0_45\bin>jvisualvm.exe. With this tool you can check performance counters of the test:
 - CPU, allocated memory, used memory, threads, etc....

 

Conclusion

Jmeter gives are flexible work and with powershell gives us a very good opportunity to check results of older test and compare improvements which they were made.  With this combination tests can be listed in the execution. Most of the work is automatic, this saves us a lot of manual work.