CI-CD Pipeline implementation of the Continuous Integration/Continuous Deployment software is the backbone of the modern DevOps environment. Most of the organization using DevOps practices and using Jenkins. Jenkins is a open source CI tool which can be used to execute the UFT tests. Jenkins is a Continuous Integration tool which does this verification periodically or whenever the developer pushes the code to the repository.

Jenkins can act like this task scheduler which can pull the source code from your source code manager & do a specific task on it. Like compiling the source code, executing the scripts etc. Once the task is done, Jenkins can also archive the results, sending out an email to the team members. Jenkins is a web based application.

UFT tests can be executed by Jenkins in two ways :

1. By Using Jenkins plugin – Execute HP tests from file system
2. By using AOM(Automation Object Model) script.

How it works

• Automation engineers push their changes to the SVN/Git/ALM.
• Jenkins can trigger a test whenever a commit was made / periodically / manually by the user.
• Jenkins gets the latest code from the source control system, deploys it in the Slave machines.
• Jenkins executes the test in the Slave machines, archive the results.
• User can see the results in Jenkins directly.

Prerequisite

1. If you dnt have Jenkins install then refer this How to install Jenkins.
2. Create slave machine for the execution of the UFT scripts(Machine where UFT is installed), and install the following plug-ins

Setting up Jenkins Execution Nodes/Slave Machine

Creating an execution node is necessary to run UFT tests.

From the Jenkins Dashboard:

1. Click Manage Jenkins
2. Click Manage Nodes
3. Click New Node

• Name the Node
• Choose Permanent Agent
• Click OK

4. Set up the Remote Root Directory, which is where test results will be saved. This can be any directory of your choosing. I created a special directory as C:\Jenkins.
5. In the Usage field, choose “Use this node as much as possible”
6. In the Launch Method, choose “Launch agent via Java Web Start”
7. For Availability, choose “Keep this agent online as much as possible”
8. Click Save
9. At this point, you’ll have to choose how to launch the Jenkins agent on the “slave” machine. I chose “Launch agent from browser.”

When you click the Launch button, you’ll get a dialog asking to save the slave-agent.jnlp file. Save this file in a location accessible from the “slave” machine. Note that this file is customized for the particular slave machine that the node was created for. You can’t use the same file on different machines.

10. Launch the agent on the slave machine. From the agent, you can click the File menu and choose to install the agent as a service. This is convenient because it will always run when the system starts and you don’t have to manually launch it. (Note: on some machines the service is not reliable, so just running the agent may be a better option for some environments.)

Once this is complete, you can go back to the Nodes management screen in Jenkins and the new slave machine should be listed and connected. You should now be ready to run tests on the nodes you created.

List of required plugins for the setup

• Copy Artifact Plugin
• Dashboard View Plugin
• Discard Old Build Plugin
• Email Extension Plugin
• Extra Columns Plugin
• Flexible Publish Plugin
• HP Application Automation Tools Plugin
• Test Result Analyzer Plugin
• Time Stamper Plugin
• Workspace Cleanup Plugin

Create a Jenkins Job

Once, you are able to open the Jenkins server, Click on “New Item”

Provide any Item name, select “freestyle project” and click on “OK” button

Once, you click on OK button, New Job configuration page will open. This is the main configuration page, where we will provide all of the configuration details about how the Job should execute.

Fill out the following information within the job configuration page

1) Description: Description about the job and the UFT test which is tied to this job.

2) Check on “Restrict where this project can be run” and provide the slave machine name with UFT installed. This is a very important step, because otherwise Jenkins will choose a slave machine randomly, and might choose Jenkins master machine (where Jenkins is installed) for execution. This will cause the job to fail.

3) Select appropriate build triggers, if you are using any source code control for storing UFT tests (Except ALM) e.g. GIT, SVN, TFS.

4) Under Build environment select “delete workspace before build starts”. This will help us in cleaning up the workspace and have a fresh start for each execution.

5) Click on Add build step button under build and select “Execute HP Tests from file system”. This will allow you to execute the tests from file system e.g. “C:\” or network drives.

6) Provide UFT tests or Solution path in the “Tests” text box. If you provide UFT’s solution file path then all tests under solution will be executed one after another. Leave Time out seconds blank.

7) Click on “Add post build action” under post build actions, and select “Publish HP test results”. Select “Always Archive HP test Reports”. This step will generate and archive Junit results for the execution on the Jenkins server, which can be used for analysis. we will discuss this in detail later.

8) If you have configured the email server then Add another post build step and “Editable email notification”. This option will send automated emails after the execution is over. We can attach any files in the email or we can send any files as the part of the email body.

Run UFT Tests

After filling out the above information, Click on Save button and start the job by clicking on “Build now” button within job details page.

Once the execution is started, click on Build number, under build history, to check the current execution details

While the test is executing, you can click on Console output, to check the current execution status.

Once the execution is completed, you can see the test results on the Build page, under test results

General Issues faced during test execution

There are several issues you may encounter during the execution of UFT tests from Jenkins

1) If scripts have any “MsgBox” statement or any “InputBox” statement, scripts will directly fail.

2) If the machine is unlocked, then there is a chance for complex applications to fail.

3) We have to make sure that machine is cleaned up before and after the execution of tests(Closing all browsers and any other types of files like excel sheets or word documents).

4) We have to make sure that UFT scripts have a proper error handling mechanism or UFT will throw an error message and Jenkins Job will fail as soon as Popup appears on the page.

5) If we have a proper framework for our scripts, then most likely we will be executing multiple tests within one UFT test. Because Jenkins HP Automation tool can only count UFT tests and there is no way to indicate no. of tests, if you execute 2 UFT tests having 20 tests each, Jenkins dashboard will only display 2 tests executed, not 40 tests executed.

6) If UFT test report’s a failure in any of the tests, you will see the failure of Jenkins complete Job instead of a particular test case.

Resolution of Issues

1) Replace any statement which requires human intervention
We must scan our scripts and replace any statements which require human intervention like MsgBox statements must be commented and should only be used during debugging. For any steps which use Input boxes for accepting inputs, use test parameters and action parameters. For receiving information back, use reporter.reportevent instead of msgbox statements.

2) Use Physical desktops, instead of virtual machines
If we use virtual machines as Jenkins slave machines, then we will have to make sure that the machines are up and running all the time, because virtual machine can go blank without any prior warning, if the user disconnects them.

3) Unlocking remote slave machine before executions.

4)Include Machine Cleanup Framework in UFT

5) Include proper error handling mechanism in your scripts.

Executing UFT tests using AOM

UFT can be automated programmatically using its COM interface.  Using this COM interface we can launch UFT, run a test, store the results in a specific location etc. This COM library is registered in your machine when you install UFT. We can execute our tests from our UFT framework (Having a driver script) using VB script .

creating a simple runner.vbs file as given below.

On Error Resume Next

Dim wshShell: Set wshShell = CreateObject( "WScript.Shell" )
Dim strComputerName: strComputerName = wshShell.ExpandEnvironmentStrings( "%COMPUTERNAME%" )

Dim Arg, tsBrowser,tsEnvironment,tsFolder
Set Arg = WScript.Arguments
tsBrowser = Trim (Arg(0))
tsEnvironment = Trim (Arg(1))
tsFolder = Trim (Arg(2))

Dim boolFlag: bolFlag = False
Dim arrFiles(5)
Set fso=createobject("scripting.filesystemobject")

arrFiles(0) = tsFolder & "\TestData\DataSheet.csv"
arrFiles(1) = tsFolder & "\TestSettings\Global_Test_Settings.xls"
arrFiles(2) = tsFolder & "\TestData\DataSheet.xls"
arrFiles(3) = tsFolder & "\TestRunner\BatchRunner.xls"

For i = 0 to 4
 If Not fso.FileExists(arrFiles(i)) Then
   Wscript.Echo "Required File Missing - "arrFiles(i)
   boolFlag = True
 End If
Next
Set fso = Nothing


If Err.Number > 0 or tsFolder = "" or boolFlag Or tsBrowser = "" Or tsEnvironment = "" Then
    ConsoleOutput "Please provide the 'UFT Framework Folder' folder path as script argument. Ensure all arguments and files in folder are available." & Err.Description
Else
    ConsoleOutput "UFT Framework is found in '" & tsFolder & "' directory"
    ConsoleOutput "Setting test execution environment on machine:" & strComputerName

    Call KillProcess("UFT.exe")
    Call KillProcess("iexplore.exe")
    Call KillProcess("chrome.exe")
    'Call KillProcess("firefox.exe")
    Call KillProcess("werfault.exe")

    WScript.Sleep 1000

    Dim scriptDir
    scriptDir = CreateObject("Scripting.FileSystemObject").GetParentFolderName(WScript.ScriptFullName)

    'Declare the Application object variable
    Dim objUFT 

    'Dim qtResultsOpt 'As QuickTest.RunResultsOptions ' Declare a Run Results Options object variable
    Dim startTime, endTime, runStatus

    'Set objUFT = CreateObject("QuickTest.Application", "hostName") ' Create the Application object for remote execution
    Set objUFT = CreateObject("QuickTest.Application") ' Create the Application object 

    'Launch UFT
    If objUFT.Launched <> True Then
        objUFT.Launch 'Start UFT
    End If
    ConsoleOutput("Launching UFT Application...")

    'Make UFT visible
    objUFT.Visible = False  'True

    'Set QuickTest run options 
    objUFT.Options.Run.ImageCaptureForTestResults = "OnError"
    objUFT.Options.Run.RunMode = "Fast" 'Hidden
    objUFT.Options.Run.ViewResults = False

    ConsoleOutput("Opening Test...")
    objUFT.Open scriptDir&"\Tests\GUI_Test", True 

    'Create Environment Variables to pass the results from UFT to BatchRunner.vbs
    objUFT.Test.Environment.Value("Browser") = tsBrowser
    objUFT.Test.Environment.Value("Environment") = tsEnvironment
    objUFT.Test.Environment.Value("JenkinsFlag") = "N"
    objUFT.Test.Environment.Value("JenkinsTestCaseDescription") = ""
    objUFT.Test.Environment.Value("JenkinsTestCaseNumber") = ""
    objUFT.Test.Environment.Value("JenkinsTestCaseResult") = ""
    'objUFT.Test.Settings.Run.IterationMode = "rngIterations" ' Run only iterations 2 to 4
    'objUFT.Test.Settings.Run.StartIteration = 1
    'objUFT.Test.Settings.Run.EndIteration = 1
    'objUFT.Test.Settings.Options.Run.StepExecutionDelay = 20
    objUFT.Test.Settings.Run.ObjectSyncTimeOut = 20000
    objUFT.Test.Settings.Run.DisableSmartIdentification = True
    objUFT.Test.Settings.Run.OnError = "NextStep" ' Instruct QuickTest to perform next step when error occurs

    Set UFTResultsOpt = CreateObject("QuickTest.RunResultsOptions") ' Create the Run Results Options object 
    UFTResultsOpt.ResultsLocation = scriptDir&"\Test_Results\UFTReport" ' Set the results location

    ConsoleOutput("Start of Test Execution")
    startTime = Now()
    objUFT.Test.Run UFTResultsOpt, FALSE 

    'Header of test execution detail on jenkin console

    WScript.StdOut.WriteLine "-----------------------------------------------------------------------------------------------------------------------------------------------------------"
    WScript.StdOut.WriteLine "| SNo |     TIME      |       TCID        |                                       TEST CASE DESCRIPTION                                           | RESULT |"
    WScript.StdOut.WriteLine "-----------------------------------------------------------------------------------------------------------------------------------------------------------"

    TC_counter = 1
    'Write the result in the console        
    While objUFT.Test.isRunning
        If objUFT.Test.Environment.Value("JenkinsFlag") = "Y" Then
            objUFT.Test.Environment.Value("JenkinsFlag") = "N"

            ' Show TC ID and Description
            WScript.StdOut.Write "|  "&TC_counter&"  | "&Time()&" | "&objUFT.Test.Environment.Value("JenkinsTestCaseNumber")&" | "&objUFT.Test.Environment.Value("JenkinsTestCaseDescription")&" | "

            'Wait till the test is executedresult is updated
            While (objUFT.Test.Environment.Value("JenkinsTestCaseResult") = "" AND objUFT.Test.isRunning)
                    WScript.Sleep 1000
            Wend

            'Show the Result
            WScript.StdOut.WriteLine objUFT.Test.Environment.Value("JenkinsTestCaseResult")&" |"
            TC_counter = TC_counter + 1
        End If
        WScript.Sleep 1000
    Wend
    WScript.StdOut.WriteLine "-----------------------------------------------------------------------------------------------------------------------------------------------------------"
    ConsoleOutputBlankLine(1)
    'Close UFT
    ConsoleOutput("End Of Test Execution")
    objUFT.Test.Close ' Close the test

    'Set qtResultsOpt = Nothing ' Release the Run Results Options object 
    'Set qtTest = Nothing ' Release the Test object
    Set objUFT = Nothing ' Release the Application object
    endTime = Now()

    set Arg = Nothing
    Set wshShell = Nothing

End if
Sub KillProcess(ByVal ProcessName)

 On Error Resume Next
 Dim objWMIService : Set objWMIService = GetObject("winmgmts:{impersonationLevel=impersonate}!\.\root\cimv2")
 Dim colProcesses : Set colProcesses = objWMIService.ExecQuery("SELECT * FROM Win32_Process WHERE Name='"&ProcessName&"'")

 ConsoleOutput("Terminating Process : "&ProcessName)

 For Each objProcess in colProcesses
    intTermProc = objProcess.Terminate
 Next
 On Error GoTo 0

End Sub

'Comment when running without a console
Sub ConsoleOutput(ByVal MessageToBeDisplayed)
 WScript.StdOut.WriteLine Time()&" :: "&MessageToBeDisplayed
End Sub

• Save this as “runner.vbs” in this location “c:\workspace\UFT_Jenkins\” on the machine.
• Open command prompt – Enter below command by updating the path for the runner.vbs.

CScript "c:\workspace\UFT_Jenkins\runner.vbs"

Ensure that this runner.vbs works fine. It should launch UFT, runs the tests selected in BatchRunner file and store the results in the specific location.

Also use below functions in your driver script /function to send the Test case execution flag , test case Id, test case description and test case result (pass or fail).

Function fn_TCDetailForJenkinsConsole(ByVal TestCaseNumber, ByVal TestCaseDescription)


Environment.Value("JenkinsFlag") = "Y"
Environment.Value("JenkinsTestCaseResult") = ""
Environment.Value("JenkinsTestCaseNumber") = TestCaseNumber
Environment.Value("JenkinsTestCaseDescription") = TestCaseDescription   
Wait 2

End Function

Function fn_TCStatusForJenkinsConsole(ByVal Status)

If Trim(Ucase(Status)) = "PASS" Then
    Environment.Value("JenkinsTestCaseResult") = "PASS"
Else
    Environment.Value("JenkinsTestCaseResult") = "FAIL"
End If
Wait 2
End Function

Create a Jenkins Job

Create a simple freestyle job. I am naming this job as QTP_JOB

In Job details page, Add a build step as given below. It should be ‘Execute Windows Batch Command‘

Add the Batch Command.  CScript ‘path of the runner’  [ CScript “c:\workspace\UFT_Jenkins\runner.vbs” ]

It is basically the same command whatever you give in your command prompt to run the vbs file.

This should do to run the UFT test.  (Jenkins can pull the UFT framework code from source code manager like ALM/SVN/Git etc.  But I assume we have the script in the local machine. Our aim is just to run UFT script using Jenkins to see how it works. 

Running UFT Scripts

Click on the Build Now to launch UFT & run the test.

References:

https://sumeetkushwah.com/2015/03/24/implementing-ci-using-jenkins-and-uft/

https://www.vinsguru.com/executing-qtpuft-scripts-using-jenkins/