Web Services Testing using soapUI

soapUI is a free and open source cross-platform Functional Testing solution. With an easy-to-use graphical interface, and enterprise-class features, soapUI allows you to easily and rapidly create and execute automated functional, regression, compliance, and load tests. In a single test environment, soapUI provides complete test coverage and supports all the standard protocols and technologies.

soapUI is available in free and pro versions. Pro Version provides more technical support and it has more features avaialble to help the testers in automation of web services testing. In this website we will mainly talk about the automation using Pro version of soapUI but at the same time we will try to give guidance how to implement the same using free version as well.

Invoke soapUI TestRunner from Command Prompt

Let us see how to execute soapUI test cases using command prompt.

In this soapUI Tutorial, we will see how can we invoke and execute soapUI test cases from command prompt. If you are familiar with soapUI application, you might have used/ noticed an option named "Launch Testrunner". We are calling this feature from a command prompt window to execute our project/ test suite or test cases. Let us peform the below steps to achieve this.

1. Open a command prompt

2. Navigate to soapUI bin folder. For example "C:\Program Files\SmartBear\soapUI-Pro-4.5.0\bin".

3. Now type the step testrunner.bat "C:\Project-Path\project-file.xml"

4. Hit enter

This will invoke and execute soapUI project in the specified project file. The above mentione is the simple way of executing a soapUI project using command prompt. But there are different options which we can provide to the soapUI testrunner.bat to configure the test execution. Here are the parameters which can be used to invoke soapUI testrunner in command prompt.
e : The endpoint to use when invoking test-requests, overrides the endpoint set in the project file

h : The host:port to use when invoking test-requests, overrides only the host part of the endpoint set in the project file

s : The TestSuite to run, used to narrow down the tests to run

c : The TestCase to run, used to narrow down the tests to run

u : The username to use in any authentications, overrides any username set for any TestRequests

p : The password to use in any authentications, overrides any password set for any TestRequests

w : Sets the WSS password type, either 'Text' or 'Digest'

d : The domain to use in any authentications, overrides any domain set for any TestRequests

r : Turns on printing of a small summary report (see below)

f : Specifies the root folder to which test results should be exported (see below)

j : Turns on exporting of JUnit-compatible reports, see below

a : Turns on exporting of all test results, not only errors

o : Opens the generated report in a browser (soapUI Pro only)

i : Enables soapUI UI-related components, required if you use the UISupport class for prompting or displaying information

t : Sets the soapui-settings.xml file to use, required if you have custom proxy, ssl, http, etc setting

x : Sets project password for decryption if project is encrypted

v : Sets password for soapui-settings.xml file

D : Sets system property with name=value

G : Sets global property with name=value

P : Sets project property with name=value, e.g. -Pendpoint=Value1 -PsomeOtherProperty=value2

S : Sets to save the project file after tests have been run

I : Do not stop if error occurs, ignore them

R : Selects which report to generate for the test objects executed, for example if running the entire project, this could specify the name of a test-suite-level report that would be generated for each TestSuite. The report is saved as specified with the -F option to the folder specified with the -f option. (soapUI Pro only)

F : Sets the format of the report specified with the -R option, for Printable reports this is one of PDF, XLS, HTML, RTF, CSV, TXT, and XML. For Data Export this is either XML or CSV (soapUI Pro only)

g : Sets the output to include Coverage HTML reports ( soapUI Pro only )

E : Sets which environment to use (soapUI Pro only)

soapUI Test Runner from Command Prompt Example

testrunner.bat -rjf "Results" "C:\Project-Path\project-file.xml"

soapUI Load Test Thread Strategy

In soapUI, performance or load testing of web services can be done using different load testing strategies. A high level idea on this can be viewed in my earlier post "Web services Load Test Using soapUI". In this soapUI tutorial for load testing, we will discuss the use of "Thread Strategy" available in soapUI. The main configuration settings available in soapUI load testing Thread Strategy are "Start Threads", "End Threads" and the "Limit". Now lets see one by one in detail.

Start Threads : We have to mention the number of threads we need when we start our soapUI load/ performance test.
End Threads : This is where we set the number of threads we expect at the end of our soapUI load/ performance test.
Limit : We can configure this in 3 ways, either we can set the duration of the load test in seconds, or we can set the total number of runs the soapUI test case needs to be executed as part of the load/ performance test. The other option available is Runs per thread, here we can control the total number of time the soapUI test case will be executed on each thread.

Lets see an example on soapUI Load Test - Thread Strategy

Say we want to do our load test for 5 minutes with a starting thread count of 1 and ending thread count of 10, then our configaration should be like below.

Start Threads : 1
End Threads : 10
Limit : 300 seconds

soapUI will start load test with 1 thread and increase the thread count in equal intervals to reach the thread count to 10 until the time limit of 300 seconds are reached.

Read data from excel in soapUI like a database

Treating Excel as a database is one of the most popular method used in automation testing. Web services testing is also not much different than that. There might be instances that we might need to treat an excel sheet as a database and read data from this excel sheet in soapUI for data validation or input data or for configuration purposes. In this example, instead of relying on groovy, we will be using a bit of java code to connect to an excel sheet and read data from this excel sheet. In order to connect to an excel as a database, we need a driver. We will be using "sun.jdbc.odbc.JdbcOdbcDriver" in our example to connect to an excel fom soapUI. Lets say we have a file named "soapUI-Excel-Example .xls" and it has a work sheet named "ExcelSheetName". Now, create a coulmn named "Name" in the sheet and enter some data in the column. Its time for us to look into the code now. Lets see the below example on how to connect to excel from soapUI.

import java.sql.Connection;
import java.sql.DriverManager;
import java.sql.ResultSet;
import java.sql.Statement;

def soapUI_Excel = "C:\\soapUI-Excel\\soapUI-Excel-Example.xls"
Class.forName("sun.jdbc.odbc.JdbcOdbcDriver");
soapUIDataConnection = DriverManager.getConnection("jdbc:odbc:Driver={Microsoft Excel Driver (*.xls)};DBQ=" + soapUI_Excel + ";ReadOnly=0", "", "");
Statement stmt = ((Connection) soapUIDataConnection).createStatement();
recordSet = stmt.executeQuery("Select * from [ExcelSheetName\$] where Name like 'soapUI%'");

while (recordSet.next())
{

log.info(recordSet.getString("Name"))

}

The above example will select all the Names like "soapUI" and print it to soapUI log.

soapUI Composite Project

In this soapUI tutorial we will explore a nice feature in soapUI called "Composite Project". This is a property in soapUI project which takes boolean values true or false. Click on soapUI project and you will see the project properties tab in soapUI and there you can see the option "Composite Project" and by default the value will be false.
Now lets discuss what this exactly means to us. First create a project on a work space and save the project. Make sure you have atleast one test suite and one test case in the project. Go back to the directory where you saved the project and see how many files you have. You will see a work space XML file and also a project XML file. In a way this is good as the number of files as less, it is easy to maintain. But imagine if multiple people are working on web services automation using soapUI and everyone is working different test cases. In this case, it will be hard to maintain a single file and merge it always. Here comes the use of soapUI "Composite Project" property. Create another project and change the "Composite Project" property to "true". Now create a test suite and create a test case inside that and save the project. Go back to the folder where the project is saved and check for the files. You will see a seperate folder for project with soapUI project file and settings files along with a folder for test suite and the interface folders. Now navigate to the soapUI test suite folder, you will see settings file, element order file, and the soapUI test case file. In this case each test case will be mainted in different soapUI XML files and each test suite will be a different folder. This way multiple people can work different test cases at the same time on the same project. This is more useful if you are working on a large web service automation project with hundreds of operations are present. Version control is also more easier with this approach.

Groovy Script Library in soapUI

Do you know in soapUI we can bundle all the frequently used scripts/ functions into a script library and use it where ever we want without duplicating the code? Lets take a look into this topic. Lets assume we need to use the below block of groovy script in many places like soapUI groovy test steps, soapUI assertions etc.

def soapUI_String = "soapUI Tutorial";
/* Do some processing on the soapUI_String variable here */
log.info(soapUI_String);
We can make all this into a function and call it when required. Doesn't that sound good? To do this, first we need to create a class and make the above tasks into a member function. Lets create a Groovy class now.

package soapUIScripts
class Scripts
{
def log;
Scripts(log)
{
this.log = log;
}
def soapUI_groovy_function()
{
def soapUI_String = "soapUI Tutorial";
/* Do some processing on the soapUI_String variable here */
log.info(soapUI_String);
}
}

Create a folder with the name of the package (in this case "soapUIScripts") and place the above code into a file and save inside the folder with name "Scripts.groovy"

We have created a groovy class and now we need to include the above class into soapUI for us to use it. In soapUI Pro, go to "soapUI Preferences -> soapUI Pro -> Script Library" mention the path of the parent folder (name of the folder which has the folder "soapUIScripts"). We can't have just the functions inside the groovy file, it should always be attached to a class. soapUI is all set to use your custom code now. Navigate to any soapUI test case and create a groovy script test step and use the below code inside that step to call the above created function.

def myscript = new soapUIScripts.Scripts(log);
myscript.soapUI_groovy_function();

If you are using any soapUI built in objects in your groovy class/ script, make sure you are first passing that as a constructor. The same way we passed the "log" object to our class in the above example.

If you are not using soapUI Pro you will need to compile your scripts into a standard java library (ie .jar file) and add that file to the classpath by putting it in the bin\ext folder to use the functions.

Capture soapUI Logs to File

In this Groovy Script example, lets see how can we capture the soapUI logs and save it into a file. At the bottom of soapUI, you might have noticed different tabs named soapUI log, http log, jetty log, error log, wsrm log, memory log and script log. Each tab will have the logs based on log category. For example, if you are invoking a Web Service request/ response, you will be able to see the web service request and response with the http headers under http tab. Same way if you have groovy script to on your soapUI test steps and if those groovy scripts are printing any dynamic values while test execution, you might also see that in the tab "script log". Lets see the below groovy script to capture the logs and save it into a text file.

//The below groovy script step is to capture the soapUI log area.

// "soapUI log" can be replaced with http log, jetty log, script log, error log etc based on the need.

def logArea = com.eviware.soapui.SoapUI.logMonitor.getLogArea( "soapUI log" );

//Below two lines of groovy script is to get the project directory, we will be saving the soapUI log contents on a file in that directory.

def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)

def projectPath = groovyUtils.projectPath

def logFile = new File(projectPath + "\\soapUI-Logs-to-file.txt")

//This is the first line in the soapUI log file.

logFile.write("soapUI Logs In a file.\r\n")

if( logArea !=null )

{

def model = logArea.model

if( model.size > 0 )

for( c in 0..(model.size-1) )

logFile.append(model.getElementAt( c ))

}

soapUI JDBC Connection configuration settings using groovy Script

In soapUI Pro, there is an option to configure a JDBC database connection and it can be accessed in many places like DataSource test step, JDBC Request step, DataSink step etc. But in some cases, we might need to get the same JDBC connection object and connection details in groovy script as well. In this post we will take a quick look on how can we get the JDBC connection properties in groovy script editor (Script test step).

First of all, double click on your project on the left side pane of soapUI.
Now, navigate to the tab "JDBC Connections" on the right side pane.
Click on the plus sign and provide the necessary details and configure the database connection.
Name the connection as "JDBCTest1" (we will be referring this JDBC connection later on the groovy script.)
Test the JDBC connection before closing the dialog box.

Now use the below code to create a connection object and get the properties to use it in groovy script.import groovy.sql.Sql;
// Below line will create an object of the already created JDBC connection.
def ConObj = context.testCase.testSuite.project.databaseConnectionContainer.getDatabaseConnectionByName('JDBCTest1');
//Below line is using the connection String from already created JDBC connection.
def Con = Sql.newInstance(ConObj.getConnectionString());
def res = Con.rows("select *from tablename;");
Con.close();

Parse and Read XML file using Groovy Script

While testing web services, we might end up parsing and reading XML files for different purposes. Let us see how can we use groovy script effectively to parse/ read XML file.

In this example, let us consider the below XML file named "DATASHEET.xml".

Now we need a groovy script to read the data from the above XML file. Let us see the below Groovy Script.

def alert = com.eviware.soapui.support.UISupport;

//Define a file pointer for groovy to handle the file operations.

def inputFile = new File("C:\\SOAPUI\\DATASHEET.xml")

if(!inputFile.exists())

{

//Display an alert if the file is not found.

alert.showInfoMessage("Input File 'DATASHEET.xml' not found!");

}

else

{

//Read and parse XML file and store it into a variable

def InputXML = new XmlParser().parseText(inputFile.text)

//Find/ Filter XML nodes based on a condition

def InputRow = InputXML.Employee.findAll{

it.Age.text().toInteger() > 19;

//We are finding all XML nodes where the age > 19

}

InputRow.each{

//Display the value of name node from the filtered record

log.info(it.Name.text());

}

If you want to do data driven testing using groovy script in soapUI to test web services, then you might consider the above script to do the same. Se if you have an XML file with test case name, parameters and an action column to control the execution flow, you can read the XML file by using this groovy script and set the parameters as properties of a test case and use it dynamically. This can be achieved by using the script in conjunction with the soapUI setup and teardown features.

List testSteps using Groovy Script in soapUI

Let us see how can we get a list of all the test steps available in a particular testCase in soapUI. This is a simple script and might be useful if you need more control on the testSteps individually.

See the example below.

// This will get the list of all testSteps in the testCase
def testSteps = context.testCase.getTestStepList()
testSteps.each{
// This block will loop through the test steps and print the test step name to the log
log.info(it.name)
}

Create Retrieve testCase testSuite Properties using Groovy Script

If you are familiar with automation testing you might know the importance of dynamically storing or retrieving property values. In soapUI, we can story properties on different levels, say on project level, testSuite level and testCase level. Either we can hardcode the property even before the test run or we can create update and retrieve the property values dynamically using groovy script. Let us see how can we achieve this using groovy in soapUI.

The below sample groovy script will allow you to add, set and retrieve properties.

/*To make this for a test case level property, change the below line to
def testSuite = context.testCase;
*/
def testSuite = context.testCase.testSuite;
def PropertyName = "UserName"
/*Below line checks if there is already a property named "UserName"*/
if(!testSuite.hasProperty(PropertyName))
testSuite.addProperty(PropertyName)
/*Below line of groovy code will set the value "user01" for the property*/
testSuite.setPropertyValue(PropertyName,'user01')
//Below line will retrieve the property value
def propertyValue = testSuite.getPropertyValue(PropertyName)

JDBC Connection example using Groovy Script in soapUI

soapUI Pro has a built in test step named JDBC Request to directly establish a connection to the database and get the results back in XML format. Connection to a database can also be achieved using Groovy Script. Let us see an example on how can we do a database connection using groovy script and get the records back using soapUI.

To do this, first create a Groovy Script test step in your soapUI test case (This can also be used in Script Assertion) and use the below code.

import groovy.sql.Sql;

//Establish JDBC Connection to the DB

def con = Sql.newInstance("jdbc:oracle:thin:@<dbserver url>:<port>:<SID>", "<db user name>", "<db password>", "oracle.jdbc.driver.OracleDriver");

def res = con.rows("SELECT sysdate FROM dual")

log.info(res[0].sysdate.toString())

con.close()

The above lines of code is making use of the method newInstance in the package groovy.sql.Sql to establish a connection to the database. This select the system date from oracle table dual and print to the log.

Capture the list of Assertions using groovy script

While doing automation testing of web services using groovy and soapUI, we might need to get the list of assertions and its status of a test request. By using groovy this can be achieved easily in soapUI. See the below example.

def obj = context.testCase.getTestStepByName("GetCustomer");
def assertions = obj.getAssertionList()
assertions.each{
log.info(it.name + ' --> ' + it.status)
}

The above code is creating an object for the test request named "GetCustomer" in the first line. In second line it is storing all the assertions into a variable name "assertions". The assertions.each block iterates through the list of all assertions attached to the test request "GetCustomer" and the log statement prints the name of the assertions along with it status.

Below line of code will give the total number of assertions attached to a test request.

def obj = context.testCase.getTestStepByName("GetCustomer");

log.info(obj.getAssertionCount())

Project Path Using Groovy in soapUI

While doing automation testing we might need to get the project path/ folder to access other files or resources. soapUI provides this feature in the below way.

def groovyUtils = new com.eviware.soapui.support.GroovyUtils(context)
def projectDir = groovyUtils.projectPath

log.info(projectDir)

Now the variable projectDir has the project path which can be set as one of the project properties and can be used in later stages.

soapUI SOAP Fault and Not SOAP Fault Assertion

There are two assertions named SOAP Fault and Not SOAP Fault in soapUI. For a test request, there are instances that the server returns a fault error instead of a valid response. To capture this faults these two assertions can be used. If a fault response is returned, then the assertion "Not SOAP Fault" will fail. This is because, the assertion expects a valid response and it received a fault response. If other case, if a fault is returned, "SOAP Fault" assertion will pass, because it is expecting a fault response. To make sure if a "test request" test step received a valid response, either one of these assertions can be used.

Below are the elements present in a SOAP fault message.

<faultcode> A code for identifying the fault
<faultstring> A human readable explanation of the fault
<faultactor> Information about who caused the fault to happen
<detail> Holds application specific error information related to the Body element.

The Fault Code values are as follows.

VersionMismatch - Found an invalid namespace for the SOAP Envelope element
MustUnderstand - An immediate child element of the Header element, with the mustUnderstand attribute set to "1", was not understood
Client - The message was incorrectly formed or contained incorrect information
Server - There was a problem with the server so the message could not proceed

Web services Load Test Using soapUI

Performance of a web service is really important especially if the response has huge volume of data returned. soapUI has a nice feature named "Load Test" to do this task. In this section, we will see how we can use soapUI to do load test/ performance test for web services.
In soapUI, every test case has a section named "Load Test" to create test cases for doing load test/ performance test. Right click on it and select "New LoadTest". This will create a test case for load test for that specific test case. Double click on the load test test case and you can see all the test steps in that test case are grouped together and ready for load test. Basically it uses the web service functional test case/ test steps for load/performance tests. If you are doing this on your web services automation test case, then make sure you don't have any heavy validations attached to your test requests or other steps. This can impact the calculation of the response time of the web services. I would suggest to create a new test case with only the test request test step and the required step need to parameterize the requests.

Eventhough the load test includes all the test steps in the test case, it displays the performance numbers/ response time for each test steps individually. So, one should be able to tell exactly what is the response time of individual test steps including the test request.

Now we have a test case to do performance/ load testing of web services, but we have to configure them based on our specific needs. To do that soapUI provide different Load Test Strategies. We will see those in detail on later posts, but will list it here just to keep that in mind.

Burst
Fixed-Rate
Grid
Script
Simple
Thread
Variance

We have to also understand few terms in the results as well.By the way all the timings in this report are displayed in milli seconds.

min - Minimum response time
max - Maximum response time
avg - Average response time
last - response time of the last run
cnt - Total number of times a step executed
tps - Transactions per second
bytes - Total bytes transferred/ processed
bps - Bytes per second
err - Error occured

soap ui Groovy Script to find testCase status

While doing automation testing, there will be cases one need to know the status of a test case on the fly. In soapUI, while doing automation for web services testing, I came across to this scenario different times. soapUI provides a unique feature called TearDown Script for its testCase. A groovy script can be written here to get the status of the testCase on the fly. Further action can be done via scripting based on the status or this can even be used for custom reporting. Let us see the groovy script samples for the same.

testRunner.status.toString();

This line of code will return the status of soapUI testCase. The status will be "FINISHED" if it is successfully completed and it will be "FAILED" if the test case failed for any reason.

soapUI Setup TearDown Script

If you are familiar with automation testing, you know the importance of an initial setup/ initialization option in any automation framework. soapUI also provides the feature of executing an initial setup and tear down script for web services testing.

Initial Setup Script : soapUI initial setup is just a place holder/ an editor for writing your initial script before start your testing. This setup is available in two levels, TestSuite level and testCase level. The groovy script written in the TestSuite option will be the first one to trigger just before the execution of that testSuite. Same for testCase, the setup script written in the testCase level will be the first to trigger just before the execution of that particular test case. The main purpose of the Initial Setup section in soapUI is to provide the user the initial property creation, initialization, etc even before the test case execution.
TearDown Script : This is same as soapUI initial setup, the only difference is the code/ script written in this section will trigger only after the completion of testCase / testSuite depending upon where the script is written. The main purpose of this is to cleanup the automation suite like delete unwanted file, kill common objects , custom reporting etc.

log, runner, context and testSuite variables are available in both setup and teardown scripts.

Mock Service in soapUI with Example

soapUI provides a great feature called MockService for testing web services. This is really helpful for a tester if he plans to automate the web services testing. By using the MockService feature, one can start working on soapUI automation script just with the WSDLs. In other words, the web service doesn't have to be connected to a web server or a back end database server to start the automation.

MockService mimics the behavior of a server and send the response back to soapUI as if it is coming from a server. Let us see an example below.

You can create a mock service while creating a project or you can right click on the interface and select create Mock Service at any point. See the below image.

In this case we have selected two services "CelsiusToFahrenheit" and "FahrenheitToCelsius" for creating a mock service. Checking the "Add Endpoint" will add the end point to the request in the test case. After creation of this mock service, your local machine will act as a server hosting these two web services and you can access those services by using the end point of the mock service. Once you have the mock service created, double click on the Response of each method and specify value for all the elements in the response. This way once you hit the end point, you will have some data returned from the mock service. During the initial stages of development, you might ask the development team to provide just the WSDL and you can create MockService using the WSDL and start working on your soapUI automation scripts.

WS Security

WS-Security (Web Services Security, short WSS) is a flexible and feature-rich extension to SOAP to apply security to web services.While doing Web Services/ SOA Testing, one has to make sure that the services are secured enough. Proper authorization and authorization needs to be implemented for web sercvices. Lacking of this can provide access to data and non public information of an organization to outside world.
WS Security can be implemented by creating different security policies. In this post, let us see how can we test WS Security using username and password token of a web service using soapUI. There are two ways to attach the security header/ profile to the outgoing request via soapUI.

Configure WS Security

Configure a WS-Security Profile on the soapUI Project level and attach to all outgoing test requests.

Double click on the project and navigate to the tab "WS-Security Configurations" and then to "Outgoing WS-Security Configurations". Click on the "+" sign and give a name to the security profile we are creating say "WSConfiguration1" Click on the "+" sign on the tab below the current section.Selcet "Username" from the WSS Entry drop down and click OK.Select the "Add Nonce" check box, this is to add a random number to the security header and will be used in conjection with the security encryption/decryption. Check "Add Created" check box and select the password type as well. Now you have a security profile setup for soapUI by name "WSConfiguration1" which can be used on an test requests. Double click on the test request and select the newly created profile from the "Outgoing WSS" drop down from the "Aut" tab of a test request.

Supply the Username and Password in the Test Request itself and apply it each time before invoking the request.

Double click on the test request from the test steps. Navigate to the tab "Aut" at the bottom of the request. Give the username, password and domain. Now go to the XML version of the request, right click there. Click on the tab "Add WSS Username Token. Select "Password Text" to send the password in plain text or select "Password Digest" to use an encrypted password. You might also right click and select "Add WS-Timestamp" to add a creation/ expiration time stamp to the security header.This will create an entry like the one below under the <Security> element for the request.

<wsu:Timestamp wsu:Id="Timestamp-2" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">

<wsu:Created>2011-10-19T18:20:37.540Z</wsu:Created>

<wsu:Expires>2011-10-19T18:21:37.540Z</wsu:Expires>

</wsu:Timestamp>

<wsse:UsernameToken wsu:Id="UsernameToken-1" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">

<wsse:Username>username</wsse:Username>

<wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">pwd</wsse:Password>

<wsse:Nonce EncodingType="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-soap-message-security-1.0#Base64Binary">M769jEEqYIUEDKP+dvH+ZA==</wsse:Nonce>

<wsu:Created>2011-10-19T18:20:25.899Z</wsu:Created>

</wsse:UsernameToken>