Performance Testing Tips : Writing test script (aka Vuser scripting)

In this article we are going to know about some tips and tricks on writing scripts for performance testing in popular performance testing tools.(aka vuser script).

This tips might be helpful for writing performance test script. I would recommend you to read my previous post.

Tips :1. I prefer, separate vuser script for separate test cases. I have seen some engineers use same test case having multiple steps based on condition to perform multiple steps to perform a scenario. That makes the test case very complex and after a time , it is hard to understand. So, better, if you have different verity of test step for a single scenario, i suggest put them separate test case. And mix-up those test steps during execution. Example : If you site has 5 ways to log in transaction, try to make 5 different test cases for each way. Like in Jmeter, I use test fragment and mix those scenarios while test execution.

Tips :2. For debugging and checking all of you correlation values , use logging or debug sampler(jmeter). It helps to get the problems quickly.

Tips :3. Keep your application architecture matching tools with you which you will need frequently
For regular expression : Regular expression tester.
For Data Format conversion (based on what your application uses): Base64 encoder/decoder, URL Encoder/Decoder, Unix Time Stamp converter.
For File comparing or string comparing : Compare it, winDiff, Beyond compare.
For smart string view and search : Notepad++(my favorite)

Tips :4. If your application is web application, use browser based tools (with plug ins to get best out of that). I even use blazemeter chrome extension.

Tips :5. Try to familiar with different type of data format like how base46 encoding look like, How URL encoding look like. How they converts string. It will help you to find what is inside.

Tips :6. Understand all locally executable functions clearly. When you are creating Vuser, that simulate load, you have to understand how can you simulate those executable functions. Example, let say for a chat application, you might have simulated log in, but after login , local client set some parameter(based on implementation) before initiate communication. You will find same behavior for web application where you use JS/Ajax calls for various functionality. And , this is become complex if you are testing single page application. You need to keep track on all of those JS execution as well as full sequence and logics. Don’t trust on some thing happens automatically. Be clear on all of your gray area of understanding the system.

Tips :7. Try same test with another performance tools to have different view of the system that will help you clear understanding on what you are doing. This helps spatially on correlation suggestion. I use WAPTPro(personal choice). You may see my post to know about it from basic level.

Tips :8. Do not trust auto co relation. For co relation, you tool need to determine about you application architecture which is impossible. Tools can help to suggest but, try co relation manually. Often, tool suggestion are good for platform you are using (like , for web, if you use asp.net, load runner or wapt will suggest you to co relate view state as it is part of asp.net by default). My suggestion will be, if you ever use co-relation which are suggested , verify this with regular expression tester. To , do that, just copy recorded response data and past on regular expression tester body and test the regular expression that the tool suggested. Load runner by default use Left & Right boundary to make co relation function. Just convert that left and right boundary expression meaning to single line regular expression to verify.

Tips :9. Be master on pattern matching from the mind. It will help applying regular expression (you need to understand the regular expressions too) as well as heuristics which defines changed behavior among requests. I think , from a skill prospective, this is the most hard and important part. If you are good at pattern matching, performance testing is easy. It will help you to solve all complex algorithm understanding as well as co relation difficulties. Bonus part is, you will learn the system from business to deployment( Application architecture, DB, infrastructure, networking, Operating system internals, deployment system and everything related to your system).

Tips 10: You should store test script in  version management tools similar to general codes. It will keep you track and knowledge about your script more clear.

Tips 11 : Based on your requirements, try to group your request under Transaction which will give you sum of all child request time. Example : keep all request for a log in under LogIn transaction which will help to know what it total log in time while getting the results.

Tips 12 : If you are writing custom function for logic behind application, use script language which keep less overhead. Example: I prefer writing code in Groovy rather than bean shell.

This is an ongoing post. I will add more ideas incrementally…

Thanks..:)

Qualities of a Performance test script(aka vuser)

In this article, we are going to see some basic quality of a performance test script which is also know as VUser script. This post actually continuation of this main post

Qualities of a VUser Script :

1.Usages/Load Model :  Vuser script should reflect a real user. That means, the real user usages model(work load model).

Example : when we add item in shopping cart(eCommerce site), our shopping card will add the item and user see that added item after some time(ex : 500ms or more). Or, when we provide log in request, user will wait for at least some logical time to log in(Ex : 200ms or more). That means, based on your performance goal and requirement. Understanding the usages model is very important. So, add relative delays based on your scenario with feedback from business users or real users(or from stakeholders) while collecting the Performance goal and requirement and add those delays or conditions respectively.

2. Follow User Steps: This means referring user steps inside script. Vuser script must reflect business functions. Script should have reference from Performance Goal and Requirements. And, organize in such way so that a person can understand the scripts when he knows about those Goals.

3.Efficient use of Resource:  Vuser should occupy less memory and processing power. We need to careful about un necessary pre/post processor usages, new variable initialization. Re using same variable as well as do multiple processing in same execution will make the script more efficient.(Example : Try to reuse same correlation variable if it is changing or when you are parsing, get multiple data same time , try to run same post processing multiple time. Incase of Regular Expression, multiple grouping works very well)

4. Initiation and Cleanup : Vuser should have cleanup steps. More often when we do test, there are lots of test data. Depending on system, a smart vuser should have cleanup steps to get those un necessary data out of the system. This is similar to TearDown/ @aftertest functionality if you are familiar with unit testing.

5. Comments : Always use comments with good naming convention. In agile, it is said , your code should be self explanatory. Added to that, if you follow good naming convention and comments that reflects to requirements, it will help you as well as your successors to follow up your script. Both Jmeter and Load Runner(C code) has comment options that really helps for keeping track.

6. Code Best Practices : Vuser script is similar to code. Some tool like Load runner use C code in background. So you should follow best practices, avoid code smell, organize in maintainable way and try to get rid of all of anti patterns.

7. Re Use build in functions : Reuses tool in built in functions, avoid creating new functions which are available inside tool.

8. Assertion/ Validation: VUser should have assertion or validation after each step or transaction of group of transaction. This is fully based on you domain application. Example : if you are doing log in transaction, try to keep assertion with any item from the page that get loaded after login. This way, it will prove, you server responses after your login request and loads the page(send all items) after the request.
In some cases when you need to check whether server is responsive or not(http 200 message) you can avoid this. I used to avoid this for load test but I add this for stress testing. Example : when I need to check all the page of a website links and responses are ok or not, I avoid this. But, I use this when I need to verify after log in request, there is a real log in or after credit transaction, the change value is coming.
[Thanks to Joerek van Gaalen , he pointed out (i forgot) ]

Note : Don't forget to deduct the time taken due to assertion/validation which is taken by tool not your server.

This is ongoing post. I will add incrementally. Please have comments.

Thanks…:)

What is VUser ? How to write VUser or Performance Test Script?

In this article we are going to see how to write performance test case which is know as VUser script. We will try to learn generic way so that we can apply to Jmeter, Load runner, WAPT or any other tools in the market.

The steps might change in order but the idea behind all are almost same for every tools.

And, We will discuses about performance test script for server side performance. So, there will be no reference for client side performance testing.

What is VUser ?
VUser means virtual user . A real user behavior will be simulated by virtual user for performance test.

What is VUser Script?
When we write real user behavior or activity simulation through a virtual user, then it is referred as VUser script. This name came from load runner world. In Jmeter, it is called Test Plan. In generic, we can say Performance Test Script.

Thought VUser is not not real user, but, it should reflect real user scenario. In Jmeter we use different component to create a series of requests (may connected to each other or not) which are executed serially one after another. In load runner, we write code in C using load runner function. We might record those steps, but make sure that the script performs the activity of a scenario which we target for.

How it works?
Vuser scripts simulate the functionality on the target application from a non GUI(with background process model). This is because GUIs are heavy as well as not possible to run with multiple(thousands) instances in running environment(like running 100s browsers with separate session of same application or generating 10000 chat client connected to server via chat GUI). So, Vuser script use a system (provide by respective test tool) to simulate target number of users. Some time this system can be build on multiple PC/Server, specific network setting, various Infrastructure configurations.

Example : Load runner has vugen for writing scripts and controller(combining with multiple pc) to run and they have a analyzer for showing test results in smart way. 
Jmeter has everything build in, you can use jmeter for writing script, again configure with multiple PC running jmeter as slave(remote execution model) to simulate load and after running you can use jmeter to see and analysis the results with different Graphs(Listeners)
This part actually fully depend on tool. See some of my blog posts on Jmeter. 

How to write VUser or Performance Test Script?

This part is totally tool specific, but I will try to keep it more generic from multiple tool using experiences.

Step 1 : Write manual steps of the test following performance goal, requirement and with test data.

Step 2 : Have a plan for test data initiation and cleanup

Step 3 : Record the full test plan with your tool and verify every step with another tool. If you application use multiple protocol, record in multiple time or multiple tools. Be careful with request you are working with. (Example, if you are recording web request, verify recording steps with proxy like fiddler with plugins or wire shark or charlesproxy.)

Step 4 : Record script twice with exact same step and exact same data

Step 5 : Compare those two steps and get the differences.

Step 6 : Analysis those differences and get dynamic parameters.

Step 7 : Again, Record and compare with previous recording with
->Different credential to know more dynamic parameter based on credential
->Different test data to see dynamic parameter based on data.
->Different PC/Browser/network environment(will change the client and change the dynamic parameter)
->Different Version/release of the application (effective on web, example : our current application changes some parameter based on released version)

Step 8 : Analysis the data coming and going from client and server. This is the main area of learning the system with RAW request from a transport level. This is definitely hard but, tools are there to help you understanding they system.
I should say, Jmeter is better to view each item clearly, compare to load runner but, load runner has advantages over co relation data(next step).
These are some points that are follow for analysis for a web application
a. From request, what are the changing sectors for your protocol.( For http , data changes in URL, Header , Body)
b. For a specific part of your request, what are the changing items.  Usually, you see parameter and value. Focus on is value changes. How it changes, and why it changes.
And, for dynamic elements, you might face, the parameters are also changing. Get to know, why changing, how changing?
c. So, after getting over all knowledge on changing parameter and value, you need to make a list of parameter and data that you need to generate to simulate the request. You need divide them like this
->What are the items I am getting from my test data. (so, you have use static parameter)
->Which are coming from previous request(so, you have to co relate them)
->Which are based on environment (like web, we see ajax post fix keep IE for Internet explorer recoding , FF for firefox)
If there are some request which are not from there, so we have generate these data applying same logic which is run by our client. Be prepare to make code or get help from development team to write that.(We use JavaScript in Jmeter which runs on via Mozila Rhino)
Before writing, I suggest you to do some research on those not found data based on..
->why we are using?
->Is it avoidable(i remember to send some CSS values . we avid but tests were ok).Re-run and check.
->What are the items inside the data? Simple value or encoded –> encrypted sterilized object. Make those in plain format and try to grab the logic behind that.

Step 9 : Based on analysis create static parameters and co-relate dynamic parameters which are changing based on
->Test Steps
->Test Data
->Application version/release/environment
->Client Environment where test will be executed(for web, browser agent, browser version, OS version etc)
->Credential
->Session/Date & Time /Cookie
->SSO(if you are using) or any security system

Step 10 : Apply those manual algorithm inside script to complete script.

Step 11 : Apply data initiation and clean up script either in script or by external process.

Step 12:  Verify the script by replay with logs as well as debugging. (if tool supports)

Step 13 : (optional)Arrange more than one created script for a single business required scenario to complete the full scenario. (like in jmeter, if you have 5 log in test cases, arrange them in separate test fragment and make a single scenario thread and use under that). Or, in load runner , organize the Run settings.

Step 13 :Add minimum reporting elements and you are ready.

This is continuous post, I will add more incrementally. Feel free to comment. your ideas.

Thanks..:)

Performance Bugs: How to find? How to report?

In this article we are going to know how to find bugs related to performance as well as how to report them. This is very basic checklist which will differ based on your project. I will try to give example with asp.net web application.

To know about basic bug and reporting , you can see one my previous posts which tells us all about a bug report.

Performance Bugs :

This is kind of admiring that Performance bugs are hard to find. Very difficult to get the evidence. So, if you are finding bugs, you have to be preferred with full performance goal and requirements. 
And, you have to monitor your application as well as resources(CPU/Memory/Disk/IO etc).
While monitoring resources, you have to see the pattern of resource uses and application behavior.
You need to make heuristics analysis either with pen and paper or inside monitor tool to observe the change behaviors.

This way you can get the scenario or scope of area for possible bug. It's more like investigation and waiting for possible incident(like Sharlok Homes).

Before doing that, you must know the Architecture of your
->Application (Development and Deployment model, framework used, platform architecture, security modules etc)
->Infrastructure(How deployed, environment, which code/modules executes where & how)

And you also should know, why you are testing in every aspect (Goals and Requirement)

Example : I consider following for an individual request to find bug for my last project(web)

1.    What is the size of the request?

2.    How much time it needs for going to server and coming back with http 200 messages(server get the request) and with responses.

3.    What are parameters we are sending as request in Http body, URL and Headers?

4.    Why we are sending which parameters?
  a.    Categorizing those parameters,
        i. Which parameters are based on client behavior/step
        ii. Which parameters are based on client data?
        iii. Which are based on environment
        iv.Which are based on our application implementation

  b.    Finding unnecessary parameters which browser send from client and how can we eliminate them.(optimization)
    i. Eliminating unnecessary parameters(based on business logic)
   ii. Optimize necessary parameters(like view state, call backs)

  c.    How can we make those requests more precise and optimize for communication.
 
  d.    Organizing resources(JS/AJAX/Image/CSS/JQuery)

  e.    Compress the requests and responses

  f.     Using long time cookie for static resources with expire ending.

5. Why we are sending request in this manner? can we send in any other way to keep the application faster?(like replacing post back calls with Ajax)

How to get the Evidence? 
Difficult bugs comes with much harder traceability...:)
So, as it is hard to find, you have prepare for evidence. Usually, performance bug evidence can be found in these aspects.

a. While debugging application from client and server for defined test steps. This should be done with tools as well as manually. (before performance test script generation in tool)

b. Monitoring resources and application while performance testing(like : IIS/apache, hosted PC, Application , DB for a web application)

c. Analyzing Reports generated by performance tools after test execution

d. Client side single user execution during high /peak load test is going on. You have to have debug tools enabled along with browser monitoring tools(web product)/proxy based tools.

So, to get evidence,
->I suggest to be prepared for Screenshots/Video recording for analysis later on.
->Make some Log analysis scripts or parsers to have quick log analysis.
->Be prepare with Pen & Paper to make quick notes.
->Make you mind for observing interesting important items which you see and related to goal
->Try not to interrupt test execution, better to let it finish
->Try long term test execution , when you need to test capacity of your application . I prefer 12/24 hour run over weekends.
->Try mixing up load scenario for different execution, More user, less time, more user long time, constant user increment, user high-low combination(like spike) , combining high load and security scanning etc
This is fully depend on application that you are working with. It may not be based on my experiences. Try to get more load scenarios involving with your application architecture & performance goals.

How to write a performance bug report? :
This is interesting, usually bugs are very specific to UI or user interactions. But, as performance testing is more related to application architecture testing. So, performance bugs should also have reference with application execution as well as how it happened from architectural prospective. That means, this standard procedure of bug reporting and + Something. And This "something" part should be developer friendly as well as full detail with references(as developers will read and get find what you are trying to explain). Let's see more detail on the "Something" with example. 

Example , for a web application with http protocol, when you click a button, a function bug will refer to only the use case, but a performance bug should also refer to what request we send to server & received from server in details.
You should add reference on the requested URL, header details, body details. It should have detail explanation on those areas.

For this example, let say a http request  Body has 25 parameters & 21 of them with values. So,
- What are those parameters,
- What are the reference of the parameters,
- Why those values are there.

Same justification for request header also. Up to this, it is only description.

So, what will be inside evidence? It should reflect your requirements. Let's say you need to know size, time and capacity for this request and you get the bug. So, in bug report you should provide

a. The total size of the request send to server

b. Time taken for server processing(http 200 response) for this requests

c. How the time varies in case of different load scenario

d. How many responses are received for this request.

e. Individual and total size of all of those responses.

f. Individual and total time  of all of those responses for different load scenarios

g.What is the rendering time for those request(Time from individual single user perform the same task in browser – server response time calculated from server)

h. For Rendering, which response takes how much time in browser(from browser tool, or fiddler or proxy)

And, From individual request, you need to break down each parameter detail and possible cause for taking the amount of time that the request is taking.

For advance QAs : (Root Cause Analysis & Suggestions) :

You can go more further, for that you might need to have access of the code. You have to include profiler and should see what cause taking more time for that single request processing.

For web application, use profiler in IIS as well as in code base to pinpoint each request impact. Like, we are doing in our project , tracing from IIS, prove with profiler in IIS, tracing respective event inside code with code analysis tools and provide evidence on what cause the extra time.

Some, senior QAs can go more further. They can provide possible solution to overcome situation.

Example : In our project, we have found , we are not using viewstate(asp.net) properly that cases a huge time. So, we are in need for optimizing view state. But, there are several ways to do. So, we are suggesting different ways as well as impact of the ways for solving. This involve retesting ,that verifies the bug . This part should consider business requirements as well as security requirements.

So , summary, a bug should contains steps to reproduce and standard items mandatory for a function bug , as well as Screenshots, Logs, Charts, test results, Analysis reports, Code snipped, Demo projects, tool references etc.

Thanks...:)

Performance Testing in a Nutshell

In this article we are going to see Performance testing activities at a glance. The basics of performance testing.
The goal of this post is to have basic idea on how they start and follow-up.

So, here is a small flow chart type(not strictly following flow chart rules) description to illustrate what are the basic steps for a performance testing from my project experience.

Viewers can add their comments and their experiences too.

Performance Testing

Test closure activity is fully based on which methodology you are using in development. I have added small agile practice that I have worked on.

Google Drive Share for this UML..

Thanks…:)

Jmeter webdriver sampler code style: Jmeter Client Side Performance Testing

In this article we are going to see the Jmeter Webdriver sampler code style. Code style refers, how it is constructed, how it is run, how we can apply different ways to resolve issues.
 
Jmeter supports a verity of scripting language to run on it. Webdriver sampler simply use Java Script execution. That means, it follows
This specifications to Run code on running JVM.
From my previous post, you might have idea on how the script was construct. Let me explain details.
image
Rule 1 : no need semicolon(;) ending
 
Rule 2 : We can use any run time Java library by Java Importer. Like here we are importing Selenium and Time
var selenium = JavaImporter(org.openqa.selenium)
var time = JavaImporter(java.util.concurrent.TimeUnit)
So, We can get all Java framework classes as well as any other added jars classes by this way.
 
Rule 3 : How to parameterize? Its simple, like as Jmeter variable call ${variable name}
 
Rule 4 : WDS functions , Just go this link and you will get the details. Let me describe them in overview on how to use them WDS.name, WDS.parameters,WDS.args
image 
WDS.log – Write anything in Jmeter Logs. It helps for debugging.
WDS.browser – This is the Driver Object which represents the Browser. .
WDS.sampleResult – This have all methods applied on results. Like , measuring time, sub sampling, failure state capture. This actually extended functionality of HTTPSampleResults. We can see, we can apply

->Assertion over results

->Sub sampling (breaking big test case in small results)

->Measure time and size for any step

->Get Meta data information of page(like encoding, data types, thread, header, label information like this)

For detail, the API link should be useful. .  So, basically, we have to work with WDS.browser which is same as driver object (firefoxdriver) from selenium webdriver.

We need to careful when we are relating selenium code style in webdriver sampler..

1. This is single file code structure, so avoid page object pattern and other utilities . You may use simple screenshots within single script.

var screenshot = WDS.browser.getScreenshotAs(selenium.OutputType.FILE)

screenshot.renameTo(java.io.File('screen.png'))

2. Don’t forget to add wait time which we do when initialize driver class in selenium
WDS.browser.manage().timeouts().pageLoadTimeout(10, timeunit.SECONDS);
It will need associate two package, add them also.
 
3. Try to use screenshot to debug, spatially Complex UI items. Keep this same important as selenium regression tests.
 
It is an incremental post, I will update those point step by steps.
 
Thanks…:)

How to debug steps in Jmeter Webdriver sampler?Jmeter Client side performance testing

In this article we are going to see how we can debug steps written in webdriver sampler.

This will help us finding specific step time as well as fail conditions. That means, if you are making script for long term project, it is must.

To make a good debug-able client side script, we can organize code in two ways.
1. Sub sampling the steps: 
 
In my previous post you know how to active this. (by adding  webdriver.sampleresult_class=true in user.properties). This is not needed for latest edition (plug in version 1.1.3)
So we will see how to divide them in sub samples.

Step 1 : Add WDS.sampleResult.subSampleStart(‘label of the step’) , this will create sub sample with the label you specified.

Step 2: Write your step to be performed in the browser.

Step 3 : Add WDS.sampleResult.subSampleEnd(true) to close this subsample.
Its very simple. Lets apply to the scripts described in previous post. I will every process separate, like this
   1: var selenium = JavaImporter(org.openqa.selenium)
   2: var time = JavaImporter(java.util.concurrent.TimeUnit)
   3: WDS.browser.manage().timeouts().implicitlyWait(30, time.TimeUnit.SECONDS)
   4: WDS.sampleResult.sampleStart()
   5: WDS.sampleResult.subSampleStart('Goto Home Page')
   6: WDS.browser.get('https://www.sumazi.com/')
   7: WDS.sampleResult.subSampleEnd(true)
   8: WDS.sampleResult.subSampleStart('Click About')
   9: WDS.browser.findElement(selenium.By.linkText("About")).click()
  10: WDS.sampleResult.subSampleEnd(true)
  11: WDS.sampleResult.subSampleStart('Click Uses')
  12: WDS.browser.findElement(selenium.By.linkText("Uses")).click()
  13: WDS.sampleResult.subSampleEnd(true)
  14: WDS.sampleResult.subSampleStart('Click Team')
  15: WDS.browser.findElement(selenium.By.linkText("Team")).click()
  16: WDS.sampleResult.subSampleEnd(true)
  17: WDS.sampleResult.subSampleStart('Log In request')
  18: WDS.browser.findElement(selenium.By.linkText("Login")).click()
  19: WDS.browser.findElement(selenium.By.name("username")).clear()
  20: WDS.browser.findElement(selenium.By.name("username")).sendKeys("shantonu_oxford@yahoo.com")
  21: WDS.browser.findElement(selenium.By.name("password")).clear()
  22: WDS.browser.findElement(selenium.By.name("password")).sendKeys("1234567890")
  23: WDS.browser.findElement(selenium.By.linkText("Submit")).click()
  24: WDS.sampleResult.subSampleEnd(true)
  25: WDS.browser.navigate().back()
  26: WDS.sampleResult.subSampleStart('Click Blog')
  27: WDS.browser.findElement(selenium.By.linkText("Blog")).click()
  28: WDS.sampleResult.subSampleEnd(true)
  29: WDS.sampleResult.sampleEnd()

 And if we run, we can see like this from view result tree.

image


This is because of this parts in the code

image

Now, lets see those individual test results from view result tree.

image

image

image

image

image

image
 
And, Finally the main sampler :

image

You might have noticed that those sub samplers do not have size value as it is part of main sampler.

Main, sampler is working like as Transaction which will be shown in table and results.

Benefits :
->You can get time for specific step without doing massive change in code. Just adding those boundary.
->It will not show in any listener table or graph, so your report format will be same. No effect. 

Not Useful for :
-> If you want to measure size of the particular request. As it is part of a bigger request, it dont show specific size of the request. For this, you can follow like other method..

2. Keeping all steps in separate webdriver sampler :

Now, lets divide those steps among multiple webdriver sampler. Like this.

image

Let see Go to Home Page: webdriver sampler  which contains only request to visit home page.

   1: var selenium = JavaImporter(org.openqa.selenium)
   2: var time = JavaImporter(java.util.concurrent.TimeUnit)
   3: WDS.browser.manage().timeouts().implicitlyWait(30, time.TimeUnit.SECONDS)
   4: WDS.sampleResult.sampleStart()
   5: WDS.browser.get('https://www.sumazi.com/')
   6: WDS.sampleResult.sampleEnd()

or Log in request.

image

you might have followed, this is the time start and end. No sub sampling. BTW, you can apply sub sampling here also.

The full JMX file is here from Google Drive Share. Please see details scripts in there.

Now, lets run the script and see the view result tree. I am adding Synthesis Report to have more clear view. Form there..
image

Let see view result tree :

image

image

image

image

image

image
You need to look at this. This step is only for click Back from Browser, so time became Zero as it is not request from application , browser just gets preloaded page.

image

I use this method to write my script from beginning so that others can understand clearly and we can apply particular step specific formulas(like JS run or any function call that applies on that step only).

Benefits :
->We can see individual request size with time
->You can represent multiple request to gather using Transaction Controller like we do for server scripts. In this way, it is easy to measure transaction time.

Should be careful when
->Large Script to maintain
->Have per-defined report format.

Note : For difference in business, you might need to make this more specific on using. I am showing some ways to have more clear on code. These are easy to start. I will post on code style separately.  

Thanks…:)

JMeter Client side performance testing : How to write web driver sampler code?

In this article we are going to see how to write code in JMeter webdriver sampler(from jmeter plugins). It is part of Jmeter Client side performance testing.

As we know from previous post that webdriver plugins use selenium. So, make sure that you have selenium server(selenium-server-standalone-2.44.0) and firefox 26 in environment. 
Now, Typically I would say, lets follow 2 min tutorial before start. You can get that from jmeter plugins sites.

So, please follow that and have a run.
I am using view result tree to have detail explanation. And, after following the link, It will be looking like this.
image

Let see firefox config , select default unless you have any specific settings.
image

In firefox tab, you can see changing user agent . This is for simulation of different version of firefox.
And, the experimental feature
image

First one is, it will create new browser. It is necessary when you need to initiate session /cookie/browser data for each iteration. Useful when you are running multiple firefox in a single test execution.

Second one is for only debugging. Spatially when selenium don’t get the element, it is difficult for us to find the exact spot. It will stop execution and you can see where firefox holed. 
Now, let's see webdriver sampler. Its pretty basic
image

BTW, as I told in my previous post , this start and stop part will calculate time. for the requests in between them.

Now, lets see view result tree, from jmeter plug in site, they use view result table. But , we are going to see view result tree to have more detail idea on what just happened.
image

From sample result, we see load time, this is in millisecond. Which is exactly the time for going to Google from browser.

The size shows page size. And, see the response code. Though this is client side performance, but it is validating the page based on http 200 response not based on full page rendering. This is because, webdriver sampler is extension of http sampler which validates with response http 200 message.
 So, your site UI elements loading time will be calculated but, it will not show any error in case of more time taken or no resource found. That means, for validation , you have to work manually. What I follow is, i used to check any text value after page loading which gives me exact time measured in my steps. And If I cant see the text(jmeter could not see), jmeter shows error and i get to know the previous request all data are not available yet in browser. This helps me in finding the exact delays and steps to follow.
So, now lets test a famous social media site https://www.sumazi.com/

Step 1 : Create a thread group (Test Plan –>Threads) and name that as Sumazi-Client Execution

Step 2 : Add, jp@gc - Firefox Driver Config under your thread. (right click thread –>add->config element ->jp@gc - Firefox Driver Config)

Step 3: Add jp@gc - Web Driver Sampler under your thread. (right click thread –>add->sampler ->jp@gc - Web Driver Sampler)

Step 4 : Add view result tree.
Note : I prefer to add a jp@gc - Parameterized Controller to keep my variables separate from test steps. You will see that from image in this post.

Step 5. Select use system proxy in Firefox driver config.

Step 6. Now, past the following code in webdriver sampler.
   1: var selenium = JavaImporter(org.openqa.selenium)

   2: var time = JavaImporter(java.util.concurrent.TimeUnit)

   3: WDS.browser.manage().timeouts().implicitlyWait(30, time.TimeUnit.SECONDS)

   4: WDS.sampleResult.sampleStart()

   5: WDS.browser.get('https://www.sumazi.com/')

   6: WDS.browser.findElement(selenium.By.linkText("About")).click()

   7: WDS.browser.findElement(selenium.By.linkText("Uses")).click()

   8: WDS.browser.findElement(selenium.By.linkText("Team")).click()

   9: WDS.browser.findElement(selenium.By.linkText("Login")).click()

  10: WDS.browser.findElement(selenium.By.name("username")).clear()

  11: WDS.browser.findElement(selenium.By.name("username")).sendKeys("shantonu_oxford@yahoo.com")

  12: WDS.browser.findElement(selenium.By.name("password")).clear()

  13: WDS.browser.findElement(selenium.By.name("password")).sendKeys("1234567890")

  14: WDS.browser.findElement(selenium.By.linkText("Submit")).click()

  15: WDS.browser.navigate().back()

  16: WDS.browser.findElement(selenium.By.linkText("Blog")).click()

  17: WDS.sampleResult.sampleEnd()

It will be like this

image

Now , Run the script. You should see browser opens and you can get the the all methods are working.
Now, lets open view result tree listener and see what is in there.

image


So, the time is 10.851 second for all steps and size 23905 byte for full requests.

JMX file from Drive Share

This script only made for going to all URL and measures the full browsing time. This is combined of steps, like specific transaction. In next blog, we will see how to debug and break steps to get actual user time for particular request.


Thanks…:)