Performance Testing Tips : Writing test script (aka Vuser scripting)

In this article we are going to know about some tips and tricks on writing scripts for performance testing in popular performance testing tools.(aka vuser script).

This tips might be helpful for writing performance test script. I would recommend you to read my previous post.

Tips :1. I prefer, separate vuser script for separate test cases. I have seen some engineers use same test case having multiple steps based on condition to perform multiple steps to perform a scenario. That makes the test case very complex and after a time , it is hard to understand. So, better, if you have different verity of test step for a single scenario, i suggest put them separate test case. And mix-up those test steps during execution. Example : If you site has 5 ways to log in transaction, try to make 5 different test cases for each way. Like in Jmeter, I use test fragment and mix those scenarios while test execution.

Tips :2. For debugging and checking all of you correlation values , use logging or debug sampler(jmeter). It helps to get the problems quickly.

Tips :3. Keep your application architecture matching tools with you which you will need frequently

For regular expression : Regular expression tester.

For Data Format conversion (based on what your application uses): Base64 encoder/decoder, URL Encoder/Decoder, Unix Time Stamp converter.

For File comparing or string comparing : Compare it, winDiff, Beyond compare.

For smart string view and search : Notepad++(my favorite)

Tips :4. If your application is web application, use browser based tools (with plug ins to get best out of that). I even use blazemeter chrome extension.

Tips :5. Try to familiar with different type of data format like how base46 encoding look like, How URL encoding look like. How they converts string. It will help you to find what is inside.

Tips :6. Understand all locally executable functions clearly. When you are creating Vuser, that simulate load, you have to understand how can you simulate those executable functions. Example, let say for a chat application, you might have simulated log in, but after login , local client set some parameter(based on implementation) before initiate communication. You will find same behavior for web application where you use JS/Ajax calls for various functionality. And , this is become complex if you are testing single page application. You need to keep track on all of those JS execution as well as full sequence and logics. Don’t trust on some thing happens automatically. Be clear on all of your gray area of understanding the system.

Tips :7. Try same test with another performance tools to have different view of the system that will help you clear understanding on what you are doing. This helps spatially on correlation suggestion. I use WAPTPro(personal choice). You may see my post to know about it from basic level.

Tips :8. Do not trust auto co relation. For co relation, you tool need to determine about you application architecture which is impossible. Tools can help to suggest but, try co relation manually. Often, tool suggestion are good for platform you are using (like , for web, if you use asp.net, load runner or wapt will suggest you to co relate view state as it is part of asp.net by default). My suggestion will be, if you ever use co-relation which are suggested , verify this with regular expression tester. To , do that, just copy recorded response data and past on regular expression tester body and test the regular expression that the tool suggested. Load runner by default use Left & Right boundary to make co relation function. Just convert that left and right boundary expression meaning to single line regular expression to verify.

Tips :9. Be master on pattern matching from the mind. It will help applying regular expression (you need to understand the regular expressions too) as well as heuristics which defines changed behavior among requests. I think , from a skill prospective, this is the most hard and important part. If you are good at pattern matching, performance testing is easy. It will help you to solve all complex algorithm understanding as well as co relation difficulties. Bonus part is, you will learn the system from business to deployment( Application architecture, DB, infrastructure, networking, Operating system internals, deployment system and everything related to your system).

Tips 10: You should store test script in  version management tools similar to general codes. It will keep you track and knowledge about your script more clear.

Tips 11 : Based on your requirements, try to group your request under Transaction which will give you sum of all child request time. Example : keep all request for a log in under LogIn transaction which will help to know what it total log in time while getting the results.

Tips 12 : If you are writing custom function for logic behind application, use script language which keep less overhead. Example: I prefer writing code in Groovy rather than bean shell.

This is an ongoing post. I will add more ideas incrementally…

Thanks..:)

Qualities of a Performance test script(aka vuser)

In this article, we are going to see some basic quality of a performance test script which is also know as VUser script. This post actually continuation of this main post

Qualities of a VUser Script :

1.Usages/Load Model :  Vuser script should reflect a real user. That means, the real user usages model(work load model).

Example : when we add item in shopping cart(eCommerce site), our shopping card will add the item and user see that added item after some time(ex : 500ms or more). Or, when we provide log in request, user will wait for at least some logical time to log in(Ex : 200ms or more). That means, based on your performance goal and requirement. Understanding the usages model is very important. So, add relative delays based on your scenario with feedback from business users or real users(or from stakeholders) while collecting the Performance goal and requirement and add those delays or conditions respectively.

2. Follow User Steps: This means referring user steps inside script. Vuser script must reflect business functions. Script should have reference from Performance Goal and Requirements. And, organize in such way so that a person can understand the scripts when he knows about those Goals.

3.Efficient use of Resource:  Vuser should occupy less memory and processing power. We need to careful about un necessary pre/post processor usages, new variable initialization. Re using same variable as well as do multiple processing in same execution will make the script more efficient.(Example : Try to reuse same correlation variable if it is changing or when you are parsing, get multiple data same time , try to run same post processing multiple time. Incase of Regular Expression, multiple grouping works very well)

4. Initiation and Cleanup : Vuser should have cleanup steps. More often when we do test, there are lots of test data. Depending on system, a smart vuser should have cleanup steps to get those un necessary data out of the system. This is similar to TearDown/ @aftertest functionality if you are familiar with unit testing.

5. Comments : Always use comments with good naming convention. In agile, it is said , your code should be self explanatory. Added to that, if you follow good naming convention and comments that reflects to requirements, it will help you as well as your successors to follow up your script. Both Jmeter and Load Runner(C code) has comment options that really helps for keeping track.

6. Code Best Practices : Vuser script is similar to code. Some tool like Load runner use C code in background. So you should follow best practices, avoid code smell, organize in maintainable way and try to get rid of all of anti patterns.

7. Re Use build in functions : Reuses tool in built in functions, avoid creating new functions which are available inside tool.

8. Assertion/ Validation: VUser should have assertion or validation after each step or transaction of group of transaction. This is fully based on you domain application. Example : if you are doing log in transaction, try to keep assertion with any item from the page that get loaded after login. This way, it will prove, you server responses after your login request and loads the page(send all items) after the request.
In some cases when you need to check whether server is responsive or not(http 200 message) you can avoid this. I used to avoid this for load test but I add this for stress testing. Example : when I need to check all the page of a website links and responses are ok or not, I avoid this. But, I use this when I need to verify after log in request, there is a real log in or after credit transaction, the change value is coming.
[Thanks to Joerek van Gaalen , he pointed out (i forgot) ]

Note : Don't forget to deduct the time taken due to assertion/validation which is taken by tool not your server.

This is ongoing post. I will add incrementally. Please have comments.

Thanks…:)

What is VUser ? How to write VUser or Performance Test Script?

In this article we are going to see how to write performance test case which is know as VUser script. We will try to learn generic way so that we can apply to Jmeter, Load runner, WAPT or any other tools in the market.

The steps might change in order but the idea behind all are almost same for every tools.

And, We will discuses about performance test script for server side performance. So, there will be no reference for client side performance testing.

What is VUser ?

VUser means virtual user . A real user behavior will be simulated by virtual user for performance test.

What is VUser Script?

When we write real user behavior or activity simulation through a virtual user, then it is referred as VUser script. This name came from load runner world. In Jmeter, it is called Test Plan. In generic, we can say Performance Test Script.

Thought VUser is not not real user, but, it should reflect real user scenario. In Jmeter we use different component to create a series of requests (may connected to each other or not) which are executed serially one after another. In load runner, we write code in C using load runner function. We might record those steps, but make sure that the script performs the activity of a scenario which we target for.

How it works?

Vuser scripts simulate the functionality on the target application from a non GUI(with background process model). This is because GUIs are heavy as well as not possible to run with multiple(thousands) instances in running environment(like running 100s browsers with separate session of same application or generating 10000 chat client connected to server via chat GUI). So, Vuser script use a system (provide by respective test tool) to simulate target number of users. Some time this system can be build on multiple PC/Server, specific network setting, various Infrastructure configurations.

Example : Load runner has vugen for writing scripts and controller(combining with multiple pc) to run and they have a analyzer for showing test results in smart way. 

Jmeter has everything build in, you can use jmeter for writing script, again configure with multiple PC running jmeter as slave(remote execution model) to simulate load and after running you can use jmeter to see and analysis the results with different Graphs(Listeners)

This part actually fully depend on tool. See some of my blog posts on Jmeter. 

How to write VUser or Performance Test Script?

This part is totally tool specific, but I will try to keep it more generic from multiple tool using experiences.

Step 1 : Write manual steps of the test following performance goal, requirement and with test data.

Step 2 : Have a plan for test data initiation and cleanup

Step 3 : Record the full test plan with your tool and verify every step with another tool. If you application use multiple protocol, record in multiple time or multiple tools. Be careful with request you are working with. (Example, if you are recording web request, verify recording steps with proxy like fiddler with plugins or wire shark or charlesproxy.)

Step 4 : Record script twice with exact same step and exact same data

Step 5 : Compare those two steps and get the differences.

Step 6 : Analysis those differences and get dynamic parameters.

Step 7 : Again, Record and compare with previous recording with

->Different credential to know more dynamic parameter based on credential

->Different test data to see dynamic parameter based on data.

->Different PC/Browser/network environment(will change the client and change the dynamic parameter)

->Different Version/release of the application (effective on web, example : our current application changes some parameter based on released version)

Step 8 : Analysis the data coming and going from client and server. This is the main area of learning the system with RAW request from a transport level. This is definitely hard but, tools are there to help you understanding they system.

I should say, Jmeter is better to view each item clearly, compare to load runner but, load runner has advantages over co relation data(next step).

These are some points that are follow for analysis for a web application

a. From request, what are the changing sectors for your protocol.( For http , data changes in URL, Header , Body)

b. For a specific part of your request, what are the changing items.  Usually, you see parameter and value. Focus on is value changes. How it changes, and why it changes.

And, for dynamic elements, you might face, the parameters are also changing. Get to know, why changing, how changing?

c. So, after getting over all knowledge on changing parameter and value, you need to make a list of parameter and data that you need to generate to simulate the request. You need divide them like this

->What are the items I am getting from my test data. (so, you have use static parameter)

->Which are coming from previous request(so, you have to co relate them)

->Which are based on environment (like web, we see ajax post fix keep IE for Internet explorer recoding , FF for firefox)

If there are some request which are not from there, so we have generate these data applying same logic which is run by our client. Be prepare to make code or get help from development team to write that.(We use JavaScript in Jmeter which runs on via Mozila Rhino)

Before writing, I suggest you to do some research on those not found data based on..

->why we are using?

->Is it avoidable(i remember to send some CSS values . we avid but tests were ok).Re-run and check.

->What are the items inside the data? Simple value or encoded –> encrypted sterilized object. Make those in plain format and try to grab the logic behind that.

Step 9 : Based on analysis create static parameters and co-relate dynamic parameters which are changing based on

->Test Steps

->Test Data

->Application version/release/environment

->Client Environment where test will be executed(for web, browser agent, browser version, OS version etc)

->Credential

->Session/Date & Time /Cookie

->SSO(if you are using) or any security system

Step 10 : Apply those manual algorithm inside script to complete script.

Step 11 : Apply data initiation and clean up script either in script or by external process.

Step 12:  Verify the script by replay with logs as well as debugging. (if tool supports)

Step 13 : (optional)Arrange more than one created script for a single business required scenario to complete the full scenario. (like in jmeter, if you have 5 log in test cases, arrange them in separate test fragment and make a single scenario thread and use under that). Or, in load runner , organize the Run settings.

Step 13 :Add minimum reporting elements and you are ready.

This is continuous post, I will add more incrementally. Feel free to comment. your ideas.

Thanks..:)