How to run multiple version of firefox in windows for automation?

In this article we are going to see the tricks to run multiple version of firefox in windows 7. You can have multiple version but only one version should be in path variable.

This is spatially needed for running Jmeter with firefox 26 and selenium with latest supported firefox.

Before starting, we have to be sure that there is no firefox install in PC for all users. This is important as by default all firefox driver path are pointed to Program Files (or x86 for 32 bit). If firefox found in this folder, this method will not work. So,

Step 1. Un install firefox from the system if it is available for all users. Delete program data and temp files (for firefox) also.

Step 2. Install Latest firefox in particular user directory or any other drive. I have installed in my user directory.In My PC , updated firefox location is "C:\Users\Shantonu\Mozilla Firefox"




Step 3. As , firefox maintain all settings globally, so run firefox and disable all update from option

 
You might disable all health reports also form option ->advance->data choice

Step 4. Close firefox an open again to check update disabling works.

Step 5. Close firefox and install firefox 26 in either user directory or other folder. (in image my firefox 26 is installed in user directory). Open firefox and make sure that you see update settings are disabled (from option). In my PC, firefox 26 location is "C:\Users\Shantonu\Mozilla Firefox 26"

Now, technically you are ready to go. But, there might be some issues with firefox profile due to no firefox is installed in default location (mentioned in driver).

 

For this, you need to create a firefox profile. So,

Step 1 : Close all firefox and open latest firefox profile manager. For that , from start menu -> run this "<path to latest firfox>" -profilemanager
For my PC, it was "C:\Users\Shantonu\Mozilla Firefox\firefox.exe" -profilemanager




Step 2 : Create a profile , following the UI instructions. (Create Profile ->


 ->Next

 

Give name (I gave name JmeterSelenium) and choose a path (my path is "D:\Env\JmeterPlugins\firefoxProfile"). And click Finish.

So, you have created a firefox profile.

Now, if you close all and run , you can run firefox 26 (which should be in path variable) with jmeter and latest firefox with eclipse.
As well as working with latest firefox, you can run jmeter test with firefox 26. 

And, for those people who run selenium , use this code (if u r using java)
System.setProperty("webdriver.firefox.bin", path);
before creating firefox driver.


Thanks...:)

How to perform client side web performance testing in JMeter?

In this article we will see how can we do client side performance testing using Jmeter Plugins.

I will be using jmeter webdriver plugins. Before starting this topic, please have some basic on what is client side performance testing from my previous post.So, lets start with

Installation :

1. Install Jmeter and Plug ins from below links following this post.

->Jmeter

-> Plugins(you may choose only web driver, but I prefer all of them)

2. Download Selenium Server from here. (you need java to run that)

3. Download Firefox 26 from Archive. Why 26? because jmeter webdriver plugins supports firefox 26. Here is link where you see supports the details.

Note: This can be tricky if you have updated Firefox version. In that case you can do like me.

->Disable firefox update checking

-> Install in new folder in separate directory name.

->When you run this one for the first time , just cancel initial update process. As you have disabled firefox update(in your updated firefox), make sure you see the update settings disabled in this firefox 26 too.

Note : This part is little tricky, I have provided separate post to resolve it.
For Jmeter Remote Execution or local, it is better to to have only one firefox (version 26) with no auto update settings which will minimize complexity for test execution.

4. Keep firefox 26, selenium server in path variable. To check, type firefox from command line and run. You should see, firefox 26 launched in desktop.

5. Setting Jmeter: Usually , we do not need any extra things for webdriver sampler. But, as we need debugging, we might use this following property in user.properties file.

It enables sub sampling which is good for debugging. 

webdriver.sampleresult_class=true

Let me explain more in how it works: JMeter webdriver sampler is just an extension of http sampler, not alternative, with a script editor. When it runs, it actually calls firefox driven by webdriver. That means, it send its instruction to mainly webdriver and, webdriver does every thing. Now, you might be wondering how code goes to web deriver. Like as other code support, webdriver core run as external code following JSR specification. It is actually JavaScript execution. And, you see, it is just like as webdriver java code with some basic modification due to jmeter adoption. I will provide separate blog or coding.

And after you write the steps as webdriver script, use listeners to get time. Like as other samplers, you use listeners to debug sensibly.

Browser supporting : 
Just follow this link which tells the configurable browser names that are supported by webdriver sampler. You can see this from jmeter also,

Time Measurement:

webdriver sampler calculates time from this line of code

WDS.sampleResult.sampleStart()

to this line of code

WDS.sampleResult.sampleEnd()

So, for debugging, we need sub samples which will be shown as child of main sample. For this purpose, we need to activate sampleresult_class true. After activation we can do sub sampling like

WDS.sampleResult.sampleStart()

//Brows to a URL

//Add user name and password

WDS.sampleResult.subSampleStart('Log In request')

//Click Log in
WDS.sampleResult.subSampleEnd(true)

//Check title

//do some another process

WDS.sampleResult.sampleEnd()

In view result tree listener, you can see , main sample contain sub sample name “'Log In request'”. And a main sample can have multiple sub sample. That means, we can calculate each time separately from result tree.

Note that, sub samples will not be shown separately in tabular format listeners or graphs.

And, if we need to measure particular transaction . we can split single test among multiple webdriver sample. Like, a sampler for log in, a sampler for doing some work in home page, a sampler for messaging. In this way, we can see results in reports. Usually each business transaction is measured in separate sample where as detail steps are sub sampled.

Writing first script :

To write webdriver sampler script , you need to add webdriver sampler(sampler –>webdriver sampler) with any browser (driver)configuration. See image from browser supporting section to get the driver configuration elements.

I will provide separate post for how to write webdriver test script with an example. You can see some nice guidelines form wedriver sampler wiki.

Test Planning :

As we know from my previous post on client side performance test, this test should run from single user or thread. As jmeter sampler populate browser with webdriver, this has me particular hardware requirement. That is, it will occupy a single thread of a processor. That means, if you want to run webdriver sampler, you need at least 2 core CPU. Why 2 core, another one is for Jmeter. So, if you have 8 core CPU, you can run only 7 threads for webdriver samplers. So, for testing we have to add separate thread group or test block for webdriver sampler.

We will run for measuring client execution time on

1. When there is not much user load, .

2. When average load on server ,

3. When High load(considered as peak load)

Sometimes , It is also better to test under

1. Beyond capacity , where error can be occurred or may just after error condition.

2. As continuous performance testing. Usually, people run selected regression test with jmeter daily or weekly.

Again, formula is simple, 1 thread of CPU for single client performance testing. 

And, you run the test simply like as Jmeter test, that's all. More detail will be in my blog with example.

Thanks …:)

What is Client side performance testing in Client Server Application?

Moved Here

What is Performance Reporting? Example with web application.

What are Performance requirements?

This blog has been moved here

Why Performance Test? Performance test goals.

This post has been moved here

What are Performance Tests?Basic Performance Test Types.

This blog has moved here

How to get all Method information under a Class in Java Reflection?

This post has been moved here

How to get all Constructor information under a Class in Java Reflection?

This post has been moved here

How to get Class Info in Java Reflection? Class Name and Signer Names

This post has been moved here

How to get Class Info in Java Reflection? Packages and Class Modifier.

This post has been moved here

How to get Class Information with Java Reflection? Interfaces and Super Class

This post has been moved here

What is Reflection? How it works in Java?

This blog has been moved

Introduction to codeception(Acceptance,Functional,Unit) tests.

In this article we are going to learn about Codeception Tests. What is Codeception test? How it works?

What is Codeception?

Codeception is full test framework for TDD and BDD styled test cases. That means we can write both traditional Unit Test formatted test as well as Behavior Driven Development formatted tests.

Codeception is build on PHP. So, we need to have PHP running in the PC as well as in the path. You may read these posts to install environments which is needed for codeception. 

1. How to install PHP in windows?
2. How to install PHP Extensions in windows?
3. What is composer? How to install composer in windows?

Codeception contains mainly 3 suits.

1. Acceptance Test suite

2. Functional Test Suite

3. Unit Test Suite

We can add API test suits manually. So, codeception is complete testing framework. It has supports for different modules.. You can get detail module description from this link.

For different type of tests, codeception has different configurations you can get full configuration file lists from here.

How It works?

Codeception is build on

a. PHPUnit

b. Yii

c. Mink

d. Bahat

and several other modules as utilities..

Codeception (the BDD style) works like as user prospective Guy Format. Guy format refers to BDD style code. In Codeception there are mainly 3 type of Guy. WebGuy, TestGuy and CodeGuy. We can have another guy, named APIGuy. When we create a test case, we create a new guy object(of the guy class). The webguy responsible for acceptance test, testGuy for functional tests , codeGuy for unit tests and the apiGuy for API tests. Example : for webguy (I have used a free site from another blog and hosted locally, the site has a text box and a button, what ever we insert in the text box, when we click the button the site converts the input text into upper case letter. )

   1:  <?php

   2:  $I= new WebGuy($scenario);

   3:  $I->wantTo("TestMyFirstApp");

   4:  $I->amOnPage('toupper.html');

   5:  $I->see('Convert Me!');

   6:  $I->fillField('string','ShantonuSarker');

   7:  $I->click('Convert');

   8:  $I->amOnPage('toupper.php');

   9:  $I->see('To Upper!');

  10:  #$I->see('String converted: SHANTONUSARKER');

  11:  $I->click('Back to form');

  12:  $I->see('Convert Me!');

  13:  $I->fillField('string','');

  14:  $I->click('Convert');

  15:  $I->amOnPage('toupper.php');

  16:  $I->see('To Upper!');

  17:  $I->see('No string entered');

  18:  $I->reloadPage(); 

When we write tests in CEPT format(BDD Style), It actually compiled two times before execution. That means, first it will compiled with depended modules and again compiled during execution. So, if we write our custom functions in test case files, it will be run twice, so, it is not proper to write custom functions in the test case. Codeception has separate way to write custom module. Like as other module, we have to write our custom module and include in configuration file to enable that. Then if we build, it will accessible from our Test Files with Guy Class. 

There is another format named as CEST format which is actually PHPUnit format(where we have before and after function).

It is woks like PHPUnit. See my other posts describing CEST & CEPT format test cases.

We can run codeception acceptance tests in JS free PhpBrowser or with Selenium Server(any browser supported with selenium server). For functional test, phpbrowser is good. And for other test, we might not need any browser(as they are code level tests)

There are some basic rules for using codeception. Usually codeception tests are run along with main code base of the project that we need to work on.  This is because of dependency. That means, if codeception tests have dependency on the modules used in main code base, we have to use the tests with code. If not, we can have separate codebase. Usually, functional tests , unit tests and api tests are depend on main code base. So, when we are writing those we need to have code base . And we run the test in same server where our code in running. But , acceptance tests suite can be run independently even with a real browser. That means, if we do not have codebase of main project, we can still make test cases for that site and run. So, we can say like that

Acceptance Suites = Can run the test Externally(can be run from outside of the codebase)

Unit/Functional Suites = Should be run internally (with source code)

You may see codeception supporting modules where we can write tests from this post. We can write test for those modules.

Note ; Now a days, selenium IDE has a plug-in for Codeception Formatter which recode and convert tests in codeception format.

Thanks…:)

How to Configure Codeception?

In this article we are going to see how can we configure codeception for different configurations.

As we know, codeception has different modules (you can know more detail from this post) and we can basically have three main suits if we initiate test cases(see this post to know how to initiate test cases)

If we initiate the full test suite we might see these three YML file for configurations.

A. codeception.yml : This is a global configuration container written in simple markup language. It contains
->All test settings and configurations
->Central DB configurations
->Bootstrap settings and result view configurations.
->Global Module Configurations.

Here is the default configuration which created while first test suits are created.


B. acceptance.suite.yml : In the tests folder , we will get this configuration YML file for acceptance test. In here, we see configuration specific for acceptance test suites. We get
-> The enabled Guy class name (webGuy)
-> The enabled modules and module related configurations
->Custom Module names(which we create for acceptance suits)

A default acceptance.suite.yml is


C. functional.suites.yml : In the tests folder , we will get this configuration YML file for functional tests. In here, we see configuration specific for functional test suites. We get
-> The enabled Guy class name (TestGuy)
-> The enabled modules and module related configurations
->Custom Module names(which we create for acceptance suits)

A default functional.suite.yml is


D. unit.suites.yml : In the tests folder , we will get this configuration YML file for unit tests. In here, we see configuration specific for unit test suites. We get
-> The enabled Guy class name (CodeGuy)
-> The enabled modules and module related configurations
->Custom Module names(which we create for acceptance suits)

A default unit .suite.yml is


Note : You can see my this post for getting different settings for those configuration ymls.

Thanks ..:)

What are the modules in Codeception? Why we use those?

In this article we are going to see the all available modules of codeception and we will try to find out why we use which one.

As we know, codeception works in 3 ways(so 3 type test suites are there). I am adding modules under the suit category

1. Acceptance Test : Codeception test with browser. So, it works externally. We do not need application codebase to test that. All type of acceptance tests, user level functional test can be in this type. And available module 

A. WebDriver:  Previously it was Selenium and Selenium2. We need selenium selenium server and supported browsers. We can also add phantomJS browser in the server to run test. Note, we need to start the server before running tests.

B. PhpBrowser : Used in acceptance tests with non-javascript browser. One of the popular test browser in Codeception.

C. SOAP : For testing SOAP WSDL web services with PhpBrowser or Frameworks. It sends requests and check if response matches the pattern.

D: REST : To test REST WebService with PhpBrowser or Frameworks.

E: XMLRPC : For testing XMLRPC WebService with frameworks or PHPBrowser

2. Functional Test : Codeception test with command line frameworks. So, all functional and API tests are in this category. We need project code to test them.

A. Dbd: This replaced Db module (in functional and unit testing). It requires PDO instance to be set.

B. Laravel 4 : For writing functional tests for Laravel

C. Symfony1 : For working with Symfony 1.4 application. Doctrine and sfDoctrineGuardPlugin are used to Authorization features.

D. Symfony2 : Uses Symfony2 crawler and HttpKernel to emulate requests & get responses.

E. Yii1 : it provides integration with Yii framework. We need to install Codeception-Yii Bridge which have component wrappers.

F: Yii2: This module provides integration with Yii2 Framework.

G: ZF1 : To run tests inside Zend Framework. It is like ControllerTestCase with Codeception syntax.

H: ZF2 : This module allows you to run tests inside Zend Framework 2.

I : Silex : For testing Silex applications like you would regularly do with Silex\WebTestCase

J : Phalcon1 : This module provides integration with Phalcon framework (1.x) for testing.

K : Kohana : Used for integration with Kohana v3 functional tests.

3. Unit Test : Like as functional test, codeception run that internally with PHPUnit. So , we need code to test.

A: Dbh : (described in functional test)

4. API Test : For testing api with wither acceptance test or functional tests

A. Facebook : Provides testing for projects integrated with Facebook API

Independent Module : Some spatial modules can be used commonly or independently with any suit. 
A. AMQP : This module interacts with message broker software that implements the AMQP(Advanced Message Queuing Protocol) standard. 
B. CLI : Which is a wrapper for basic shell command. 
C: Doctrine1 : Allows integration and testing for projects with Doctrine1.x ORM. 
D: Doctrine2 : Allows integration and testing for projects with Doctrine2 ORM. 
E: Db: This module works with SQL Database. 
F. Filesystem : Module for testing local file system. (extendable for FTP/Amazon s3, others ) 
G. MongoDb : Works on MongoDB database. In order to have your database populated with data you need a valid js file with data 
H. Redis : Works on Redis Database. 
I. Sequence : It works on data clean up. This is used for initiating data in database for testing and we can clean up. It has no effect on testing but have on initiating test data. 
J. Memcache : Connects to memcached to perform cleanup by flashing all values after each test run.
K : Asserts : A module for different assertions (validations)

Thanks…:)

How to Run Test in Codeception?

In this article we are going to see the Test running command. From my previous article we know about initiating test case. Now its time to run. I will write commands for running in different ways.

In command prompt (change directory to root of the project where we have the codeception.yml) write those (In all cases I have use 2 type commands , a codeception with phar , and codeception with composer)

1. Run All Tests :
A: php codecept.phar run
B: ./vendor/codeception/codeception/codecept run
2. Run Only a Particular Suit :
A: php codecept.phar run <suitName>
Example  : php codecept.phar run acceptance
B: ./vendor/codeception/codeception/codecept run <suitName>
3. Run Only a particular test file under a particular suit
A: php codecept.phar run <suitName> <testFileName>
Example : php codecept.phar run acceptance regWithEmailCept.php
B: ./vendor/codeception/codeception/codecept run <suitName>  <testFileName>

Options :
And Now, with Every Run Command We can use different options. Day by day, the number of options are getting increased. I am mentioning some necessary options that we may use.

1. To see test running step by step , we have to add –steps in the command . (I am giving example for only this option, others are similar to this.) , From previous example if we want to add the steps option, the command will be
php codecept.phar run acceptance regWithEmailCept.php –steps

2. To run tests with no stepping use --silent

3. To use custom path for configuration file (suite specific YMLs) . we need to add --config or –c , like
--config = “path to configuration yml”

4. To debug test and get scenario output we need to use –debug or –d

5. To run Tests in Groups, we need to add --group or –g , like
--group=”name of the group”

6. To run tests with skipping group use --sg or --skip-group=”group name to skip”

7. To run tests by skipping test suit use –skip or –s. (we need to mentioned suit name like as group)

8. To run tests with specific Environment use --env=”desire environment values ”

9. To keep the test running (no finish with exit code) we use --no-exit

10. To stop after first fail , we use –f or --fail-first

11. To display any custom help message use –h or --help=”our desired message”

12. To stop all message showing, –q or --quiet

13. To skip interactive questioning use --no-interaction or -n

Coverage Options :
1. Run with code coverage  use  --coverage
2. Generate CodeCoverage HTML /XML /text report (in path) use 
--coverage-html
--coverage-xml
--coverage-text

Report /output Options :
1. To show output in compact style use –report
2. Generate html/xml/Tap/json log use
--html 
--xml
--tap
--json
3. To use/hide colors in output , use –colors or --no-colors
4. For Ansi or no ansi output use, -- ansi /--no-ansi

Thanks…:)