Extracting info from java Exception-Trace messages by Regular Expression

In this article, we are going to see how can we extract information from a Exception/Trace messages using Regular Expression. This is regular basis need, spatially when you are analyzing a trace message that you found after a automated test process or unit test.For basic idea about regular expression, see my previous post.

So, when we analysis a trace message, we have to retrieve information from a single trace message from trace stack. I am providing an simple example with an android trace message.

at widget.List.fillFrom(ListView.java:709)

In here, the format of the trace is packageName.className.methodName(FileName:lineNumber)

So, by construction all type of names that are used in java are made of alphabets, so the expression of every name will be [a-zA-Z]+.(any character, single of multiple which are are under rage of a-z, A-Z)
Every line number may have single character to 4/5 character(if we write very long class). So , for line number the expression will be \d+(any digit more than 1 time)

Now, let's look at the message, it begins with at and have a space then the original message.
And in the end it closes with ).
And after every name there is a Character(dot character or braces) .
So if we add those conditions together, we get these

Now we group our necessary items that we need to get from the expression. As we want package  name, class name, method name, file name, line number

If we do not want package name and class name , we just omit the braces.

So, in this way we can get our necessary information from a expression .

Thanks ...:)

How to extract data from Email by Regular Expression

In this article, we are going to see how we can retrieve different information from an email address.

Normally we have email address like sarker.shantonu@gmail.com and some time we may find
sarker.shantonu+abcedd@gmail.com , but due to implementation rules , we omit the characters after+ . So, we need to retrieve information from a valid email address. What info can be retrieve?

1. Domain Name(in example gmail.com)
2. User name

So, let's take the complex data as  sarker.shantonu+abcedd@gmail.com
In here, the domain name start from @ and then small letter words, here can be capital and numbers. and end will be up to dot(not including). So, our regular expression will indicate a alphabet/ numeric number from beginning and end which can be one or more(to accurate 2 or more)

So, for domain name extraction, it will be @[a-zA-z0-9]{2,} - in here max limit is not mentioned, so I keep it blank {2,}. And [a-zA-z0-9] means, any character a-z or A-Z or 0-9.
To make is simplify, @[a-zA-z0-9]+ : This will results same but + refers to 1 or more existence of the character set.

Now, we need to get the user name. An user name start with any of numeric or alphabetic character, ends with numeric or alphabetic character, and in between them it can contains dot(.) , Dash(-), underscore(_). So, our regular expression will be

[0-9a-zA-Z] means any character a-z or A-Z or 0-9.
* means previous charter set can be repeated 0or more time
[-._\w] means it can have - or . or _ or any alphanumeric character
*means previous charter set can be repeated 0or more time
[0-9a-zA-Z] means any character a-z or A-Z or 0-9.

If we now, use grouping both where first group will show user name and the 2nd group domain. Then our expression will be like as below

so, we have our user name and domain in separate group.


Introduction to Regular Expression

In this article, we are going to learn basic Regular Expression. Regular expression is one of the daily necessary items for a developer as well as tester. My focus will be on how to cover both side. I personally use regular expression
1. Jmeter , for extracting information from response data
2. Data Scraping, for extracting information from any HTML/CSS/XML element
3. Selenium webdriver, for finding element
4. Robotium, for finding element
5. Genie, For finding element.

So, my focus will be the basic terms and examples which are used by me.

First, let's have an idea about what is Regular Expression. A regular expression is the set of characters containing symbolic and literal meaning, together they create an expression that matches like as pattern match from a given character set. Detail description, you may find in wiki. The main objective is , getting objective character set from given set of characters.
So, a regular expression activity will have 3 parts.
a. Given text or set of characters or a doc( text/html/xml or any)
b. Pattern to match( the regular expression)
c. Matched Array(the result set)

Before start writing Expressions, we need to know some string meaning(literal) in the regix. Let's use see in the rgular expression the meaning of the symbols. 
a-z - Letters
0-9 - Digits
A-Z  - Capital Letters
. - any character

\d - any digit(single character)
\D - any Non Digit character
\. - Period
\s - white space
\S - any Non White space
\w - any alpha numeric character 
\W - any non alpha numeric character 

^ - starts with indicator
$ - ends

( ) - makes/captures in a groups
(a(bc)) - makes/captures in a sub groups
(.*) - makes/captures variable content
a|b - matches a or b

[abc] - only one of a, b or c
[^abc] - Not of [abc], means no a/b/c
[a-z] - any single character of range a to z
[A-Z] - any single character of range A to Z
[0-9] - any single number between 0-9, it is similar to \d

{m} - m Repetitions
{m,n} - m to n Repetitions
* - 0 or more Repetitions
+ - 1 or more Repetitions
? - optional

So, what are those. Those are meaning in the expression. Example - if our data string is like abcdefgh123ijklmnopqrstuvwxyz
and we need to find 123 from there. so, our regular expression will be simple  
we can add some intelligence here..  what if we do not know the value is 1,2 or 3. If it was told to use that 3 adjacent digits. so , our expression will be  
If we do more smarter way, as there is 3 times digit we ca write the same thing in this way
If, it was mentioned that we will have a digits but there can 1 digits minimum and max 3 digits. Then expression :
If we make a stile change , say, we want to have at least 1 digit to and up to infinite(all digits serially), so the expression will be

I will provide several useful regular expression example with explanations in separate post.
Here are some useful tutorial
1. For learning basic with JAVA
2. To practice
3. To do experiment or test before apply , use this tool "The Regex Coach"

And for advance users , you may see this. It's a very enlarge support for JAVA.


PerfMon-JMeter Server Monitoring PlugIn

In this article we are going to see what are additional features/items added after installing jmeter plugin (Google Plugin) and what we can do with it. I will discuses about Listeners only. To know about basic of jmeter see my this page. And, the process to install plug in , see this post.

What is PerfMon plugins? It is a server monitor plugin , where we can check the health of the server from jmeter while running. The supported list is here. And we can test on web server that we are generating load on as well as remote servers where our remote agents are running.
So, by the objective of the plug in, we clearly understand there will be two component. One is, jmeter listener which will be running with jmeter. And another one will be on the server, which will send info to jmeter listener about the server.
From this download link, download server plugins(currently version 2.2.1), and install plugins to jemter.

-After download the server agent on server, and run startAgent.bat in windows or startAgent.sh in Linux server. It will open UDP/TCP server ports where jmeter will communicate.

-Now, we have to add jp@gc - PerfMon Metrics Collector  this listener in our jmeter test plan. (controller PC, where we are running jmeter)

-Select jp@gc - PerfMon Metrics Collector and click add a row
-In a row, we have to add server IP/address( where the server agent is present),Port number, the information that we want (CPU/memory) from a selectable comb and a metric parameter. So, here is a generic screenshot.

So, now we can run, our test and see the log file. I have used a sample dummy sampler(not real test) to get some viewable report.
Running screen shot1
Running screen shot2

Metric parameter:
Metric parameter are sub type or options under main metric controller. It is basically a string constructor that goes as parameter. It helps for making understandable metrics as well as helps to get specific info that we need. So, for Nine type of metric controllers we have nine metric parameters. Screenshots are here.
1. Metric parameter-CPU
2. Metric parameter-Memory
3. Metric parameter-Disk IO 
4. Metric parameter-Network IO
5. Metric parameter-Swap 
6. Metric parameter-TCP
7. Metric parameter-JMX 
8. Metric parameter-EXEC
9. Metric parameter-TAIL

Agent Tweaking :
1. We can enable or disable any port by shell command. Port 0 for disabled. We have to run our startAgent.sh/bat in following manner 
Syntax : $ ./startAgent.sh --<ProtocallPort> <port number>
Ex : To enable UDP at port 5000 : we need to run this in command prompt.
$ ./startAgent.sh --udp-port 5000
Or, to disable TCP port
$ ./startAgent.sh --tco-port 0

2. We can Auto Shutdown agent after test completion
Syntax : $ ./startAgent.sh --<ProtocallPort> <port number> --auto-shoutdown
 EX- $ ./startAgent.sh --udp-port 5000 --auto-shutdown

3. We can use system Info showing for system object while running, 
$ ./startAgent.sh --sysinfo

Notes : 
-No Need Admin privilege in the server
-Server agent needs JRE 1.4+ version. So, you need that installed in server.(or you can add jre package inside the folder. In that case, you have to edit bat file)
-We can use multiple servers. To do that , multiple server should have the agent.

Agent Extension : Following this interface kg.apc.perfmon.client.Transport , we can use this plug in for other software using, like custom performance monitor. I will make a separate post on how can we make a class on JAVA to use this plugin for getting information.


Tips-Tricks in Robbotium(Android Automation Framework)

In this article we are going to see some tecniques that can be applied in robotium to use it smartly. This is not a complete post, I will add more tips & tricks as the time goes. For basic ideas on robotium, please see my Mobile Testing section. Details , step by steps posts are there.

Sample test template :

I have created two projects for same test case where both are two ways to test same app(one with code, and another is without code). The concept is, there will be a base class maintaining all communication with android test runner. And all test classes will extend that class . All test classes will only contain test methods where as the base class will contain the main android unit test extension , setup, tear down. Here is a screenshot for more details.
Project with source code
Project without source code

How to run Robotium Test Case?
-Run Emulator or device
-Check that the device is connected using "LogCat" /"Device from eclipse or using adb commands in command prompt.  In "LogCat" or"Device", we can see device responses.
-Install debug version of app if you are using apk file to test(no source codes)
-Right click the the package you want to run in side eclipse(or the whole project)
-Select run as Android JUnit Test. If it is not in here, select Run Configuration , and use android junit test.
-Now , we see Emulator activity from log cat and test will be executed. In eclipse, we will see junit runner describing the test progress & results.

Debugging from Emulator and Eclipse : 
- Use "LogCat" and "Device" in eclipse to see app's activity. Spatially what it is doing or interacting. screenshot.
To add "LogCat" or "Device, from eclipse, go to window > show view . You will find device and logcat. If not , click other and you will see full list. Screenshot.
- Try to avoid filter in log cat and use verbose type logcat view so that you can see detail activity of emulator . Spatially test application's activity and responses from android system . If you need to monitor specific activity , then use filter.
- From device, we can do a lot of advance and interesting stuffs like
a. Method profiling : We can see a method's resource using , run time and all other information.
b. Dump View Hierarchy : We can dump view Hierarchy for automated UI tests.
c. Take screenshot : We can take screen shot any time we want. It will help for identifying bugs.
d. Capture system wide trace :

Thanks...to be continued...:) 

Android APK(No Source Code) Test in Robotium

In this article we are going to see how can we write unit tests with Robotium Framework when we do not have the source code. That means we are going to test an application using its installer apk file.

Please read my previous post for robotium setup basic

In this post we will only see the test Class used for testing stand alone APK (not with code). So, please see my this post for code start basic. I will mainly focus on test class here.

To test a stand alone application, we need to have debug version of APK file. To convert an apk to debug version, see my this post.

Like as following this post, we will not add dependency project. In stead of this step, we have to install the application in the emulator.
And add android test project in eclipse with robotium library. Now add a class(APKTestCase).

So, after adding a test class( my example class name is APKTestCase ), we will extend from ActivityInstrumentationTestCase2 without parameter(as we don't have codes, so we do not know)
So, my test class will be.

public class APKTestCase extends ActivityInstrumentationTestCase2{
}And the constructor 
public APKTestCase() throws ClassNotFoundException{

This constructor needs class name as parameter. So, how to get the class name ? We have to be very careful here. While testing with robotium, only activity name is not enough , we have to specify the class name. The screen shot is showing how to define the class name with pacakage name. This is called full class name.
So, if we do not have source code, so how can we get the class name. This is a bit tricky.
-Run eclipse and open an emulator.
-Run your installed application(if not installed, then install it first using ADB commands, use my this post)
-From eclipse, see log cat messages in no filtered mode.(usually in verbose)
After starting activity, system process shows in text section.
you can see the message like this <activity neme>/.<Class Name> : +<time> . See the screen shot
Here is the screen shot

So our full class name will be. com.example.android.notepad.NotesList
Now, we have to convert this class name into class by an identifier. To do this we will use a spatial class named as Class to identify. We will use it as static as it is accessing static class name. So for this part , our code will be
private static final String Launcher_Activity_ClassName  = "com.example.android.notepad.NotesList";   
private static Class launcherActivityClass;
            launcherActivityClass = Class.forName(Launcher_Activity_ClassName);
        }catch (ClassNotFoundException ex) {
            throw new RuntimeException(ex);

And the constructor will become
public APKTestCase() throws ClassNotFoundException


Like as standard test case, we need to override setup & tear down. So, finally class will be

public class APKTestCase extends ActivityInstrumentationTestCase2{  
    private Solo mySolo;
    private static final String Launcher_Activity_ClassName  = "com.example.android.notepad.NotesList";  
    private static Class launcherActivityClass;
            launcherActivityClass = Class.forName(Launcher_Activity_ClassName);
        }catch (ClassNotFoundException ex) {
            throw new RuntimeException(ex);

    public APKTestCase() throws ClassNotFoundException
    public void setUp() throws Exception
        mySolo = new Solo(getInstrumentation(), getActivity());
    public void tearDown() throws Exception


Now we are ready for writing test case. We can add any sample test case. You may see a sample project link in here.


How to write test case in Robotium with source code

In this article we are going to see how can we write unit test case for a android application. We will discuses about a small demo example with detail explanation. The objective of the post is to have clear idea on how to get started with robotium uni test case writing for a invoice.

First we need a project which we will test.We can download a notepad project which is used for learning purpose with robotium from here. We will get two projects there, one project containing application code base another one is robotium test case. Here is the link of the same project from my drive. We will use the code base.We will add the code base to eclipse. To do that, just import the downloaded code base. In eclipse, From, File >Import > Select Archive file under General > show the downloaded ZIP file and finish the process.

To setup robotium environment, see my this post. So, we will start from adding a android test project.[ As all previous steps are shown in my post]

After adding a test project, we get a package. We can edit the package names from android manifest in case we want to change the package name(for test project). Here is the screen shot.

Now, we have to add target package in current application manifest. To do that,
-Click AndroidManifest.xml
-Click Instruments tab
-Click the android.test.InstrumentationTestRunner
-From Target Package, provide the target package name.
(from this example : com.example.android.notepad)
Now save this. We can do the same thing by adding this
 <instrumentation android:targetPackage="com.example.android.notepad" />
directly to AndroidManifest.xml

So,after renaming I have made my project as com.notepad.test this package and target package is com.example.android.notepad. Screen shot

Now, we need to include the main project(which we will test) as dependent of our test project. So that, when we will run the test case, the application apk will be installed in the test environment.  To do that.
-Right click on test project and select properties
-Click Java build path
-Select project tab and click add
-Select NotePad project and press ok

So Now, Add a class under a package of src folder .
After a class creation , extend the class from base class of " ActivityInstrumentationTestCase2 ".
So what is this "ActivityInstrumentationTestCase2" ?
This is a unit test case template(base class) that provides all controlling over a app activity for unit testing. For more detail you may see the declaration. We need to override setup and tear down functions. For those who are new at unit testing, please see junit or nunit unit test case basic. I have an article on nunit.

When extending , we will use our target activity class name(assuming that we have source code) as parameter. In here, we have to be careful as wrong class name will cause error while execution. That means, we have to choose the functions to test. From this example, we are going to test adding/deleting/editing notes. So, from code we have check which class is active(actively responsible) for those functions. in here Notelist class is responsible for showing all notes and it creates or handle all types of note operating events. So, we need this class as parameter while extending "ActivityInstrumentationTestCase2". My testing class is TestNoteAdministration So the code will be
public class TestNoteAdministration extends ActivityInstrumentationTestCase2<NotesList>{

-And add the constructor.
public TestNoteAdministration() {

This is spatial constructor for test case. This actually calls super class's constructor with test class activity as parameter. When the test run, this activity get initiated.

-Add a Solo Class's object. This is our main robotium functional test operator object. We have to initialize in setup.
private Solo mySolo;

-Now we need to override Setup and Teardown Methods.

 public void setUp() throws Exception {
        mySolo= new Solo(getInstrumentation(), getActivity());

In here , getInstrumentation used for initializing settings from manifest and getActivity initiates the test activity.

 public void tearDown() throws Exception {

This will close all the operating activity.

Now we are ready to write a test case. Like as a Junit test method, we will write the methods. Example -
public void AddNoteTest() throws Exception {

A sample test case with the test project will be found in this link. Like as importing note pad project, you may import and see the test code.

- We will see how not to use parameter in separate post. And, I will try to provide some sample code in separate post that I have worked with different projects.
- Before running, you may fix android properties. To do that , right click the project > android tools > Fix Project Properties.


Setting up Robotium in Eclipse

In this article, we are going to see how to install Robotium In Eclipse.We will use robotium for android application testing using code base or stand alone apk. I will provide separate post for how to code. Let's make environment ready.

Step 1. Install android SDK. See my this post.

Step 2. Download Robotium : Download latest robotium and java doc from this link . As we will perform detail analysis on robotium, you may download the source from this github link. I use zipped source.

Step 3 : Adding Robotium to the project : In this step, we will see how to create robotium project. Robotium is an extended Android JUnit package. So,
-Run Eclipse
-From file > New >Project
-Select Android Test Project [Under Android Folder, with Junit logo]
 Screenshot Link 
-Add a project name(ex-ShantonuTestRobo)
-Select This Project(not under any existing project)
-Select an android version( I choose 4.1.2) and click Finish. So we will find like this.

Now , we have to add downloaded library. For this, right click on project and select properties.
 -From property window, select Java Build Path.
-Click Libraries
-Click Add external JARs and show the downloaded path of robotium solo jar.
-You may add Java Doc and downloaded source files. It will help advance unit testers.
Screenshot for JavaDoc
Screenshot for Source Adding.

Step 4 : Add a class under package of the source file . We will see two ways to write robotium unit test cases. I will post separately with codes.

Now we are ready environment to start coding with Robotium for Unit testing.

Note : To prevent java.lang.NoClassDefFoundError export the robotium library. To export any library go to project properties > java build path > order and export . Now select robotium and press ok. 


Setting up Android SDK in Eclipse

In this article we are going to see how to set up android SDK with eclipse.We are doing this for mainly two purposes.
1. When we try Robotium to for android unit testing, we need that
2. When we do code review or android junit unit test case writing
Beside that, for development we need to set up android sdk.

Step 1 : Download Eclipse 
We need to download Eclipse from this link.  I use eclipse classic, yo may use j2ee or java developer edition.

Step 2 : Download Java
We need to have JAVA(sdk) in the PC. I use x86 SDK, you may also try x64 SE editions. Download link. To check if java installed in you pc or not, you may check C:\Program Files\Java or C:\Program Files (x86)\Java folder having java SDK(for windows).
From command prompt if we use following command , we can also know , does the PC have Java or not.
java -version

Step 3 : Download Android SDK 
WE need to download android SDK from this link. 
For windows
For Linux

Step 4 : Install SDK 
Install the SDK that we have just downloaded. 

Step 5 : Add to OS path (as system variable)
We need to add sdk path as environment variable. To add this in windows 7/8
Right click My computer > properties >advance system settings , then click Environment variable . Select Path > click Edit then add the path at the end with semi colon(;) . Now , press ok.
To check, it is working or not, open command prompt and write
adb devices
(please connect an android device to see the device ID). If you see, no command found, that means not done. If command is found , it will show device ID or blank (if no device)

Step 6 : Install Eclipse Android Plugins : We need to install android plugins in eclipse.
-To do that, open eclipse
-Click Help from Menu
-Click Install New Software
-Click Add button
-In name (name of the plug in), write ADT plugins
-In Location, write : https://dl-ssl.google.com/android/eclipse/
-Press ok, you will see the list after loading from Internet. Select developer tools and press next, next up to finishing process to install(with agree the agreements). Wait for the plugins to be downloaded. Then close eclipse and run again(Eclipse may prompt for restart it self, you can restart).

Step 7 : Locate android and update sdk : 
After installation, we will see this two icon in the tool bar as well as menu under Window.

When we click window > Preferences , we will see android.
Now, click window > Android SDK manager to get latest SDK tools
And, you may Click window > Android Virtual Device Manager to make your desired Virtual device.

So, now we are done with Android SDK setup. We may develop or test android codes.

Note : Now a days, android sdk can be downloaded as bundle(Android SDK+eclipse).