How to install docker in Ubuntu 14

In this post we are going to learn about how to install docker in ubuntu 14.04 LTS. We need docker for deploying application or monitoring system. I will write a series to use docker as Jmeter server, so this post will help.

Before getting stared , I would like to say all of the items here are mainly from docker documentation. And those who want to know what process will be for windows or Mac, here is the basic difference which we need to know before installation.



That means, you need additionally a VM(usually virtual box) on top of docker if it is not Linux. The virtual box actually work as boot2Docker.

We sill follow the things needed for Linux only. And all of the command here are in command line.

First, docker needs minimum Kernel version 3.10.

To check kernel version :
uname -r 




As both cases, they are 3.10+, we are set to go.

There are two ways to install , I am including both ways.

Primary Installation Type (Tested in Ubuntu 14.04 LTS) : Preferable method for error free installation. 

Check APT work with HTTP :
If you dont have https enable for APT, just download certificate
sudo apt-get install apt-transport-https ca-certificates

Add public key for server to download :
sudo apt-key adv --keyserver hkp://p80.pool.sks-keyservers.net:80 --recv-keys 58118E89F3A912897C070ADBF76221572C52609D

Adding docker repository source : Go to this folder (/etc/apt/sources.list.d) and find a file docker.list.
If you find, open this as root and delete any entry if you have.
If you don't have this file, create this file.

sudo touch  docker.list

Now , edit (sudo gedit docker.list) this file


and add this following repository name & save it
deb https://apt.dockerproject.org/repo ubuntu-trusty main

Optional : If you want to use multiple architecture(x64 & x86) , you may specify  like this

deb [arch=amd64,i386] https://apt.dockerproject.org/repo ubuntu-trusty main

This will enable x86 & x64. I am only using x64.

Note : If you go that repository you will see following folders
debian-jessie/                             
debian-stretch/                            
debian-wheezy/                             
ubuntu-precise/                            
ubuntu-trusty/                             
ubuntu-utopic/                             
ubuntu-vivid/                             
ubuntu-wily/

If you Ubuntu version is not 14 (trusty), then you may use other destination. On this part is different then other Ubuntu versions.

Now, Update APT sudo apt-get update

-> verify APT is pulling from right repository
sudo apt-cache policy docker-engine 



-> To check, if APT is not getting from old repo,
sudo apt-get purge lxc-docker




Get Linux Image Extra kernel, To do that
sudo apt-get install linux-image-extra-$(uname -r)

It will update the current extra image kernel. If you are in latest , you dont need. to run this. You might see something like this.



We need apparmor, for application security. So,  if you dont have apparmor, you can install
sudo apt-get install apparmor

By default apparmor used to present in ubuntu 14. So, you may see like this.


We need both generic and required kernel. So, Update APT and
sudo apt-get install linux-image-generic-lts-trusty

Press Y for new installation prompt and your PC is ready for installing

Reboot PC .

Now, finally we are in installation step!

sudo apt-get install docker-engine

Press Y 2 times when it asked for permission.

Now, Docker is installed. Let's test . Type
sudo docker run hello-world



Optional Steps :

If you want to run docker as service , run this as service
sudo service docker start

If you want to add an user for docker , (I am putting my name)
sudo usermod -aG docker shantonu

Reboot PC(or log out) . Now you can use docker without root rights. If you try

docker run hello-world 




We are done.

Alternative Installation Type:(tested in Ubuntu Mate 14, based on LTS)

Step 1 : Install CURL down-loader if you don't have it. (usually no need as it comes with Ubuntu, but you can update APT before going forward )

Step 2 : Download & install using CURL by this command

curl -fsSL https://get.docker.com/ | sh

It will prompt for root credential , you need to provide that.

Optional step : Adding an user (i am adding myself, so that I do not need root rights each time I use docker) :

sudo usermod -aG docker shantonu

And now, try hello world : (add sudo if you haven't add user)

docker run hello-world



We are done.

So, this following part is necessary to use Docker . Actually , it will allow to use docker complete for development. These are mainly permission or admin related things. 

Using swap memory if you have less Ram :

Edit grub as root etc/default/grub
sudo gedit /etc/default/grub

And add this line
GRUB_CMDLINE_LINUX="cgroup_enable=memory swapaccount=1"

Now , save & update grub
sudo update-grub


UFW firewall entry :

To know the status of you PC of UFW, type this (you may avoid verbose)
sudo ufw status verbose

I am not using after by default , so inactive.


if you are using firewall(you should) , Change DEFAULT_FORWARD_POLICY,
"DROP" to  "ACCEPT"
sudo gedit /etc/default/ufw



Save & close file. And, now Reload UFT
sudo ufw reload


Add Docker port for incoming container request.
sudo ufw allow 2375/tcp



So, we are done adding in firewall. Now,

Configure DNS : 

We need to add DNS entry for our Docker outgoing calls. Usually /etc/resolv.conf shortcut used to have local host entry.
gedit /etc/resolv.conf


So, Docker access this & when try to use, it cant as it is used by our Ubuntu. So, Docker try using default Google Public DNS servers [8.8.8.8 8.8.4.4].

We need to make this DNS entry to our Local host IP address. So, know your IP address by ifconfig and use that. In my case it is 192.168.0.19. So,

sudo gedit /etc/default/docker

 

And add your local address. Now, as you can see we can add multiple DNS address, so, if your PC connected to multiple network (wifi, lan etc) , you should add those address so that each time you change network, you dont need to add dns.

Save & close file.  And, restart docker

sudo restart docker

Done, we are ready to use docker in Ubuntu.

Note : in all command, I use either gedit editor , you can use your favorite one like nano.


----- Thanks.. :)

Live Performance Dashboard: Jmeter-Grafana-influxdb in windows 8x64

This is continuing post of Live Performance Dashboard: Jmeter-Grafana-influxdb only this time with windows(as I have promised). I would suggest, please visit that post to continue.

In this example, I am going to setup influxdb & Grafana in windows 8x64 PC in VM. You can use real PC/AWS. And like as before, I will run jmeter in my local PC to test keeping monitoring system isolated.
My Local Address (in this whole example) : 172.16.244.135

[As influxdb/grafana will be accessed from outer network, make sure you put in outbound firewall rules so that others can access this]

Step 1 Installing InfluxDB : 

Download Influxdb from this link (nightly build, updated up to now). It is standalone zip. If you extract, you will see something like this.(i renamed in to  influxdb-0.11.0)



Now, open command prompt and get default configuration (in bin all exe present)
(like as previous post)  

influxd.exe config

So, you will get default configuration. Copy this to a notepad and save as influxdb.conf. [to make life easy, i save my own influxdb.conf in bin , so default one in etc, my one in bin]

Now, if you run this you will see error messages due to local host address resolver is not working for windows. Github discussion.

Easy to solve , add local host with port number. I change the under
[meta]
  bind-address = "localhost:8088"
  http-bind-address = "localhost:8091"

That's all to resolve the error.

Now, add graphite configuration at our influxdb.conf.

[[graphite]]
enabled = true
bind-address = ":2003"
database = "jmeter"

So, open command line , goto bin directory (where influxd.exe & influxdb.conf are present )

influxd.exe -config influxdb.conf



Now,  if we goto browser and see http://172.16.244.135:8083 , we will be able to see home page. Like as before , create a table 'Jmeter'.



And , if we see show database ,we should be able to see DBs.
So, our influxDB setup is done.

My Windows influxdb.conf

Step 2 : Installing Grafana :

Download grafana standard from this link. (I am using 2.6, latest)
Extract it in your suitable location. (i also renamed in grafana-2.6.0 to keep things simple)

Now , Open command line
-> goto bin directory under grafana-2.6.0 and run grafana-server.exe
You will see server running.

Goto browser http://172.16.244.135:3000/
And you should able to see grafana home page for login prompt. (use admin, admin to login) 

Like as previous post, configure influxDB settings into grafana.
 

 Like as previous post, make a dashboard. (I am using same request count) 
 

So, we are done setup, now lets test it. 

Jmeter Tests :
Like as previous post , I am using same test case just changed the URL to 172.16.244.135
 

Now, if we run jmeter 
-> goto http://172.16.244.135:8083
-> select Jmeter from right upper corner.
->from query template SHOW MEASUREMENTS
we will see like this.

 

This means DB is getting Data. 

Now if we goto grafana dashboard (our dashboard name Jmeter  Live )
we should be see graph for request sending


So, Like as Ubuntu , we have completed our setup a live Jmeter result monitoring system with Grafana & InfluxDB.

Note : As I am installing in my PC, ignore my user name, for your PC , it will be your's. So, don't just copy past my configurations. 

----- Thanks.. :)

Generics in java : How read and write Generics?

In this article we are going know about Java generics, basic idea, how to implement, how to use with different scenarios.
This post is intend for QAs and novice Java developers who need to read code frequently and most of use it. I will clear on concepts as well try to give some good examples. I will try to make statements / interpretation for each generic expression.

What is generics ?
Avoiding bookish statement, Generics is a technique which allowed programmers implement features/functions not specific to particular data types rather more general ways.
So, the implemented functionality become general use and can be applied beyond of specific data types. Good example, think of sorting (we know sorting from university), if we apply sorting only for integers then other data types will need separate implementation of sorting. And, if sorting is applied in such way so that every type of data type may use this, then it can be used by all . So this is generic implementation of sorting.

Now , start with declaration. To make things manageable, I am making in 3 parts.

Class/ Interface Level Generics : (Typed) 

In collection classes or other, we will see some thing like this
List<String> = It is a list that contains String type Data,

Now, if we want to make our own type.
List<OurOwnType> = It is list (a collection) of OurOwnType.

Now , in language as well as community, this is applied like List<T> where T is OurOwnType. (So, keep in mind , when ever you see T or any other class name, it is just our own type that we define, we will also see , undefined /wildcards in the end)

So, how to declare a class that will provide common functionality for all other classes? just adding class parameter in class declaration level .

class House<T>{
}
= it means, House is a class that can contains any type.

if we want to have multiple generic type , it is simply
class House<T,H,J,K..., > {
}

You may have seen multiple generic types for collection like
HashTable<k,v> ,

Hashtable< String, Integer>.

Note : when using the generic type, if you don't use parameter then it is called Raw type.
public House = new House(); 

You might be confused, why only T. Well, that is just a meaningful name. You can use your own. In community, you might see following as best practices.

E - Element (any)
K - Key
N - Number
T - Type
V - Value
S,U,V etc.

How compiler handles ?
Very important note here. Java compiler manage this generic notifier T as object. That means, in compiler, it assumes this <T extends Object> .

Unless we tell compiler specifically.

Note : For all preemptive data types , java compiler automatically boxing  & un-boxing when we use them. (using int when declared as Integer)

Now, lets define this following two terms that we will use next

1. Sub type = When a class is implemented interface(s) or extended a class, then that class is call subtype of those interfaces or super class. Example
class A extends B, implements C, D {} =  A will be called subtype of B as well as C & D.
It is also part of SOLID , the L = Liskov Substitution Principle

2. Super type : Just inverse of subtype. A subtype's parent or definer is called super type. That means, the class which is extended or interfaces which are implemented are called super type. From the same example, B, C & D are super type of A.

Java Generic support super typing or sub typing. It means, we can define our generic type for their sub & super types like this.
Super type can be used only in method level . So, here is subtype example.

class House<T extends AnyOtherDeclearedClass> = It means, House Class takes T which should be subtype of AnyOtherDeclearedClass.

BTW, This time, there is an interesting thing happens. Compiler treat this differently than previously T . This time compiler handle this T object as AnyOtherDeclearedClass instance. So, we are bounding compiler for treating generic T.
And This AnyOtherDeclearedClass can't be another generic type, should be declared. 

Now it comes intersection types. By the name you can understand it is like Set theory of math. When we want to define our generic class T is a subtype of multiple types, then we use intersection. To define intersection , we simply use "&" operator or comma "," . Example - Let assume we have InterfaceA, superClass, InterfaceB. So, we can use

<T extends superClass & InterfaceA & InterfaceB> or
<T extends superClass,  InterfaceA,  InterfaceB>

Important Note : Among sub classes, the expandable class should be at the beginning.  If we put super class after interface, compiler will throw error as class get more priority than interface. (tightly coupled)

Now, how compiler will treat this T. It's simple, following math of set theory. T represents superClass,  InterfaceA,  InterfaceB as common part.
So, you can access all methods/fields from superClass,  InterfaceA,  InterfaceB.

Notes :
1. We cant use super type in class level like this class House<T super AnyOtherDeclearedClass>
2. Super type can only be used with wildcards.

In summary :
<T extends A>= T class is subtype of A class
<T extends A & B > =  T is a subtype of A and B
<T extends A , B > =  T is a subtype of A and B


Interface : 
Like as Class, we can use generics in interface. Here is a declaration
public interface InterfaceAB<T> extends Collection<T> {
}
= InterfaceAB is an interface that takes extends Collection of generic type T .
or even simple one
public interface GenericInterface<T> {
}
= GenericInterface is generic type interface.


Note : In type level , (class/interface) we can not use wildcards. (what is it, see detail in method section)

Method level :
Like as class level (type) , we can declare method level generic parameter.
public void doSome(T aT ){
}
= doSome method takes a generic parameter T. Here is an important note, this T should be defined in class level. That means, in a class House<T> , we can define T as parameter.

So, what happens when class does not take generic parameter but we need to implement method with generics. We simply need to define T in the method declaration.

Method level declaration : T should be defined before return type of the method. for doSome method, it should be like this
public<T> void doSome(T aT ){
}
= doSome method takes a T.

So, declaration in the beginning. Let' see if we have multiple type parameter and one of them used in return.
public<T,K> K doSome(T aT, K aK){       
return aK;
}
= It means, doSome takes T & K type parameter and returns K type.

Now, when we are using generics in method, it follows all rules from Sub Typing. So, it can have multiple subtype as well as inner subtypes. Let see some subtype example.

public <T extends AbastractA> void doATask(T t){   
......}
= doATask method takes a generic type T which is extended from AbastractA (it can be class or interface, in my code it is class)

public <T extends InterfaceA> T doSomeWork(List<T> items){
=In here , this method takes a list of generic type which should implement InterfaceA.

Now as we have seen in class level, the keyword extends is used for both interface & Class subtypes.

Super type can be used with wildcard. So what is wildcard? It is very simple in Java. Just using ? instead of T. So, why we use this. Actually , wildcard has separate meaning of usages. When we use they type it self , we define the type(like as T), so, when we actually use wildcard , we wont be able to use the type. For example, if we make a function that actually shows its collection no matter what type of item inside collection, we can use Collection<?>

public void printInUI(Collection<?> items){...}

Let see some example and try to understand : Let assume we need to get Max value from a collection. so, first thing in mind.
public <T> T getMax(Collection<T> items){
....} 

But, the generic type T should be able to compare to it self, so that we can actually get the max. So, now the following will be our mind.

public <T extends Comparable<T>> T getMax(Collection<T> items){
...}
= This means, T will be treated as comparable to java compiler. So, it will loose its authenticity (type will be changed, so we will loose functionality.) . To overcome this, we will intersect this with Object  (intersection type)

public <T extends Object & Comparable<? super T>> T getMax(Collection<T> items){
...}
= As java compiler peak first intersection type, so it will treated as object.

Now, If we handling with legacy code with not generic type and this method needed to be used with classes from legacy code, wildcard can be used to do that. It will make this method more compatible to different type. Let's think from a user prospective.

Initially, we wanted to use getMax method to take a collection of T.
And then we wanted comparable collection of T.
And now we want this T to be flexible. And, inside this getMax method we will do
1. compare with collection of items(working with references)
2. Return one Item.

So, if we apply wildcard here, we get something like this.
public <T extends Object & Comparable<? extends T>> T getMax(Collection<Comparable<? extends T>> items){
...} 
=This means, any type that extends T can be accepted to use this.

Now, when we are using legacy code, most of the time we have to use excising type or at most we can extend it. So, how can we return such object that actually be more generic. Here comes super. If we return by any type which is a super type of our generic type T, so, that any existing can actually call our method. So, we will get this.

public <T extends Object & Comparable<? super T>> T getMax(Collection<Comparable<? extends T>> items){
...}

=  you can see, I am returning T = <T extends Object & Comparable<? super T>> =  means, this function will return an object which was actually a subtype of legacy class(super of T) that we don't even know.

This is one of the most useful example I found which may looks weired but very very useful for using.

[this example is widely used for understanding generics, you may check VJug session of generic to have more expert opinion, I love VJug ]

Why T or ? : This is one of the most commonly asked or found confusing. Actually, when you are defining ? (not T) that means you can not use the generic class or its referenced object. That means, there is no implementation with any function depends on that class. But, when you use T, you can reference instance of T and use it in you function. So, when to use what? simple, if you are writing any function that works on your generic type, you need to use T, if not , simply ? (wildcards).
Perfect example , comparable object collection. If you need to short an list of object which are comparable to make the list short you may use like this.

public List<? extends Comparable<?>> shortThis(List<? extends Comparable<?>> items){
...}
= this will take any List of object that is comparable(subtype of comparable)

And again you can keep this for you own generic type with interface

public <T extends InterfaceA > List<? extends Comparable<? extends T>> shortThis(List<? extends Comparable<? extends T>> items){
.... }
= Your generic type T, extends, is the super type of ? do the same thing. You just include restriction, where previously only using ? refers to simple object by compiler.

When sub type when super type :  I like the get/put principle describe in -> Java Generics and Collections , By: Maurice Naftalin; Philip Wadler

" The Get and Put Principle: use an extends wildcard when you only get values out of a structure, use a super wildcard when you only put values into a structure, and don’t use a wildcard when you both get and put"

The concept is simple,

When you need to use only type item references(the object items), use extends 

When you need to to return item reference(as super type) use super. 

Wildcards(?) has lots of limitation, I am not going to detail on that as it is not the target of the post.

Constructor level :

Declaring inside constructor is simple, it will take the type as parameters.
class MyClass<T, U> {
private T first;  private U second;// final is best practice
public MyClass(T first, U second) // get info from Class declaration
{ 
this.first=first; 
this.second=second; 
}
}

-> If we need new item here, we can define type same like as method.  Let's add Key

public <K> MyClass(K key, T first, U second) {
        this.first = first;
        this.second = second;
    }
-> It means , K is new type can be used as parameter , but remember , as it is not declared in class level, we have use in the scope of constructor.

All rules of method are applied here (as compiler treat constructor as spatial type of method)


Rules for static Items :
As, static items are context of class(shared by all), we cant declare member variable as static. 


Usages of Generics in Test automation : Being an automation engineer for more than 7 years, I use generics in very limited scopes, mainly in testing and framework building. I am using  in following scenarios

1. Making a generic test class to test all web page common tests like page static validation, style sheet validation, resolution checks for compatibility etc.
2. In selenium with page object principle, i use generics for common events, like click, right click, mouse scroll , mouse hover, drag & drop. It helps me to have quick development as well as no repeat of code.
3. Browser Behaviors validation for compatibility testings(common for all browsers)
4. Error Handling in generic way with screen shots + log + action/steps+ exceptions messages.
5. Utility classes having common functionality.


Please comment if you have any question.
Thanks .. :)