Hi Shantonu! thanks for good and detailed guide on this. However I failed to found how to see in influxDB transactions response times without percentiles. Is it ever possible?
in jmeter response time comes in Max,Min, or any %tile.. In influx db , data stored in all of those fields. So, just select proper parameter for the graph. Check details here on which items are stored. http://jmeter.apache.org/usermanual/realtime-results.html Select among them.
----------- if you want to dig inside------------- If you check what jmeter sends , see this https://github.com/apache/jmeter/blob/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListener.java) Method name ( static void sendToListener) And , in sample result(https://github.com/apache/jmeter/blob/trunk/src/core/org/apache/jmeter/samplers/SampleResult.java) , it has every thing.
So, practically it send everything as results , you need to see your query in grafana. And , if you want some more info , you need to make a plugins that extends backend listener and use this predefined statistics. https://github.com/apache/jmeter/blob/trunk/src/components/org/apache/jmeter/visualizers/backend/SamplerMetric.java https://github.com/apache/jmeter/blob/trunk/src/components/org/apache/jmeter/visualizers/backend/UserMetric.java
Hey Shantanu, this is simply great, relatively new to this area. Was looking for this kind of tool. Just wanted to confirm few things
1- Does Influxdb picks up the data from Webserver to track the traffic/load real time? 2- Not sure if we can configure this for realtime servers with no UI
1. influxdb doesn't pick, we need to send. I am sending from jmeter backend listener to the influxdb host's 2003 port. This port is opened for graphite format input. (see the configuration) it is possible to send /receive data without any formating/plugins. I use 8086 port to actually read data from influxdb to show in grafana. you can send data using that. in fact, infludb supports multiple clustered DB , so you can have distributed DB for data collection which scales.
2. Yes, this is full non UI, see my docker example http://shantonusarker.blogspot.com/2016/04/jmeter-grafana-influxdb-docker-live-monitoring.html and I test with jmeter which can be run in non gui or even from jenkins jobs(maven, ant, sh)
We are trying to disply active threads with multiple jmx script that are controlled by Ant Build.properties But Grafana is just displaying the Max value of active threads amongst the jmx scripts as running Users. How do i handle this? I want to display all running VUsers of all the jmx scripts
Suppose i have 3 scripts 1. 4 Users 2. 5 Users 3. 6 users
now i just see 6 Users as running with test.startedT
I have followed all the steps mentioned above. These are the Errors I am facing: Jmeter: #1 java.lang.NoClassDefFoundError: javafx/embed/swing/JFXPanel 2018-02-14 15:51:11,234 INFO o.a.j.s.FileServer: Set new base='/home/dinesh/apache-jmeter-4.0/bin/Result' 2018-02-14 15:52:15,516 INFO o.a.j.e.StandardJMeterEngine: Running the test! 2018-02-14 15:52:15,517 INFO o.a.j.s.SampleEvent: List of sample_variables: [] 2018-02-14 15:52:15,517 INFO o.a.j.s.SampleEvent: List of sample_variables: [] 2018-02-14 15:52:15,573 INFO o.a.j.v.b.BackendListener: Backend Listener: Starting worker with class: class org.apache.jmeter.visualizers.backend.graphite.GraphiteBackendListenerClient and queue capacity: 5000 2018-02-14 15:52:15,587 INFO o.a.j.v.b.BackendListener: Backend Listener: Started worker with class: class org.apache.jmeter.visualizers.backend.graphite.GraphiteBackendListenerClient 2018-02-14 15:52:15,665 INFO o.a.j.v.b.g.TextGraphiteMetricsSender: Created TextGraphiteMetricsSender with host: localhost, port: 2003, prefix: jmeter. 2018-02-14 15:52:15,673 INFO o.a.j.g.u.JMeterMenuBar: setRunning(true, *local*) 2018-02-14 15:52:15,992 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group 2018-02-14 15:52:15,992 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Thread Group. 2018-02-14 15:52:15,992 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error 2018-02-14 15:52:15,993 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 perThread=1000.0 delayedStart=false 2018-02-14 15:52:16,011 INFO o.a.j.t.ThreadGroup: Started thread group number 1 2018-02-14 15:52:16,011 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started 2018-02-14 15:52:16,031 INFO o.a.j.t.JMeterThread: Thread started: Thread Group 1-1 2018-02-14 15:52:16,100 INFO o.a.j.p.h.s.HTTPHCAbstractImpl: Local host = dinesh-VirtualBox 2018-02-14 15:52:16,153 INFO o.a.j.p.h.s.HTTPHC4Impl: HTTP request retry count = 0 2018-02-14 15:52:16,452 INFO o.a.j.p.h.s.LazySchemeSocketFactory: Setting up HTTPS TrustAll Socket Factory 2018-02-14 15:52:16,462 INFO o.a.j.u.JsseSSLManager: Using default SSL protocol: TLS 2018-02-14 15:52:16,462 INFO o.a.j.u.JsseSSLManager: SSL session context: per-thread 2018-02-14 15:52:16,535 INFO o.a.j.u.SSLManager: JmeterKeyStore Location: type JKS 2018-02-14 15:52:16,537 INFO o.a.j.u.SSLManager: KeyStore created OK 2018-02-14 15:52:16,537 WARN o.a.j.u.SSLManager: Keystore file not found, loading empty keystore 2018-02-14 15:52:16,713 ERROR o.a.j.v.b.g.TextGraphiteMetricsSender: Error writing to Graphite: Connection refused (Connection refused) 2018-02-14 15:52:17,666 ERROR o.a.j.v.b.g.TextGraphiteMetricsSender: Error writing to Graphite: Connection refused (Connection refused) 2018-02-14 15:52:18,611 INFO o.a.j.t.JMeterThread: Thread is done: Thread Group 1-1 2018-02-14 15:52:18,611 INFO o.a.j.t.JMeterThread: Thread finished: Thread Group 1-1 2018-02-14 15:52:18,629 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test 2018-02-14 15:52:18,629 INFO o.a.j.v.b.BackendListener: Worker ended 2018-02-14 15:52:18,631 ERROR o.a.j.v.b.g.TextGraphiteMetricsSender: Error writing to Graphite: Connection refused (Connection refused) 2018-02-14 15:52:18,636 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, *local*)
InfluxDB: Created jmeter database and using the database. Typer SHOW MEASUREMETS. It is just saying "Success! (no results to display)"
Hi Shantonu, I am running test using jmeter on two machines inorder to distribute the load properly. I am sable configure influxdb and Grafana in two machines and fetch the report for the scripts that are run in that machine. When I try to write data to influxdb from machine 1 to machine 2 I am getting the below error.
2018/09/17 14:55:32 INFO - jmeter.engine.StandardJMeterEngine: Running the test! 2018/09/17 14:55:32 INFO - jmeter.samplers.SampleEvent: List of sample_variables: [] 2018/09/17 14:55:32 INFO - jmeter.visualizers.backend.BackendListener: Backend Listener:Starting worker with class:class rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient and queue capacity:5000 2018/09/17 14:55:32 INFO - jmeter.visualizers.backend.BackendListener: Backend Listener: Started worker with class:class rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient 2018/09/17 14:55:42 ERROR - jmeter.JMeter: Uncaught exception: java.lang.IllegalStateException: Failed calling setupTest at org.apache.jmeter.visualizers.backend.BackendListener.testStarted(BackendListener.java:341) at org.apache.jmeter.visualizers.backend.BackendListener.testStarted(BackendListener.java:297) at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:204) at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:325) at java.lang.Thread.run(Unknown Source) Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: connect timed out at com.google.common.base.Throwables.propagate(Throwables.java:160) at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:270) at org.influxdb.impl.InfluxDBImpl.describeDatabases(InfluxDBImpl.java:246) at rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient.createDatabaseIfNotExistent(JMeterInfluxDBBackendListenerClient.java:249) at rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient.setupInfluxClient(JMeterInfluxDBBackendListenerClient.java:207) at rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient.setupTest(JMeterInfluxDBBackendListenerClient.java:142) at org.apache.jmeter.visualizers.backend.BackendListener.testStarted(BackendListener.java:339) ... 4 more Caused by: java.net.SocketTimeoutException: connect timed out
My Expected Result: Run test in two machines and the scripts should always report to influxdb located in one machine so that I collate the results easily.
After running Jmeter scripts ,Measurements showing:- Success! (no results to display)
ReplyDeleteSelect DB from right upper corner and do it again.
Deletemake sure your influxdb has graphite plugins configured.
Hi Shantonu!
ReplyDeletethanks for good and detailed guide on this.
However I failed to found how to see in influxDB transactions response times without percentiles. Is it ever possible?
Thanks,
Artem
Hi Artem,
Deletein jmeter response time comes in Max,Min, or any %tile.. In influx db , data stored in all of those fields. So, just select proper parameter for the graph.
Check details here on which items are stored. http://jmeter.apache.org/usermanual/realtime-results.html
Select among them.
----------- if you want to dig inside-------------
If you check what jmeter sends , see this https://github.com/apache/jmeter/blob/trunk/src/components/org/apache/jmeter/visualizers/backend/BackendListener.java) Method name ( static void sendToListener)
And , in sample result(https://github.com/apache/jmeter/blob/trunk/src/core/org/apache/jmeter/samplers/SampleResult.java) , it has every thing.
So, practically it send everything as results , you need to see your query in grafana.
And , if you want some more info , you need to make a plugins that extends backend listener and use this predefined statistics.
https://github.com/apache/jmeter/blob/trunk/src/components/org/apache/jmeter/visualizers/backend/SamplerMetric.java
https://github.com/apache/jmeter/blob/trunk/src/components/org/apache/jmeter/visualizers/backend/UserMetric.java
thanks
Hey Shantanu, this is simply great, relatively new to this area. Was looking for this kind of tool. Just wanted to confirm few things
ReplyDelete1- Does Influxdb picks up the data from Webserver to track the traffic/load real time?
2- Not sure if we can configure this for realtime servers with no UI
thanks
Delete1. influxdb doesn't pick, we need to send. I am sending from jmeter backend listener to the influxdb host's 2003 port. This port is opened for graphite format input. (see the configuration)
it is possible to send /receive data without any formating/plugins. I use 8086 port to actually read data from influxdb to show in grafana. you can send data using that.
in fact, infludb supports multiple clustered DB , so you can have distributed DB for data collection which scales.
2. Yes, this is full non UI, see my docker example http://shantonusarker.blogspot.com/2016/04/jmeter-grafana-influxdb-docker-live-monitoring.html
and I test with jmeter which can be run in non gui or even from jenkins jobs(maven, ant, sh)
Hi Shantonu,
ReplyDeleteUseful article.
Keep it up.
Thanks ..:)
DeleteWe are trying to disply active threads with multiple jmx script that are controlled by Ant Build.properties
ReplyDeleteBut Grafana is just displaying the Max value of active threads amongst the jmx scripts as running Users.
How do i handle this? I want to display all running VUsers of all the jmx scripts
Suppose i have 3 scripts
1. 4 Users
2. 5 Users
3. 6 users
now i just see 6 Users as running with test.startedT
how do i get sum of all these 3 i.e 15 Users
V users in jmeter are the threads. So, include thread counts.
DeleteAwesome article Shantonu dada.
ReplyDeleteRegards,
Jobayer
Is there any way to pass number of threads to influxDB??
ReplyDeletepossible as variable . But i prefer , send me JMX, let me try first
DeleteHi Shantonu,
ReplyDeleteI have followed all the steps mentioned above. These are the Errors I am facing:
Jmeter:
#1 java.lang.NoClassDefFoundError: javafx/embed/swing/JFXPanel
2018-02-14 15:51:11,234 INFO o.a.j.s.FileServer: Set new base='/home/dinesh/apache-jmeter-4.0/bin/Result'
2018-02-14 15:52:15,516 INFO o.a.j.e.StandardJMeterEngine: Running the test!
2018-02-14 15:52:15,517 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2018-02-14 15:52:15,517 INFO o.a.j.s.SampleEvent: List of sample_variables: []
2018-02-14 15:52:15,573 INFO o.a.j.v.b.BackendListener: Backend Listener: Starting worker with class: class org.apache.jmeter.visualizers.backend.graphite.GraphiteBackendListenerClient and queue capacity: 5000
2018-02-14 15:52:15,587 INFO o.a.j.v.b.BackendListener: Backend Listener: Started worker with class: class org.apache.jmeter.visualizers.backend.graphite.GraphiteBackendListenerClient
2018-02-14 15:52:15,665 INFO o.a.j.v.b.g.TextGraphiteMetricsSender: Created TextGraphiteMetricsSender with host: localhost, port: 2003, prefix: jmeter.
2018-02-14 15:52:15,673 INFO o.a.j.g.u.JMeterMenuBar: setRunning(true, *local*)
2018-02-14 15:52:15,992 INFO o.a.j.e.StandardJMeterEngine: Starting ThreadGroup: 1 : Thread Group
2018-02-14 15:52:15,992 INFO o.a.j.e.StandardJMeterEngine: Starting 1 threads for group Thread Group.
2018-02-14 15:52:15,992 INFO o.a.j.e.StandardJMeterEngine: Thread will continue on error
2018-02-14 15:52:15,993 INFO o.a.j.t.ThreadGroup: Starting thread group... number=1 threads=1 ramp-up=1 perThread=1000.0 delayedStart=false
2018-02-14 15:52:16,011 INFO o.a.j.t.ThreadGroup: Started thread group number 1
2018-02-14 15:52:16,011 INFO o.a.j.e.StandardJMeterEngine: All thread groups have been started
2018-02-14 15:52:16,031 INFO o.a.j.t.JMeterThread: Thread started: Thread Group 1-1
2018-02-14 15:52:16,100 INFO o.a.j.p.h.s.HTTPHCAbstractImpl: Local host = dinesh-VirtualBox
2018-02-14 15:52:16,153 INFO o.a.j.p.h.s.HTTPHC4Impl: HTTP request retry count = 0
2018-02-14 15:52:16,452 INFO o.a.j.p.h.s.LazySchemeSocketFactory: Setting up HTTPS TrustAll Socket Factory
2018-02-14 15:52:16,462 INFO o.a.j.u.JsseSSLManager: Using default SSL protocol: TLS
2018-02-14 15:52:16,462 INFO o.a.j.u.JsseSSLManager: SSL session context: per-thread
2018-02-14 15:52:16,535 INFO o.a.j.u.SSLManager: JmeterKeyStore Location: type JKS
2018-02-14 15:52:16,537 INFO o.a.j.u.SSLManager: KeyStore created OK
2018-02-14 15:52:16,537 WARN o.a.j.u.SSLManager: Keystore file not found, loading empty keystore
2018-02-14 15:52:16,713 ERROR o.a.j.v.b.g.TextGraphiteMetricsSender: Error writing to Graphite: Connection refused (Connection refused)
2018-02-14 15:52:17,666 ERROR o.a.j.v.b.g.TextGraphiteMetricsSender: Error writing to Graphite: Connection refused (Connection refused)
2018-02-14 15:52:18,611 INFO o.a.j.t.JMeterThread: Thread is done: Thread Group 1-1
2018-02-14 15:52:18,611 INFO o.a.j.t.JMeterThread: Thread finished: Thread Group 1-1
2018-02-14 15:52:18,629 INFO o.a.j.e.StandardJMeterEngine: Notifying test listeners of end of test
2018-02-14 15:52:18,629 INFO o.a.j.v.b.BackendListener: Worker ended
2018-02-14 15:52:18,631 ERROR o.a.j.v.b.g.TextGraphiteMetricsSender: Error writing to Graphite: Connection refused (Connection refused)
2018-02-14 15:52:18,636 INFO o.a.j.g.u.JMeterMenuBar: setRunning(false, *local*)
InfluxDB:
Created jmeter database and using the database.
Typer SHOW MEASUREMETS. It is just saying "Success! (no results to display)"
Please help me
Thanks
Dinesh
its connection errors... check port & connections
DeleteHi Shantonu,
ReplyDeleteI am running test using jmeter on two machines inorder to distribute the load properly. I am sable configure influxdb and Grafana in two machines and fetch the report for the scripts that are run in that machine. When I try to write data to influxdb from machine 1 to machine 2 I am getting the below error.
2018/09/17 14:55:32 INFO - jmeter.engine.StandardJMeterEngine: Running the test!
2018/09/17 14:55:32 INFO - jmeter.samplers.SampleEvent: List of sample_variables: []
2018/09/17 14:55:32 INFO - jmeter.visualizers.backend.BackendListener: Backend Listener:Starting worker with class:class rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient and queue capacity:5000
2018/09/17 14:55:32 INFO - jmeter.visualizers.backend.BackendListener: Backend Listener: Started worker with class:class rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient
2018/09/17 14:55:42 ERROR - jmeter.JMeter: Uncaught exception: java.lang.IllegalStateException: Failed calling setupTest
at org.apache.jmeter.visualizers.backend.BackendListener.testStarted(BackendListener.java:341)
at org.apache.jmeter.visualizers.backend.BackendListener.testStarted(BackendListener.java:297)
at org.apache.jmeter.engine.StandardJMeterEngine.notifyTestListenersOfStart(StandardJMeterEngine.java:204)
at org.apache.jmeter.engine.StandardJMeterEngine.run(StandardJMeterEngine.java:325)
at java.lang.Thread.run(Unknown Source)
Caused by: java.lang.RuntimeException: java.net.SocketTimeoutException: connect timed out
at com.google.common.base.Throwables.propagate(Throwables.java:160)
at org.influxdb.impl.InfluxDBImpl.execute(InfluxDBImpl.java:270)
at org.influxdb.impl.InfluxDBImpl.describeDatabases(InfluxDBImpl.java:246)
at rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient.createDatabaseIfNotExistent(JMeterInfluxDBBackendListenerClient.java:249)
at rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient.setupInfluxClient(JMeterInfluxDBBackendListenerClient.java:207)
at rocks.nt.apm.jmeter.JMeterInfluxDBBackendListenerClient.setupTest(JMeterInfluxDBBackendListenerClient.java:142)
at org.apache.jmeter.visualizers.backend.BackendListener.testStarted(BackendListener.java:339)
... 4 more
Caused by: java.net.SocketTimeoutException: connect timed out
My Expected Result: Run test in two machines and the scripts should always report to influxdb located in one machine so that I collate the results easily.
influx cant connect , (time out).. see if you can connect from browser . without debugging , i cant give you any feedback
Delete