Jmeter -To start with

JMETER

Apache JMeter is a 100% pure Java Desktop application designed to load-test functional behavior and measure performance. It was originally designed for testing Web Applications but has since expanded to other test functions. JMeter can be used as a unit test tool for JDBC database connections, FTP, LDAP, Web services and JMS.

If you are not aware of the basics related to JMETER then please refer to the following link for 10 min before reading this complete article.

http://jmeter.apache.org/usermanual/build-web-test-plan.html

The JMETER tool looks difficult to understand quickly since the new user does not have the basic knowledge to create a load test environment. This article will concentrate only on one of the best ways to record a web application using Workbench (file provided in code files) and run load tests in a similar way, like fro example QTP or Visual Studio - CodedUI.

Difference between performance and load testing


Load Testing Tools


Various types of load test variants


Example of Requirement of load testing:

• Given a load of 50 requests per second for this URL and no other requests
• Within database CPU utilization and under 20% and Web Server CPU utilization of under 10%

To analyze the load testing results:

• To check the performance of the web page the average value of receiving the page is an important parameter to check
• Samples: A sample means one sampler call. One request to a web page in our case. So the value of 51 means a total of 51 web page requests to the page, for example "http://www.google.com", were made by JMeter
• Average: This value is the average time taken to receive the web pages. There were 51 values of receiving time that were added and divided by 51 and this value is arrived by JMeter. This value is a measure of performance of this web page. This means on an average 334 milliseconds is required to receive this web page for our network conditions.
• Min and Max: These are the minimum and maximum values of time required for receiving the web page.
• Std. Dev: This shows how many exceptional cases were found that were deviating from the average value of the receiving time. The lesser this value more consistent the time pattern is assumed.
• Error %: This value indicated the percentage of error. For example 51 calls were made and all are received successfully this means 0 error. If there are any calls not received properly they are counted as errors and the percentage of error occurrence against the actual calls made is displayed in this value.

Terminology
Before we dive into the step-by-step instructions, it's a good idea to define the terms and ensure the definition is clear.

Master: the system running Jmeter GUI, that controls the test
Slave: the system running jmeter-server, that takes commands from the GUI and send requests to the target system(s)
Target: the webserver we plan to stress test

Note: Starting the Test
Before start load testing, if you want to double check that the slave systems are working then open jmeter.log in Notepad. You should see the following in the log. Jmeter.engine.RemoteJMeterEngineImpl: Starting backing engine. If you do not see this message then it means the JMETER server did not start correctly.

1. Download JMETER binary release from http://jmeter.apache.org/download_jmeter.cgi
2. Extract the zip file and go to bin folder.
3. Double-click "jmeter.bat"
4. This will start the JMETER application and the following screen will be displayed:
   
   
   
   Part 1: How to record your web applications using Workbench for load test.
   
   As mentioned above, JMETER looks like a difficult tool to understand quickly, since the new user does not have the basic knowledge to create a load test environment. The following provides one of the best ways to record the web application using Workbench and run load tests like for example QTP or Visual Studio - CodedUI.
    
5. Go to File Open.
6. Browse the sample test plan attached with this article for TestPlan2.jmx.jmx or Sample.jmx and click open.
7. This will open the screen as shown below:
   
   
8. Click on TestPlan2 on the left panel and edit the field "Number of Threads(users)": 25 on the right panel. Provide the desired figure with which testing is required to be conducted.
9. Click on "HttpRequestDefaults" on the left panel and edit the Server Name or IP : 152.425.306.120 and Port Number:8080 (if the Tomcat Server is running on port other than 8080) on the right panel. Do not give the Server Name as "localhost". The field should contain the machine name or IP.
10. There are already 3 http requests created. In other words Login, File Browser and Search. You can run the test plan to test performance for these 3 requests. You can also add new requests to test against specific criteria. This is discussed later on.
11. In order to run the test plan, go to Run Start. Please ensure the Tomcat Server is running.
12. This will start executing the test plan.
13. Once the test plan execution is complete, you can see the result in various formats as explained below.
14. Click on Graph Results on the right panel. This displays the time taken for each request by plotting the requests on the x-y axis.
    
    
15. Click on "View Results" in the table to see the results in tabular format
    
    
16. Click on View Results Tree to view request and response data for each request.
17. Click on "Aggregate Graph". Then click on "Display Graph" on the right panel. This will render the results in a graphical view.
    
    
18. In order to create your own request for a specific search criteria, a proxy server needs to be set up that will record the request to be tested.
19. Right-click on WorkBench and select "Merge". Browse through the file "WorkBench.jmx" and click on "Open".
20. Click on "Http Proxy Server" on the left panel. By default, the proxy server is configured to run at 9080 port. If you want to run on a different port then edit the Port field on the right panel and then click on the Start button present towards the end. This will start the proxy server.
    
    
21. Now open Internet Explorer and login into OpenEDMS. Again ensure you access the application with server name or IP and not localhost.
22. Click on the "Advance Search Tab".
23. In order to start recording the search, the proxy server needs to be set up in IE.
24. Go to "Tools" | "Internet Options" | "Connections LAN Settings"
    
    
25. Tick the checkbox "Use a proxy server for your LAN" and provide the address and port as per the configuration done previously.
26. Now as soon as you click on "Search" after filling the search criteria, a new request will be added in JMeter as shown.
    
    
27. You can provide some meaningful name instead of "/edms/servlet/edms.do" for example Report Search in the Name field on the right panel
    
    
28. Now drag on the Report Search item on the left panel and place it above the Graph Results as shown below.
    
    
29. This way you can create your own search criteria to be tested. Now follow the same procedure to run the test as was mentioned earlier and view the results in desired formats.  

Response time Vs LatencyVs Processtime

Latency is the delay incurred in communicating a message (the time the message spends “on the wire”). The word latent means inactive or dormant, so the processing of a user action is latent while it is traveling across a network.
Changes in latency are typically unavoidable through changes to your code. Latency is a resource issue, which is affected by hardware adequacy and utilization.
Example: The latency in a phone call is the amount of time it takes from when you ask a question until the time that the other party hears your question. If you’ve ever talked to somebody on a cell phone while standing in the same room, you’ve probably experienced latency first hand, because you can see their lips moving, but what you hear in the phone is delayed because of the latency.
Response time is the total time it takes from when a user makes a request until they receive a response.
Response time can be affected by changes to the processing time of your system and by changes in latency, which occur due to changes in hardware resources or utilization.
Example: The response time in phone conversation is the amount of time it takes for you to ask a question and get a response back from the person that you’re talking to.
Processing time is the amount of time a system takes to process a given request, not including the time it takes the message to get from the user to the system or the time it takes to get from the system back to the user.
Processing time can be affected by changes to your code, changes to systems that your code depends on (e.g. databases), or improvements in hardware.
Example: The processing time in a phone conversation is the amount of time the person you ask a question takes to ponder the question and speak the answer (after he hears the question of course).
In these terms:
Latency + Processing Time = Response Time
In many cases, you can assert that your latency is nominal, thus making your response time and your processing time pretty much the same. I guess it doesn’t matter what you call things as long as everybody involved in your performance analysis understands these different aspects of the system. For example, it is useful to make a graph latency vs. response time, and it is important for all the parties involved to know the difference between the two.

Short Interview with Jmeter

Quality Analyst: Hello JMeter can you tell us something about yourself?
Apache JMeter: Hello!  I am a load testing tool.  If you tell me how to perform a certain action I can simulate multiple users doing that action.  This is like cloning users, isn't it?  I am a child of Apache Jakarta project, technically sub project of Apache Jakarta project.  I was created initially to test performance of web sites, but you know what, now I help people test many more things like database performance, FTP server performance and much more.

Quality Analyst: We heard that sometimes you talk Java and Javascript language?  Is this true, and do we Quality Analysts have to know Java or JavaScript to talk with you?
Apache JMeter: Well, simple scenerios of testing can be done with no knowledge of JavaScript but for advanced functional testing you should know JavaScript.  Java is needed only if you are using Java Sampler part of my abilities and if you want to extend my capabilities.
Quality Analyst: You mentioned functional testing just now, so can you also help us testing our software functionally?
Apache JMeter: The answer is yes.  I can help you test the software functionally.  This requires thorough knowledge of the Sampler you are using.  For example in case of web site testing HTTP Request Sampler is used so the request and response patterns of this sampler shall be known to the user.  They can then write assertions on the response to check whether the response was as expected and hence functionally check the response.

Quality Analyst: Do you like a particular browser?
Apache JMeter: This question is confusing.  Let me clear some points first.  I am not a browser.  No, I am not a browser in traditional sense.  Yes I can visit a web site just like a browser do (through my HTTP Request Sampler and HttpClient Sampler) but I do not show that web site on screen exactly as a browser do.  My job is to visit a web site and capture the response sent by that web site.  I can show you the response as TEXT, XML or even in terms of HTML but without executing any JavaScript on that page.

So there is no question of me liking any browser. Browsers and I are different things.  My behaviour is different than any existing browser.

Quality Analyst: I know that you help us record our actions and then you can play them again.  My question is, are those recordings OS dependant?  In other words if I record something on Linux, can I play those actions on a Windows workstation?
Apache JMeter: Oh yes you can.  I can save Test Plans for you and those test plans are in XML format so they have nothing to do with any particular OS.  You can run those test plans on any OS where I can run.

Core Performance testing Activities

Core Performance Testing Activities

1. Activity 1. Identify the Test Environment.  Identify the physical test environment and the production environment as well as the tools and resources available to the test team. The physical environment includes hardware, software, and network configurations. Having a thorough understanding of the entire test environment at the outset enables more efficient test design and planning and helps you identify testing challenges early in the project. In some situations, this process must be revisited periodically throughout the project’s life cycle.
2. Activity 2. Identify Performance Acceptance Criteria.  Identify the response time, throughput, and resource utilization goals and constraints. In general, response time is a user concern, throughput is a business concern, and resource utilization is a system concern. Additionally, identify project success criteria that may not be captured by those goals and constraints; for example, using performance tests to evaluate what combination of configuration settings will result in the most desirable performance characteristics.
3. Activity 3. Plan and Design Tests.  Identify key scenarios, determine variability among representative users and how to simulate that variability, define test data, and establish metrics to be collected. Consolidate this information into one or more models of system usage to be implemented, executed, and analyzed.   
4. Activity 4. Configure the Test Environment.  Prepare the test environment, tools, and resources necessary to execute each strategy as features and components become available for test. Ensure that the test environment is instrumented for resource monitoring as necessary.
5. Activity 5. Implement the Test Design.  Develop the performance tests in accordance with the test design.
6. Activity 6. Execute the Test.  Run and monitor your tests. Validate the tests, test data, and results collection. Execute validated tests for analysis while monitoring the test and the test environment.
7. Activity 7. Analyze Results, Report, and Retest.  Consolidate and share results data. Analyze the data both individually and as a cross-functional team. Reprioritize the remaining tests and re-execute them as needed. When all of the metric values are within accepted limits, none of the set thresholds have been violated, and all of the desired information has been collected, you have finished testing that particular scenario on that particular configuration.

Performance Testing Types

Performance Testing

Performance testing is defined as the technical investigation done to determine or validate the speed, scalability, and/or stability characteristics of the product under test. Performance-related activities, such as testing and tuning, are concerned with achieving response times, throughput, and resource-utilization levels that meet the performance objectives for the application under test. Because performance testing is a general term that covers all of its various subsets, every value and benefit listed under other performance test types in this chapter can also be considered a potential benefit of performance testing in general.

Key Types of Performance Testing

The following are the most common types of performance testing for Web applications.

Term	Purpose	Notes
Performance test	To determine or validate speed, scalability, and/or stability.	A performance test is a technical investigation done to determine or validate the responsiveness, speed, scalability, and/or stability characteristics of the product under test.
Load test	To verify application behavior under normal and peak load conditions.	Load testing is conducted to verify that your application can meet your desired performance objectives; these performance objectives are often specified in a service level agreement (SLA). A load test enables you to measure response times, throughput rates, and resource-utilization levels, and to identify your application’s breaking point, assuming that the breaking point occurs below the peak load condition. Endurance testing is a subset of load testing. An endurance test is a type of performance test focused on determining or validating the performance characteristics of the product under test when subjected to workload models and load volumes anticipated during production operations over an extended period of time. Endurance testing may be used to calculate Mean Time Between Failure (MTBF), Mean Time To Failure (MTTF), and similar metrics.
Stress test	To determine or validate an application’s behavior when it is pushed beyond normal or peak load conditions.	The goal of stress testing is to reveal application bugs that surface only under high load conditions. These bugs can include such things as synchronization issues, race conditions, and memory leaks. Stress testing enables you to identify your application’s weak points, and shows how the application behaves under extreme load conditions. Spike testing is a subset of stress testing.  A spike test is a type of performance test focused on determining or validating the performance characteristics of the product under test when subjected to workload models and load volumes that repeatedly increase beyond anticipated production operations for short periods of time.
Capacity test	To determine how many users and/or transactions a given system will support and still meet performance goals.	Capacity testing is conducted in conjunction with capacity planning, which you use to plan for future growth, such as an increased user base or increased volume of data. For example, to accommodate future loads, you need to know how many additional resources (such as processor capacity, memory usage, disk capacity, or network bandwidth) are necessary to support future usage levels. Capacity testing helps you to identify a scaling strategy in order to determine whether you should scale up or scale out.

The most common performance concerns related to Web applications are “Will it be fast enough?”, “Will it support all of my clients?”, “What happens if something goes wrong?”, and “What do I need to plan for when I get more customers?”. In casual conversation, most people associate “fast enough” with performance testing, “accommodate the current/expected user base” with load testing, “something going wrong” with stress testing, and “planning for future growth” with capacity testing. Collectively, these risks form the basis for the four key types of performance tests for Web applications.

Summary Matrix of Benefits by Key Performance Test Types

Term	Benefits	Challenges and Areas Not Addressed
Performance test	Determines the speed, scalability and stability characteristics of an application, thereby providing an input to making sound business decisions. Focuses on determining if the user of the system will be satisfied with the performance characteristics of the application. Identifies mismatches between performance-related expectations and reality. Supports tuning, capacity planning, and optimization efforts.	May not detect some functional defects that only appear under load. If not carefully designed and validated, may only be indicative of performance characteristics in a very small number of production scenarios. Unless tests are conducted on the production hardware, from the same machines the users will be using, there will always be a degree of uncertainty in the results.
Load test	Determines the throughput required to support the anticipated peak production load. Determines the adequacy of a hardware environment. Evaluates the adequacy of a load balancer. Detects concurrency issues. Detects functionality errors under load. Collects data for scalability and capacity-planning purposes. Helps to determine how many users the application can handle before performance is compromised. Helps to determine how much load the hardware can handle before resource utilization limits are exceeded.	Is not designed to primarily focus on speed of response. Results should only be used for comparison with other related load tests.
Stress test	Determines if data can be corrupted by overstressing the system. Provides an estimate of how far beyond the target load an application can go before causing failures and errors in addition to slowness. Allows you to establish application-monitoring triggers to warn of impending failures. Ensures that security vulnerabilities are not opened up by stressful conditions. Determines the side effects of common hardware or supporting application failures. Helps to determine what kinds of failures are most valuable to plan for.	Because stress tests are unrealistic by design, some stakeholders may dismiss test results. It is often difficult to know how much stress is worth applying. It is possible to cause application and/or network failures that may result in significant disruption if not isolated to the test environment.
Capacity test	Provides information about how workload can be handled to meet business requirements.   Provides actual data that capacity planners can use to validate or enhance their models and/or predictions. Enables you to conduct various tests to compare capacity-planning models and/or predictions. Determines the current usage and capacity of the existing system to aid in capacity planning. Provides the usage and capacity trends of the existing system to aid in capacity planning	Capacity model validation tests are complex to create. Not all aspects of a capacity-planning model can be validated through testing at a time when those aspects would provide the most value.

Website Performance testing using VSTS

This blog post gives information about identifying, creating performance testing strategy along with implementation using Visual studio (VS) 2010. I would not be covering as “what is meant by” concepts of performance testing but how to part of it.
First and foremost challenge you could face while starting performance testing is
- Identifying the goals/Scenarios for performance testing: For example it could be,

• Concurrency test scenario: Monitor the performance of the website for total of 100 concurrent users performing operations New Enrolment, Change My Enrolment and Un-registration
• Scalability test scenario: Website should be scalable for 5000 users performing New Enrolment operation

- Requirements to be met to pass the performance testing criteria: For example it could be,

• Response time of the web pages should be <= 5 sec
• For reports response time should be <= 15 sec

You need to get the above details defined from business/Program manager before you proceed any further.
Performance testing for a website using VS would involve the following steps:

• Create web test
• Customize the web test (data binding, validation rules..,)
• Run the web test and fix errors if it has
• Create load test
• Configure web test and load test scripts
• Run the load test
• Analyze the results

Lets go through each step in detail.

• Create web test
  • Open the Visual studio team system and create a test project
  • Add web test
  • Type the URL of the application
  • Record the scenario which you want to load test. In the left pane you can see the URL’s being recorded
  • Once you are done with recording the scenario stop the recording
  • You can see the URL’s (get and post requests) it has recorded
• Customize the web test
  • In this section you would data bind users, create validation rules, set the response time goal for each link, set the think time.,
• Run the web test and fix errors if it has any
• Create load test
  • Add load test
  • Give name of the scenario
  • Click on next
  • Select scenarios (scripts) click on next
  • Select network type click on next
  • Enter constant load of 100 (you can give ramp up if you want)
  • Add IIS and DB server name
  • Finish
• Configure web test and load test scripts
  • Configuration for web test scripts:
    • Response time goal of 5 seconds for each page
  • Configuration of Load test scripts:
    • Scenario1: 100 concurrent users
      • Constant load pattern: 100 users
      • Run duration: 20 minutes
      • Web test connection model: Connection per user
      • Web test: one row per data source
    • Scenario2: Scalability
      • Constant load pattern: 25 users
      • Run duration:  480 minutes
      • Web test connection model: Connection per user
      • Web test: one row per data source
      • Uncheck test time outs settings
• Run the load test
• Analyze the results (samples summary result documents attached below)
• Important statistics for us:
  • Response time for each page should be < 5 seconds
  • Web Server Processor usage < 70%
  • Web Server Available MB >128 MB
  • DB Server Processor usage < 70%
  • DB Server Available MB >128 MB

Finally use code profiler to identify the functions which are taking lot of time and memory.

QAAutomation_Easylearning

How Apache JMeter simulates multiple users?

1. Introduction to different elements of a JMeter Test Plan.
2. Difference between real world and JMeter world while doing a web request.
3. Workflow of a typical web server request and response.
4. JMeter's role in a web testing work flow.

Let us take a view at how a request for web page is sent to the Web server from a User's browser, and how a web page is finally displayed.  Understanding this workflow will give us opportunity to imagine how JMeter fits in this workflow and how it can do same thing as a browser does.

Now introducing Apache JMeter into the picture.  The actions taken by the User are no longer required.  The User action trigger some HTTP requests which are recorded in a JMeter HTTP Request Sampler for multiple uses.  

See how JMetr replaces a browser, but how the working is a little different.

So what makes one User transaction in terms of JMeter?  

When we add a HTTP Request Sampler in JMeter Test plan it stores one Web Page calling and receiving sequence.  We can call this as a single user transaction.  As soon as we tell this to JMeter it is capable of doing the same thing any number of times.  It also does this multiple times simultaneously.

As JMeter is not exactly a browser the HTTP Request Sampler makes all the received data available for further analysis.  We use View Tree Listener for actually visualizing the received response.  View Tree Listener is a good way to know what request was sent, what was the response and the result of the transaction.

When JMeter simulates multiple users by making the same call multiple times, in different operating system threads (we can say execution paths) that time more analytical data is available.  The Summary Report Listener can show us the additional calculated data line Minimum time and Maximum time taken to receive the response.  It also calculates the average time taken using all the results together.

So we have seen that for web site testing we need many items like

1. HTTP Request Sampler
2. View Tree Listener
3. Summary Report Listener

To use these items provided by JMeter we will also need items like

1. Test Plan and
2. Thread Group

For some advanced user transactions and logical conditions there are Controllers.  Once we learn how to combine these elements we are ready to write our first Test Plan in JMeter.

Yes initially this may seem like a lot of jargon.  Eventually we will get used to these elements and then testing will be a piece of cake.  Ok, let us re-visit what they all mean in JMeter world.

We will write our first Test Plan step by step in the next article but let us first get introduced formally with the different elements we have seen till now.

Now we know some elements which will be used in creation of a Test Plan in JMeter.  It will be worthwhile checking the difference between real world transactions and the transactions that happen in JMeter world.  Is there one to one mapping between actions performed in real world and while executing a JMeter test plan?

Thats all for this article.  Thanks for reading

PerfTest of Webpage using Jmeter

Performance  Testing of web page with Apache JMeter

Topics covered in this article:

1. Step by step method to test single web page in Apache JMeter
2. Manually creating a test plan in Apache JMeter
3. Simulating 50 users through Apache JMeter test plan

Let us see what we want to achieve first.  We want to create a Test Plan in Apache JMeter so that we can test performance of one web page say page shown by the URL: http://www.google.com/.  We will need many elements in the JMeter Test plan let us see what are they:

So this is what we shall see when we are done with creating the Test Plan.  We have to start with a blank JMeter window

Before adding anything more to this window let us change the name of Test Plan.  We can type a new name for the node in the Name text box.  In this version of JMeter the name does not get reflected immediately when you type it in the text box.  You have to change focus to workbench node and back to Test Plan node to see the name getting reflected.

Let us give new name "First Test" to the Test Plan node.

Now we will add our first element in the window.  We will add one Thread Group, which is placeholder for all other elements like Samplers, Controllers, Listeners.  We need one so we can configure number of users to simulate.  The Thread Group represents one set of actions performed by User.

In JMeter all the node elements are added by using the context menu.  You have to right click the element where you want to add a child element node.  Then choose the appropriate option to add.

We will click the Thread Group option and it will get added under the Test Plan (First test) node.

Like we changed the name of Test Plan let us also change the name of Thread Group.  For us this is the element which represents multiple users.  So let us name it "Users".  For us this element means Users visiting the Google Home Page.

Now we have to add one Sampler in our Thread Group (Users).  As done earlier for adding Thread group, this time we will open the context menu of the Thread Group (Users) node by right clicking and we will add HTTP Request Sampler by choosing Add > Sampler > HTTP request option.

This will add one empty HTTP Request Sampler under the Thread Group (Users) node.

 We will need most configuration in this node element, HTTP Request Sampler.  The settings we have to configure are:

1. Name: we will change the name to reflect the action what we want to achieve
1. This will will name "Visit Google Home Page"
2. Server Name or IP: here we have to type the web server name
1. In out case it is www.google.com
2. Note http:// part is not written this is only the name of the server or its IP
3. Protocol: we will keep this blank
1. That will mean we want HTTP as the protocol
4. Path: we will type path as / (slash)
1. This means we want the root page of the server
2. This also means now the server will decide what page will be sent to us and it is the default page of the web site

After add the HTTP Request Sampler we will need one Listener.  Let us add View Results Tree Listener under the Thread Group (User) node.  This will ensure that the results of the Sampler will be available to view in this Listener node element.

Adding a listener is also same like adding a new node under Thread Group.  Open the context menu and choose Add > Listener > View Results Tree option to add the listener.

With this much of setup we are now ready to run our first test.  Coming back to the configuration of the Thread Group (Users) we have kept it all default values.  This means JMeter will execute the sampler only once.  It will be like a single user and only one time.

This is similar to like a user visiting a web page through browser, only here we are doing that through JMeter sampler.  We will execute the test plan using Run > Start option.

Apache JMeter asks us to save the test plan in a disk file before actually starting the test.  This is important if we want to run the test plan again and again.  If we say not to save by clicking No option it will run without saving.  Let us save it first so we can reuse it again.

We have kept the setting of the thread group as single thread (that means one user only) and loop for 1 time (that means run only one time), hence we will get result of one single transaction in the View Result Tree Listener.

This means JMeter was successful in fetching the web page at the given URL just like a browser.  It has stored all the headers and the response sent by the web server and ready to show us the result in many ways.

Let us take a closer look at the three tabs available in the View Result Tree Listener view.

The first tab is Sampler Results.  It shows JMeter data as well as data returned by the web server.  Usually browsers hide this data from us as it is not related to showing the web page.  This data is utilized internally by the browser, for example when there is a Response Code: 404 then the browser shows a page not found error page, and when the Response Code is 200 the browser shows the received web page HTML.

The second tab is Request, where all the data which was sent to the web server as part of the request is shown.

The last tab is Response data.  In this tab the listener shows the data received from server as it is in text format.  It also have facility to show the data in XML, HTML formats but that will be just the data in other way (no JavaScript will be executed).  This may look different that the page actually seen in the browser.

We can select different ways of viewing this response data like as below:

This was just one request but JMeter's real strength is in sending the same request like many users are sending it.  To test the web servers with multiple users we will have to change the Thread Group (Users) settings.

We will set 50 users and these requests will be sent with some delay of (10/50) seconds as we have set the ramp up time to 10.

To visualize the results, a better aggregating listener will be required so we will add Summary Report Listener in the Thread Group node.  Adding Summary Report Listener is similar to adding a View Results Tree Listener.  Just right click on the Thread group node and choose Add > Listener > Summary Report.

We will run the test plan again after adding the Summary Report Listener ans when the test is done we will see the following results.  Your results may be different than mine due to different environments but mostly similar.

This is how we tried testing one single page with JMeter.  The summary report shows values about the measurement JMeter has done while calling the same page as if many users are calling the page.  Let us look at some of the value headings and what they mean:

• Samples: A sample mean one sampler call.  One request to web page in our case.  So the value of 51 means total 51 web page requests to the page http://www.google.com were made by JMeter 
• Average: This value is the average time taken to receive the web pages.  In our case there were 51  values of receiving time which were added and divided by 51 and this value is arrived by JMeter.  This value is a measure of performance of this web page.  This means on an average 334 milliseconds time is required to receive this web page for our network conditions.
• Min and Max: These are the minimum and maximum value of time required for receiving the web page.
• Std. Dev:  This shows how many exceptional cases were found which were deviating from the average value of the receiving time.   The lesser this value more consistent the time pattern is assumed.
• Error %: This value indicated the percentage of error.  In our case 51 calls were made and all are received successfully this means 0 error.  If there are any calls not received properly they are counted as errors and the percentage of error occurrence against the actual calls made is displayed in this value.

Performance Testing metrics explained

To understand the load test results better, Hope they will help interpret results better:

What is “Response Time”?

Response Time measures the performance of an individual transaction or query and is the amount of time from the moment that a user sends a request until the time that the application indicates that the request has completed.

What is the “90% Line”?

90% Line (90 percentile of the response times)represents one of the most interesting metrics and shows that “90% of the samples finished within 0-x time” or “90% of the samples took no more than this time”

What is the “Average Response Time”?

The “Average Response Time” is a good number to know, however in a business-to-business service application, it doesn’t matter if the average response time is 500ms, if the 90% line is 1.5 seconds. From a software developer perspective, if the 50% and 90% lines are far appart, the application fluctuates a lot.

What is “Latency”?

Latency , a synonym for delay, is the time for a specific block of data to get from one point to another. Latency is measured by sending a packet that is returned to the sender and the round-trip time is considered the latency.
Latency + Processing Time = Response Time

What is “Throughput”?

Throughput indicates the number of transactions per second an application can handle, the amount of transactions produced over time during a test.

Transactions per second

Transactions per second refer to the number of database transactions completed in one second.

Selenium WebDriver Architecture

In this article let’s see the Architecture of Selenium WebDriver.

Selenium WebDriver:

Selenium-WebDriver makes direct calls to the browser using each browser’s native support for automation.
Before starting the automation using any automation tool, it is very important to know how that tool works and how it is architecture. This will helps to take the good advantage of the tool at the same time it will helps to make right automation framework.
In my further posts I will start explain how to use selenium and how to create selenium framework in details but before that let’s get an overview of Selenium webdriver architecture. Selenium can be a little bit confusing. As a beginner you will find how simply you can record and play the selenium scripts but it is not straight forward to how it’s doing that. At first glance it might appear that Selenium is actually driving the browser directly from your code but there’s actually a little bit more going on here and it’s going to help us understand how we can remote execute our test by looking at this basic architecture. So here’s a picture of the architecture for Selenium WebDriver which is the current version of Selenium which I going to explain here.

Selenium webdriver architecture mainly divided into three parts

1. Language level bindings
2. Selenium Webdriver API
3. Drivers

1) Language Level Bindings :
You can see at the Left hand side here we’ve got some bindings and these are language level bindings and with which you can implement the Selenium webdriver code. In simple words these the languages in which are making an framework, will interact with the Selenium Webdriver and work on various browsers and other devices. So we have a common API that we use for Selenium that has a common set of commands and we have various bindings for the different languages. So you can see there’s Java, Java, Python, Ruby, there’s also some other bindings and new bindings can be added very easily.
2) Selenium Webdriver API:
Now these bindings communicate with Selenium Webdriver API and and This API send the commands taken from language level bindings interpret it and sent it to Respective driver. Right now don’t worry about how it works. I will explain them in upcoming posts. In basic term it contains set of common library which allow to send command to respective drivers.
3) Drivers:
Drivers here at the right hand side, you see we have various internet browser specific drivers such as IE driver, a Firefox, Chrome, and other drivers such as HTML unit which is an interesting one. It works in headless mode which make text execution faster. It also contains mobile specific drivers as well. But the basic idea here is that each one of these drivers knows how to drive the browser that it corresponds to. So the Chrome driver knows how to handle the low level details of Chome browser and drive it to do things like clicking button, going into pages, getting data from the browser itself, the same thing for Firefox, IE, and so on.


How all blocks work together?
So what’s happening here is you’re going to write your test in let’s say in Java and you’re going to be using common Selenium API and that Java binding is going to be sending command across this common WebDriver API. Now on the other end is going to be listening a driver, It’s going to interpret those commands and it’s going to execute them on the actual browser and then it’s going to return the result backup using the WebDriver API to your code where you can look at that result.


Let’s take more closure look that how exactly that works
Let say you have written test using java (binding code) against Selenium API and that binding code is going to issue commands across WebDriver wire protocol this is a rest-based web service that is able to interpret those commands. The driver server is just a little executable that runs each one of the drivers has this driver server that basically listens on a port on your local machine when you run your tests and it’s waiting for these commands to come in. And when these commands come in it interprets those commands and then automates the browser and then returns those results back.
I hope this will give some clear idea about how Selenium Webriver being architect.

CSS Selectors for Selenium Automation testers 

Using Selenium WebDriver whenever we want to automate a particular task we have to identify the UI Elements. Selenium WebDriver is providing built-in locators like By ID, Class Name, CSS Selectors, XPATH Selectors … In this article I am going to explain about the CSS Selectors.
First of all what is CSS?
Cascading Style Sheets is a language used to specify the visual appearance of a webpage. The main difference between HTML & CSS is, HTML is a mark-up language structure of a document for web distribution, where as CSS tells the browser how to it should look? (Like font style, back ground colour …). CSS can accomplish this by using CSS pattern matching rules, properties & values.
Syntax:
CSS pattern matching rules {Property: Value ;}
Example:
h1 {color: green;}
In the above example will make all the h1 tags to look in green colour.
In CSS, pattern-matching rules determine which style should be applied to elements in the DOM. These patterns, called CSS Selectors. If all conditions in the pattern are true for a certain element, the selector matches the element and the browser applies the defined style (Style is defined by properties & values) in CSS syntax.
Selenium WebDriver uses same principles of CSS selectors to locate elements in DOM. Selenium WebDriver’s By class provides the cssSelector () method for locating elements using CSS selectors. Let’s explore few of the CSS Selectors.
For examples purpose I am using the following HTML File

Default

<form name="loginForm"> <h1> Login Form</h1> <label title="username"> UserName:</label> <input type="text" id="input1"/> <label title="password">Password:</label> <input type="password" class="password" id="input2" /> <input type="password" class="password" id="input3" /> <input name="login" type="submit" value="Login" id="button1" /> </form>

1 2 3 4 5 6 7 8 9

<form name="loginForm"> <h1>Login Form</h1> <label title="username"> UserName:</label> <input type="text" id="input1"/> <label title="password">Password:</label> <input type="password" class="password" id="input2" /> <input type="password" class="password" id="input3" /> <input name="login" type="submit" value="Login" id="button1" /> </form>

Element Selectors:
In the style sheet, Element selectors are written using by the tag name. Element selectors also referred to as “type” selectors, select by matching elements (HTML tag name). The following line of code will identify the elements whose tag names matches to “input”.
IWebElement Element = Driver.FindElement(By.CssSelector("input"));
Class Selectors
In the style sheet, class selectors are written using the dot (“.”) preceding a custom name. Class selectors are extremely useful selectors that allow to add a class attribute to a given element in the mark-up, with a custom value. Then using that value preceded by a dot, write a corresponding rule using the class name.
The following line of code will identify the elements whose class name matches to “username”.
IWebElement Element = Driver.FindElement(By.CssSelector(".username"));
 ID Selectors:
In the style sheet, ID selectors are written using the hash “#”preceding a custom name. Since ID name can be used exactly one time in a DOM, ID’s are particularly useful in CSS layout when identifying significant portions of the document as they are unique.
The following line of code will identify the elements whose ID matches to “input1”.

IWebElement Element = Driver.FindElement(By.CssSelector("#input1"));
Attribute Selectors:
Apart from the class and id attributes, CSS selectors also enable the location of elements using other attributes of the element. In the following example, the Name attribute is used to locate an element whose tag name is “input” and having an attribute “name” of value “username”.
IWebElement Element = Driver.FindElement(By.CssSelector("[name=username]"));
There are four Attribute selectors available in CSS.

Attribute Selector	Pattern Matching	Example
[name^='ctrl']	Starting with	For example, if the name of an element is ctrl_12, this will locate and return elements with ctrl at the beginning of the name attribute.
[name$='word']	Ends with	For example, if the name of an element is password, this will locate and return elements with word at the end of the name attribute.
[name*='user']	Starting with	For example, if the name of an element is 1_username, this will locate and return elements that are containing user in the name attribute.

We can use ID as a attribute, which will be useful in case of dynamic controls in which ID’s will be generated by appending numbers. For example a particular element is having ID like this username_1_2345. In order to identify these kind of controls we can write selector as [ID^=username_1_].
Chaining of Selectors:
Till now we have seen basic selectors, we can chain those selectors in such a way we can identify the element uniquely in a much simpler way in complex DOM.
Few of chaining examples are listed in the below table.

Syntax	Description
*	Matches any Element
E	Matches all elements with tag name E
E C	Matches all elements with tag name E that are descendents of C
E>C	Matches all elements with tag name C that are direct descendents/children of E
E.C	Matches all elements E, that are having class name of C
E#i	Matches all elements E, that are having ID of i
E[a]	Matches all elements E, that are attribute of a, of any value
E[a=v]	Matches all elements E, that are attribute of a, of value v
E[a^=v]	Matches all elements E, that are attribute of a, whose value starts with v
E[a$=v]	Matches all elements E, that are attribute of a, whose value ends with v
E[a*=v]	Matches all elements E, that are attribute of a, whose value contains v

How to write Css and Xpath for an Element

This page will show you some CSS rules and pseudo-classes that will help you move your XPATH locators to CSS, a native approach on all browsers.

I: Simple

Direct child

A direct child in XPATH is defined by the use of a “/“, while on CSS, it’s defined using “>”
Examples:

//div/a

css=div > a

Child or subchild

If an element could be inside another or one it’s childs, it’s defined in XPATH using “//” and in CSS just by a whitespace.
Examples:

//div//a

css=div a

Id

An element’s id in XPATH is defined using: “[@id='example']” and in CSS using: “#”
Examples:

//div[@id='example']//a

css=div#example a

Class

For class, things are pretty similar in XPATH: “[@class='example']” while in CSS it’s just “.”
Examples:

//div[@class='example']//a

css=div.example a

II: Advanced

Next sibling

This is useful for navigating lists of elements, such as forms or ul items. The next sibling will tell selenium to find the next adjacent element on the page that’s inside the same parent. Let’s show an example using a form to select the field after username.

</input> </input>

Let’s write a css selector that will choose the input field after “username”. This will select the “alias” input, or will select a different element if the form is reordered.

css=form input.username + input

Attribute values

If you don’t care about the ordering of child elements, you can use an attribute selector in selenium to choose elements based on any attribute value. A good example would be choosing the ‘username’ element of the form without adding a class.

</input> </input> </input> </input>

We can easily select the username element without adding a class or an id to the element.

css=form input[name='username']

We can even chain filters to be more specific with our selections.

css=input[name='continue'][type='button']

Here Selenium will act on the input field with name=”continue” and type=”button”

Choosing a specific match

CSS selectors in Selenium allow us to navigate lists with more finess that the above methods. If we have a ul and we want to select its fourth li element without regard to any other elements, we should use nth-child or nth-of-type.

• <p>Heading</p>
• Cat
• Dog
• Car
• Goat

If we want to select the fourth li element (Goat) in this list, we can use the nth-of-type, which will find the fourth li in the list.

css=ul#recordlist li:nth-of-type(4)

On the other hand, if we want to get the fourth element only if it is a li element, we can use a filtered nth-child which will select (Car) in this case.

css=ul#recordlist li:nth-child(4)

Note, if you don’t specify a child type for nth-child it will allow you to select the fourth child without regard to type. This may be useful in testing css layout in selenium.

css=ul#recordlist *:nth-child(4)

Sub-string matches

CSS in Selenium has an interesting feature of allowing partial string matches using ^=, $=, or *=. I’ll define them, then show an example of each:
^= Match a prefix

css=a[id^='id_prefix_']

A link with an “id” that starts with the text “id_prefix_”
$= Match a suffix

css=a[id$='_id_sufix']

A link with an “id” that ends with the text “_id_sufix”
*= Match a substring

css=a[id*='id_pattern']

A link with an “id” that contains the text “id_pattern”

Matching by inner text

And last, one of the more useful pseudo-classes, :contains() will match elements with the desired text block:

css=a:contains('Log Out')

This will find the log out button on your page no matter where it’s located. This is by far my favorite CSS selector and I find it greatly simplifies a lot of my test code.

Sauce Labs - Selenium Testing on the Cloud

Sauce Labs’ cloud-based testing platform allows developers to automatically or interactively test mobile and web applications on more than 300 browsers and platforms including iOS, Android & Mac OS X. With built-in video recording of every test, debugging tools, and secure tunneling for local or firewalled testing, Sauce makes running, debugging and scaling test suites easier than ever. Sauce supports automated functional testing with Selenium and Javascript unit testing in all the popular programming languages, frameworks and CI systems.