I have used a number of tools in the past to perform load testing of web sites and web applications, including STA, TestComplete and WAPT.

Each of these tools has certain weaknesses (mostly related to authentication and the quality of reporting) that have required additional work or resulted in a lack of confidence in the results.

Incredibly (to a confirmed Microsoft sceptic), Visual Studio 2008 includes a powerful load testing component with many useful features:

Contents

Load tests

Load tests are built from a collection of web tests linked to a load pattern and return results as a number of counters.

Web tests in VSTS 2008

Unlike most other test tools, VSTS 2008 only records traffic between the client and server for a web test - there is no direct interaction with objects in the browser or the web page.

For each interaction with a web site that results in a call to the server (including AJAX calls) a URL is recorded. For each URL you can specify QueryString Parameters (to change the input data to the tests) and Validation Rules to check the results.

File:WebTest.png

I'll use a Sharepoint app as an example. Checking a user's profile in the AUT results in a visit to the URL http://server/Person.aspx passing the QueryString Parameter accountname with the value DOM/gordon.bennett

To check that the correct page is reached when you replay the test you can specify Validation Rules, such as Find Text Gordon Bennett.

Data binding in web tests

So far, so good but this becomes much more useful when you bind QueryString Parameters to a data source, rather than a single value - allowing the test to be repeated with different test data.

For instance, to check ALL user's profiles you can create a CSV file containing all user IDs as individual lines and include "ID" as the first line. You can then add the CSV file as a Data Source to the webtest. Now you can edit the value of a QueryString Parameter to select the ID value in the Data Source.

File:WebTest_QSPdataBound.png

Now when you re-run the web test it will take the first value from the data source as the input data and perform a single run. If this works you can edit the web test's run settings to perform one run for each row in the data source. This is a good way to check the validity of your test data before you run a load test.

File:WebTest_RunDataBound.png

You aren't limited to a single value of course. For example, you can easily set up a CSV file with user ID and password and any other user-related details to use as either authentication credentials, input data or master results to compare the test output to. You can also use a database or XML file as a data source.

Building a load test scenario

Now you have a working web test and valid test data, you can build a load test using the wizard which asks you to specify the following:

All the above can be left at the default values unless you are trying to trigger specific problems.

Note that when we talk about a load of 100 virtual users, this is not the same as 100 real users navigating to the site at the same time. A virtual user load measures a web site's ability to sustain simultaneous connections for a number of users. For instance, a load of 100 could generate anything from a few hundred to a few thousand requests for a page every minute. A single virtual user represents the load of many real-world users.

File:LoadTestScenario.png

File:LoadTestMixedScenario.png

Load Test Counters

From an end-user perspective, the most important results of a load test are the timings - the response time of the page and the time to render the content. VSTS2008 records these for each page (Response time and page time, respectively) but if you have the correct permissions in your account, you can also record many valuable server-side statistics, such as CPU load and memory use.

Caching in

First-time users to a web site make far higher demands on web servers than returning visitors as the latter should have most of the site's static content cached in their browser. This can be reflected in a load test by setting a value for Percentage of New Users in the scenario's properties.

Load Test Run Settings

The most used run settings in a load test are likely to be those that determine how long the test will run. This can either be a fixed time - Run Duration, a fixed number of runs - Test Iterations, or can be limited to the number of test data items (by setting Test Iterations to a high enough value and going back to set the web test's Data Source Access Method property to Unique).

Load Test Results

With web tests recorded and checked with valid test data, a load test scenario built and counters chosen, you are good to go.

The results will appear in real time as a set of graphs and tables:

File:LoadTestResults.png

Note that the scale for each line in each graph may be different

The graphs can show the results for any counter. In the example above you can see that response times and page times increase gradually as the load leaps upwards but, encouragingly, the CPU and memory load on the SQL and web servers is flat.

Exporting results

You can select a graph and click the Excel button in the toolbar and a new workbook is created based on the figures recorded in the web test.

It is then a simple measure to create a report in Excel:

File:LoadTestResults_Excel.png

Agents and IP switching

If you run a load test on your machine, it will only use a single core. Depending on the test and available hardware, this should be OK for up to 200 virtual users.

If you want to increase the load, or just reduce the strain on your own desktop, you can install the optional (and eye-wateringly expensive) Controller and Agent components on other PCs.

Using agents also allows you to enable IP switching, which is useful if you need to test a load balancer (which will only direct traffic to different web servers if requests are made using different IP addresses).

Despite the ruinous expense of purchasing agent licences, the expected easy installation process never materialises. You need to ensure that you install both the agent and controller with an account that has Administrator and Run As Service privileges on each machine. Even after specifying this account during the install process, you will still need to go back into Admin Tools and check that this has been set up. You also need to check that the account is included in the TeamTestAgentService, TeamTestControllerAdmins and TeamTestControllerUsers groups on the Controller machine. You will also need to install VSTS SP1 on all machines. Finally you will need to reboot each machine 42 times. Probably.

Managing Testing Resources

Using the Solution Explorer you can include pre-existing files such as test data or your hosts file. Existing files can be added as links if you cannot move them into the project folder. A good example would be creating a link to your hosts file C:\WINDOWS\system32\drivers\etc\hosts so that you can edit it within VSTS if you need to map URLs to alternative IP addresses to bypass load balancers, etc.

Further reading

I have only provided a brief introduction to load testing in VSTS 2008 - there are far more settings here than I could hope to cover, but these might apply to only a few performance test scenarios.

For more information, see the Quick Reference Guide and keep an eye on the blogs:



Generation Y further reading