Test Mix Model in Visual Studio 2008

Test Mix Model in Visual Studio 2008

In Visual Studio Team System 2008 Test Edition, the Test Mix Model option were added to the load test feature to let you more easily create accurate load.

You use load modeling options to more accurately predict the expected real-world usage of a Web site or application that you are load-testing. It is important to do this because a load test that is not based on an accurate load model can generate misleading results.

Percentage Based on Tests Started

For each test in the mix, you can specify a percentage that determines how often the test is selected as the next test to run. For example, you might assign the following percentage values to three tests:

• TestA (50%)

• TestB (35%)

• TestC (15%)

If you use this setting, the next test to start is based on the assigned percentages. This is done without taking into consideration the number of virtual users who are currently running each test.

Percentage Based on Virtual Users

This model of test mix determines the percentage of virtual users who will run a particular test. If you use this model of test mix, the next test to start is based not only on the assigned percentages but also on the percentage of virtual users who are currently running a particular test. At any point in the load test, the number of users who are running a particular test matches the assigned distribution as closely as possible.

When do you choose Percentage test mix and when do you choose Percentage based on virtual users?
The difference between these two choices is important when some tests in the test mix have a much longer duration than other tests. In this situation, you should probably choose Percentage based on virtual users. This choice helps avoid a test run in which the probability increases that too many users will be running long-duration tests.
However, if the tests all have similar durations, you can more safely choose Percentage test mix.

Pacing Test Mix

If you specify a pacing test mix, you set a rate of test execution for each virtual user for each test in the test mix. For each test, this rate is expressed as tests run per virtual user per hour. For example, you might assign the following pacing test mix to the following tests:

• TestA: 4 tests per user per hour

• TestB: 2 tests per user per hour

• TestC: 0.125 tests per user per hour

If you use the pacing test mix model, the load test runtime engine guarantees that the actual rate at which tests are started is less than or equal to the specified rate. If the tests run too long for the assigned number to be completed, an error is returned.

The “Think Time Between Test Iterations” setting does not apply when you use a pacing test mix, so it is not displayed as a property on the scenario. Rather, it is a property on the Percentage Test Mix.