This section is about applying known performance test techniques in the cloud context. In addition, we’ll describe an approach to new performance test techniques, such as testing elasticity. However, not all traditional performance tests are run. For example, executing a stress test is in most cases not possible because it jeopardizes the stability of the service for the other customers. User experience of response times and the system load are key in testing performance. Depending on the identified risks, this leads to a limited or thorough review of different circumstances in which certain performance is desired (during the day, working hours, affects of other users, etc.). Also to be considered is the performance in the case of an (unexpected) increase in user numbers, peaks in load, and prolonged periods of high load. To determine whether a service meets the performance requirements and expectations, acceptance criteria are needed. What are requirements regarding response times and at which loads do these requirements have to be met? When “going into the cloud,” one has to realize that with services it can be difficult to achieve the same response times one is accustomed to. Introducing more communication steps and the fact that services are not specifically made for the customer are probable factors in slower response times. If it is needed to determine the response times as a result of going into the cloud, the performance on the current infrastructure has first to be measured. The resulting outcome is used as a reference (baseline) with which to compare the cloud performance (see figure below).
With SaaS the center of gravity for performance lies fully with the service provider: they are responsible for the performance that is experienced by the end users (not including equipment and Internet connection speed). With PaaS and IaaS this is more complex because the application and software environments are partially or fully the responsibility of the customer. The performance as experienced by the end user is dependent on the efficiency of the application software (customer), the speed/efficiency of the software platform (customer and/or supplier), and the speed of the infrastructure (supplier). Most performance tests are applicable to all service models. If necessary, the effect the ser- vice model has on the preparation and execution of performance tests should be discussed.The following subsections provide overviews of the different kinds of performance tests and include practical examples for applying these tests to services. In addition, the various aspects of testing performance, such as creating performance test cases, the use of tools, and test setups, will be addressed.