Testing cloud services – Testing functionality
Acquiring SaaS can, in terms of functionality, be compared to buying software packages. There is a standard service that is provided and it has to fit in the intended processes in one way or another. The functionality of platform (PaaS) or infrastructure (IaaS) services is in an entirely different league than that of SaaS. The PaaS user group is to be found in the development department that uses the platform in the cloud for different environments and the department that rolls out the in-house software and keeps it operational. IaaS users are people that configure the infrastructure in the cloud, roll out software on the infrastructure, and keep the environment operational. This is traditionally a task for the data center.
Just as with testing packages, a number of different test objectives can be determined. For a start, a standard service usually does not fit exactly to the business processes that are going to use it. Flexible services can be configured to match the business processes. Customization is needed when no further configuration is possible and the match to the business processes is not yet completed. But where should the customization take place? Is the supplier going to expand the service or modify it for a certain customer or group of customers (for instance, for a community cloud)? Or is the customer going to make the changes, for instance, to connect the service to other systems? Test activities for both cases are required and are discussed in this chapter. In addition to changing the service, it may be decided to also change the business processes.
The software quality of a service is the supplier’s responsibility, and they have to go through a proper software development process to make sure customers do not suffer from defects in the software. As a result, it is not the primary task of customers to test software quality or service stability. However, because of the business risks, a customer still needs to perform an acceptance test on the service for critical parts of the business process. The outcome will weigh heavily on the selection decision.
There are a number of functional test objectives:
- Does the service fit the business processes?
- Do the business processes fit the service?
- Is the service quality sufficient (number of bugs)?
- Is the service sufficiently user friendly?
- Is the service configuration done correctly?
- Does potential supplier customization function properly?
- Does potential customer customization function properly?
- Do interfaces between the service and other systems work properly?
- Are platforms that are relevant to the customer being properly supported?
- Does everything work after changes (is there no regression)?
Checklist test measures ‘testing functionality’
- 5.6.1 Compatibility of service with business processes
- 5.6.2 Testing service quality
- 5.6.3 Testing user-friendliness
- User documentation
- 5.6.4 Testing interfaces to other systems
- Technical aspect
- Syntax aspects
- Semantical aspects
- Nonfunctional aspects
- 5.6.5 Testing service configuration; such as:
- Configuring authorizations
- (Possibly) configuring the workflow
- Configuring several general options (language, communication protocols, date formats)
- Configuring look and feel (company style, company colors, logos)
- Detailed configuration of the service (what is used, what is not, mandatory fields)
- 5.6.6 Customization by the supplier
- 5.6.7 Customization by the customer
- 5.6.8 Testing web services and API’s
- 5.6.9 Multi-platform testing
- 5.6.10 Testing of and testing with apps
- 5.6.11 Testing for working offline
- 5.6.12 Testing for regression
- 5.6.13 Creating a test basis
- Process flows
- Use cases
- Classification tree
- Authorization table
- Interface specifications (agreements)