Testing as a Service – a new dimension of software testing

I have been working as a tester for many years and I often come across the need to justify the costs of testing. Inasmuch as performing simple automatic or manual tests has become a kind of standard, more complicated tests and activities are becoming hard to justify from an economic standpoint, so they are pushed to one side during projects. Of course, I am not saying that the situation always turns out like this; however, having years of experience in many smaller-scale projects, I have observed this kind of “craft” management attitude.

Such management of the application development process is based on several talented “tester-craftsmen” on whom the entire test process depends. This is not unequivocally wrong; however, it means that all test activities are undertaken by a group of QA specialists who provide manual, automation, security or performance tests, and manage all the tests. 

The main question is – does such a small team have all the required competences? Especially when you consider that, during the project, eventually there will probably come a time when highly specialized performance or security engineers and other experts will be needed, but having them employed on a permanent basis at our organization is inefficient in terms of cost-effectiveness.

Elevate Your Application Development

Our tailored Application Development services meet your unique business needs. Consult with Marek Czachorowski, Head of Data and AI Solutions, for expert guidance.

Schedule a meeting

Software testing – challenges

Very often during the software development process, it turns out that testers’ competences are insufficient for conducting tests, or the number of specialists is too small to manage a heavy workload. What can happen as a result? Due to recurring issues, the client focuses on analyzing problems or conducting the recruitment process, instead of managing standard tasks.

The recruitment process is time-consuming, hence it is not easy to employ a dozen senior testers each with a specific skill set that suits the prerequisites in terms of software. This may be problematic as we usually need assistance “as soon as possible”. What is more, we have to bear in mind that the onboarding process and training take time as well, not to mention the costs of downtime and having redundant testers once the workload gets lower.

Stoppages and “mountains” (surpluses) in the project

Another problem is the fact that the number and type of tasks during the project development phase can fluctuate. The assumption that three in-house employees will always have the same amount of work during the course of the project is just plain wrong. Stoppages and surpluses in projects are a totally natural phenomenon.

In the first case, we can assume that employees with nothing to do will undertake other activities which they don’t have time for during project development, such as putting things in order. At first glance, it makes sense – the only question is whether the activities left at the time are really necessary. It is much better to apply good practices that will not lead to situations where such actions are necessary. Of course, the issue of training and self-improvement remains, but I will leave this case to each company to deal with subjectively.

Much more cumbersome are these “mountains”: for example, with three testers working on a project, our resources are necessarily limited. Let’s assume that our test team was expecting to tackle tasks that had to be delivered. The team has made the necessary preparations, as well as all other activities. Unfortunately, the previous stage of the project preceding the testing stage ran long, and it turns out that the tests which were planned to last two weeks should be done within a week, and it is absolutely impossible to put the deadline back. Therefore, our three employees have to do the work of six people.

What can be done in this situation? We can bring test engineers across from less urgent projects if possible. Having people do overtime is another solution. However, in this case, the scale of overtime would be very large, far exceeding 50% of regular work time. This will not only increase costs, as overtime means higher rates, but will also lead to testers being overworked. Here we have to remember that an overworked tester will not work as efficiently as a rested employee. This means that even if our team tries to accomplish all of the tasks on time, this does not give us a guarantee of a suitable result.

Industrial, not craft, approach

Therefore, it is necessary to change the way of thinking about how we manage a test team, and look at the employees’ skill levels. A company that uses the craft model needs the most talented tester who has all the necessary skills to work on the appropriate level. However, in the real world, it is difficult to successfully implement such a model.

We have to remember that a specialist who has all the competencies from all specializations probably does not exist. Even a specialist with many years of experience only has the competences and skills gained from projects he has participated in. For example, if a test engineer has 15 years of experience and 10 years ago he was very good at testing, but has not done any testing since then, he doesn’t really have any valid level of competences at present.

In a traditional, craft approach, we only have access to the specific skills of one tester or an agile QA team, and not always when we need them.

More challenges in ensuring software quality

Another problem is that the challenges associated with IT project development are getting bigger, and the scope of its use is increasing. It is difficult for test engineers to be a specialist in every specialization. An engineer who is excellent at web testing will not necessarily be a specialist in Big Data testing. Of course, a good test engineer with basic knowledge should be able to cope with any project – but we have to ask ourselves questions about the effectiveness of his work.

One of the methods of solving this type of problem is the testing center of excellence – specialized testing departments which conduct tests as a service for other departments within the company. However, this solution will only be effective if the entire potential of such a department is actually used, e.g. if the company has a wide portfolio of projects that can be supported in this way. Otherwise, such a department will generate excess costs.

Smaller companies, or those that don’t have a sufficient number of diversified IT projects, can solve this problem by deciding on Software Testing as a Service (TaaS).

What is TaaS?

Testing as a Service is a model in which particular tasks are outsourced instead of hiring specific specialists. This means that the recipient does not make use of a limited pool of employee competences, but rather benefits from the cloud of their qualifications. This way, we avoid a situation in which a senior manual test engineer performs the automatic tests that require the skills of a junior automation tester, which unfortunately our manual tester does not have. This is definitely an inefficient use of the employee’s potential.

In this model, tests are provided on demand as and when required. However, tests should be scalable, have a high level of independence, and also not burden the customer with costs related to tools, hardware, software, employee training, or building software application testing environments. Using cloud-based tools at work is also common practice.

TaaS provider is able to deliver all types of testing: from manual to automated tests, performance, penetration, or even improvement of the test process, as well as conducting training for the client’s employees.

Benefits and key TaaS features

Thanks to the “cloud of competences” of the IT Partner, the client gains access to a pool of different qualifications and several experienced specialists with:

• unrestricted access to a large testing laboratory,
• the best testing tools on the market like Cucumber, Selenium, Serenity, Jenkins, TestComplete, SoapUI and many others,
• a team certified at various levels of expertise,
• working standards with the use of Tmap Next, ISTQB, TPI Next, ISO 29119, TMMi.

With such a service, the client is provided with access to specialists with grounded testing knowledge. An IT partner with a wide range of competences will be able to prepare for implementing the necessary changes by assigning a senior Quality Assurance team, which will be ready to build a tailored and secure application testing strategy or create an improvement plan in order to solve recurring problems. 

The partner will work closely with the client to create the organizational testing strategy and policy. But what are the benefits for the client? They will surely benefit from their partner’s mentoring, supervising, and training skills to guide their employees.

Software Testing as a Service vs. standard approach

Many companies outsource entire teams; however, it very often turns out that what they need is access to certain competencies during a particular period of time. With a standard approach, it is hard to maintain such a group of specialists all the time and justify these costs from an economic standpoint. 

This is where TaaS service comes in handy since it is flexible and available on demand. The client receives the whole range of test automation, non-functional testing and functional testing including regression tests, stress tests, acceptance tests, etc. In this model, only particular activities are outsourced to the IT partner, who specializes in simulating real testing environments according to the client’s requirements.

This model features the following attributes:

• on demand – it is used when needed,
• suited to specific business requirements,
• fully scalable,
• very high level of testing independence,
• no additional cost of tools and testing environments,
• cloud testing environments.

The delivery model of TaaS – how it works?

The form of the service should be as simple as possible for the client. The contract is signed before the details of the service are established. Subsequently, the client should provide all the documents, passwords and other artifacts necessary for implementing the testing process.

Theoretically, his role should end at this point. In practice, it looks a bit different, as Testing as a Service is still classic testing, so the service shouldn’t be performed without any contact between the client and the third-party service provider. Efficient communication and exchange of knowledge are a must so that the service provided is of the highest quality. The TaaS service is finalized by presenting the results in the form of a report.

The above scenario is quite obviously a simplification, but the idea is to provide relief for the recipient firm, to make the form of cooperation most convenient and “trouble-free”.

How do you get started using TaaS?

Each company’s specific preferences and requirements should lead to implementing an individual model of engagement with an IT partner. The client’s expectations in terms of the nature of the problems he is facing calls for a bespoke approach to the work of his service provider. It is very common for the client’s prerequisites to change during collaboration with the partner, which requires a distinctive approach. This is why searching for a service provider who offers a wide range of engagement models is recommended.

Which strategy will meet your requirements? For those clients who are still considering the implementation of Testing as a Service, I would propose beginning with 1-2 months of trial support for a single work package or separate tests. During that time, you can choose particular testing activities or support for a particular area, and decide afterward which strategy is adequate. As you can see, TaaS is a flexible, on-demand and adjusted range of testing services, which makes it easily scalable in terms of the evolution of cooperation.

Summary

The IT environment is changing and testing must change with it. It is not possible for the old model with a single “craftsman” test engineer to solve our problems. STaaS wants to be flexible, cost-effective, and provide the best solutions available for any company regardless of its size. It is a big challenge, but the final effect is worth it.