AI in the Software Testing Process

Pablo Ramirez

Project Manager

In this article:

First things first: what is AI?

The term artificial intelligence (AI) refers to the intelligence displayed by machines. However, the term is often applied when a machine mimics the cognitive functions associated with human capabilities, such as learning and problem-solving.

A world without borders

We live today in a world where almost nothing surprises us; the line between reality and science fiction gets thinner every day, and we often find ourselves not knowing if we are interacting with a human or a robot. Artificial intelligence and its rapid evolution, is with no doubt, one of the key disruptors of this era. 

For many, it all began with the arrival of smartphones in 2007 which allowed everyone to use information in a different, much simpler way and increasingly became a necessity – like it’s the case with smart assistants, facial recognition, or the GPS built into our devices. All this information becomes available through our smartphones, and it’s quickly processed through specific artificial intelligence algorithms to help us every day, often without even realising it.

And this not only applies to people, but also to large organisations:

  • Retail companies use AI to allow customers to virtually try on clothes in front of AI-powered mirrors;
  • Banks have integrated smart ATMs for customers to access all services that were previously carried out by a cashier or by a customer service agent; and
  • Large hotel chains use it to book their guests with bots based on intelligent IVR. 

Thanks to AI’s significant advances in this area, it is very difficult for guests to realise that they are being served by a bot.

For all this, in this digital age, organisations are constantly being pressured to find the right balance between cost and benefit, and to increase process efficiencies while still delivering an excellent experience to the end-user. Artificial Intelligence can play a crucial role in making this happen.

AI and Software Testing

This same pressure applies to the process of developing and testing new solutions and software – especially now, with the pandemic having accelerated the digital transformation for businesses worldwide. The biggest challenge in application testing has always been around not having enough time to test and develop the correct test methods and procedures. Still, organisations are required to run more tests, find incidents quicker and release products faster.

Is it possible to produce high-quality digital assets such as e-commerce platforms, supply chain systems, or engineering and management solutions without spending a lot of time and money ensuring their quality? In other words, can you test a system without testing it?

While that may seem like a dream, the software industry has already started talking about developing systems and processes with intelligent quality engineering (QE) capabilities. Time will tell to what extent testing AI systems become a reality; but it is already clear that it’s possible to achieve significant efficiencies and speed by applying these smart technologies; driving growth and improving results across industries.

However, while there are high expectations and some evidence of the application of supervised learning as a central part of machine learning (ML) to make quality engineering (QE) smarter, the adoption and application of these methodologies have not yet reached the maturity required to show visible results.

High expectations for the benefits of AI 

The latest World Quality Report 2020-2021 highlights that a large part of those surveyed are enthusiastic about the possibilities offered by artificial intelligence: almost 90% say AI testing and testing with AI are the largest growth areas within their companies. And 80% stated their intention to increase the number of AI-based trials and proofs of concept.

Use cases

The good news is that some companies are now working to change traditional models, leading the way in applying artificial intelligence to QE for unsupervised models, natural language processing (NLP) and computer vision technology. 

We are now witnessing the emergence of new use cases for this type of test. For example, real-time analysis of production events and application logs not only helps to perform an in-depth intelligent what-if analysis, but also helps to predict future quality and determine the necessary steps in development activities and proof. This improves testing by incorporating, in a smart way, actual usage patterns into the test and it also supports methods like ‘shift-left testing’.

Another use case that has gained ground is the use of AI to generate and manage data. For example, we can use this type of test to identify coverage gaps compared to real user experience patterns. The same can also be successfully applied to create synthetic data, for example, to comply with the rules of handling personal data (GDPR).

In summary, for organisations to reap the most benefit from AI in quality engineering, they will need their teams to strengthen their knowledge and experience of the tools, the overall Quality Assurance (QA) and IT strategy and the business objectives. It is a great opportunity, not only for companies, but also for QA people.

Quality control teams should have QA engineers with skills in data science, analytics and artificial intelligence. If necessary, they should collaborate with other parts of the organisation to acquire these skills.

The role of testers is not threatened by the development of this technology. On the contrary, it will be favoured since AI requires constant interaction of human testers with them.

Another important point: to train artificial intelligence, we need a good mix of input / output (which we call a training data set). So to work with modern software, we must choose this training dataset carefully, as artificial intelligence begins to learn from it and creates relationships based on what we deliver to it. Also, it is important to monitor how the AI ​​is learning. This is crucial to know how the software will be tested. Human participation in artificial intelligence training is therefore necessary. Last but not least is to make sure that, when working with artificial intelligence, the ethical, security, and privacy aspects of the software are not compromised. Again, we need humans for all this.


Although artificial intelligence continues to advance, it’s unlikely it will ever be able to imitate the human brain. People are the ultimate users of the applications and technology innovations so it must be considered that understanding, creativity and the human context are necessary characteristics to ensure high-quality products. 

In other words, manual testing remains essential; automation and artificial intelligence must complement each other. They are entirely different functions and should be used appropriately according to their respective advantages, rather than being compared.

Pablo Ramirez

Project Manager

In this article:

Related articles

Talk with one of our experts

If you're considering a new digital experience, whatever stage you're at in your journey, we'd love to talk.