Software testing future in the era of AI

In the past few months, I have been reading a lot of contradicting articles on the AI, ones have

apprehension that machines trump humanity in their usefulness, others promising

that technology is here to work for us and not against us. Both extremities have their own

valid lines of argument, backed by numbers and statistics and a whole lot of technobabble,

ones you’ll find very difficult to counteract. But as I found myself piecing together both sides

of the argument, a question pressed on to me, what should I believe?

There are many issues which are related to the emergence of Artificial Intelligence. I have

been in software testing for more than 10 Years, so its natural for me to discuss the impact of

AI on software testing jobs. I am an optimist and forward-looking woman, but sometimes it

troubles me to envisage that AI is going to replace Software Tester in the near future. For me

like AI, the field of software testing is extremely broad Artificial Intelligence (AI) is

transforming the nature of almost everything which is connected to human life e.g.

employment, economy, communication, warfare, privacy, security, ethics, healthcare etc.

However, we are yet to see its evolution in long-term, whether it's leading humanity towards

making this planet a better place to live or a place which is full of disaster. For AI we are not

yet sure whether in the long-term positive effects will always keep outweighing the negative

effects and if that is not the case then we are in serious trouble. If we look around us, on the

one hand, we seem to embrace the change being brought by technology, be it smart home,

smart healthcare, Industry 4.0 or autonomous cars.

On the other hand, we often found ourselves protesting against the government in the

context of unemployment, taxes, privacy etc. As AI development is speeding up, more robots

or autonomous systems are being born and replacing the human labour. This is the current

situation; however, in long-term, results seem to get more interesting. Statistics say that 47%

of all employment opportunities will be occupied by machines within the next two decades.

For as long as there has been software, there has been software testing. Despite the rise of

other approaches such as formal verification and model checking, testing remains the primary

means of validation and verification (V&V) used in the software industry today. The tester

needs to be smarter than the programmer to find problems in the code.

Software testing is a process that constitutes a very fundamental aspect in the area of

development. Testing is fundamentally about inputs and expected outputs. Testing combines

lots of human and machine-generated data. Folks in testing often don't have much exposure

to AI, but that will change quickly, just like everyone else in the world is waking up to the

power of AI. So, Testing is a ripe field for applying AI.

Artificial Intelligence for Software Testing (AIST) is an emerging field aimed at the

development of AI systems to test software, methods to test AI systems, and ultimately

designing software that is capable of self-testing and self-healing.

The real value of human-powered testing is the creativity required to either identify problems

that are subjective or discover bugs that software engineers didn’t think of or weren’t able to

predict at the time of implementation.

In my experience, more than 70 percent of testing is repetitive. You’re often just checking

that things work the same way they did yesterday. This work is solvable by AI and automation.

That other 20 percent of a tester’s time today, the creative, questioning, reasoning part—that

is what people should really be doing, and that rarely happens in today’s fast-moving and

agile app teams.

I think in this context Robots means (AI-powered) automated testing tools or Testbots. In my

opinion, Testbots will definitely reduce the number of testers but not replace all of them.

Testbots will lift experienced testers to higher levels at which they will control the testbots

and give guidance to them (if needed) to achieve the best results. For mobile

applications, TestGrid.io has already unveiled a testing software that can functionally test

mobile applications without written scripts.

Let us examine the advantages of AI testing in answer to the shortcomings of manual testing.

Manual testing often relies on the tester role-playing as the end user to ensure correct

application behaviour. On the other hand, an AI approach to testing is built on the solutions

to the very problems that are inherent to manual testing and thrives on change.

Functional and Regression Testing are already automated. UI and aesthetic testing are harder

to automate, and manual tests are a better solution for that.

 However, human testers working alongside smart machines can certainly test modern

software in a better way that is done today. Here are four implications that AI has for software

testers:

I. Leave the exhaustive testing to AI. Leave tapping every button, inputting obvious valid and

invalid data into text fields, etc. to the machines.

II. Focus on the qualitative aspects of software testing that is specific to their specific app and

customer.

III. Focus on creative and business-specific test inputs and validations. Be more creative and think

of email address values that a machine with access to thousands of possible email test inputs

wouldn’t think to try. Verify that cultural- or domain-specific and expectations are met. Think

of test cases that will break the machine processing for your specific app (e.g., negative prices,

disconnecting the network at the worst possible time, or simulating possible errors).

IV. Record these human decisions in a way that later helps to train the bots. Schematized records

of input and outputs are better than English text descriptions in paragraph form.

V. I realized that generalized AI-powered bots could address some of the major pain points in

software testing today.

I. Problem 1—Performance: Improved app performance is the number one priority of

app teams today. They can’t improve what they can’t measure, and the best solutions

today measure performance in noisy production environments, depend on SDKs and

require teams to look at raw data and charts to figure out what is slow. Worse,

performance regressions are often caught weeks after the offending code change.

Solution: Automated test bots could test the performance of every action in an app,

many times, and catch regressions within minutes of each new build. Rather than
charts, the bots could take easily to understand pictures of the slowest part of your

app and show them to the app team.

Problem 2—Release Velocity: Every app team wants to move faster thanks to

competition, as well as adopt agile, lean and continuous build environments. Manual

testing isn’t fast enough anymore, and today’s test automation is expensive, slow, and

often breaks when you need it most.

Solution: Bots can generate 100 times the test coverage of most test teams. Even

better, with a little bit of AI mixed in, the bots could automatically discover new

features and test new behaviours. If the change in the app is too complex for a bot to

know it is a bug, it simply sends a before and after picture to a human to make the

bug-or-feature decision.

Problem 3—No Money, Talent, or Time: Many teams can’t afford legions of test

automation engineers or the infrastructure they need. Most teams can’t wait for six

to eighteen months for an automated test suite to be coded up and be running. Most

interestingly, there is far more demand for software test development engineers than

there are test engineers.

Solution: AI-powered bots could start basic testing of an app right away. Machines are

far less expensive than the cost to hire a team and to write and maintain basic test

code. Machines can also provide test coverage and execute in parallel, enabling all this

work to be done in just minutes.

Everyone has the problem. Current solutions are painful and very expensive, but with

changes to cloud computing, storage, and AI, the problem was finally tractable.
Whether testers will be replaced by AI bots as our field moves forward appears to be as

unclear as the new problem sets we’re beginning to encounter as testers. How will we as

testers adapt? What is it we need to do to stay ahead of the machine learning curve? How

will we continue to reduce the risk for our companies in an age where uncertainty is certain

to proliferate?

I believe it’s only a matter of time before AI and robotics start making an impact on the

software development world. We're already seeing some of the early adaptors of AI with test

automation companies like Appdiff, ReTest and Testim.

I predict that “writing software is a field ML will conquer before it will conquer testing.” The

advent of new technology(AI) strips many people of their livelihoods, what menial source of

earnings they have. But in the long run, it also improves the society as a whole and raises the

nation’s standard of living. I realize that sounds pragmatic, probably even a bit cold, but I’d

like to believe that the introduction of new technology does more good than harm.

By working in harmony with AI in the future, human testers will see the most interesting and

valued aspects of testing open up. Therefore, the best way forward is for humans and

machines to exist simultaneously, leaning on each other’s strengths.

So no worries if you have a good testing background and experience in test automation. With

the essential knowledge, you can control the Testbots and they will make your testing work

more enjoyable :)

AI won’t take over testing. But testing is going to get much harder as we introduce machine

learning into applications because we won’t know what the application is supposed to do in

all cases. With the most difficult problems, machine learning will be making choices based on
likelihood, not certainties. So, I can infer that testing job is about to change dramatically.

Every tester need to think: The machines are learning , are you?