As a research project to see why test automation of web applications is so hard, and why Selenium scripts seem to break so frequently, we ran an experiment to analyze the top 500 (ranked by Moz) web sites to see what patterns we would find that we could use to help automation engineers succeed more easily.
Unlike many other presentation technologies, web applications are written in a plain text markup language (HTML) that follows many conventions and rules, but browsers are so forgiving, that many of the rules are ignored since they are not enforced in any way. This causes a huge challenge for automation because the developers of the apps often use frameworks that make it hard for automated test scripts to reliably locate elements and perform user actions.
In this talk we will present some background on the problem, then detail our findings from the Moz 500 research experiment, showing how we crunched the data regarding which attributes, CSS, ARIA, IDs sites are using to see how best practices developed in theory will work in practice.
The talk provides suggestions and ideas derived from the data about how we can create more reliable tests. It discusses tools and techniques can be employed to make automation scripts easier to create and maintain. Finally, it suggests ways in which we can use a combination of Artificial Intelligence and Machine Learning to automate some of these solutions, and make tests self-healing.
