Perceptual diff is one of the most important concepts and a tool that I have encountered in recent times for successful front end functional testing.
It provides visual regression testing and bridges the important gap in the automated testing for digital by focusing on functional and layout aspect of the page templates unlike most automated test which focuses only integration testing.
Due to large number of page templates in this digital applications, many times we have seen that changes in layout like CSS will work in one template but breaks different template due to some dependency between them. This is only caught in production by end users as manual testing just cannot find that difference if we have to deploy at speed. Thus doing regression testing of layout changes is extremely important part of the automated testing.
At a high level, concept is that you take one screenshot per page template before the release and take one more screenshot after the release and then compare them as image bitmap. Any differences should be analysed for typo, layout errors, wrong formatting, styling issues and so on. All dynamic pages should work as you have taken the screenshot with same set of data before and after, thus any change is due to build deployment.
Perceptual diff considerably increases confidence in the deployment process and thus helps in achieving shorter cycle and greater success.
There are various tools available like google perceptual diff tool (https://github.com/myint/perceptualdiff), Diffux (http://causes.github.io/blog/2014/02/19/visual-diffs-with-diffux) , GNU pdiff , QTP bitmap compare and the likes. Essential features needed in any chosen tool:
- Capture expected and actual image
- Compare specific section
- Deals with dynamic pages
- Result analysis
- Complexity of installation
- Knowledge available
- Time to implement the same
In summary Percetual diff is one of the most important concepts and tool that I have encountered in recent times. It provides visual regression testing and bridges the important gap in the automated testing for digital by focussing on functional and layout aspect of the page templates.
Please share any additional thoughts on other approaches used for front end functional automation.