Testing front end changes quickly and easily
In this post I will show you how to take lots of screenshots at lots of breakpoints, compare live sites to local changes, then use a diff tool to compare them all. The outcome, a quick and easy way to test front end changes. This is achieved by using a screen shot service, an image diff tool and a browser, screenshot as a service, PhantomJS and ChunkyPNG. You will also need to have NodeJS installed.
Having a large device lab is great, you can check changes against every browser at every resolution, though the downside is that you have a lot of manual checking which takes up time. Having a faster way to check all your changes without devices all over your desk can be a time saver, but also a simpler way to test changes.
The Tools
PhantomJS is a headless browser that allows you to test without the need for a UI. This is great for automating tests, such as taking lots of screenshots. The second part of the process involves taking 2 screenshots and comparing them using a diff tool, Chunkypng. This tool can output different visual cues as to how your changes have affected your site, the outputs we are going to cover are a bounding box and a colour map.
The process is dependant on you checking work locally against your current live site, in my case, my sandbox Vs the BBC news site.
I have missed one more part out, which is a free tool called screenshot as a service. This includes an app that uses Nodejs to host an internal PhantomJS server, as well as all the configs required to take advantage of PhantomJS.
Installation of tools
gem install chunky_png gem install phantomjs git clone https://github.com/fzaninotto/screenshot-as-a-service.git
The How
We take our 2 URLS and curl them using terminal. You can see below the different options I have included in the command, the localhost server address, the URL I want to screenshot, the max width of the browser, a delay to allow the page to load and the file name of the screen shot. What you can do here is set the width to loads of different sizes to gain lots of screenshots. There are additional options that you include, which can be found in the screenshot documentation. For RWD, this approach is probably going to help the most.
In the case below, I have added an L to indicate the live site. The second URL is my sandbox and will take the screenshot 5 seconds after the first.
curl http://localhost:3000/?url=m.live.bbc.co.uk/news\&width=320\&delay=2000\ > bbc1L.png sleep 5 curl http://localhost:3000/?url=http://pal.sandbox.dev.bbc.co.uk/news\&width=320\&delay=2000\ > bbc1.png
What you end up with are 2 images in the folder that are of your 2 URLs. That’s great, but the next step is really what we want to do, compare the 2 images automatically and output a single image to show where the page has differed. For that we use ChunkyPNG.
Special Sauce time
A simple Ruby script runs that compares the 2 files, then outputs the diff you want. My approach is to take 2 diffs, one which shows a bounding box around the area of the page that has changed, another shows a black page with changes highlighted in colour.
This method has been developed by a few different people, thanks to Jeff Kreefmeijer for a great blog post on the subject. This is good for skimming over the first set of images to see what’s changed, then looking at more detail with the colour map.
In my case, the script would be ruby compare.rb bbc1 bbc1L diff1
require 'chunky_png'
require 'rubygems'
include ChunkyPNG::Color
file1, file2, file3 = ARGV
images = [
ChunkyPNG::Image.from_file("#{file1}.png"),
ChunkyPNG::Image.from_file("#{file2}.png")
]
images.first.height.times do |y|
images.first.row(y).each_with_index do |pixel, x|
images.last[x,y] = rgb(
r(pixel) + r(images.last[x,y]) - 2 * [r(pixel), r(images.last[x,y])].min,
g(pixel) + g(images.last[x,y]) - 2 * [g(pixel), g(images.last[x,y])].min,
b(pixel) + b(images.last[x,y]) - 2 * [b(pixel), b(images.last[x,y])].min
)
end
end
images.last.save("#{file3}.png")
Further automation
In working out how to best approach this type of regressions testing, I first looked at existing tools, PhantomCSS was one of them. PhantomCSS does essentially what we have acheived here, though the way in which the image is captured is different with more screenshot variables available via curl. The processing of the image is also different, it is completed by js-imagediff, rather than using ChunkyPNG. PhantomCSS is a little simpler in setup, so it might be something you also consider instead of my method.
The automation benefits really come into play when writing longer scripts, here is a simple script that will take 6 screenshots at 3 different sizes, then compare them. This type of automation is going to be beneficial to those working on lots of types of pages. This is simply a .sh file that I run by using the following in terminal - sh compare.sh
curl http://localhost:3000/?url=m.live.bbc.co.uk/news\&width=320\&delay=2000\ > bbc1L.png sleep 5 curl http://localhost:3000/?url=http://pal.sandbox.dev.bbc.co.uk/news\&width=320\&delay=2000\ > bbc1.png sleep 5 curl http://localhost:3000/?url=m.live.bbc.co.uk/news\&width=360\&delay=2000\ > bbc2L.png sleep 5 curl http://localhost:3000/?url=http://pal.sandbox.dev.bbc.co.uk/news\&width=360\&delay=2000\ > bbc2.png sleep 5 curl http://localhost:3000/?url=m.live.bbc.co.uk/news\&width=400\&delay=2000\ > bbc3L.png sleep 5 curl http://localhost:3000/?url=http://pal.sandbox.dev.bbc.co.uk/news\&width=400\&delay=2000\ > bbc3.png sleep 5 ruby compare.rb bbc1 bbc1L diff1 echo "Diff1 complete" ruby compare.rb bbc2 bbc2L diff2 echo "Diff2 complete" ruby compare.rb bbc3 bbc3L diff3 echo "Diff3 complete"
Conclusion
Where you may spend a couple of hours testing 15 different mobile devices, you can now look at a small amount of diff images that will indicate the changes within minutes. For testing CSS changes, this is the closest you can get to automated tests.
The output you can see above is the front page of the BBC news site, I waited for a story change before running the second curl. This is a good example of differences, the image, the headline, the time stamp have all changed.
For BBC news, we have the front page, story pages with video, without video, correspondent pages, media asset pages and live event pages. That is six pages, at 8 different resolutions, with JS on and off, that totals 192 individual screenshots with 96 diff screenshots. If there are lots of front end changes, we can leave the bounding box output and just look at the colour map, this is simply because i know there will be changes.
The workflow of how you use diffs is really up to you, but the ability to have a single image showing differences between how your site looks now, with how it will look, is one of the best ways i’ve found to test front end changes.
