Testing For Dummies

IE help from Microsoft - Modern.ie

Thu, 31 Jan 2013 13:59:00 +0000

Microsoft have released a series of tools aimed at better testing and support of Internet Explorer. http://www.modern.ie/

What Microsoft are offering is actually pretty cool, a 3 month trial to browser stacks, a remote testing VM. This allows you to connect to a browser running on a Windows OS using a particular version of IE, from IE6 to 10. The only concession is 3 fixed resolutions, 800x600, 1024x768 and 1280x1024. This tool will be a big help to those looking for a way not to have local VMs or spare computers. It costs $19/month for a single user if you like it after the 3 month trial.

Microsoft have also included a series of Virtual machines on several platforms for testing. You can also select different VM packages such as virtual box. http://www.modern.ie/en-us/virtualization-tools

The best feature available here is the IE report, a quick way to test your websites browser rating according to IE’s abilities. It outputs a list of areas you can improve your site such as vendor prefixes, browser plugins, compatibility modes and more.

Hopefully this will help those looking to better serve IE.

Testing front end changes quickly and easily

Mon, 10 Dec 2012 10:00:00 +0000

In this post I will show you how to take lots of screenshots at lots of breakpoints, compare live sites to local changes, then use a diff tool to compare them all. The outcome, a quick and easy way to test front end changes. This is achieved by using a screen shot service, an image diff tool and a browser, screenshot as a service, PhantomJS and ChunkyPNG. You will also need to have NodeJS installed.

Having a large device lab is great, you can check changes against every browser at every resolution, though the downside is that you have a lot of manual checking which takes up time. Having a faster way to check all your changes without devices all over your desk can be a time saver, but also a simpler way to test changes.

The Tools

PhantomJS is a headless browser that allows you to test without the need for a UI. This is great for automating tests, such as taking lots of screenshots. The second part of the process involves taking 2 screenshots and comparing them using a diff tool, Chunkypng. This tool can output different visual cues as to how your changes have affected your site, the outputs we are going to cover are a bounding box and a colour map.

The process is dependant on you checking work locally against your current live site, in my case, my sandbox Vs the BBC news site.

I have missed one more part out, which is a free tool called screenshot as a service. This includes an app that uses Nodejs to host an internal PhantomJS server, as well as all the configs required to take advantage of PhantomJS.

Installation of tools

gem install chunky_png
gem install phantomjs
git clone https://github.com/fzaninotto/screenshot-as-a-service.git

The How

We take our 2 URLS and curl them using terminal. You can see below the different options I have included in the command, the localhost server address, the URL I want to screenshot, the max width of the browser, a delay to allow the page to load and the file name of the screen shot. What you can do here is set the width to loads of different sizes to gain lots of screenshots. There are additional options that you include, which can be found in the screenshot documentation. For RWD, this approach is probably going to help the most.

In the case below, I have added an L to indicate the live site. The second URL is my sandbox and will take the screenshot 5 seconds after the first.

curl http://localhost:3000/?url=m.live.bbc.co.uk/news\&width=320\&delay=2000\ > bbc1L.png
sleep 5
curl http://localhost:3000/?url=http://pal.sandbox.dev.bbc.co.uk/news\&width=320\&delay=2000\ > bbc1.png

What you end up with are 2 images in the folder that are of your 2 URLs. That’s great, but the next step is really what we want to do, compare the 2 images automatically and output a single image to show where the page has differed. For that we use ChunkyPNG.

Special Sauce time

A simple Ruby script runs that compares the 2 files, then outputs the diff you want. My approach is to take 2 diffs, one which shows a bounding box around the area of the page that has changed, another shows a black page with changes highlighted in colour.

This method has been developed by a few different people, thanks to Jeff Kreefmeijer for a great blog post on the subject. This is good for skimming over the first set of images to see what’s changed, then looking at more detail with the colour map.

In my case, the script would be ruby compare.rb bbc1 bbc1L diff1

require 'chunky_png'
require 'rubygems'

include ChunkyPNG::Color

file1, file2, file3 = ARGV

images = [
  ChunkyPNG::Image.from_file("#{file1}.png"),
  ChunkyPNG::Image.from_file("#{file2}.png")
]

images.first.height.times do |y|
  images.first.row(y).each_with_index do |pixel, x|

    images.last[x,y] = rgb(
      r(pixel) + r(images.last[x,y]) - 2 * [r(pixel), r(images.last[x,y])].min,
      g(pixel) + g(images.last[x,y]) - 2 * [g(pixel), g(images.last[x,y])].min,
      b(pixel) + b(images.last[x,y]) - 2 * [b(pixel), b(images.last[x,y])].min
    )
  end

end

images.last.save("#{file3}.png")

Further automation

In working out how to best approach this type of regressions testing, I first looked at existing tools, PhantomCSS was one of them. PhantomCSS does essentially what we have acheived here, though the way in which the image is captured is different with more screenshot variables available via curl. The processing of the image is also different, it is completed by js-imagediff, rather than using ChunkyPNG. PhantomCSS is a little simpler in setup, so it might be something you also consider instead of my method.

The automation benefits really come into play when writing longer scripts, here is a simple script that will take 6 screenshots at 3 different sizes, then compare them. This type of automation is going to be beneficial to those working on lots of types of pages. This is simply a .sh file that I run by using the following in terminal - sh compare.sh

curl http://localhost:3000/?url=m.live.bbc.co.uk/news\&width=320\&delay=2000\ > bbc1L.png
sleep 5
curl http://localhost:3000/?url=http://pal.sandbox.dev.bbc.co.uk/news\&width=320\&delay=2000\ > bbc1.png
sleep 5
curl http://localhost:3000/?url=m.live.bbc.co.uk/news\&width=360\&delay=2000\ > bbc2L.png
sleep 5
curl http://localhost:3000/?url=http://pal.sandbox.dev.bbc.co.uk/news\&width=360\&delay=2000\ > bbc2.png
sleep 5
curl http://localhost:3000/?url=m.live.bbc.co.uk/news\&width=400\&delay=2000\ > bbc3L.png
sleep 5
curl http://localhost:3000/?url=http://pal.sandbox.dev.bbc.co.uk/news\&width=400\&delay=2000\ > bbc3.png
sleep 5
ruby compare.rb bbc1 bbc1L diff1
echo "Diff1 complete"
ruby compare.rb bbc2 bbc2L diff2
echo "Diff2 complete"
ruby compare.rb bbc3 bbc3L diff3
echo "Diff3 complete"

Conclusion

Where you may spend a couple of hours testing 15 different mobile devices, you can now look at a small amount of diff images that will indicate the changes within minutes. For testing CSS changes, this is the closest you can get to automated tests.

The output you can see above is the front page of the BBC news site, I waited for a story change before running the second curl. This is a good example of differences, the image, the headline, the time stamp have all changed.

For BBC news, we have the front page, story pages with video, without video, correspondent pages, media asset pages and live event pages. That is six pages, at 8 different resolutions, with JS on and off, that totals 192 individual screenshots with 96 diff screenshots. If there are lots of front end changes, we can leave the bounding box output and just look at the colour map, this is simply because i know there will be changes.

The workflow of how you use diffs is really up to you, but the ability to have a single image showing differences between how your site looks now, with how it will look, is one of the best ways i’ve found to test front end changes.

Viewing network traffic on any device

Thu, 15 Nov 2012 11:40:00 +0000

In this post, i will talk about how to view network traffic and the HAR (HTTP Archive) file. This is useful for doing several things, my initial intention was to discover network traffic from mobile devices. This was something i wanted to view in order to find out whether our vary headers and stats were correctly being logged, but there is also an interesting way to measure performance.

Requirements

The basic usage is to capture the HAR file, you will use tcpdump to capture TCP traffic and save it to PCAP file. The website above, you guessed it, shows you the PCAP file in it’s HAR form. This will allow you to view the HAR timeline, request headers, UA string etc.

Run this command in terminal. This takes the tcpdump from my bridged adaptor, the best way to do this is to share your wired conniption through your wireless and connect any mobile device to the new wireless access point.

sudo tcpdump -i bridge0 -n -s 0 -w nameoffile.pcap tcp or port 53

Once you have done all of you browsing, in some cases this will simply be loading a web page, hit control + C and the file is created in the directory you are in. The next step either requires you to have a local instance of the HAR viewer on your machine, or use the online version. Which ever you decide to use, the output will seem very familiar to most.

By using this tool, we can see how each device loads the page, where you could improve upon for certain devices and what, if any, are the issues with your website.

While this isn’t as good as Chrome for Android that shows you all of this in real time, for very old devices, this is the best way to inspect your traffic.

A week online with the Surface

Sat, 10 Nov 2012 22:04:00 +0000

For a week now, I have been using the Microsoft Surface, the latest gadget in the mobile world. A tablet computer in it’s basic form, but also a work device, featuring office applications and the Windows explorer. This device is not the first tablet that I have owned, I purchased a HP Touchpad at reduced cost over a year ago, though I stopped using it after a few months. Ultimately, it didn’t offer all the features I wanted, not to mention the terrible web browser, so thought now was the time to make a choice, the Microsoft Surface was my new tablet.

The Device

Unlike most tablets, the Surface is intended for use in landscape, but also to be used with a keyboard, the Touch or Type cover. This is a different type of interaction than any tablet I have used before, it is strange, but I switch between the touch screen and the keyboard seamlessly, without thinking and without looking at the keyboard. It offers the accuracy of a laptop, with the simplicity of a touchscreen.

The Surfaces large, 10 inch screen at 1366x768, is a little unwieldily in portrait mode, but offers the best landscape browsing experience. There have been a few articles detailing the usage of tablets and web browsing, with most people using a tablet in landscape mode. This is how I found myself using the tablet, though for long pages, portrait is a curiously nice experience.

The Operating System

The Surface uses Windows RT, which is not quite Windows, but offers a lot of the same functionality of the core OS, without the ability to install ANY Windows applications. Only Windows store applications that run in the Modern UI can be installed, so that offers a big limitation for people wanting to replace a laptop. It also offers a big limitation for software in general, so you are reliant on the Microsoft store for everything, problem number two. The store doesn’t have a lot of the basics, no Facebook, no Twitter, no Instgram…

This means ultimately the average person is going to use something a lot of people don’t really like to talk about, Internet Explorer, known to the average user as “The Internet”

The browser

Internet Explorer is no slouch, so don’t think I am at odds with it, in fact, I’m very impressed. Since 2003, I have been using Firefox or Chrome, so going back to IE is something that I initially dreaded, but the Modern UI took away all of the clutter that IE used to come with. The performance is excellent and the browser has some of the latest web standards. The browser also has a few different size options, full screen, mostly full screen, and sidebar style size. This is great for working on the desktop on a document, while having BBC News open with the latest live event rolling throughout the day.

In that one short paragraph we have unveiled one big problem with the Surface, it appears a lot of people weren’t really ready for such a device.

Issue number 1 - Is it a desktop?

One of my favourite new websites is Polygon.com, it is a great responsive website with some terrific gaming journalism. It is a beautiful look, with a simple navigation, sort of. On a touch screen phone you hit the navigation button to display the various sections, the first time I visited this site was on a phone, I think that is the first time I have done that on a mainstream site. In any case, I touched the section I wanted and away I went, though once in the office was confronted by an issue, my Surface couldn’t use the navigation. The screen is wide, the resolution is big, it runs Internet Explorer, it must be a desktop right? This means the hover style navigation for desktop users was being forced upon me :(

While using a navigation on my Android phone, pushing the navigation button didn’t open it and leave it there for me to peruse the sections, rather it opened a closed in a flash, leaving me unable to navigate. I was actually surprised that this style of navigation was used, as i continued to use the Surface, it seems clear many websites are bound to the idea that if it is IE10 using your site, the user must be using a keyboard and mouse input.

This is actually a problem with many sites, the fact is, the device shouldn’t matter, therefor the input method should be universal, but this type of device is so new, it blurs the lines between what is a desktop and what is a tablet. Does that show that some websites are only building for phone, tablet and desktop, with devices slipping through the cracks…Maybe, but it should be a good example that IE 10 is not a desktop computer, it is a browser that can run on a host of different devices with different inputs.

Issue number 2 - The sidebar browser

This is a great idea from Microsoft, multi tasking in terms of a mobile OS is actually quite a difficult thing to grasp for some, so putting two things on one screen is a neat idea, something to quickly glance at. This issue has been summed up quite nicely by Tim Kadlec, but many websites simply haven’t implemented the @-ms-viewport rule, thus leaving Windows 8 users high and dry. My concern here is that many websites simply won’t bother to do this, leaving this potentially useful way of browsing to those who have the time and availability to test.

Closing thoughts

After using my Surface for the past week or so, it has shown me first, how bad most Android tablets are, but secondly, that navigating the web is hard when something thinks you are a desktop computer. Imagine being at a high school reunion with the wrong name tag on, if people don’t remember your face, it can be a pretty awkward experience, which is how i found the Surface while using the web. In time, things will change, but in the mean time, the web is going to be a interesting experience for Microsoft tablet users.

IE 10 on Windows8 - A few Windows tips

Sun, 28 Oct 2012 15:01:00 +0000

With Windows 8 now out, the final release of IE 10, both Desktop and Modern UI(formally Metro) browsers. They are the same, but have very different user interfaces, one touch based, the other a conventional Desktop browser. The RT verions, this is not Windows 8 RT, rather a seperate OS callled Windows RT, has an ARM implementation of the browser to.

The best devices to get in order to test IE 10 on are probably the Samsung Ativ or the Microsoft Surface. Both available now.

In testing out the browser, there are a few things to help you in terms of gestures and shortcut keys.

Touch in IE 10:

Swipe down from top of screen to show tabs and URL bar
Swipe left from off the right side of screen to show the Charms menu for IE settings
Swipe right from off the left side of screen to show other apps
Swipe right from off the left side of screen to drag and pin another app.

The app pinning is quite cool because you can have your favourite site open while you do work, such as BBC News :)

Keyboard Shortcuts

This list is available from the Microsoft site, as is other information about Internet Explorer 10. This is the complete list as most of you are either using on windows machine, or putting Win8 into a VM for testing, so other keyboard short cuts might help. On OSX, this would just be right CMD key and any of the keys if you use Virtual Box.

Windows key: Switch between Modern Desktop Start screen and the last accessed application
Windows key + C: Access the charms bar
Windows key + Tab: Access the Modern Desktop Taskbar
Windows key + I: Access the Settings charm
Windows key + H: Access the Share charm
Windows key + K: Access the Devices charm
Windows key + Q: Access the Apps Search screen
Windows key + F: Access the Files Search screen
Windows key + W: Access the Settings Search screen
Windows key + P: Access the Second Screen bar
Windows key + Z: Brings up the App Bar when you have a Modern Desktop App running
Windows key + X: Access the Windows Tools Menu
Windows key + O: Lock screen orientation
Windows key + . : Move the screen split to the right
Windows key + Shift + . : Move the screen split to the left
Windows key + V: View all active Toasts/Notifications
Windows key + Shift + V: View all active Toasts/Notifications in reverse order
Windows key + PrtScn: Takes a screenshot of the screen and automatically saves it in the Pictures folder as Screenshot
Windows key + Enter: Launch Narrator
Windows key + E: Open Computer
Windows key + R: Open the Run dialog box
Windows key + U: Open Ease of Access Center
Windows key + Ctrl + F: Open Find Computers dialog box
Windows key + Pause/Break: Open the System page
Windows key + 1..10: Launch a program pinned on the Taskbar in the position indicated by the number
Windows key + Shift + 1..10: Launch a new instance of a program pinned on the Taskbar in the position indicated by the number
Windows key + Ctrl + 1..10: Access the last active instance of a program pinned on the Taskbar in the position indicated by the number
Windows key + Alt + 1..10: Access the Jump List of a program pinned on the Taskbar in the position indicated by the number
Windows key + B: Select the first item in the Notification Area and then use the arrow keys to cycle through the items Press Enter to open the selected item
Windows key + Ctrl + B: Access the program that is displaying a message in the Notification Area
Windows key + T: Cycle through the items on the Taskbar
Windows key + M: Minimize all windows
Windows key + Shift + M: Restore all minimized windows
Windows key + D: Show/Hide Desktop (minimize/restore all windows)
Windows key + L: Lock computer
Windows key + Up Arrow: Maximize current window
Windows key + Down Arrow: Minimize/restore current window
Windows key + Home: Minimize all but the current window
Windows key + Left Arrow: Tile window on the left side of the screen
Windows key + Right Arrow: Tile window on the right side of the screen
Windows key + Shift + Up Arrow: Extend current window from the top to the bottom of the screen
Windows key + Shift + Left/Right Arrow: Move the current window from one monitor to the next
Windows key + F1: Launch Windows Help and Support

PageUp: Scroll forward on the Modern Desktop Start screen
PageDown: Scroll backward on the Modern Desktop Start screen
Esc: Close a charm
Ctrl + Esc: Switch between Modern Desktop Start screen and the last accessed application
Ctrl + Mouse scroll wheel: Activate the Semantic Zoom on the Modern Desktop screen

Alt: Display a hidden Menu Bar
Alt + D: Select the Address Bar
Alt + P: Display the Preview Pane in Windows Explorer
Alt + Tab: Cycle forward through open windows
Alt + Shift + Tab: Cycle backward through open windows
Alt + F: Close the current window Open the Shut Down Windows dialog box from the Desktop
Alt + Spacebar: Access the Shortcut menu for current window
Alt + Esc: Cycle between open programs in the order that they were opened
Alt + Enter: Open the Properties dialog box of the selected item
Alt + PrtScn: Take a screen shot of the active Window and place it in the clipboard
Alt + Up Arrow: Move up one folder level in Windows Explorer (Like the Up Arrow in XP)
Alt + Left Arrow: Display the previous folder
Alt + Right Arrow: Display the next folder
Shift + Insert: CD/DVD Load CD/DVD without triggering Autoplay or Autorun
Shift + Delete: Permanently delete the item (rather than sending it to the Recycle Bin)
Shift + F6: Cycle backward through elements in a window or dialog box
Shift + F10: Access the context menu for the selected item
Shift + Tab: Cycle backward through elements in a window or dialog box
Shift + Click: Select a consecutive group of items
Shift + Click on a Taskbar button: Launch a new instance of a program
Shift + Right-click on a Taskbar button: Access the context menu for the selected item
Ctrl + A: Select all items
Ctrl + C: Copy the selected item
Ctrl + X: Cut the selected item
Ctrl + V: Paste the selected item
Ctrl + D: Delete selected item
Ctrl + Z: Undo an action
Ctrl + Y: Redo an action
Ctrl + N: Open a new window in Windows Explorer
Ctrl + W: Close current window in Windows Explorer
Ctrl + E: Select the Search box in the upper right corner of a window
Ctrl + Shift + N: Create new folder
Ctrl + Shift + Esc: Open the Windows Task Manager
Ctrl + Alt + Tab: Use arrow keys to cycle through open windows
Ctrl + Alt + Delete: Access the Windows Security screen
Ctrl + Click: Select multiple individual items
Ctrl + Click and drag an item: Copies that item in the same folder
Ctrl + Shift + Click and drag an item: Creates a shortcut for that item in the same folder
Ctrl + Tab: Move forward through tabs
Ctrl + Shift + Tab: Move backward through tabs
Ctrl + Shift + Click on a Taskbar button: Launch a new instance of a program as an Administrator
Ctrl + Click on a grouped Taskbar button: Cycle through the instances of a program in the group
F1: Display Help
F2: Rename a file
F3: Open Search
F4: Display the Address Bar list
F5: Refresh display
F6: Cycle forward through elements in a window or dialog box
F7: Display command history in a Command Prompt
F10: Display hidden Menu Bar
F11: Toggle full screen display
Tab: Cycle forward through elements in a window or dialog box
PrtScn: Take a screen shot of the entire screen and place it in the clipboard
Home: Move to the top of the active window
End: Move to the bottom of the active window
Delete: Delete the selected item
Backspace: Display the previous folder in Windows Explorer Move up one folder level in Open or Save dialog box
Esc: Close a dialog box
Num Lock Enabled + Plus (+): Display the contents of the selected folder
Num Lock Enabled + Minus (-): Collapse the selected folder
Num Lock Enabled + Asterisk (*): Expand all subfolders under the selected folder

Keep up to date!

Wed, 12 Sep 2012 12:54:49 +0100

Over the last week or so, Nokia have been pushing updates to lots of phones, these are running the latest versions of Symbian, S60 V10 or Symbian Belle. Although the support list for this version of the OS for devices is small, the main ones are covered. These phones now have access to the latest version of the Symbian browser, Nokia Belle Refresh for Nokia N8, Nokia E7, Nokia C7, Nokia C6….

What this means for us is that the new version has enough support to cut the mustard, in this case the N8 loads the smartphone version of the website. This change of browser is something we dont see often, but when it happens, it opens up a lot of possibilities for users, but also exposes some issues. As you can see in the image, the font embedding support is spotty, so worth testing all aspects of your site for other issues.

Ultimately, this is a good example of why you should be checking phones for updates all the time, ensuring you are testing the latest releases.

Colours in Android

Wed, 05 Sep 2012 16:12:15 +0100

The interesting thing about Android is it can sometimes behave differently, even on the same hardware. Here is a video playing on 2 Galaxy Nexus phones, the top is using Android 4.1(Jellybean), the bottom is using Android 4.0(ICS). When both phones are using the same auto brightness option, the phone will give a slightly different colour, note the individuals jacket. Nothing you can do here as many different phones have very different brightness options, but something to keep an eye on when testing. Hopefully the user will never notice….

No More OS4

Mon, 30 Jul 2012 15:37:00 +0100

The end has come for our little Blackberry OS4 device. Eight or nine months ago it was still relevant, it had a solid user base which was slowly shrinking. However, the pressure for Blackberry juicers to upgrade seems to have become too much, mostly bowing to Android and iOS.

While many of you will have given up testing on OS4, or perhaps not even started….It remained a more highly used OS than Windows Phone 7 until the last couple of months, so ensuring this user group remained happy was still important to me.

Finally though, this 6 year old OS with terrible browser support is nearly dead. It joins the ranks of webOS, Meego, Windows Mobile, Android 1.x and early versions of iOS. We salute your Blackberry OS4, the first real “smartphone” OS is put to bed.

Here is a question, are you still supporting BB OS4?

WEb INspector REmote

Tue, 17 Jul 2012 14:25:00 +0100

Weinre, a WEb INspector REmote tool for webkit browsers. It has been included in some tools such as the Adobe Shadow application, though this requires the device to have the application and the computer to have the app….and the browser to have the plugin.

Weinre is a more difficult set up, but once down, allows for a more immediate benefit. If you were only looking to use the inspector tool of an Adobe Shadow type application, then using basic weinre is probably best for you.

Firstly, here is the page where you will lots of information, guides, links, support etc. Check it out and have a read to get the most out of weinre

http://people.apache.org/~pmuellr/weinre/docs/latest/Home.html

Installation

Weinre can be installed anywhere, so just create a folder in your preferred location. I should say that these instruction are for OSX, Windows users should refer to the manual.

1. You first need to install node.js here http://nodejs.org/#download

2. Next you should install npm (package manager)

sudo curl http://npmjs.org/install.sh | sudo clean=yes sh

You dont have to run this full command, it just makes it easier in case you have conflicted installs or permissions.

3. Once you have that, create a folder to store wierne and cd to it, then run the following command in terminal

npm install weinre

4. You can now start the server with the following command. The main thing here is to ensure you have the right path. My folder is on my desktop so the command is

node node_modules/weinre/weinre

5. This is now running at localhost:8080, so you can have a look now, but we havent got anything going, so we can add an IP address to allow devices to access the server. We can run the following command

node node_modules/weinre/weinre --boundHost 192.168.1.78 -all-

It really depends on how you are going to work. I will talk about how this can benefit your team later

6. You may have decided not to do step 5, so you want to make weinre work, adding the following to your page will allow you to inspect.

script src="http://192.168.1.78:8080/target/target-script-min.js#anonymous">

Put it into the body and then save the file.

7. You can now browser to your page and the inspector should be working. Just visit either your local host or IP forwarded address, (localhost:8080) and click debug client user interface link. This is compatible with Android browser, Chrome, iOS Safari, webOS, Dolphin, Blackberries and more.

Summary

The benefits are that you dont need to rely on applications or support for this tool to work. It allows you simple access to debug web pages on any web kit browser. Apart from Opera and Windows Phone, it nearly gives you full coverage over smart phones. I would say the biggest benefit here is support for other browsers based on webkit but not native, which Adobe shadow uses. The next big plus is that OS’s like Blackberry and QNX, webkit browsers, are also supported. So the potential for checking browsers and editing the page on the device is great. Although you could use Ripple, a Blackberry emulator tool, to inspect browsers, there is nothing like the live experience on a device.

If you are the kind of person who wants to share your work and have other people look at it, check it, change it, you can use the IP address technique i mentioned in the guide to allow them to do this. If you use this method, the debug tools will be accessible from any computer on your local network, meaning only 1 computer needs to run the services. http://192.168.1.78:8080/ If this was an IP address of the server, type this into Chrome and you will be shown the control panel of the weinre, then clicking on the client user interface will bring up the Chrome tools. If you are working with branches and someone notices a bug, being able to fix it right there and then on any webkit device could help you speed up release.

While im sure many of you have used this tool in the past, or have heard of it, i think this is a better way to work than use Adobe shadow for inspecting. Shadow will speed up time looking at pages for sure, but it needs total device support to make it a really powerful tool, as it stands, using the normal weinre is the best way to go.

How long do mobile browsers last?

Tue, 10 Jul 2012 17:01:00 +0100

There have been a lot of articles recently that are talking about when to stop/drop support for IE, specifically IE8. This is a big deal for those who are stuck on Windows XP or Vista, but don’t have the option to update. So i started looking at what people will do with mobile browsers.

The likes of Meego, WebOS and QNX have come and mostly fizzled away. In some cases, you may decided not to even test against the device, let alone support it. We also have to deal with browsers that are not fully baked yet, Windows phone for example. Finally, the choice of when to drop support for older browsers, how long do you wait, what is the tipping point?

In a lot of ways, it will depend on your market and when you live, but i think any phone that you can into a shop and buy should be in your first line of supported devices. This may cause you a little amount of pain as you will probably dealing with at least 1 or 2 crap phones. However, as Android and iOS move on, the browser support gets a lot better, so the days of some Android browsers are numbered.

What is on the way out?

A lot of what I am about to say is speculation, it is based somewhat in trends that we have seen, what our stats indicate, research on sales figures and market data. We will look at 3 main browsers, Android 2.1, Windows Phone 7.5 and iOS. With IE8 still in the game, we will look at some mobile browsers that hanging around, but which may be reaching the end of there lives.

Android 2.1

From what we have seen from our stats, Android 2.1 is nearly gone, with most of the Android users on 2.2/3, which reflects the global stats.

When looking at Chrome and Firefox, the version a lot of people are now supporting is the latest one, as the auto update feature takes over. Android is essentially the IE of the Android world, the versions are tied to the OS version and users may not want to update the OS.

What I predict is a lot of people dropping support and not testing Android 2.1 after it hits 3 years old, October 2012, this is for a couple of reasons. Firstly, most phones are sold on 2-year contracts, with some 2.1 phones on the shelves for sale in late 2010, this prolonging the support. Also, some Android OEMs will not have launched a phone until they got the source code and updated it, which means a delay in 2.2 adoptions. Secondly, Android 2.1 users will be tempted by iOS 6, Android 4.1, Windows Phone 8 and maybe BB10 in the next few months.

Windows Phone 7.5

Windows Phone 7 hasn’t really grown to a big user base, so should you support Windows phone 7 and 8? If you take a look at the BBC news site, m.bbc.co.uk/news, on a Windows Phone, you will see our arrows are missing. Thanks to a post on ubelly, downloadable fonts, as well as other features, are missing from WP7, which is really annoying.

When WP8 comes out, it hopefully be a fuller version of IE10, so much better support than most other browsers today, but given how small the WP7 market is, we will most likely support WP8 straight away, with a more relaxed approach to WP7. As always, we will look at our stats, but given how long WP7 has been out, it seems clear that MS is putting it’s money on WP8 and it’s shiny new browser. WP7 should be put to bed in 6-8 months, knowing it helped a better browser on a better OS. The main downside of better support from WP8 is not upgrade path for existing WP7 users, which means you will be forced to support both, unless you cut the cord early and chose WP8.

Apple iOS

iOS has a 80% change in versions once a new one is released, which bucks the trend in the rest of industry. This is good for support as most of the user base are on either 4.3 or above, given excellent support. We test back to 4.0 though, as well as supporting versions in between. Anything before 4.3 doesn’t get support by our media player, which is most likely where i see our version testing moving to when iOS 6 comes out. I don’t know how many of you support below 5 still, it might be good to know, but testing against a version of iOS with just a few percent market share may start having no return in visitors.

Summary

What i see happening in the next few months is WP7.5, Android 2.1 and iOS 4 being made redundant, this will free up time testing for more modern browsers.

Based on the fact Android 2.3 phones are still on sale, with only the second Android 4.0 phone coming out in April of this year, will we be supporting Android 2.3 in 2014, 4 years after in launched? Will mobile browsers start auto updating independant of the OS? Will we see the browser move into the app arena like Google have done with Chrome on Android?

I didn’t really want to finish on a question, but if you have any for me, or how the BBC builds our site, you can tweet me and our team @davey_blooman @danscotton @kaelig @tmaslen @jcleveley @AndrewVos

Ten things to remember when testing RWD

Mon, 02 Jul 2012 15:37:00 +0100

A few things to remember when doing a bit of testing.

10 - phones are bad

A lot of phones are slow, have lots of bloatware and use the browser as a way to search Google, rather than look at your beautiful website. Symbian, Android, BBOS and Windows phone can be put onto very slow hardware, so buying that hardware is essential. More importantly though, if you only test on high end android and iphones, you will probably not realise the world of pain you are causing to bad phone users. Remember, phones are bad.

9 - browsers are bad

Older browsers may cause you a headache, so while Android 2.1 and iOS 4.0 are fading away, these two naughty browsers still need attention.

8 - rotate that device

It seems that most people use a phone in portrait unless they are playing games or watching movies. But there can be issues when using a web page in lanscape, layout changes, column placement, text alignment. It is worth spending some time on this, even if you feel most people wont interact with your website this way.

7 - android can catch you out

An android phone can be a well made, well designed customised experience, it can also be a load of bloatware and unnecessary browser tweaks. If you can buy multiple phones, the less popular phones, differentiating on software and not hardware can sometimes make a web page look odd, using single column mode by default for example.

6 - blackberry users still exist

Despite the doom and gloom of RIM, the blackberry maker, users will be around for a while yet. For that reason, we always test fully on all blackberry phones and OS versions. So when testing, remember, blackberry users still exist.

5 - loading times are good

On a 2G connection, which in a lot of areas this is still the case, it can take time to load the page and interact with it. I loaded a few websites, itv news, BBC news, the Guardian and Boston Globe on a 2G connection. BBC news and the Guardian were very quick, with itv also loading fast. The Boston Globe was actually slower than the others, with headlines and images taking longer to load in. Try out some sites for yourself.

4 - If you think it is complicated, it probably is

This is something that user testing will have to prove for you, but it can very useful. Testing your website with users from off the street can show you what things they look for in a page, how they navigate, what the steps are for them to get to content. If you learn these things, it will help you to identify if your page is going the wrong route in terms of user interaction or design.

3 - data is limited

According to recent reports, web pages are getting bigger. Mobile operators are still making everyone pay through the nose for data, so if a use visits a 1MB page 15 times a day, every day, it doesn’t leave much room for twitter or Facebook. Our maximum page weight target is 70k.

2 - everything isn’t an iphone

We test the iphone the same amount as every other phone we have defined as our core user base. Although it may seem like the phone that everyone has, it isn’t the number one browser in our stats, is it on yours?

1 - does it work on everything?

You might say this is a pillar of RWD, but does your page work on any device at any resolution, at any pixel ratio and still look good with all the functionality of non RWD page? Not sure if you want to answer that one, but the more you test, the more devices you test on and the wider browser support you employ, will hopefully lead to this goal.

Devices & Pixel Ratios

Thu, 28 Jun 2012 16:37:00 +0100

The idea of a 720p display on a phone is the norm, with 1080p probably coming over the next year or 2. With other devices like the iPad and iPhone with huge pixel counts, the use of images that exploit screens that have a pixel ratio of 2 is something everyone should look into doing.

In my device selection post a few months back, i only had 2 HD phones, no i am not going to use the term RETINA as a standard. These 2 phones were the iPhone and the Galaxy nexus, both had pixel ratios of 2. Since then, tablets and phones have appeared with similar specifications and more, such as “The New iPad”. In terms of device selection, if you have no plans to support HD images, then this is slightly less important, but for those that are, device selection can be slightly tricky.

The Samsung Galaxy Tab 7.7 for example, has a 1280x800 screen, yet uses a pixel ratio of 1. More content, but lack of HD images, so one to avoid. The iPad 3, clearly the favourite, but the big thing coming out is the Nexus device from Google. This device will be sold everywhere, use Chrome as the default browser and have a PR of 2, giving a good device to test those lovely images on.

For phones, the G Nexus, iPhone, HTC One X, Galaxy S3 are all good choices, though the iPhone has the highest PPI, followed closely but the Galaxy Nexus, so they could be your 2 bets choices.

Below we have a couple of images showing the difference in News Banner, this is a version of the site that has the enhancement and a version that doesn’t, the second image is the HD image.

If you are going to make you images HD, then picking the right device is important to test on, the pixel ratio is ultimately set by the manufacturer of the device, so high resolutions screens are not always going to be supported. For hybrid devices with strange ratios, it might be worth looking at your site traffic and seeing which devices are going to improve from getting HD images, only time what tell what frankenstein creations will be hitting the market soon with Windows 8 and Jelly Bean.

Landscape Changes

Thu, 21 Jun 2012 11:06:14 +0100

Here we have the web page in landscape view. The latest version is on the right, the older version on the left. The layout has changed in order to use the available screen width while keeping the larger image and maintaining the look and feel of the portrait layout

Before and After

Wed, 20 Jun 2012 14:37:36 +0100

A side by side of the live version of the site, right, and an early version left.

Tools

Mon, 18 Jun 2012 14:50:00 +0100

In a recent blog post i talked about what devices the BBC News team use for testing our responsive mobile site, in this post, i will talk about some of the tools you can use on these devices. This post will talk mostly about remote debugging, using Shadow and Chrome USB, but also front end work with Ripple and Proxylocal.

Chrome USB Debugging - https://developers.google.com/chrome/mobile/docs/debugging

Although Chrome for Android is only available on 4.0 (Ice Cream Sandwich), the options USB debugging and testing give you make it a valuable tool. The setup is very simple, the Android SDK and ADB(Android Device Bridge) need to be downloaded, PATH set, then load Chrome and you are away. You will need to run a terminal command to connect the device, but it is a near plug and play setup.

With Chrome loaded on the page you want to look at, Chrome desktop can then connect by using localhost:9222, this will then present the familiar inspector we have come to know and love.

So, just like the normal version of Chrome, you can edit your page with the phone updating instantly, allow for a real world version of you website, live on a device. This feature can help you develop around issues with other browsers, by finding solutions and experimenting on Chrome, you can quickly assert whether a change will work.

The other tabs in Chrome Inspector will allow you to test areas where you need reassurance, but more importantly, if real world interaction is correct. The stats for BBC use an image to identify Javascript interactions that we want to track. Switching between the 2 tabs on the front page, using the sections menu, wanting to read more live event comments, all of these creating a new response to track. The network tab in the inspector is where we can easily look at the images and requests being made, thereby ensuring the stats are being made available to be recorded.

Of course, this is just a snap shot of the possibilities of Chrome inspector. The excellent thing that Google have provide us here, is the chance to do the same desktop tasks on mobiles.

Ripple

The next tool that is still in beta is from RIM, called Ripple. It does what a lot of other responsive tools do, but makes device selection much easier. It also allows for other options like GPS and stats like PPI. One thing to note is that you can inspect the phone as you would a web page, so you can change things on the fly, good for front end stuff.

It is in beta right now, but has a lot of potential. If RIM implemented some updates like custom headers, adjustable browser reporting size and screen shot options, it could be a great testing platform. It also means you dont have to use the ridiculous Blackberry emulators anymore.

Proxylocal - http://proxylocal.com/

Something that we have been using extensively over the last couple of months is a tool to make a sandbox public for testing. Proxylocal is a simple install which allows you to take your latest build on your sandbox, or pull branches down from others, and test the functionality against devices and different screen sizes. We have found testing against our live site with early builds finds the issues and bugs much quicker than or standard build process. Although testing on a sandbox is not ideal, you may find inconsistencies, it will give you an insight into features and how they will look, which may reduce the back and forth between the team.

Adobe Shadow - http://labs.adobe.com/technologies/shadow/

Adobe Shadow is a tool that recently got an update, this is something that is useful for large scale testing of Android and iOS devices. You essentially sync up the devices to your browser via the shadow tool. This means when you visit a new page or website, the phones will follow you and display the same page. This is probably the best tool for people wanting to test multiple features or layouts across a large amount of devices at the same time. The tool allows you to refresh all the devices remotely, as well as take screenshots of the device, although it can be a little buggy.

The other feature is remote inspection, this will allow you to use weinre to inspect the devices browser, something that will help you test, but also change things on the fly to see if you decisions are correct. The network tab is not usable unfortunately, so backend services still have to be checked manually, but for front end devs, a very useful tool.

The image shows a device connected to shadow, with the control page and the weinre inspector open.

Is One Blackberry Enough?

Tue, 03 Apr 2012 16:33:00 +0100

In this post, we will look at why testing more than one Blackberry version or device may be the better way to go.

Blackberry OS

Blackberry phones, some of us love them, a lot of people don’t, but they are important to your development and testing. The Blackberry user base is quite large in the UK, but also has a solid global audience, it is mostly a non touch device group with limited screen resolutions and size. The choice to make an app for Blackberries is also a tough one for many companies, so your responsive website may have an edge in this market.

In Pictures

The picture above shows 3 different versions of Blackberry OS(BBOS), from 4 to 6. From the left, Sharon is OS4, Kenny is OS5, Len is OS6 and the final devices is also OS6. What these devices represent is the most popular Blackberries that are used on the site, but also the different versions that look to test against. OS7 has had a small impact so far, but as it picks up we will move to test it more.

What is also very important to take from the picture is how the different screen sizes and resolutions affect the end result of loading a webpage. You can also see how the page is different based upon the version, smartphone or featurephone. The OS5 device has a good screen width, 480px(reports as 465x304), so to use the extra space, the text wraps beside the image, rather than using the single larger top image. This is different to the OS4 featurephone, which uses the standard design with the top image. The OS4 device has a 360px wide screen, but a reasonable 305x224 reporting rate, which although low, means that you expect a certain level of content on the screen. When testing, it is important to have a eye for the detail, looking at whether you agree with the layout, or if you think it needs changing.

The Smartphones are quite different in the look and feel, Len, a 360px (reports as just 320x167)screen is actually receiving a much different experience. The image and font are the obvious two things, producing a smartphone experience on a very low resolution device.

Len is the more budget Blackberry, it is part of the curve range, which also feature lower resolution displays. While you may want to focus on the lovely Bold 9900, the budget range are the more popular. If you are going to buy Blackberries to test against, i would suggest getting a low resolution device, this will help ensure that users with OS6, which features a webkit browser, will get a satisfactory experience. If the Nokia N95 was the lowest horizontal width screen, the Blackberry 9300 is our lowest vertical, which is why we chose it for testing against our site. In a way, it does fall outside of what some would call an usable experience, especially for a smart phone, but it has to be considered.

I think there should be a prudance decision made on whether this type of device affects your development, but also whether certain features are required to look the same as on an iPhone. If you really don’t think Blackberries are a potential market, then sticking to the more common Bold series may be better, offering a more usable experience with higher reporting rates.

Knowing the difference

As we have discussed, unlike iOS and to some extent, high end Android phones, Blackberries are limited by the vertical height of the viewport, so you may choose to design around this to get more content on the page. This is seen on the Blackberry called Kenny, which uses the extra width to add the summary text next to the top image, but other elements like menus could be used in this space. This has a counter effect though, with more than one design, more than one layout, more time testing Blackberries is required.

The same hardware with a different OS will be different, the same OS with different hardware will also be different in some cases. This is where the extra Blackberries will help to ensure the best compatibility with the platform.

But no one uses Blackberries right?

According to some figures, RIM is in the decline, but the existing user base is compared to iPhone users in that they are very happy with their devices. This, combined with your locations stats, should give you an indication on whether testing and developing for the Blackberry platform is a high priority or not. If it is, getting more than one Blackberry should be something you decided quickly.

Food for thought

Blackberry users with OS6 and above may be the ones to catch you out in your testing, they have the capable browser, but the viewport is a big limiting factor. While supporting and testing Blackberries may seem a low priority for some, if you put in the time, ensure a solid experience, they may become a solid part of your userbase.

David Blooman - Twitter: @davey_blooman

Remember the Little Guy

Sun, 01 Apr 2012 22:04:26 +0100

It is sometimes easy to forget the devices that went head to head against the first iPhone, one such titan was the Nokia N96. A whopping 264 MHz CPU was the engine of this little gem, compared with the monster 412 MHz iPhone.

We can safely say the N96 has no business being called a smartphone, or can we?

The Why

In this post, I’m going to talk about more about the older phones and third party browsers we looked at, what they are capable of, but also how they can help you iron out the creases.

As mentioned in my previous post about devices we tested on, Opera Mobile have given phones a new lease of life for browsing this fair Internet. As you can see above, the N96 gets our lovely Smartphone BBC News site, instead of the feature phone version you will see with the stock browser.

With many older Symbian phones being recycled every day, Symbian is the OS that will most likely continue to be number 1 for a few more years. However, the browser for S60 was first rolled out in 2006, with some phones never updated. This has slowly lead to the Symbian browser becoming the IE of the Nokia world, but there is hope in the form of Opera Mobile.

The Alternative

Opera Mobile is a free, modern browser which has advanced features, including solid support for HTML5. Opera Mobile is a browser that can be installed on Symbian(S60) and Android phones, which is a big improvement for Symbian. This support means that older phones can get a website that otherwise would have been unsupported on the phone.

The Phones

Opera Mobile is a drop in the ocean in numbers compared to Opera Mini, but I think it could expose areas when you could improve a website. As well as the Opera browser, I also look at the devices it can be run on, the screen size, the CPU power, the RAM and the interface. S60 devices are now mostly the budget phones, with Nokia pushing Windows Phone as the high end, so picking low end devices is easy. Screen size should start at around 240px and your top end phone should be the Nokia N8, what seems to be the last performing Symbian phone from Nokia. By combining old phones with new browsers, you can quickly see if you website is the masterpiece you believe it is. Slow and overweight websites are only going to mean emails from users saying it takes too long to load, but they may have a point.

A Change of Plans

With such low specifications by today’s standards, some websites can cause browsers on this older hardware to slow down. It is therefore important to consider these factors when developing and testing. Assuming that older devices can’t run your shiny new website is not advised, browsers like Opera Mobile may catch you out.

Should I Care?

If you are only looking at Opera and Symbian from a suppport point of view, then for some of you, the answer is no. However, having these devices on hand is going to provide you with the answers to speed questions, whether it is OK to have a huge page weight and will also help you to find the limitations of your site. If anything, this sort of testing could just be the reassurance you are looking for in your site, so give it a try and let me know how you get on.

David Blooman - Twitter @davey_blooman

Testing

Wed, 28 Mar 2012 09:59:00 +0100

We thought we’d share some information on our testing process.

My name is David Blooman, I am the principal tester for responsive news, all questions and queries can be sent to me at @dblooman

When testing the devices, there are 4 main areas to cover, feature phones, smart phones with low level support, smart phones and tablets. The primary tablet market will be 7-inch devices up to 720x1280 pixels with the lowest smart phone being 240x360 pixels.
Currently, we are only approaching devices that are not conforming to desktops, so smart TV’s, games consoles and other exotic devices are test, desktops are not.

We do test desktop browsers, Chrome, Firefox, IE, however we typical test the version which our stats point to, or when there was a feature update. As some browsers use auto updating services, it can often be a case of waiting for the next release to switch testing to a new version.

When testing on a desktop, the hardware does play an important part. The pixel density of large desktop screens, which is typically low, means testing on a desktop screen may prove fruitless, given that any CSS issues may be masked. When testing for mobile, images may also appear over blown or scaled incorrectly, given the larger screen.

The final part of our per-requisite list is cutting the mustard http://blog.responsivenews.co.uk/post/18948466399/cutting-the-mustard

Minimum Device Testing
———————————

Symbian S60 - N96 (Symbian 9.3)
This phone represents the lowest of our requirements, mobile barlesque, our header, is 240px wide, meaning that our lowest support width is the same. The Nokia N96 is a great test case to identify the impacts to other dependencies within a webpage. This phone is also widely used, given the low cost nature of the OS(open source Symbian), as well as having a strong user base in our stats. The OS also supports opera mini/mobile for comparing a non JS/JS experience
Android 2.2 Native Browser- Galaxy S
This phone is our minimum Android OS support for smart phones, this is based upon the fact that it cuts the mustard in our tests. While the user base for this OS is dropping due to more recent versions of Android, 4.0 etc. This devices is a good hardware point to test against as well. It features a 1ghz CPU, which was less optimised when 2.2 was launched, meaning that 2.2 on a slower CPU may produce a smoother experience. With manual testing, we have discovered a consistency across Android in that, if it works on 2.2, it has a strong chance of working on later versions. Android 2.2 is the best browser to test against for maximum stability across the Android user base, from 2.1 to 2.3.3
iOS 6 Native Browser - iPhone4S
This browser has taken most of the iOS user base, being stable and having a lot of the more advanced features of HTML5. The browser produces a huge amount of web traffic globally, so testing this is a must for any large commercial website.
Blackberry OS 6 - Bold 9700
BBOS6 is not the latest version of the OS, but does feature a near identical browser as 7, with minor improvements seen in 7. Even so, the uptake of 7 is light, so the focus should be on 6. This is for 2 reasons, 1, BBOS6 supports a Webkit browser, 2, the browser cuts the mustard. What is important to note about BBO6 is the Curve range. The curve range is a budget range of phones with lower report rates or resolutions. The 9300 for example, has a strong user base, but it also is a good indicator of how the smartphone experience is impacted by a screen with a small vertical height. For this reason, testing on a 9300 could give you a skewed result, the same for only testing on the 9700. If you are only testing features that have no layouts and CSS changes, the 9700 is the better of the two, but testing on the 9300 will give you an idea of the budget uses experience.
Nexus 7
Android 4.1 was launched on the most popular Android tablet, the Nexus 7. With 4.1 came Chrome for Android by default, this represents a shift away from the native browser, which many Android OEMs customise. By having a consistent browser across all devices, Google makes testing Chrome easier too. More than that, the Nexus 7 size, resolution and pixel density make it a solid device to test tablet and phone layouts.

Review
———————————

The list above may be different to the list many others have come up with, this is because many testers will focus on existing markets and user base, which is Android and iOS oriented at the moment. This type of focus is inadvisable given the ease of switching services from apps to web, other competitors to yours. This under estimation of the market and how quickly it can change may lead you to not fully support a browser or entire range of devices. In many test devices list, you will see Android 2.1,2.2,2.3 and 4 devices listed as more priority over Symbian devices. The project will then focus on using emulators with opera installed for Symbian testing, which may provide such a different result, that it could lose market share in Africa, for example. The devices above will cover different break points for design, but also play a secondary part, third party browser testing. Opera is the most popular mobile browser, which is supported on iOS, Android and Symbian, it brings HTML5 support to even the N95. For this reason, Opera should be looked at closely on these devices if you looking into lots of different markets and countries. Opera Mini also plays a big part, the news website is reduced to feature phone look as Opera Mini uses proxy servers, disabling most of the Javascript on the page. A Galaxy Nexus with a 720p screen can make some websites look terrible when using Opera Mini, ensuring you have Opera on you test list is key to success.

Hardware Tests
———————————

Device of 240 pixels wide screen - N95
Device of more than 1000 wide screen - Any device
Touch Screen - Any device will be relevant, this is to test hit states, button placements and general usage while using a device with on screen keyboard.
Non-touch - In most cases, Symbian and blackberry phones will fall into this category, how the user can navigate the site is of importance, so testing the functionality of the site with such devices is a high priority.

These 4 additional requirements may require additional devices, but in investing in this equipment, the devices will cover most of the core stats and you will be able to test against the majority of browsers and input methods. This should allow you to see where your problem areas are going to be, as you can quickly identify which browsers have the key levels of JavaScript and technologies support required.

It is recommended that you clear all cookies, cache and reset browser to default before beginning any testing. Not doing this can lead to older versions of the site interfering with testing as you always must look at the latest version of the site. With builds occurring all the time, minor changes will often show little impact, such as back end work. Front end work can mean changes to the site that will fundamentally affect testing, as the look can be radically different on certain devices. You should always consider this when working, there is no such thing as a minor change when testing responsive web pages.

Further Testing
———————-

For further testing, the following list identifies the most common Mobile OS’s. Below is explanation of the differences between the OS’s and browsers, this could help you determine whether you testing need to extend to all the devices.
The Android 2.3 OS has the the largest user base of Android, therefor the most users on that versions browser. We use the Galaxy S as 1, it is one of the most popular devices in our stats, 2, it has a typical size screen and pixel density for that OS, meaning a more balanced approach when doing UI testing.
The Bold 9700 is again, one of the more popular devices, but it also represents a crossover device from OS5 to OS6. It’s screen is of a good pixel size for Blackberrys and its internals are one of the more high specifications for a feature phone.
Windows phone is a good way of testing IE10, as a large amount of the code base was used to create the browser on WP8, though mobile has less features.

iOS 6 - iphone 4
Android 2.2 - HTC desire
Android 2.3 - Samsung Galaxy S
Android 4 - Both stock and Chrome browsers
Blackberry OS 5 - Bold 9700
Blackberry OS 6 - Bold 9700
Blackberry OS 7 - 9380
Windows Phone 8 - Lumia 920
Symbian S60 - Symbian Browser
Symbian S40 - Nokia Browser
Nexus 7
iOS 6 - iPad

Browser and OS Analysis
———————————-

While emulators are very useful in a lab style environment, they often miss things that are only found when using handsets. There is also a lot of nuances that you will discover when you use this many devices. Below is a combination of personal experience, information from other sources and my opinion on the direction when testing responsive design.

iOS. The iphone and ipod range from the 3Gs upwards all support iOS5, which has support for most standards, the browser on iOS5 is the same on every device, therefore testing on 1 device is suitable. The second most supported version of the OS which supports a slightly lower performing browser is iOS4, however this has not performed differently except in hardware (larger screen) Here is a comparison of the 2 versions http://www.blaze.io/mobile/ios5-top10-performance-changes/ - The range of iOS devices represents nearly 20 million page views per month, hence it’s high priority status.

Android 2.1 is the lowest version of Android to cut the mustard. Android has varying levels of support, Android 2.2 browser and onwards has been the most consistent, with Android 2.2 and 2.3 being the highest user base. Android 2.1 has the ability to support many smart phone features, but lacks some font embedding and other technologies, such as Flash. This has proven to be one of the more problematic devices, but is considered a baseline phone for smart phone support. This particular version of Android is in decline as the jump from 2.1 to 2.2 is being made, or the user upgrades to a different phone. Android 2.1 represents about 25% of the Android user base with a million page views per month.

Android 2.2 was the last version to be open source, so a lot of devices used this version for Google TVs, phones, music players etc. The browser for 2.3 is also based on this version. Differences in the operating system can have some impact on the browser, but the difference between 2.2 and 2.3 is mostly down to how the particular vendor changes the phone. A change log is listed in the link below http://www.knowitsdifference.com/difference-between-android-2-2-and-android-2-3/ Android 2.2 is a measurable jump in performance over 2.1, the user base is almost 3 times as big with nearly 8 million page views per month. Android 2.3 offers an overall system update that is relevant to user, not as much for browsers though. In our test, there have been less that 1% of difference between the 2, with only 1 or 2 bugs discovered not to replicate on both versions. The 7 million combined users with 20 million page views shows us how important it is to correctly test on Android 2.x versions.
What is important to note about 2.2 is the browser features a shrunk down version of desktop Chrome’s V8 JavaScript engine, Google have integrated it into the Android 2.3 OS to drastically improve JS performance. When testing Android 2.1 and 2.2, the should be treated as different entities, rather than just a point release, such as 2.2 and 2.3.

Android 3 is the tablet skew of the OS. 3.2 and above supports lower screen sizes that mean you can have a large range of devices, so testing on this version is best. The update of Android tablets is still very small compared to iOS devices, like the iPad, however the browser is very high quality. The Android 3 OS is essentially a fork of Android 2.2, as such is inherits much of the browser from there. However there is added support for multi touch within Javascript and improved zooming options.

Android 4 is the newest and most advanced version of the OS, it currently supports 2 browsers, Chrome and the Android browser. It seems that Google intend to replace the stock browser with Chrome on upcoming devices, so testing on both until this is that case is prudent. The current version of Chrome uses a new version of Webkit than the stock android browser. Info on browser http://www.android.com/about/ice-cream-sandwich/. What it is important is the that Chrome is a Google product, therefore it may not be present on the open source version of the OS, so for future testing, stats will determine which webkit version and by extension, which browser is the most popular on the platform. Android 4.1 brought a few improvements over 4.0, better HTML5 video support, supports the updated HTML5 Media Capture specification on input elements.WebView now supports vertical text, including Ruby Text and other vertical text glyphs.

Android 4.1 had a few tweaks to the browser, but in reality, the focus is now on Chrome as the stock browser faces heavier competition. Chrome Should be seen as an equal in testing to the stock browser.

Blackberry OS 5 is a pure feature phone experience, it’s screen size support is one of the biggest factors, given the high landscape resolution, but low vertical. The browser has been the most consistent for feature phones, supporting all the requirements.

Blackberry OS 6 has patchy support, it uses a webkit browser, but the support for font embedding has been hit and miss. The browser is currently one of the highest usage of the website, meaning that this is important OS for testing. Newer versions of the Blackberry browser seem to have fixed the issues we are having with 6. Browser Info http://crackberry.com/blackberry-6-review - new-browserhttp://crackberry.com/blackberry-6-review#new-browser One of the core issues with Blackberries is the fact that they have unusual screens. The majority of phones sold between 2008 and 2011 have a 480x360 screen; this is intended for reading emails instead of web pages. What should also be tested is what the screen reports itself as, on some devices the resolution is dropped to 360 wide due to hardware limitations. It is important to identify issues with the look and feel on a webpage based upon these types of devices, especially for us with more than 4 million users and 16 million page views.

Blackberry OS 7 has improvements over the BBOS6 browser. Most of the bugs have been fixed, while adding new features such as a web inspector and better video/audio support. The full list can be found here, but the highlights include support for native touch and cursor events, web socket API support and new media support in the form of <audio> and <video> elements, as well as the <source> element which lets you specify a media resource. There is also a new JIT (Just-In-Time) JavaScript® compiler that offers better performance.

Windows Phone 7 uses the same code base as Internet Explorer 9, in a lot of cases, phone issues replicate onto the desktop. If there are issues on the mobile OS, testing on IE9 could be the fastest way to identify the issue, the mobile version is essentially a stripped down version of the desktop browser, as such, don’t expect high levels of support for technologies. The lack of font embedding is of a particular annoyance, for this reason, IE9 and WP7 fall into smartphone with low level support category. http://en.wikipedia.org/wiki/Windows_Phone_7.5 Windows Phone 8 will launch with a new, IE10 based browser with much better HTML5 support, at time of writing, it still is being developed. Windows phone 7 is now a deprecated platform, with not more updates coming, as usage goes down, so will the need to support this browser.

Windows Phone 8 has an IE10 based browser with much better HTML5 support, at time of writing, it still has some issues. IE 10 has a large bug with viewport tags, this leads to hardware pixels being detected rather than software. There is a fix in the works from Microsoft, no timeline on that though. IE 10 offers a much better support listed here, some of the things IE10 mobile can’t support are Inline video, Some new manipulation views APIs for touch panning and zooming, with the exception of –ms-touch-action, Multi-track HTML5 audio (simultaneous), ActiveX and VBScript, Drag-and-drop APIs, File access APIs with the exception of blobs which are supported on Windows Phone 8.

Symbian phones are a large combination of S60, S40 and Symbian Belle, there is massive variations in the devices as there are hardware and software limitations of the range. In most cases, the S40 phones are feature phones. Some phones are being logged with a new webkit browser; this browser can do some JavaScript, however the usage figures are low as it requires a manual update. The S60 phones are also mostly features phones, though Nokia is updating many phones with a newer browser that supports up to HTML 5.0 and JavaScript 1.8, most of these phones will receive the feature phone version of the site, however these are low usage devices. In stats, these phones can appear as either S60 or S9.x, the latest version of the S9.x is 9.5. Finally there is a new version of Symbian, sometimes seen as Nokia OS or Symbian Belle, this version is also a hit and miss support OS. While this version is low usage, it could become more common in emerging countries. Given that Nokia doesn’t use Over The Air (OTA) updates for its phones, users of the Nokia N8, the latest Symbian phone, could be running any one of 3 different versions of Symbian, each with its own version of the webkit browser, only the latest version supports HTML 5. Some phones will also come with some version coming with Opera, netfront and webkit depending on location, so Symbian testing is very limiting when trying to ensure maximum support. In our testing, we should focus on the Nokia N95 and Nokia N8, these feature different versions of S60 and have different browser versions, so provide a solid range of old browsers in Africa, with modern the UK. The most recent UK launch of a Nokia Symbian phone is the N8, which currently is classed as a feature phone, even though it support is relatively good, it does not cut the mustard. The best browser for this type of Symbian phone is Opera mobile, fast and fully feature, we do a lot more testing of Opera on Symbian S60 with Opera than any other platform. Link to different Symbian versions: http://en.wikipedia.org/wiki/Web_Browser_for_S60

Symbian 10 is also known as Symbian Belle, a new brand from Nokia. This comes with a brand new browser which is Webkit based and supports lots of good HTML5 features. There a few notable issues, such as font embed support, but it brings a few Nokia/Symbian phones into the realm of HTML5, which is a good thing. Check out my post here for more info

Silk Browser. Amazon has forked Android 2.2 to create the Kindle OS with Silk browser. This browser routes all traffic through Amazon servers and can affect some elements of the website. Form submission is something that should be thoroughly tested to ensure the server is able to receive the form data. You may also find that because of the form factor, you will have to make a decision, only use smartphone layouts or use both tablet and smartphone layouts, this is mostly due to the screen width being able to support tablet web pages. But what is most troubling for the tester is that you may have to test both of these layouts on the say device, this is going to really annoy you.

Silk Browser version 2 is a fork of 4.0 with the AOSP browser. This browser has had some changes for the cloud sync, but has the same support as Android 4 stock browser.

The iPad is currently the market leader, with the single device having 60% of the market, the market grows every month though, so the user base is ever growing. While the differences to the iPad browser are subtle to the iPhone browser, it is important to look at the iPad as the biggest device, in terms of screen size, that users will visit with. While this device can support most, if not all, desktop sites with ease, it is important to remember that speed and simplicity are often desired more than functionality. For this reason, users of iPads and other tablets, will use the mobile version of the site, especially if they are using a data service. If it is outside of the scope of your project, you should consider potential markets with the iPad, responsive news’ largest device support is 7 inches, however testing is conducted on the iPad.

Browsers
———————————-

The next section will include information about the browsers that you will need to test in addition to the native browsers we have already talked about. Just to reiterate, all the operating systems we have talked about so far have their own native browser.

Third party browsers that have big impact in the world include:

UC Browser - Information

Chrome Mobile - Information

Opera Mini/Mobile - Information

Firefox - Information

As we are test for users all over the world, UC browser and Opera Mini may be less important to you. These 2 browsers combined represent a sizeable chunk of Africa and Asia browsing traffic, so if you are in that region, you are probably already testing on it. Firefox is growing, but they are updating rapidly, so testing on the latest stable build is advisable, especially as it is more than likely the user will upgrade. The complete custom build is only available on Android. Chrome is not yet the default browser, it will also sit alongside the native Android browser, at least for now. Testing on both is probably the best approach, similar to Firefox, testing the latest build is best. There is a version of Chrome for iOS, but this uses the same version of Webkit as the version of IOS you are on. The Android version of Chrome is update frequently and is the only place to get the complete Chrome build.

Summary
———————————-

In total, you final device list could have other smaller markets included, our current device list looks something like this :

iOS 6 - iPhone
iOS 6 - iPad mini
iOS 6 - iPad
iOS 5 - iPad
iOS 5 - iPod
Android 2.1 - HTC desire
Android 2.2 - Samsung Galaxy S
Android 2.3 - Samsung Galaxy S
Android 3.2 - Motorola Xoom
Android 4.0 - Galaxy Nexus (Stock and Chrome)
Android 4.1 - Galaxy Nexus (Stock and Chrome)
Blackberry OS 4 - Curve 8520 (We have recently dropped testing this browser)
Blackberry OS 5 - Bold 9700
Blackberry OS 6 - Bold 9700
Blackberry OS 7 - Bold 9860
Windows Phone 7.5 - Lumia 800
Windows Phone 8 - Lumia 920
Symbian S60 - N95
Symbain S60 - N8
Symbian S40 - C3
Symbian S40 - Asha 303
Kindle E-Reader
Kindle Fire - Android 2.2 Fork
Kindle Fire HD - Android 4 Fork
Android 3.0 - Samsung 7.7
Android 4 - Samsung 7.0 v2
Android 4.2 - Nexus 7
IE 10 - Microsoft Surface

Edit 20/6/2012 - We have recently determined our 3 in market tablets to look at, Kindle fire, BB Playbook and Galaxy Tab 7.7. They are all webkit based browsers, but have some differences in support, as well as volume of sales. The 7.7 uses a pixel ratio of 1, whereas the other 2 dont, this means that the break points will have to be correctly established for tablets in order for users not to receive broken features.

Edit 29/8/2012 - We are no longer testing on BBOS4 anymore, we have found the share to drop below our support level and global usage has dropped to an all time low. We have added other versions of Android and other hardware sets, such as 7 inch tablets.

Edit 13/9/2012 - I have made some big changes to the testing list this month, we have changed the minimum level of support to start at Android 2.2 and BBOS6, this is due to total browser market shifts on the site as well as globally and within the UK.

Edit 13/11/2012 - I have now updated list to include changes to BBOS7, IE 10 on Windows and some Android and iOS support changes.

Edit 16/01/2013 - I have updated Windows phone to indicate our new support list, as well as changing our testing devices

the screen of a tester

Fri, 16 Mar 2012 18:51:24 +0000

the screen of a tester

Android's Ugly Family

Mon, 27 Feb 2012 12:07:50 +0000

What Android will sometimes present, is a bug you simply cannot explain. This is one of those bugs that you have tested on every version of Android, but only shows on one. In our case, Android 2.1 shows it annoying side by creating a bug that continues to overlay a text field once submitted.

How can you determine if this is a bug with your code, or a bug with the OS, or a bug with the browser? Get another phone.

Our Apples to Apples comparison is between the HTC Desire and the Sony Xperia X10, both running Android 2.1 update 1.

Once we get into it, the picture above shows what happens when we input data into a search box and hit enter. This issue seems to be intermittant on this phone, it will depend on the website. What we learn from this issue however, is that it is not persistent across all Android phones. The X10 doesnt produce this bug, so now we entertain the possibility that HTC has slightly forked the browser……

If testing a website means buying a HTC phone with sense, Samsung phone with Touchwiz, Sony phone with their skin and a Google phone with no skin, the cost of testing Android 2.1,2.2,2.3,3.2 and 4.0 suddenly spirals, it also increases test lead time.

However, thanks to our stats, we can see which phones are more popular, which will determine which phone you should buy to test. In most cases, starting with the pure Google experience is going to be a good place to start, the Nexus line of phones are what you should look for, but these pure versions are not the biggest sellers.

While Android fragmentation is an issue for the tester who is looking at so many different versions of webkit and Android, there is light at the end of the tunnel. Chrome for Android is being introduced on Android 4, which could mean that a locked down browser will provide consistency and a more reliable test environment.

For now though, Androids fragmentation will continue with more elaborate skins and changes to the core OS are going to occur, ultimately meaning more devices and more testing to be completed.