Ten things to remember when testing RWD
A few things to remember when doing a bit of testing.
10 - phones are bad
A lot of phones are slow, have lots of bloatware and use the browser as a way to search Google, rather than look at your beautiful website. Symbian, Android, BBOS and Windows phone can be put onto very slow hardware, so buying that hardware is essential. More importantly though, if you only test on high end android and iphones, you will probably not realise the world of pain you are causing to bad phone users. Remember, phones are bad.
9 - browsers are bad
Older browsers may cause you a headache, so while Android 2.1 and iOS 4.0 are fading away, these two naughty browsers still need attention.
Read More
Devices & Pixel Ratios
The idea of a 720p display on a phone is the norm, with 1080p probably coming over the next year or 2. With other devices like the iPad and iPhone with huge pixel counts, the use of images that exploit screens that have a pixel ratio of 2 is something everyone should look into doing.
Read More
Landscape Changes
Here we have the web page in landscape view. The latest version is on the right, the older version on the left. The layout has changed in order to use the available screen width while keeping the larger image and maintaining the look and feel of the portrait layout
Before and After
A side by side of the live version of the site, right, and an early version left.
Tools
In a recent blog post i talked about what devices the BBC News team use for testing our responsive mobile site, in this post, i will talk about some of the tools you can use on these devices. This post will talk mostly about remote debugging, using Shadow and Chrome USB, but also front end work with Ripple and Proxylocal.
Chrome USB Debugging - https://developers.google.com/chrome/mobile/docs/debugging
Although Chrome for Android is only available on 4.0 (Ice Cream Sandwich), the options USB debugging and testing give you make it a valuable tool. The setup is very simple, the Android SDK and ADB(Android Device Bridge) need to be downloaded, PATH set, then load Chrome and you are away. You will need to run a terminal command to connect the device, but it is a near plug and play setup.
With Chrome loaded on the page you want to look at, Chrome desktop can then connect by using localhost:9222, this will then present the familiar inspector we have come to know and love.
So, just like the normal version of Chrome, you can edit your page with the phone updating instantly, allow for a real world version of you website, live on a device. This feature can help you develop around issues with other browsers, by finding solutions and experimenting on Chrome, you can quickly assert whether a change will work.
The other tabs in Chrome Inspector will allow you to test areas where you need reassurance, but more importantly, if real world interaction is correct. The stats for BBC use an image to identify Javascript interactions that we want to track. Switching between the 2 tabs on the front page, using the sections menu, wanting to read more live event comments, all of these creating a new response to track. The network tab in the inspector is where we can easily look at the images and requests being made, thereby ensuring the stats are being made available to be recorded.
Of course, this is just a snap shot of the possibilities of Chrome inspector. The excellent thing that Google have provide us here, is the chance to do the same desktop tasks on mobiles.
Ripple
The next tool that is still in beta is from RIM, called Ripple. It does what a lot of other responsive tools do, but makes device selection much easier. It also allows for other options like GPS and stats like PPI. One thing to note is that you can inspect the phone as you would a web page, so you can change things on the fly, good for front end stuff.
It is in beta right now, but has a lot of potential. If RIM implemented some updates like custom headers, adjustable browser reporting size and screen shot options, it could be a great testing platform. It also means you dont have to use the ridiculous Blackberry emulators anymore.
Proxylocal - http://proxylocal.com/
Something that we have been using extensively over the last couple of months is a tool to make a sandbox public for testing. Proxylocal is a simple install which allows you to take your latest build on your sandbox, or pull branches down from others, and test the functionality against devices and different screen sizes. We have found testing against our live site with early builds finds the issues and bugs much quicker than or standard build process. Although testing on a sandbox is not ideal, you may find inconsistencies, it will give you an insight into features and how they will look, which may reduce the back and forth between the team.
Adobe Shadow - http://labs.adobe.com/technologies/shadow/
Adobe Shadow is a tool that recently got an update, this is something that is useful for large scale testing of Android and iOS devices. You essentially sync up the devices to your browser via the shadow tool. This means when you visit a new page or website, the phones will follow you and display the same page. This is probably the best tool for people wanting to test multiple features or layouts across a large amount of devices at the same time. The tool allows you to refresh all the devices remotely, as well as take screenshots of the device, although it can be a little buggy.
The other feature is remote inspection, this will allow you to use weinre to inspect the devices browser, something that will help you test, but also change things on the fly to see if you decisions are correct. The network tab is not usable unfortunately, so backend services still have to be checked manually, but for front end devs, a very useful tool.
The image shows a device connected to shadow, with the control page and the weinre inspector open.
Is One Blackberry Enough?
In this post, we will look at why testing more than one Blackberry version or device may be the better way to go.
Blackberry OS
Blackberry phones, some of us love them, a lot of people don’t, but they are important to your development and testing. The Blackberry user base is quite large in the UK, but also has a solid global audience, it is mostly a non touch device group with limited screen resolutions and size. The choice to make an app for Blackberries is also a tough one for many companies, so your responsive website may have an edge in this market.
In Pictures
The picture above shows 3 different versions of Blackberry OS(BBOS), from 4 to 6. From the left, Sharon is OS4, Kenny is OS5, Len is OS6 and the final devices is also OS6. What these devices represent is the most popular Blackberries that are used on the site, but also the different versions that look to test against. OS7 has had a small impact so far, but as it picks up we will move to test it more.
What is also very important to take from the picture is how the different screen sizes and resolutions affect the end result of loading a webpage. You can also see how the page is different based upon the version, smartphone or featurephone. The OS5 device has a good screen width, 480px(reports as 465x304), so to use the extra space, the text wraps beside the image, rather than using the single larger top image. This is different to the OS4 featurephone, which uses the standard design with the top image. The OS4 device has a 360px wide screen, but a reasonable 305x224 reporting rate, which although low, means that you expect a certain level of content on the screen. When testing, it is important to have a eye for the detail, looking at whether you agree with the layout, or if you think it needs changing.
The Smartphones are quite different in the look and feel, Len, a 360px (reports as just 320x167)screen is actually receiving a much different experience. The image and font are the obvious two things, producing a smartphone experience on a very low resolution device.
Len is the more budget Blackberry, it is part of the curve range, which also feature lower resolution displays. While you may want to focus on the lovely Bold 9900, the budget range are the more popular. If you are going to buy Blackberries to test against, i would suggest getting a low resolution device, this will help ensure that users with OS6, which features a webkit browser, will get a satisfactory experience. If the Nokia N95 was the lowest horizontal width screen, the Blackberry 9300 is our lowest vertical, which is why we chose it for testing against our site. In a way, it does fall outside of what some would call an usable experience, especially for a smart phone, but it has to be considered.
I think there should be a prudance decision made on whether this type of device affects your development, but also whether certain features are required to look the same as on an iPhone. If you really don’t think Blackberries are a potential market, then sticking to the more common Bold series may be better, offering a more usable experience with higher reporting rates.
Knowing the difference
As we have discussed, unlike iOS and to some extent, high end Android phones, Blackberries are limited by the vertical height of the viewport, so you may choose to design around this to get more content on the page. This is seen on the Blackberry called Kenny, which uses the extra width to add the summary text next to the top image, but other elements like menus could be used in this space. This has a counter effect though, with more than one design, more than one layout, more time testing Blackberries is required.
The same hardware with a different OS will be different, the same OS with different hardware will also be different in some cases. This is where the extra Blackberries will help to ensure the best compatibility with the platform.
But no one uses Blackberries right?
According to some figures, RIM is in the decline, but the existing user base is compared to iPhone users in that they are very happy with their devices. This, combined with your locations stats, should give you an indication on whether testing and developing for the Blackberry platform is a high priority or not. If it is, getting more than one Blackberry should be something you decided quickly.
Food for thought
Blackberry users with OS6 and above may be the ones to catch you out in your testing, they have the capable browser, but the viewport is a big limiting factor. While supporting and testing Blackberries may seem a low priority for some, if you put in the time, ensure a solid experience, they may become a solid part of your userbase.
David Blooman - Twitter: @davey_blooman
Remember the Little Guy
It is sometimes easy to forget the devices that went head to head against the first iPhone, one such titan was the Nokia N96. A whopping 264 MHz CPU was the engine of this little gem, compared with the monster 412 MHz iPhone.
We can safely say the N96 has no business being called a smartphone, or can we?
The Why
In this post, I’m going to talk about more about the older phones and third party browsers we looked at, what they are capable of, but also how they can help you iron out the creases.
As mentioned in my previous post about devices we tested on, Opera Mobile have given phones a new lease of life for browsing this fair Internet. As you can see above, the N96 gets our lovely Smartphone BBC News site, instead of the feature phone version you will see with the stock browser.
With many older Symbian phones being recycled every day, Symbian is the OS that will most likely continue to be number 1 for a few more years. However, the browser for S60 was first rolled out in 2006, with some phones never updated. This has slowly lead to the Symbian browser becoming the IE of the Nokia world, but there is hope in the form of Opera Mobile.
The Alternative
Opera Mobile is a free, modern browser which has advanced features, including solid support for HTML5. Opera Mobile is a browser that can be installed on Symbian(S60) and Android phones, which is a big improvement for Symbian. This support means that older phones can get a website that otherwise would have been unsupported on the phone.
The Phones
Opera Mobile is a drop in the ocean in numbers compared to Opera Mini, but I think it could expose areas when you could improve a website. As well as the Opera browser, I also look at the devices it can be run on, the screen size, the CPU power, the RAM and the interface. S60 devices are now mostly the budget phones, with Nokia pushing Windows Phone as the high end, so picking low end devices is easy. Screen size should start at around 240px and your top end phone should be the Nokia N8, what seems to be the last performing Symbian phone from Nokia. By combining old phones with new browsers, you can quickly see if you website is the masterpiece you believe it is. Slow and overweight websites are only going to mean emails from users saying it takes too long to load, but they may have a point.
A Change of Plans
With such low specifications by today’s standards, some websites can cause browsers on this older hardware to slow down. It is therefore important to consider these factors when developing and testing. Assuming that older devices can’t run your shiny new website is not advised, browsers like Opera Mobile may catch you out.
Should I Care?
If you are only looking at Opera and Symbian from a suppport point of view, then for some of you, the answer is no. However, having these devices on hand is going to provide you with the answers to speed questions, whether it is OK to have a huge page weight and will also help you to find the limitations of your site. If anything, this sort of testing could just be the reassurance you are looking for in your site, so give it a try and let me know how you get on.
David Blooman - Twitter @davey_blooman
Testing
We thought we’d share some information on our testing process.
My name is David Blooman, I am the principal tester for responsive news, all questions and queries can be sent to me at @dblooman
When testing the devices, there are 4 main areas to cover, feature phones, smart phones with low level support, smart phones and tablets. The primary tablet market will be 7-inch devices up to 720x1280 pixels with the lowest smart phone being 240x360 pixels.
Currently, we are only approaching devices that are not conforming to desktops, so smart TV’s, games consoles and other exotic devices are test, desktops are not.
We do test desktop browsers, Chrome, Firefox, IE, however we typical test the version which our stats point to, or when there was a feature update. As some browsers use auto updating services, it can often be a case of waiting for the next release to switch testing to a new version.
When testing on a desktop, the hardware does play an important part. The pixel density of large desktop screens, which is typically low, means testing on a desktop screen may prove fruitless, given that any CSS issues may be masked. When testing for mobile, images may also appear over blown or scaled incorrectly, given the larger screen.
The final part of our per-requisite list is cutting the mustard http://blog.responsivenews.co.uk/post/18948466399/cutting-the-mustard
Minimum Device Testing
———————————
- Symbian S60 - N96 (Symbian 9.3)
This phone represents the lowest of our requirements, mobile barlesque, our header, is 240px wide, meaning that our lowest support width is the same. The Nokia N96 is a great test case to identify the impacts to other dependencies within a webpage. This phone is also widely used, given the low cost nature of the OS(open source Symbian), as well as having a strong user base in our stats. The OS also supports opera mini/mobile for comparing a non JS/JS experience
- Android 2.2 Native Browser- Galaxy S
This phone is our minimum Android OS support for smart phones, this is based upon the fact that it cuts the mustard in our tests. While the user base for this OS is dropping due to more recent versions of Android, 4.0 etc. This devices is a good hardware point to test against as well. It features a 1ghz CPU, which was less optimised when 2.2 was launched, meaning that 2.2 on a slower CPU may produce a smoother experience. With manual testing, we have discovered a consistency across Android in that, if it works on 2.2, it has a strong chance of working on later versions. Android 2.2 is the best browser to test against for maximum stability across the Android user base, from 2.1 to 2.3.3
- iOS 6 Native Browser - iPhone4S
This browser has taken most of the iOS user base, being stable and having a lot of the more advanced features of HTML5. The browser produces a huge amount of web traffic globally, so testing this is a must for any large commercial website.
- Blackberry OS 6 - Bold 9700
BBOS6 is not the latest version of the OS, but does feature a near identical browser as 7, with minor improvements seen in 7. Even so, the uptake of 7 is light, so the focus should be on 6. This is for 2 reasons, 1, BBOS6 supports a Webkit browser, 2, the browser cuts the mustard. What is important to note about BBO6 is the Curve range. The curve range is a budget range of phones with lower report rates or resolutions. The 9300 for example, has a strong user base, but it also is a good indicator of how the smartphone experience is impacted by a screen with a small vertical height. For this reason, testing on a 9300 could give you a skewed result, the same for only testing on the 9700. If you are only testing features that have no layouts and CSS changes, the 9700 is the better of the two, but testing on the 9300 will give you an idea of the budget uses experience.
- Nexus 7
Android 4.1 was launched on the most popular Android tablet, the Nexus 7. With 4.1 came Chrome for Android by default, this represents a shift away from the native browser, which many Android OEMs customise. By having a consistent browser across all devices, Google makes testing Chrome easier too. More than that, the Nexus 7 size, resolution and pixel density make it a solid device to test tablet and phone layouts.
Review
———————————
The list above may be different to the list many others have come up with, this is because many testers will focus on existing markets and user base, which is Android and iOS oriented at the moment. This type of focus is inadvisable given the ease of switching services from apps to web, other competitors to yours. This under estimation of the market and how quickly it can change may lead you to not fully support a browser or entire range of devices. In many test devices list, you will see Android 2.1,2.2,2.3 and 4 devices listed as more priority over Symbian devices. The project will then focus on using emulators with opera installed for Symbian testing, which may provide such a different result, that it could lose market share in Africa, for example. The devices above will cover different break points for design, but also play a secondary part, third party browser testing. Opera is the most popular mobile browser, which is supported on iOS, Android and Symbian, it brings HTML5 support to even the N95. For this reason, Opera should be looked at closely on these devices if you looking into lots of different markets and countries. Opera Mini also plays a big part, the news website is reduced to feature phone look as Opera Mini uses proxy servers, disabling most of the Javascript on the page. A Galaxy Nexus with a 720p screen can make some websites look terrible when using Opera Mini, ensuring you have Opera on you test list is key to success.
Hardware Tests
———————————
- Device of 240 pixels wide screen - N95
- Device of more than 1000 wide screen - Any device
- Touch Screen - Any device will be relevant, this is to test hit states, button placements and general usage while using a device with on screen keyboard.
- Non-touch - In most cases, Symbian and blackberry phones will fall into this category, how the user can navigate the site is of importance, so testing the functionality of the site with such devices is a high priority.
These 4 additional requirements may require additional devices, but in investing in this equipment, the devices will cover most of the core stats and you will be able to test against the majority of browsers and input methods. This should allow you to see where your problem areas are going to be, as you can quickly identify which browsers have the key levels of JavaScript and technologies support required.
It is recommended that you clear all cookies, cache and reset browser to default before beginning any testing. Not doing this can lead to older versions of the site interfering with testing as you always must look at the latest version of the site. With builds occurring all the time, minor changes will often show little impact, such as back end work. Front end work can mean changes to the site that will fundamentally affect testing, as the look can be radically different on certain devices. You should always consider this when working, there is no such thing as a minor change when testing responsive web pages.
Further Testing
———————-
For further testing, the following list identifies the most common Mobile OS’s. Below is explanation of the differences between the OS’s and browsers, this could help you determine whether you testing need to extend to all the devices.
The Android 2.3 OS has the the largest user base of Android, therefor the most users on that versions browser. We use the Galaxy S as 1, it is one of the most popular devices in our stats, 2, it has a typical size screen and pixel density for that OS, meaning a more balanced approach when doing UI testing.
The Bold 9700 is again, one of the more popular devices, but it also represents a crossover device from OS5 to OS6. It’s screen is of a good pixel size for Blackberrys and its internals are one of the more high specifications for a feature phone.
Windows phone is a good way of testing IE10, as a large amount of the code base was used to create the browser on WP8, though mobile has less features.
- iOS 6 - iphone 4
- Android 2.2 - HTC desire
- Android 2.3 - Samsung Galaxy S
- Android 4 - Both stock and Chrome browsers
- Blackberry OS 5 - Bold 9700
- Blackberry OS 6 - Bold 9700
- Blackberry OS 7 - 9380
- Windows Phone 8 - Lumia 920
- Symbian S60 - Symbian Browser
- Symbian S40 - Nokia Browser
- Nexus 7
- iOS 6 - iPad
Browser and OS Analysis
———————————-
While emulators are very useful in a lab style environment, they often miss things that are only found when using handsets. There is also a lot of nuances that you will discover when you use this many devices. Below is a combination of personal experience, information from other sources and my opinion on the direction when testing responsive design.
iOS. The iphone and ipod range from the 3Gs upwards all support iOS5, which has support for most standards, the browser on iOS5 is the same on every device, therefore testing on 1 device is suitable. The second most supported version of the OS which supports a slightly lower performing browser is iOS4, however this has not performed differently except in hardware (larger screen) Here is a comparison of the 2 versions http://www.blaze.io/mobile/ios5-top10-performance-changes/ - The range of iOS devices represents nearly 20 million page views per month, hence it’s high priority status.
Android 2.1 is the lowest version of Android to cut the mustard. Android has varying levels of support, Android 2.2 browser and onwards has been the most consistent, with Android 2.2 and 2.3 being the highest user base. Android 2.1 has the ability to support many smart phone features, but lacks some font embedding and other technologies, such as Flash. This has proven to be one of the more problematic devices, but is considered a baseline phone for smart phone support. This particular version of Android is in decline as the jump from 2.1 to 2.2 is being made, or the user upgrades to a different phone. Android 2.1 represents about 25% of the Android user base with a million page views per month.
Android 2.2 was the last version to be open source, so a lot of devices used this version for Google TVs, phones, music players etc. The browser for 2.3 is also based on this version. Differences in the operating system can have some impact on the browser, but the difference between 2.2 and 2.3 is mostly down to how the particular vendor changes the phone. A change log is listed in the link below http://www.knowitsdifference.com/difference-between-android-2-2-and-android-2-3/ Android 2.2 is a measurable jump in performance over 2.1, the user base is almost 3 times as big with nearly 8 million page views per month. Android 2.3 offers an overall system update that is relevant to user, not as much for browsers though. In our test, there have been less that 1% of difference between the 2, with only 1 or 2 bugs discovered not to replicate on both versions. The 7 million combined users with 20 million page views shows us how important it is to correctly test on Android 2.x versions.
What is important to note about 2.2 is the browser features a shrunk down version of desktop Chrome’s V8 JavaScript engine, Google have integrated it into the Android 2.3 OS to drastically improve JS performance. When testing Android 2.1 and 2.2, the should be treated as different entities, rather than just a point release, such as 2.2 and 2.3.
Android 3 is the tablet skew of the OS. 3.2 and above supports lower screen sizes that mean you can have a large range of devices, so testing on this version is best. The update of Android tablets is still very small compared to iOS devices, like the iPad, however the browser is very high quality. The Android 3 OS is essentially a fork of Android 2.2, as such is inherits much of the browser from there. However there is added support for multi touch within Javascript and improved zooming options.
Android 4 is the newest and most advanced version of the OS, it currently supports 2 browsers, Chrome and the Android browser. It seems that Google intend to replace the stock browser with Chrome on upcoming devices, so testing on both until this is that case is prudent. The current version of Chrome uses a new version of Webkit than the stock android browser. Info on browser http://www.android.com/about/ice-cream-sandwich/. What it is important is the that Chrome is a Google product, therefore it may not be present on the open source version of the OS, so for future testing, stats will determine which webkit version and by extension, which browser is the most popular on the platform. Android 4.1 brought a few improvements over 4.0, better HTML5 video support, supports the updated HTML5 Media Capture specification on input elements.WebView now supports vertical text, including Ruby Text and other vertical text glyphs.
Android 4.1 had a few tweaks to the browser, but in reality, the focus is now on Chrome as the stock browser faces heavier competition. Chrome Should be seen as an equal in testing to the stock browser.
Blackberry OS 5 is a pure feature phone experience, it’s screen size support is one of the biggest factors, given the high landscape resolution, but low vertical. The browser has been the most consistent for feature phones, supporting all the requirements.
Blackberry OS 6 has patchy support, it uses a webkit browser, but the support for font embedding has been hit and miss. The browser is currently one of the highest usage of the website, meaning that this is important OS for testing. Newer versions of the Blackberry browser seem to have fixed the issues we are having with 6. Browser Info http://crackberry.com/blackberry-6-review - new-browserhttp://crackberry.com/blackberry-6-review#new-browser One of the core issues with Blackberries is the fact that they have unusual screens. The majority of phones sold between 2008 and 2011 have a 480x360 screen; this is intended for reading emails instead of web pages. What should also be tested is what the screen reports itself as, on some devices the resolution is dropped to 360 wide due to hardware limitations. It is important to identify issues with the look and feel on a webpage based upon these types of devices, especially for us with more than 4 million users and 16 million page views.
Blackberry OS 7 has improvements over the BBOS6 browser. Most of the bugs have been fixed, while adding new features such as a web inspector and better video/audio support. The full list can be found here, but the highlights include support for native touch and cursor events, web socket API support and new media support in the form of <audio> and <video> elements, as well as the <source> element which lets you specify a media resource. There is also a new JIT (Just-In-Time) JavaScript® compiler that offers better performance.
Windows Phone 7 uses the same code base as Internet Explorer 9, in a lot of cases, phone issues replicate onto the desktop. If there are issues on the mobile OS, testing on IE9 could be the fastest way to identify the issue, the mobile version is essentially a stripped down version of the desktop browser, as such, don’t expect high levels of support for technologies. The lack of font embedding is of a particular annoyance, for this reason, IE9 and WP7 fall into smartphone with low level support category. http://en.wikipedia.org/wiki/Windows_Phone_7.5 Windows Phone 8 will launch with a new, IE10 based browser with much better HTML5 support, at time of writing, it still is being developed. Windows phone 7 is now a deprecated platform, with not more updates coming, as usage goes down, so will the need to support this browser.
Windows Phone 8 has an IE10 based browser with much better HTML5 support, at time of writing, it still has some issues. IE 10 has a large bug with viewport tags, this leads to hardware pixels being detected rather than software. There is a fix in the works from Microsoft, no timeline on that though. IE 10 offers a much better support listed here, some of the things IE10 mobile can’t support are Inline video, Some new manipulation views APIs for touch panning and zooming, with the exception of –ms-touch-action, Multi-track HTML5 audio (simultaneous), ActiveX and VBScript, Drag-and-drop APIs, File access APIs with the exception of blobs which are supported on Windows Phone 8.
Symbian phones are a large combination of S60, S40 and Symbian Belle, there is massive variations in the devices as there are hardware and software limitations of the range. In most cases, the S40 phones are feature phones. Some phones are being logged with a new webkit browser; this browser can do some JavaScript, however the usage figures are low as it requires a manual update. The S60 phones are also mostly features phones, though Nokia is updating many phones with a newer browser that supports up to HTML 5.0 and JavaScript 1.8, most of these phones will receive the feature phone version of the site, however these are low usage devices. In stats, these phones can appear as either S60 or S9.x, the latest version of the S9.x is 9.5. Finally there is a new version of Symbian, sometimes seen as Nokia OS or Symbian Belle, this version is also a hit and miss support OS. While this version is low usage, it could become more common in emerging countries. Given that Nokia doesn’t use Over The Air (OTA) updates for its phones, users of the Nokia N8, the latest Symbian phone, could be running any one of 3 different versions of Symbian, each with its own version of the webkit browser, only the latest version supports HTML 5. Some phones will also come with some version coming with Opera, netfront and webkit depending on location, so Symbian testing is very limiting when trying to ensure maximum support. In our testing, we should focus on the Nokia N95 and Nokia N8, these feature different versions of S60 and have different browser versions, so provide a solid range of old browsers in Africa, with modern the UK. The most recent UK launch of a Nokia Symbian phone is the N8, which currently is classed as a feature phone, even though it support is relatively good, it does not cut the mustard. The best browser for this type of Symbian phone is Opera mobile, fast and fully feature, we do a lot more testing of Opera on Symbian S60 with Opera than any other platform. Link to different Symbian versions: http://en.wikipedia.org/wiki/Web_Browser_for_S60
Symbian 10 is also known as Symbian Belle, a new brand from Nokia. This comes with a brand new browser which is Webkit based and supports lots of good HTML5 features. There a few notable issues, such as font embed support, but it brings a few Nokia/Symbian phones into the realm of HTML5, which is a good thing. Check out my post here for more info
Silk Browser. Amazon has forked Android 2.2 to create the Kindle OS with Silk browser. This browser routes all traffic through Amazon servers and can affect some elements of the website. Form submission is something that should be thoroughly tested to ensure the server is able to receive the form data. You may also find that because of the form factor, you will have to make a decision, only use smartphone layouts or use both tablet and smartphone layouts, this is mostly due to the screen width being able to support tablet web pages. But what is most troubling for the tester is that you may have to test both of these layouts on the say device, this is going to really annoy you.
Silk Browser version 2 is a fork of 4.0 with the AOSP browser. This browser has had some changes for the cloud sync, but has the same support as Android 4 stock browser.
The iPad is currently the market leader, with the single device having 60% of the market, the market grows every month though, so the user base is ever growing. While the differences to the iPad browser are subtle to the iPhone browser, it is important to look at the iPad as the biggest device, in terms of screen size, that users will visit with. While this device can support most, if not all, desktop sites with ease, it is important to remember that speed and simplicity are often desired more than functionality. For this reason, users of iPads and other tablets, will use the mobile version of the site, especially if they are using a data service. If it is outside of the scope of your project, you should consider potential markets with the iPad, responsive news’ largest device support is 7 inches, however testing is conducted on the iPad.
Browsers
———————————-
The next section will include information about the browsers that you will need to test in addition to the native browsers we have already talked about. Just to reiterate, all the operating systems we have talked about so far have their own native browser.
Third party browsers that have big impact in the world include:
UC Browser - Information
Chrome Mobile - Information
Opera Mini/Mobile - Information
Firefox - Information
As we are test for users all over the world, UC browser and Opera Mini may be less important to you. These 2 browsers combined represent a sizeable chunk of Africa and Asia browsing traffic, so if you are in that region, you are probably already testing on it. Firefox is growing, but they are updating rapidly, so testing on the latest stable build is advisable, especially as it is more than likely the user will upgrade. The complete custom build is only available on Android. Chrome is not yet the default browser, it will also sit alongside the native Android browser, at least for now. Testing on both is probably the best approach, similar to Firefox, testing the latest build is best. There is a version of Chrome for iOS, but this uses the same version of Webkit as the version of IOS you are on. The Android version of Chrome is update frequently and is the only place to get the complete Chrome build.
Summary
———————————-
In total, you final device list could have other smaller markets included, our current device list looks something like this :
- iOS 6 - iPhone
- iOS 6 - iPad mini
- iOS 6 - iPad
- iOS 5 - iPad
- iOS 5 - iPod
- Android 2.1 - HTC desire
- Android 2.2 - Samsung Galaxy S
- Android 2.3 - Samsung Galaxy S
- Android 3.2 - Motorola Xoom
- Android 4.0 - Galaxy Nexus (Stock and Chrome)
- Android 4.1 - Galaxy Nexus (Stock and Chrome)
- Blackberry OS 4 - Curve 8520 (We have recently dropped testing this browser)
- Blackberry OS 5 - Bold 9700
- Blackberry OS 6 - Bold 9700
- Blackberry OS 7 - Bold 9860
- Windows Phone 7.5 - Lumia 800
- Windows Phone 8 - Lumia 920
- Symbian S60 - N95
- Symbain S60 - N8
- Symbian S40 - C3
- Symbian S40 - Asha 303
- Kindle E-Reader
- Kindle Fire - Android 2.2 Fork
- Kindle Fire HD - Android 4 Fork
- Android 3.0 - Samsung 7.7
- Android 4 - Samsung 7.0 v2
- Android 4.2 - Nexus 7
- IE 10 - Microsoft Surface
Edit 20/6/2012 - We have recently determined our 3 in market tablets to look at, Kindle fire, BB Playbook and Galaxy Tab 7.7. They are all webkit based browsers, but have some differences in support, as well as volume of sales. The 7.7 uses a pixel ratio of 1, whereas the other 2 dont, this means that the break points will have to be correctly established for tablets in order for users not to receive broken features.
Edit 29/8/2012 - We are no longer testing on BBOS4 anymore, we have found the share to drop below our support level and global usage has dropped to an all time low. We have added other versions of Android and other hardware sets, such as 7 inch tablets.
Edit 13/9/2012 - I have made some big changes to the testing list this month, we have changed the minimum level of support to start at Android 2.2 and BBOS6, this is due to total browser market shifts on the site as well as globally and within the UK.
Edit 13/11/2012 - I have now updated list to include changes to BBOS7, IE 10 on Windows and some Android and iOS support changes.
Edit 16/01/2013 - I have updated Windows phone to indicate our new support list, as well as changing our testing devices
Android’s Ugly Family
What Android will sometimes present, is a bug you simply cannot explain. This is one of those bugs that you have tested on every version of Android, but only shows on one. In our case, Android 2.1 shows it annoying side by creating a bug that continues to overlay a text field once submitted.
How can you determine if this is a bug with your code, or a bug with the OS, or a bug with the browser? Get another phone.
Our Apples to Apples comparison is between the HTC Desire and the Sony Xperia X10, both running Android 2.1 update 1.
Once we get into it, the picture above shows what happens when we input data into a search box and hit enter. This issue seems to be intermittant on this phone, it will depend on the website. What we learn from this issue however, is that it is not persistent across all Android phones. The X10 doesnt produce this bug, so now we entertain the possibility that HTC has slightly forked the browser……
If testing a website means buying a HTC phone with sense, Samsung phone with Touchwiz, Sony phone with their skin and a Google phone with no skin, the cost of testing Android 2.1,2.2,2.3,3.2 and 4.0 suddenly spirals, it also increases test lead time.
However, thanks to our stats, we can see which phones are more popular, which will determine which phone you should buy to test. In most cases, starting with the pure Google experience is going to be a good place to start, the Nexus line of phones are what you should look for, but these pure versions are not the biggest sellers.
While Android fragmentation is an issue for the tester who is looking at so many different versions of webkit and Android, there is light at the end of the tunnel. Chrome for Android is being introduced on Android 4, which could mean that a locked down browser will provide consistency and a more reliable test environment.
For now though, Androids fragmentation will continue with more elaborate skins and changes to the core OS are going to occur, ultimately meaning more devices and more testing to be completed.
