Usability testing reveals the Good, not just the Bad or Ugly

If you are considering a re-design of your website usability testing is a great way to inform what to focus on because it reveals user experience issues. But, just as importantly, it can also reveal what is working – so you avoid “improving” something that wasn’t broken in the first place! In this website usability review we highlight some great user experiences on the Comic Relief website.

It is an unfortunate feature of these post-financial crash times that charities are feeling more than their fair share of the squeeze from people tightening their belts. Fund raising is a tough business in which every single penny must be made to count. These organisations do not have the luxury of inadvertently losing custom due to poor usability or underwhelmed users and are unlikely to experiment. We thought we’d take a quick look at this sector  to find out how people are rising to this challenge.

User Selection and Test Set Up

We decided to ask some potential users to test out Comic Relief’s Red Nose Day. We screened the users to ensure that those who responded would be open to the thought of donating to charity. This was an important step since we wanted the results to be as accurate as possible. A study performed in 2004 by “The Giving Campaign”  found that certain sections of society would never be likely to donate. (I was somewhat horrified but sadly not surprised to learn that these demographic types were “higher-rate taxpayers with expensive leisure interests and fashionable homes” and “well-educated, stylish young professionals”.)  Thus we ensured the screening questions revealed behavioural attitudes given that we could not go on socio-economic factors alone.

The task itself was simple. We asked people to explore the site and find some activities for Red Nose Day that they would genuinely consider undertaking.

The Site http://www.rednoseday.com/

The look and feel of the site is very graphically rich – with large photographs of the celebrities and others involved. The design elements are bright and bold but with an overlaid appearance on a grey background and plenty of  white space it looks clean and easy on the eye. The overall effect is that it is a cheery, approachable site. There are plenty of ideas for activities and events the organisers have thought hard how to make it easy for people who may not have experience of organising charitable events. The site also targets information at people who may be organising activities at work and in schools.

Findings

All of our users successfully found their way through the suggestions for fund raising activities and also to the free fund raising packs for order. They commented positively on the visual design and the overall look and feel. The site succeeded well in providing an engaging experience for these users.

►

►

 

Conclusion

As we know, usability testing is an extremely successful  way to revealing the problems in the UI , but just as important is knowing what works so the good stuff isn’t thrown away in the next redesign. Testing little and often is great way to ensure your designs are still on track. Just imagine filling those PowerPoint presentations with good news for a change!

New Feature: mixed participants

Our clients told us that we could make WhatUsersDo even easier if they could combine  different user demographics in a single order. We listened and have released this important feature on the online order form:

Easily select different user profiles using the dropdowns

Now it’s even easier to get a balance of gender, age and Socioeconomic Groups when setting up online User Experience research with WhatUsersDo.

We’d love to know what you think – please leave a comment or contact us.

New features on WhatUsersDo

Just in case you missed them, we’ve just added some new features for all WhatUsersDo clients:

1. Multiple-logins

You can add multiple-logins to your WhatUsersDo account (even if you only buy one user video) and start sharing your insights with colleagues.  If you already have an account, you can do it right here:  http://www.whatusersdo.com/viewaccount.php

2. User filtering

Now you can filter our UK user panel by Socio-economic Group and get even more relevant users and therefore insights. Use the drop-down on the order form or read more about Socio-economic Groups.

That’s all for this week! As ever, do let us know what you think by leaving a comment or emailing support@whatusersdo.com.

Disneyland Paris User Experience Review

There’s no better way to chase away the Winter blues than by booking a holiday and having something fun to look forward to.  If you can book one that is. This usability review of the Disneyland Paris website left me thinking “How can a multi-billion dollar supposedly consumer-centric business get things so wrong on their website”?

The Task

We asked pre-screened users from the WhatUsersDo panel to imagine that they were thinking of treating the family to a weekend away at Disneyland Paris. As they used the site to find a package that they’d genuinely consider buying, their screens and spoken thoughts were recorded into online videos.

Home Page

Things did not start well. On first entering the site, users are assailed by a musical cinematic fanfare that is so loud that, should you happen to be at work, you can be sure your boss, and probably even your boss’s boss will be alerted to the fact that you are having a few minutes down time on some ‘personal admin’. It’s very unusual for someone to go straight to a site and make a purchase of this size on that first visit so you can expect the trumpeting accompaniment to annoy you not once but each time you enter the site thereafter.

Home Page Fanfare

►

Once the auditory senses have recovered, the eyes can start to take in the confusing array of options on the page. The large moving graphic distracts attention from what are a competing set of navigational options. It is difficult to know not only which option to start from but whether going down one path will preclude you from taking advantage of ‘offers’ from another.

For example, which is the most appropriate top navigation route – ‘Choose your Experience’ or ‘Find your ideal offer’? Within ‘Choose your Experience’ if I have a young family should I pick ‘Families with toddlers’ or ‘Family Adventure’? The user by now is already being asked to think and weigh up far too much .

Language and terminology used in the navigation feels very internal to the Disney organisation, but not that helpful for users: ‘Disney Parks’, “Disney Village’  - can anyone but a seasoned Disneyland visitor instantly know the difference here? The site doesn’t help you to understand what there actually is at Disneyland Paris.

“It is not very user friendly…you have to work it out as you go”.

Things did not improve for our users who made it into the booking process. There are several paths into this, each with its own problems. There is a booking form on the home page which several users started with however each ran into different issues.

There is a distinct lack of clear direction through the process and users were unclear of next steps making it unlikely outside the test environment that they would ever be confident enough to complete a booking.

Booking Process Unclear

►

“I don’t want to phone I want to book online”

In another example of being confused throughout the booking process, this user was told that the holiday could not be booked on-line and that she must complete the booking by phone . It has been at least 10 years since I have seen that most unhelpful of instructions on a website!

Booking Process Confusion 2

►

Family friendly?

One user, trying to book for two adults and 3 children, was surprised to see that the maximum number of children only went up to two.

Family of five or more? Tough – it seems. Had this user thought to look at the bottom of the form at a badly labelled link and clicked on it, she could have read about a ‘workaround’ for this – make two separate bookings and then ‘link’ the bookings in the order summary screen.

So, the booking system is not flexible enough to cope well with this common scenario and it requires the user to do the leg work. Would it really be so hard to include the number of rooms in the booking criteria?

Preventing errors – not creating them

Elsewhere in the site this simple test uncovered yet another common usability mistake from circa 2001 – not designing to prevent errors and displaying unhelpful error messages when the unwitting inevitably fall into the trap.

The booking form on the home page loads with today’s date as default. Should the user not change that (a blank field is more of a prompt to enter information) then on the next page when the user is required to choose a hotel,  they find that all the hotel links are disabled and greyed out. And no, there is no message to inform the user to choose a date – the only instruction on the page is to choose a hotel.

Error Prevention – Default Date

►

A Mickey Mouse validation error

A further example of this painful lack of form validation is in entering dates of birth for children. Should you choose not to do so the form will simply not submit leaving the user hanging on and staring at the unchanging screen.

A further example of this painful lack of form validation is in entering dates of birth for children. Should you choose not to do so the form will simply not submit leaving the user hanging on and staring at the unchanging screen.

Error Prevention – Mandatory Fields

►

Offers are off!

Users also uncovered an annoying issue that the site would let you get so far into booking an offer, even to the stage of booking transportation, before telling you that the offer was not available. When you have invested time and effort on the understanding that you had all the information available this is clearly annoying and off-putting.

Conclusion

Booking a holiday can be a nerve-racking process, especially when you’re responsible for other people. There is a lot at stake: time, money and not least happiness. It is important for users to be confident not only that they have made the right decisions but also that their requests will be carried out to the letter once their holiday starts.

Disneyland Paris’ customers undoubtedly get a great experience at the park itself and the strength of the brand will mean many users will put up with a poor online user experience.  But, should they have to?

How we did it

We posted a simple task on WhatUsersDo asking users to imagine that they were thinking of treating the family to a weekend away at Disneyland Paris in Summer and to use the site to find a package that they’d genuinely consider buying which included travel. We screened users specifying that they should have children under 12. We then inserted tags and notes at interesting points during the recording and made our conclusions.

New feature: get users in your target market

Now you can get feedback from people in your target market with WhatUsersDo and make your usability test results even more relevant.

In addition to filtering by age and gender you can set a qualifying question (or screener) that users must answer before they start your tasks. And only users who choose answers that match your target market will be selected.

For example, if Sky wanted to test their website with only Virgin Media or BT Vision customers they could setup a question like this:

… users would be shown a multiple choice question before seeing the task instructions and only those who answered Virgin Media or BT Vision would be included.

It’s really quick to setup your own pre-qualifying question on the order form.

Please contact us with any feedback or just leave a comment below as we’d love to know what you think.

Snow Disruption: Gateshead.gov.uk usability review

As the snow spreads across the UK, people are turning to their local authority websites for the latest local information. With the adverse weather now into its fifth day in the North East of England we were not expecting to find too many usability issues at Gateshead.gov.uk. How wrong we were!

We set a single user three simple tasks to complete.

1. Is my child’s school Open or Closed?

The website’s home page has a prominent link to information about school closures and navigating to a specific school is relatively easy (they are listed alphabetically). Against each school a box displayed the status either Open or Closed but for many that box was empty.

In the following clip the user simply could not tell if their child’s school (Kells Lane) was Open or Closed and the link to school page did not work.

2. When will my bins be collected?

We thought that finding out whether bins are being collected or not would be a key task and might merit a link from the Home Page (in the same area as the School closures link), but it isn’t. In the following clip the user eventually found the information by using search, after navigating to the Collection and Disposal section of the website which did not yield the expect information.

3. Request for Gritting

Citizens can request that a street or path is gritted. Naturally, all councils need to prioritise gritting at these times, but we were surprised to find a broken link on this area of the website.

For some reason the Request Gritting page is a secure (https) page which only adds to the confusion.

Watch the following clip to “feel the user’s pain” and watch out for the broken link!

Summary

This single user test revealed some obvious areas where Gateshead Council could improve how information is both found and displayed. Getting this right will help build trust with Citizens who, according to central government policy, will be expected to increasingly transact online to reduce costs.

We normally recommend testing with five users, but in this case some of the issues were so obvious we stuck with just one.

Online Christmas Shopping – Firebox.com review

Avoid annoying your users to increase sales

So the Christmas cards are in the shops, the first TV ads starting to appear and awkward conversations about whose turn it is so host the big day are starting to happen. Without a doubt Internet shopping has made the whole experience of Christmas shopping a heck of a lot easier for those of us who work. Sometimes however an online experience can be the equivalent of leaving an overcrowded shop empty handed and in a huff such is the lack of usability on the site. Obstacles that exist in the real world e.g. not being able to find an item, crowded shelves, or worse, rude shop assistants,  unclear price tags and poorly signposted cash desks all have their equivalents online.

We decided to take a look at Firebox.com, a popular gift site to see how it fared in the run up to Christmas. Firebox sells a huge range of gadgets, toys and games for prices starting from under £5 to over £150. We asked some users to imagine they were buying a gift for a partner or friend for under £10.

First Impressions

First impressions of the site were that it was very busy and overwhelming. This may be a deliberate design on the part of Firebox as it does convey the wealth of items available for sale, but it does not help the user know where to start. However, with the competition only a click away the overwhelmed user may not hang around.

First Impressions

►

Allow users to easily scan content

There are several ways to narrow down the range of items available. As well as ‘Top 10’ and ‘What’s New’ lists, on the left there are price brackets and options for who you are buying for. Most of our users opted for a ‘Gift Finder’ option which lets you narrow by person, price and type of gift.

This is another very busy page requiring some effort to sift through the items. The page shows small thumbnail images with item name and price available only when the user mouses over. This was annoying to some users who would have preferred to have that small amount of text in the page itself. The current design makes it impossible to quickly scan the page and mentally select or dismiss items. With such a wide range of stock this design simply adds more work to the purchase process.

Scanning items

►

Design to build trust

With the price limit of £10 the test users altered the price slider on the left to be a maximum of £10. What they found however was that this limit seemed to reset itself to £15 when they came back to this page after viewing the item details page. This caused confusion and annoyance with some believing it was a deliberate ploy on behalf of the retailer. Hopefully this is just a bug in the slider code rather than something more calculated but issues like this can erode the trust a user must have in an online retailer and should be avoided.

Price Slider Rest 1

►

Price Slider Reset 2

►

Avoid annoying customers on the way to the cash desk

On the home page, all the test users had noticed a banner stating delivery was free in October and remembered this when going through the purchase process.  On the delivery options page however,  the delivery type defaulted to ‘Standard’ delivery which was not free. Not all the users noticed this at first and so were confused as to why their total was more than they expected. This also caused annoyance with one users saying this was “sneaky”  - another example of a simple design decision damaging the user’s trust.

Default Delivery Setting 1

►

Default Delivery Setting 2

►

Conclusion

The site sells wide range of stock which is perhaps its greatest strength. It also provides, on the face of it, easy tools with which to narrow down the options. However when it comes to the actual usage of these tools there are clearly some serious flaws – in particular the unexpected behaviour in some features and options  which could certainly put users off committing to a purchase and more importantly returning to the site in future. Keeping and converting current customers is always less costly than attracting new ones through marketing and advertising and this is a great example of how making a few very simple changes to the usability of the site could achieve this.

Firebox.com could benefit from the following:

• A multivariate (A/B comparison) test between the current home page and an alternative simpler design to find out which option results in more sales would indicate the best design direction.
• Layout of  product list pages (e.g. Gift Finder) should be improved in order to facilitate scanning and enable users to choose between options quickly. An alternative design to the Gift Finder page which displays text inline (instead of on hover as in current design) could also be tested in the same way as above.
• The possible bug which resets the price slider’s minimum price should be checked out and fixed.
• When free delivery is offered, use this as the default delivery option. Users will appreciate it.

How we did it

We posted a simple task on whatusersdo.com asking users to imagine they wanted to find a gift for a partner or friend under £10 and gave the firebox website address. We specified an age range of 18-35 and any gender. Each user’s session was recorded into an online video. We then inserted tags and notes at interesting points during the recording and made our conclusions.

BBC iPlayer redesign – user experience review

The new version of iPlayer launched a month ago. I imagine it was a nervous time for the people involved over at the Beeb. Going about changing something so well-loved would I suppose be a bit like considering another actor to play Alfie Moon. I am an infrequent user of the service, relying mainly on my PVR but last week the inevitable happened and I managed to miss my all time favourite programme Mad Men. I suppose this must be one of, if not the most common user goals that the iPlayer designers have in mind when approaching their designs – allowing users to easily find an episode of something they have missed. It’s testament to the designers that I found my programme easily and can’t actually remember the steps I took to do so.  However that was one person and one scenario.

Test Set Up

In order to better understand how successful the redesign has been we tested it with 6 other users and more scenarios (using whatusersdo.com). We selected a range of ages to recognise the BBC’s reach, several of those being over 50.

The tasks were firstly to find a favourite programme of the user’s choice and in order to delve a bit deeper “a recent documentary about Scientology” which wouldn’t be so obvious to find. We also included tasks relating to the new features for this release: sharing recommendations and favourites.

Searching for a programme

There is absolutely no substitute for seeing real users in action on a website. I have yet to experience a usability test that did not throw up something new. On this occasion, the first finding to take me by surprise was the way in which people went about initiating their favourite programme search. Three out of the six users entered the programme name in the main search box on the home page without first navigating to iPlayer. The results page has two sections – on the left programme and content links and on the right results from iPlayer. Users who went down this route tended to focus on the left hand list and therefore missed several possibly good matches from the iPlayer links on the right.

These two examples show users entering their search and their reactions to the standard search results page.

►

►

Search Results

The iPlayer redesign has deliberately given the same priority to radio content as for TV. However the two types of iPlayer links are not distinct from one another which could cause problems. Several of our users were not confident that they were going the right way to find their programme when seeing radio programme titles in their results. In my view there should be more clear visual indication of the result type in or around the link itself so it is more obvious at first glance.

There is an autosuggest/autocomplete tool on the search box which takes users to a results page different from the one if autocomplete has not been used. If there are no plans to standardise the pages in future then this seems inconsistent. This layout of the autocomplete results page makes the different types of results more prominent and better differentiated and users on this page found the iPlayer links more easily.
This extract shows the autocomplete results page and the prominence of iPlayer links:

The autocomplete tool only seems to be populated with index headings and series titles rather than programme titles which caused some confusion for our users who weren’t aware that the documentary “Secrets of Scientology” was a Panorama programme. Thus typing “Scien…” brought up Science in Action and Church of Scientology but not the programme title as shown in this clip.

Embedded iPlayer

Another issue  highlighted by those users who used the main search field was that the experience of iPlayer differs between accessing it embedded within programme pages and accessing it in the iPlayer pages. In the embedded version one user could not tell if she was looking at the most recent episode of Songs of Praise as there is no obvious date label. When asked to find how to share the programme with friends the users who had accessed iPlayer from the programme pages had great difficulty finding this feature.  For those users searching for their programmes in this way, it’s possible that they may never even access the iPlayer pages themselves.

Additional Features

This next clip shows one user struggling to find how to share or recommend the programme and looking through the Help pages for it:

Those users attempting to browse for the documentary had some difficulty. Category headings do not seem to reflect users’ own perceptions and may be too internally focused. For example, the Panorama documentary on Scientology is contained in ‘News & Current Affairs’ whereas our users tried to find it in ‘Factual’. The programme lists are sorted by date by default but this is not immediately obvious nor is the option to change this. Fortunately the iPlayer’s own search and results pages proved a successful back up in these instances but browsing could be improved to support users without a particular programme in mind.

One of the new features is the ability to indicate Favourites and iPlayer will then collect them in your Favourites collection. This concept was well understood by most but none of the users indicated that they would know how to go about doing this. There is a star icon underneath the video window that can be clicked and toggled on or off. However this does not appear to be available in the embedded version of iPlayer in the programme pages which is why several of our users missed the feature.

TV Favourites

►

The new Last Played feature was intended to allow users to view programmes from where they left off. However our tests showed that the feature did not display any programmes that had been only partially watched. This could be a temporary bug.

Conclusion

On the whole, it seems that the redesign is supporting the primary user goal very well. There are several ways available for users to find their programmes easily although identifying types of search results could be clearer. Featured programmes on the home page pod and in the iPlayer section surface the most popular items and aid serendipitous discovery of others. Browsing and taxonomies could be improved on in future releases to ensure it more closely matches users’ understanding rather than reflecting the BBC’s internal language and categorisation.  The embedded iPlayer could include the additional features of the full version however they could be considered more appropriate for the frequent user rather than the casual surfer.
Lastly, one of our older users (surely now a significant proportion of the BBC’s internet audience) highlighted that the readability of the site could be improved upon which would improve accessibility for the older generation.

If this could be achieved without affecting the slick visual design then together with the improvements suggested above I  think the iPlayer team could then collectively turn off their computers and book that long holiday they’ve probably been denying themselves up to now – job done.

Getting started with Conversion Rate Optimisation

CRO (or Conversion Rate Optimisation) is all about maximising the number of visitors who take action on your website, be that subscribing, buying or making an enquiry. There are numerous tools (including our own) to help you do this, but where to start if you’ve not done it before?

Google have published a terrific free e-book that’s a good place to start. You can download it here.

A Google tool called Google Website Optimizer features in the e-book (it lets you run conversion experiments on your website),  and here’s a great list of alternative tools (but Optimizer is a great place to start not least because it’s free).

Smart Insights review of WhatUsersDo

We were lucky enough to be interviewed by Dave Chaffey of Smart Insights recently where our founder (Lee Duddell) explained how people use the WhatUsersDo service and how our clients benefit from the User Experience insights it provides.

Read the interview in full.

If you’re not already following Dave or the Smart Insights blog, then we highly recommend that you do – it’s a great source of practical tips and tools to improve your online marketing. There’s a terrific post today on how you can get more sales and leads from your website.