Shaw Trust User Testing
Accessibility was a prominent feature of ClacksWeb's development plan, reflecting its status as a local authority site and my increasing awareness of it accessibility as a critical issue. As work on the site progressed in early 2005 I undertook regular checks to make sure it was on track to deliver, both by checking against the WCAG and by using very simple tests like increasing the text size, browsing the site with Lynx, and navigating with the keyboard. Yesterday it became very clear to me just how inadequate that sort of testing was in truly determining whether or not the site was accessible.
I had the pleasure of spending the day at the Shaw Trust's regional headquarters in Llandarcy, Wales, experiencing first-hand some of the user testing of ClacksWeb. Accompanied by Andrea Kennedy, the Trust's Web Accessibility Services Officer, and Grant Broome of CDSM , who co-devised the audit programme and conducts the technical audit, I met, observed and learned from 6 users with various disabilities and levels of web experience. They were:
- Linda - a Jaws user;
- Steve - an accomplished motor-impaired user of a keyboard with a guard;
- Malcolm - a Dragon NaturallySpeaking user who also tested the site with ZoomText ;
- Jamie - a Jaws user;
- Mark - an accomplished Jaws user;
- Ann - a dyslexic user who undertook readability testing.
The users are provided with scripts by Andrea - basically a series of tasks, for example yesterday the users were required to register with MyClacksWeb - and are asked to record their positive and negative experiences while working their way through them. They all use PCs running Windows XP. The atmosphere is relaxed and informal - this isn't a sterile, clinical testing lab, but somewhere the testers seem to come to socialise as well as make their services available to the Trust. They are all volunteers, although there are plans to create a social enterprise offering user testing on a commercial basis - an excellent idea in my opinion.
I spent some time with each user, observing and asking questions, or in the case of Mark listening in to his use of Jaws. This was an invaluable experience for me, seeing and hearing how these users navigated the site, what barriers they were facing and what strategies they each used to overcome them. It only took a couple of minutes of observing Mark, and having him talk me through his perception of the site, to discover a serious problem with the site's contextual navigation menus. Basically they are placed just above the destination of the site's "skip to content" link, so he never knew that contextual navigation existed, and was forced to use alternative methods such as search or the A to Z to complete the tasks.
Another serious problem was encountered by Steve, the keyboard user - I hadn't specified any focus styling on links, so when tabbing around the site it wasn't obvious which link had the focus. The browser default, faint dotted border just wasn't enough for Steve to perceive the current focus. These two examples show just how important pan-disability user testing is. Neither would have been discovered by automated testing (99% of pages on the site are valid XHTML and satisfy WCAG to at least AA), and neither user was affected by the problem the other discovered.
More testing was scheduled for today, by another group of users with different disabilities, with the technical audit to follow soon thereafter. I expect to receive the initial report late next week, and based on my experiences yesterday have no doubt that it will prove to be an extremely valuable resource.