March 2006 Archive

« February 2006 | Main | April 2006 »

March 30, 2006

Seduced by automated testing?

@ 6:39 PM

There's a wee bit of this year's Better Connected that escaped my attention on first reading, but happily an item in Headstar's very worthy E-access Bulletin (external link) this week led me back to the report. It concerns the disparity between WAI conformance claims on councils' websites and their real level of comformance.

Of the 296 sites in the transactional (T) and content plus (C+) categories which claimed a particular level of conformance, only 69 were found to achieve that level in reality, or just 23%. There are only really two explanations for such an alarming disparity - either the councils in question are deliberately over-stating their conformance level, or, more likely in my opinion, they are being led to believe that their sites are achieving a higher conformance level than they really are.

Better Connected suggests that the culprit might be automated testing:

There is no doubt that achieving Level A is hard work and that measuring it is a complex business. Many might also be lulled into thinking that passing the automated tests of Level A (and Level AA and AAA) means that you have achieved comformance at those levels.

If this is true it's fair to say that the use of automated testing is effectively damaging the accessibility of the sites in question, rather than improving it. Given the gravity afforded to the SiteMorse league tables in some quarters, it's easy to understand why councils might be seduced into developing and measuring their sites using the company's tool alone. But as has been said before (external link), the number of WAI guidelines that can be reliably tested with automated software is very small indeed, and the only way to really know if your site is accessible is to have people use it, preferably disabled people using a range of assistive technologies.

An analogy I like to use with non-technical, non-web managers is that of a car's MOT Test (for non-UK readers the MOT Test is a comprehensive safety test that cars have to pass every year). A full accessibility audit is like an MOT Test - it delves into aspects of your site's performance and accessibility that you can't reach yourself, and that you really aren't qualified to judge. An automated test on the other hand is like emailing a photograph of your car, with a note of the make, model and year, to a bloke who knows a bit about cars, and having him judge on that evidence alone if your car is road-worthy and safe to travel in. Which would you rather your passengers travelled in?

PS: I know this is a familiar refrain, and I know that I bang on about it all the time, but I've been convinced of the value of repetition (external link) by Jeff Atwood.

March 29, 2006

Guardian Inside View

@ 12:50 PM

When Better Connected (External link) was published at the beginning of the month I was asked if I'd write a short piece for the Guardian newspaper detailing our approach to web development at Clackmannanshire Council, with a slight emphasis on accessibility. I was very happy to do so, and it was finally published in today's paper, in the Epublic supplement. You can also read it online at the Guardian site - Why size doesn't matter in setting web standards (External link).

Also worth a read is the headline article in the supplement, Online, but out of touch (External link).

March 24, 2006

Where are the gatekeepers?

@ 10:53 PM

I'm a great fan of standards. They provide a constant point of reference, an ideal to measure yourself and others against. Not just the standards that are set for us by the W3C and their ilk, that have far-reaching and universal benefits, but also those we set for ourselves. Without standards how can we know for sure that we're achieving the levels of quality we aspire to?

It seems to me to be a lack of standards that has led Feather to post about the (mis)use of significant wads of public money  (external link) for the sponsorship of a conference, some of which will have gone towards developing an inaccessible website  (external link). He asks:

What if our provincial and federal governments made web accessibility a requirement for actually recieving the sponsorship money? What if organizations that get any funding from the government had to have accessible web sites? Would any of that help awareness? Would it make a difference? Is it simply that accessibility wasn't a requirement on the project, and so it just didn't happen?

My gut feeling is that awareness of web accessibility issues is still next to zero outside of the small but steadily expanding web standards clique. We are getting there, slowly, but when the websites of the agencies Derek cites, the Ontario Media Development Corporation  (external link), Canadian Heritage  (external link) and the Canadian Broadcasting Corporation  (external link), aren't accessible, what hope is there that the websites of projects they are sponsoring are going to be held to higher standards?

Here in the UK we're no less guilty. In recent months I've covered the brand new and in many cases horribly inaccessible websites of agencies either partially or wholly funded by our national government, and expressed my frustration at their disregard for (or ignorance of) the government's own standards  (external link).

The question I keep asking is where are the gatekeepers in these scenarios? Setting standards is only effective when someone is doing the measuring at the sharp end, fulfilling the quality assurance role that gives practical foundation to the commitment made in the standards. Or you could call it putting your money where your mouth is.

I'd like to see an enforcement of standards for all new domains, with domains only allocated to projects once they have demonstrated the necessary commitment and follow-through on accessibility and other standards. Currently the conditions for use of domains  (external link) states:

When you are using a domain name to deliver a web presence you are reminded that websites should comply with the e-Government Interoperability Framework, the Guidelines for UK Government websites and Framework for Local Government particularly on such issues as use of metadata, PICS labelling, accessibility and security.

Excuse my language, but screw "reminding" them, that's all just a bit too afternoon tea and bowler hats for my liking. If they don't comply then don't allocate the domain, or if it's already been allocated then withdraw the domain, after a warning shot if you want to be soft. I'm sure it would concentrate the mind wonderfully.

March 12, 2006

I can't complain, but sometimes I still do

@ 7:08 PM

Blair asks a searching question (external link) about inaccessible websites over at The Letter - just who do you complain to?

A user's first instinct might be to contact the site owner, but my personal experience is that very few even acknowledge such complaints, let alone act on them. Last year I (anonymously) emailed over a dozen local authority websites which had serious accessibility problems despite claiming AA or AAA conformance, but only one responded (kudos to East Renfrewshire Council (external link) who quickly addressed some of the problems and edited their accessibility statement).

Blair's suggestion is a link to the DRC's website inviting users who have found an inaccessible website to report it to the DRC. A fine idea in my opinion, which might pave the way to a dedicated reporting facility should the DRC see the volume of complaints increase.

March 8, 2006

PAS 78

@ 1:50 PM

It's a big day for web accessibility in the UK with the launch and publication of Publicly Available Specification (PAS) 78 (external link) in London. The specification was developed by the Disability Rights Commission in collaboration with the British Standards Institute, providing guidance to those commissioning websites to help them to understand the issues around web accessibility, and to ensure that the work they commission results in accessible sites.

It's attracted plenty of publicity today, including a good piece on BBC News (external link), so here's hoping that it makes a material difference in the months and years to come.