Like a lot of public bodies, we pay for automated website testing/ranking. We use SiteMorse, which is one of the industry leaders. I know a lot of web managers swear by competitor Site Improve and swear at SiteMorse.
I’m not passing comment.
We got the results of the latest SiteMorse survey over the weekend and we’re rather pleased to come in at number 2 out of over 400 local authorities.
Number 2 is great, especially for our KPIs (!) but I’ll admit that it’s not the be-all-and-end-all.
SiteMorse tests amongst other things:
- functionality: whether the site and navigation works
- code quality: does the site pass W3C muster?
- accessibility: against WCAG
- performance: webserver response time
There’s no user testing, so usability and content don’t come under scrutiny.
SiteMorse is a good as far as it goes. It’s not perfect, but it’s a good snapshot of whether or not the site is technically robust. We consistently score in the top quartile so I guess we’re doing something right.
In a few weeks, we’re likely to get the results of the Better Connected survey from SOCITM (and a bit of a Twitterstorm!) As with SiteMorse (or any other testing, for that matter), we treat Better Connected as a guide. It’s a snapshot, by one reviewer on one day of the year.
Where BC scores is measurement against more objective scenarios. It tests content and usability. But, my experience is that the reviews against objective tests sometimes become too subjective, as reviewer personal preferences sneak in.
SOCITM says it cross-checks and reviewers argue long and hard before coming to conclusions. And, to be fair, SOCITM is willing to respond to grumbles from council webmasters.
There have also been heated arguments in the past over the validity of SOCITM/RNIB accessibility testing which more or less say all but a handful of council’s break the law. I’ll reserve judgment until I see this year’s results.
For our part, we’ve used both surveys to feed into our work improvement programme. But, neither SOCITM nor SiteMorse actually use our website on a daily basis.
There is no substitute for user testing from real people seeking service or information.
Local user testing can be quick and cheap, possibly better value. In the current climate, I wonder whether some authorities might start looking at the cost of services like SiteMorse and SOCITM subs.
(Just to re-iterate the disclaimer on this site: any opinions here are mine and not necessarily my employers. Budget decisions are not mine!)