Guidelines and User Testing – Let’s Talk About Food Instead!

Everybody loves a good analogy and surprise surprise I’m no different! I want to revisit an issue that I’m sure has been raised numerous times before – the differences between guidelines and user testing and the benefits of each. Now feels like an important time to talk about this again given the imminent release of the next version of the Web Content Accessibility Guidelines (WCAG 2.0). At the recent Accessibility 2.0 conference held by AbilityNet, there was also some talk of how useful guidelines actually were compared to user testing, so I thought I’d take the opportunity to put my opinion out there.

So, with all that in mind, let’s talk about food instead!

Guidelines – The Recipes for a Better Web?

Recently in the office, a colleague offered me a taste of her lunch. It was the national dish of Jamaica, Ackee and Saltfish, which is a combination of a fruit called Ackee and salted cod served with rice – very nice it was too! I got her to email me the recipe so I could try my hand at cooking the dish sometime in the future.

OK, so how am I going to relate this to guidelines and user testing I hear you ask? Let’s imagine what it might be like if I hadn’t got the recipe for the dish or we didn’t have the WCAG guidelines. All we would know is that we have a goal we are trying to achieve, i.e. cooking a dish or building an accessible website. Depending on our level of expertise, we may or may not know how to go about achieving these goals.

Now, I’ve never cooked the national dish of Jamaica before in my life! In theory, the recipe should give me a fighting chance of getting it right, but that’s all it gives me. The recipe won’t guarantee I’ll get things right, but I’ll certainly have a better chance than without it. Recipes are not normally a completely foolproof set of instructions. There is a certain degree of knowledge that is taken as given when following a recipe. For example, the recipe doesn’t tell me to take the rice out of the packet before putting it in a pan – it assumes that I know at least that much!

The same goes for guidelines. They exist purely to “guide” you to creating more accessible web content. They come with no guarantees, but they can go a long way to helping people understand what some of the problems are, and the best practice solutions to those problems, which is better than nothing. There is a certain degree of knowledge that is required in order to apply them properly. This is where expert review and independent verification can be beneficial, both in terms of reassuring people about the accessibility of your site but also as a learning experience for how to apply the guidelines in the real world.

So recipes and guidelines can help you out when you don’t know how to go about doing something. If we were to abandon guidelines now, it would be like cooking something you only know the name of. The chances are, you’ll get it totally wrong and it won’t taste very nice! Similarly, if your goal is to create an accessible website, without the guidelines you may have no clue where to begin and your efforts might be very wide of the mark. With the guidelines at your disposal, you at least have a fighting chance of getting things right.

User Testing – Tasting the Results?

The age old saying goes that “the proof of the pudding is in the eating” and it certainly is! I’m very much one of the proponents that what we do with the web should be focussed around people and not technology.

What exactly do I mean by that then? Well, most of us probably spend our days building sites for our customers. The majority of the users coming to your sites won’t particularly care about the technology you’ve used to build it, they just want you to make their lives as easy as possible when using your site. As long as we are fulfilling this goal, the technology, standards and guidelines we’ve used along the way become less important. They are merely the tools we can use to achieve the end goal of satisfied customers and users.

Saying that, I’m still a strong believer in the fact that it’s important to write clean, valid code where possible and follow guidelines where appropriate, but to become too focused on this and lose sight of the overall goal of the “user experience” would be a mistake. Similarly, to ignore standards and guidelines and rely on user testing alone would be a step backwards from where we are now in my opinion. Guidelines and user testing go hand in hand, just like a recipe and tasting the finished dish go hand in hand. The important thing is to find a healthy balance between the two so they complement each other.

Guidelines are better for ensuring a site is technically accessible. Following them can help to iron out problems which may otherwise surface during user testing and nip them in the bud so to speak. This should result in fewer accessibility issues being found at the user testing stage. Following guidelines can also often naturally lead to improving the usability of a site.

User testing is useful for genuinely making sure that a site is usable in everyday use as well as being technically accessible against a set of guidelines which is an important concept. User testing is valuable in helping make sure that users can understand and find the content and features on your site. The results you get from user testing are often easy to read for non-technical people, as they offer an “account” of various peoples’ experience of using your site.

User Testing - Does This Taste Right To You?

There is one very important point to remember with user testing. The testers are rarely experts in web accessibility, and will normally only test the route to a limited number of resources to ensure that they are usable from their own perspective. It’s important to be certain that solutions applied to resolve issues highlighted by one user, don’t create problems for other users with different needs.

To ensure good accessibility for all, it’s important to have an expert to review and interpret the feedback received from users, in the same way as with guidelines. An expert can translate feedback from users into suggested solutions for developers that will ensure pan-disability access to the site (as the WCAG guidelines do).

Let’s relate the need to have an expert review the feedback from user testing to tasting a dish when it’s cooked. When I have my first attempt at cooking Ackee and Saltfish, I could taste it and think I’ve done a good job because I’m not experienced at cooking Jamaican food. More than likely, my colleague might taste my first attempt and tell me she could do a much better job, and I’m sure she would be right! It’s vitally important that the feedback from users is interpreted correctly and transformed into workable solutions by a professional.

Conclusion

To sum up, I think I can say that both following guidelines and testing with end users are beneficial practices and certainly both are a good idea to do if you have the resources.

Following guidelines can help to reduce the number of potential problems that might be uncovered by users and testing with users can help to uncover problems that might not be covered in the guidelines, so they complement each other well.

Guidelines and user testing don’t provide a “magic fix” for ensuring an accessible and usable website on their own. When used together, they still don’t provide a “magic fix” but they come a lot closer. What do you think? Comments are open!

Further Reading

Posted by Andrew on 10/07/2008 at 16:21, and filed under General, Standards, Testing, Usability, WCAG.
Follow any responses to this entry with the RSS 2.0 feed.
You can post a comment, or trackback from your site.