Sep 17

Co-founder of Obox Themes. Designer. Racing driver when I have time. Crazy about cars. Love tech. WordPress is my bread and butter.

The Do’s and Don’ts of User Experience Testing

Yesterday I wrote about my frustration with UI designers calling themselves UX designers. The post got a lot of comments (all of them good) and the general opinion was that UX design was not just one person - nor was it just about testing. There are so many angles to UX design and it involves a lot of people.


We still believe that the most important part of UX design is testing. You can write, strategise, research and design all you want but the proof is in the testing. We consider user testing so important because of the natural tendency of people to interact with your work in ways you could never predict.

We’ve done User Experience tests for four years but we don’t consider ourselves professionals, however we do have some tips on how to do it easily and in some cases, cheaply but before we get into the ‘how-to’, let’s look at the ‘how-not’.

What isn’t User Testing

Testing a fellow professional

This one is a bit controversial. We have many friends who’ve been in the game longer than we have and yet we won’t show them our work until we’re done. Why? Because we are not seeking an opinion, we are seeking to observe.

Your professional/experienced friend will always have an opinion and a bias based on their past experiences. Testing is most useful when the subject is there to truly interact with your product without knowing who you are and what you’ve done.

Testing your better half

Once again this will lead to an opinion and in some cases a fight. One of the biggest cop-outs is calling yourself a user experience designer when the only person you’ve tested is your boyfriend/girlfriend. Not only is there the threat of being offended (they don’t like my work, OMG!) but there is also a high chance that he/she is not your subject matter. Nine of out ten times this is a bad idea.

Asking Twitter

This is not uncommon and we’ve been guilty of it ourselves. Asking Twitter to have a look at your work and deduce it’s usability is a big no-no. The best you will get is someone who can quote an article they’ve read regarding form and button alignment and pass that on as useful advice.

What is User Testing

As mentioned above, seeking opinion is not user testing. In order for it to be a test there need to be parameters.

Research and define your target market

In order to get the best results you need the best test subjects and to get those you have to know who they are. There is nothing better than watching your customers/users interact with your product, you will learn so much along the way and maybe one day you’ll be able to relate. In order to gather the right group you need to do as much research as you can.

When we hired Flow Interactive to do our user testing there was a lengthy period of discovery that took place on their behalf. In our experience they have set the standard when it comes to UX and the methods they put in place prior to the actual test was a lesson in preparation.

Give your test subject space to work

The best kind of user test is one where the subject is not influenced by too much direction, they are simply there to use what’s in front of them to achieve a goal that you set out before the test began.

For example when we tested the purchase process on our old site we offered our subject a premium theme and asked them to buy it. They still had to pay (using our credit card) and they had no one telling them which theme to use. We simply gave them the web address and said, “Your task is to buy the theme you like most.” If and when they were confused we’d ask why but we wouldn’t tell them how to solve their problem. It was up to them to work it out.

Record your test subject

The most useful user experience tool is your smart phone. Ideally filming your subject in secret (not the creepy kind!) is the best way to get honest interaction however it’s not a train smash if they know you’re doing it. The aim is to store their session for future review as you won’t catch everything the first time.

There may also be times when you’re designing a new page and need to reference previous interactions to get a good sense of how you’re doing. After all testing can be incredibly expensive so it’s best to re-watch videos to get an idea of what a user may think of a new page.

We use our phones when a quick test is required. We share office space with a big web agency and when we redesigned our brand new checkout process the people who worked there were the perfect test subjects. This is where our phones came into play, we’d simply roll over to their desk and ask them to purchase a theme. During that time we’d film the whole interaction go back to our part of the office and discuss our observations.

If you have budget, hire professionals

Obox is a small team and by no means are we the experts in every facet of UX design which is why we hired professionals to do our first few tests. Our core competency is in the user interface arena however our time spent with companies like Flow has taught us enough to get the basics right.

That being said it was not a cheap exercise, and rightly so, however if we had to do it again we would. Going through the motions of researching, wireframing, observing, iterating and validating was one of the most eye opening experiences of our lives. Of course you can do it yourself but at the end of the day there’s no better form of product investment than letting the pro’s do what they do best. At the end of it all you will have a better product and happier users.

2 Comments on “The Do’s and Don’ts of User Experience Testing

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <s> <strike> <strong>