Leave nothing to chance by exploring the product testing process that leads to a delightfully smooth user-centric product.

You’ve done your user research, you’ve pinned down what kind of product you’re about to design and you know exactly why your target market needs it… right?

If you can’t give me a solid “Hell yeah!”, then check out my user research guide before continuing with this article. It’s a prequel to product testing and it’s pretty much essential.

If you’ve laid down a solid foundation, that’s awesome! Now it’s time to build prototypes and wireframes, then relentlessly test and polish them until you have a product with silky smooth UX.

That’s how you’ll maximize your product’s chances of success. Let’s take a closer look at how it’s done!

Test early, test often

The general rule of thumb during development is to test things as soon as you can. You need to trial your prototype among your target users, collect feedback, refine your prototype and repeat the process.

Catching mistakes and misassumptions at an early stage means that you can make changes as quickly and cheaply as possible. If you don’t, they will pile up and make revisions a nightmare.

You can begin testing in the week following your user research. Hold a workshop and end it by creating a medium-fidelity prototype with tools such as Sketch or InVision, or anything else that works for you. Test it with users and confirm that you’re on the right path from the very start.

Not only is this useful, it’s also super rewarding. Going from an idea to a working concept and receiving actual feedback always feels great, and it’s a fantastic way to kickstart product development.

But what kind of feedback are you looking for? And which three aspects should you always strive to achieve?

The Big 3 – Effectiveness, Efficiency and Satisfaction

You’re designing products to be effective, efficient and satisfying.

But what exactly does this mean? Let’s break it down.

Effectiveness

Does the product allow users to achieve their goals? Do users understand how the product works?

These are the main two questions you’re looking to answer when testing for effectiveness.

In order to measure it, keep track of accuracy, user error rates and quality of completion when completing the task at hand.

Efficiency

Can users find what they are looking for? How much energy did it take for the users to get there?

Nowadays, time is the most precious currency and so efficiency is crucial. People won’t do something if it takes longer than it needs to.

That’s why you should measure efficiency by monitoring the time it takes to complete the task and by asking users if they feel that the task took too long to finish.

Satisfaction

What are the users’ attitudes towards the product? Do the features look and feel right?

This is the most intangible attribute of the three. Everything might function perfectly, but that means nothing if most users dislike your product.

You can measure satisfaction through questionnaires and scale ratings, as well as asking users for their opinions. Let them explain what they dislike about the product and why.

Conduct holistic product testing

Some teams test and validate their concept early on, and that’s pretty much where their testing ends.

Even when a project is underway, you should still be running regular testing sessions as you create higher fidelity prototypes, longer flows, richer features and so on.

As your development process matures, so too should your product testing. Adapt it so that it serves your current needs. Make it fluid, not static or too rigid.

Here’s how the testing focus could shift throughout your project:

Make sure that you guide specific tests towards a desired outcome. If you’re testing the visual design of the product, let assumptions and goals focus on items directly relating to this phase.

Of course, you’re still going to encounter some feedback pertaining to other things. Write this down and integrate it into the product roadmap later on.

Avoid the dreaded user churn

User churn is the percentage of users who stopped using your company’s product or service within a certain time frame. Product testing plays a huge role in minimizing it.

The most common reason for a high churn rate is that there are problems with the new user experience. If the UX is bad, you risk losing a potential user or customer almost immediately. An example would be when you’re asked to provide credit card details in order to continue to the next step (that’s not checkout). Most visitors just leave the site, even if it’s not shady. Another example is when the visual design of the checkout is different or inconsistent, which can confuse customers and lead them to abandoning their cart.

Such cases create a fast churn effect, whereby your new customer engagement rate might be impressively high but you lose just as many every day.

For this reason, churn is very much a design problem; you need to get the experience right via testing or you may lose your users after just a short period of time.

Where should you test your product?

You can conduct a Home Usage Test (HUT) or a Central Location Test (CTL).

HUT
A HUT means that your product is shipped to target users, so the research can be conducted in their own homes.

The main advantage of HUT is that the product is being used in its natural environment – a real-life setting. It enables the user to have a longer testing period (up to a few weeks) and as a result, it produces a more accurate product evaluation. Products such as cooking ingredients, face and body cosmetics, domestic appliances, electronics and products for children can hardly be evaluated after 30 minutes of use in a controlled environment.

This method of testing can be more expensive, as it can only be carried out with high-fidelity prototypes or physical products that are close to their final form.

HUTs are usually conducted via mobile and online market research platforms or through specialist companies.

CLT
A CLT means that the test is conducted within a chosen environment, such as a lab, a mall or your office building.

CLTs offer the opportunity for a fast turnaround of results – necessary tests can be completed in the space of a day, with results following in the same week. They are also easier to moderate and allow more participants to be involved.

You can either conduct CLTs yourself or hire a market research company.

Choosing between a HUT and a CTL really comes down to the current stage of your product development, how quickly you need the results, the size of your budget and which product testing method you choose.

Speaking of methods, let’s take a look at some of the ones at your disposal.

Product testing methods

Usability testing

In usability tests, users are asked to complete specific tasks using your prototype. The user is usually asked to think out loud during the process, or to retrospectively explain their thought process while performing the task. The main goal is to detect flaws and then iterate based on direct feedback.

Usability tests can be moderated or unmoderated, in person or remote.

When

As soon as possible and whenever you need to validate a concept, test a feature or make a significant change.

Heuristic analysis

Heuristic analysis is a usability inspection method in which one or more usability experts compare a digital product’s design to a list of recognized design or usability principles (called heuristics).

Experts go over the features and flow, then identify where the product is not following these principles to highlight potential issues. It’s basically a critical review of your product.

When

The best time to perform a heuristic analysis is when you have most of the features and flows fleshed out. However, you don’t need to have a high-fidelity prototype because the experts won’t actually be using the product. I’d advise that you always conduct the analysis before the final stage of usability testing.

Card sorting

You can help to ensure that your product or service is easy to use by organizing information so that people can find what they’re looking for. That’s where card sorting comes into play. It’s a method where users are asked to group topics and labels in a way that makes sense to them.

When

At the start of the development process.

A/B testing

A/B testing is a controlled experiment in which you compare two or more versions of a page or flow to optimize a certain result or metric. It must include a hypothesis, which can be a simple statement such as: “If we change the subscription button from small and green to big and red, we will get more subscribers.”

In order to confirm or reject the hypothesis, you need to measure the user interaction with both versions of your app or site. After a certain number of participants have interacted with each version, you’ll get a result.

A/B testing is a good method for polishing your product, but you need to have a solid story, copy and UX design to make it really effective.

When
During the final stages of development or when the product is already live.

How many participants should you include per testing session?

You’ve probably heard of the “fiver users” rule before. It’s so well-known because it applies to usability tests, which is by far the most widely used product testing method.

Having five users is enough to shed light on the majority of the issues you need to know about (around 85%).

With that said, you do need to have five users per target group. You’ll know your target groups once you’ve finished with your user research and sometimes, there’s going to be more than one. If this is the case, you’ll need to test with participants from each group or you might end up with a product that’s perfect for 60% of your users and confusing for the other 40%.

Voila, a user-centered product!

User research + Product testing = Maximized chances of success

That’s the equation! Leave nothing to chance – go after that feedback and create a marvelous product that will delight your users. Trust me, it’ll all be worth it when everyone’s raving about your product. 🙂

About the author

Oh hey, I’m Romina Kavcic, Design strategist

I am a Design Strategist who holds a Master of Business Administration. I have 14+ years of career experience in design work and consulting across both tech startups and several marquee tech unicorns such as Stellar.org, Outfit7, Databox, Xamarin, Chipolo, Singularity.NET, etc. I currently advise, coach and consult with companies on design strategy & management, visual design and user experience. My work has been published on Forbes, Hackernoon, Blockgeeks, Newsbtc, Bizjournals, and featured on Apple iTunes Store.

More about me  *  Let’s connect on Linkedin  *  Let’s connect on Twitter

Comments

comments