What does the data say?

Some argue that there is a much deeper purpose to philosophy than reasoned arguments. A single right answer exists, and finding that right answer should be the aim of every philosophical inquiry. I doubt most experience designers think this way. If asked whether there is a single, right experience for a given service or product, most of us would probably say that there are too many variables, too many users, or too many behaviors to account for. We look to HCI experts, usability engineers, social scientists, etc., for help in finding the best amalgamations of utility, usability, and (gasp) business purpose.

Not many doubt that a product/service needs to be useful, usable, and profitable (or at least not cost prohibitive) in order to succeed. How to find that balance is the focus of an entire industry, and one of its key pillars – research – also happens to be its biggest challenge.

Take A/B or multivariate testing, both of which help determine a specific design based on the observed effectiveness of alternate treatments. This data is so authoritative that it has leveled the playing feel between stubborn client and frustrated designer, and between junior employee and overbearing executive. “Let the data speak for itself.”

But therein lies the problem, it’s just data. The objectivity and direct observation of the behaviors of a large population make it easy to discount the careful analysis of user research in the context of a stated hypothesis. Instead of using this sort of testing to fine tune a design, some are using it as a shortcut to avoid all the hard decisions that should be made before anything is tested.

While we all like it when things “just work”, we don’t strive to find that ideal state in the things we create. We seem more comfortable in following the process, checking the boxes, pulling the right levers, and pushing the right buttons in the hopes that, as boxes/levers/buttons improve, we are guaranteed a positive outcome.

Or maybe things shouldn’t just work since if they did, most of us wouldn’t have jobs.