As you may have read in my Twitter feed, I had a bit of a technology meltdown as I was writing the UX goal-centric design post last week. Just as I finished the post, I decided to resave one more time before doing a quick spellcheck and posting. I was working online, had saved previously and had been working for a couple of hours in an open browser window. When I hit Save this time it cued me to log in. That sinking feeling was interesting. I haven't had one of those in years.
Anyway, as I stubbornly got back in and recreated the article I left out a couple of relevant details.
Next step: making goals measurable
In last week's UX Week 2009's "Be A Strategy Team of One" workshop with Henning Fischer I was reminded how important it is to have measurable goals of success that you are working against, particularly in iterating cultures.
When working in UX goal-centric design it's a simple extension to make the worded goals extend into items that can be measured. In one of the exercises in the workshop we worked through a percentage modeling system that showed if we started with 10,000 people how many made it through to the end to accomplish the ultimate task that we set them.
Once the touchpoints are chosen (in this example I chose a sign-in process with discrete steps that would have Web statistics to map into the model) the percentages can be quickly calculated throughout the testing process. It's always sobering to realize that as few as a handful of the users that set out on our created adventures ever make it to the pinnacle to plant their flag.
It was a great reminder that our data stories don't have to be deeply complex, they just need to show us where we are succeeding or failing in our design thinking.
FAIL success - determining what's not working
And failing to meet our goals is not the end of the world in a goal-centric design process. It simply leads us to iterate. How can we streamline this sign-in process? Too many steps? Unclear language in the instructions? Lack of education about what they are getting?
Temple Grandin told us stories about "getting down into the chute" in the exact spot where the cattle were getting scared to determine what it was that was spooking them. To her, the issue was immediately obvious. For us, it's not quite that simple, even when we've developed a good set of measurable goals.
Temple pointed out that with our busy minds, that our unspoken assumptions about what and why a user is behaving in a certain way gets in the way of seeing what the real problem is. This is why all of our iterative processes can benefit from the use of user testing to allow engineers and stakeholders a chance to "get into the chute" with users.
This doesn't need to be full-blown test case user testing. In reality, all that we often need if we're on a tight turnaround is to have an informal and unrelated user come in and perform the task that we are struggling to understand. But, if the process that you are trying to tune is critical to your business success, you should consider working in formal testing all along the development path to save you time and money in the long run.
What other tools/process have you or your company used to measure success in your design process and implementation?