Your Users Don’t Know Their Requirements Until They See Them

In my previous post, I explored the dilemma of a service organization creating a website or web application: while you’re the one whose goals the site has to support, you aren’t the ultimate user of the site. This isn’t unique to nonprofits and government, but it is harder because there’s no easy metric like sales to let you know how well you’re doing. If you’ve figured this out, congratulations! You’ve gone a long way toward making your site successful by focusing on the end user. But there’s a problem: your users can’t often tell you what they need, and when they tell you what they want, they’re likely to unintentionally lie.

Humans are notoriously bad at predicting what they will respond to. That simple fact has given rise to parts or all of the fields of behaviorism, psychology, user experience, efficiency, human factors design, and psychometrics. It turns out that we often state what we’d like to be true of ourselves and then do something different when forced to act. Economists have a term for this: revealed preference. The upshot is that we’re really bad at doing something like looking at a wireframe or design for a website and accurately predicting how we’d use it and if it will solve our problems.

Agile methodology illustrationSo how do you create a successful website if you can’t test it on yourself and you can’t simply ask users what will work for them? There are of course general principles of user experience, and any good developer should steer you toward them. But those principles are notoriously hard to apply perfectly in every case. Fortunately, the software world has been aware of these problems for some time and has developed several methodologies to help, collectively known as Agile.

Even though details vary, all agile processes share one insight in common: users don’t know what they need until they try working software and their requirements change quickly over time. The goal is to quickly get out the most important features to users and let them begin working with them, and use metrics and usability tests to assess how well that version is working. By doing this over and over (iteratively, in Agile terminology), you can make small changes, measure their impact, and change accordingly until you get the feedback you want.

This is a big mind-shift from how nonprofits are used to operating, but it’s one that can make the difference between a website that you can demonstrate to funders actually fulfills its mission versus one you simply hope gives you good results by the next grant application. I’ll explore some of the challenges in adopting it unique to nonprofits in a future post, but meanwhile, Google and this article can get you started on understanding how selecting a developer who uses an agile methodology will make your project more successful.

This month, be sure to check out my company’s Kickstarter campaign and consider backing us: we’re developing two new productivity tools that will help non-profits.

Advertisements

Link

While much of my advice is for organizations dealing with outside contractors or web development agencies, many organizations choose to actually hire web developers. If you go that route, I suggest reading this excellent post by Laura Thomson on managing engineers. Engineers are like other skilled white collar professionals, but more so in many of the ways that reward manager-as-servant mentalities.

You’ll also need to hire them, and this post at Autodidacticism, while aimed at developers, covers the non-techical things you’ll want to look for in an engineering recruit. Much of the theme of this blog is that both developers and those working with them forget that success depends a lot on non-technical factors that the myth of technology as magic and engineers as robotic, mysterious, yet interchangeable resources obscures.

Present the Problem, Not the Solution

“How hard would it be to make a button…”

I can’t tell you the number of times I’ve heard sentences start with those exact works from clients. Early in my career, usually after the initial delivery of a beta, I’d get that question. I’d listen to the whole thing, put my head down, maybe ask a couple of questions about the details, think for a moment, and then come up with an estimate. Since I’m a pessimist, unlike most programmers, that number would often dissuade the client from proceeding.

After a few rounds of this, they’d complain that we weren’t as responsive as we were in the initial engagement. At first, that puzzled me. Our response time was usually just as fast, and in fact sometimes I was giving the estimate live in the meeting. They’d respond we weren’t as “flexible,” which I took to mean they were getting sticker shock and wanted the rate of feature delivery they got in the main site development even though they didn’t have a fraction of the budget.

Listening past the question

Over time I realized that’s not really what they meant. What they really meant is that we weren’t engaging with the same level of problem-solving we were in the initial engagement. In some ways, that wasn’t fair: in the initial site build, we often had strategic goals, long discussions about what they wanted, what their workflow was like, and how we could adapt the technology to meet their needs and stay within the budget. We were in contact several times a week and got updates on changes at their organization quickly.

Once the main site was done, that level of awareness was gone, so when they’d come out of the blue with a request, my automatic customer-service response was simply to answer the question as asked. Once I figured out they were frustrated with several rounds of that, I learned to listen past the question and instead of asking details, follow up with, “What’s the problem you’re having now that this would solve?”

Overcoming literalism

Unfortunately that’s not an insight that comes easily. Not only does it take experience, it takes a certain mindset. Junior people haven’t learned it yet, people from a sales background want to get an answer back quickly above all, and programmers are notoriously literal-minded and don’t often think about the meaning behind what is being said.

On the organization’s side, it’s also easy to get caught up in “feature fever,” especially as you start to understand the possibilities of web applications and how they can make your life easier. It’s incredibly tempting to envision a magic easy button that solves your problem. The only hangup is that while it looks like the flying magic unicorn pony simply drops elements on a page and they are easy to use and work perfectly, it actually takes a great deal of user experience and engineering skill to make a site function naturally and intuitively.

So the best defense against a good relationship going sour is a good offense: ask your web developer or agency to propose solutions to your problems. Explain to them the way you are doing things now, whether online or offline, let them know where the pain comes in, and ask them if there are any ways they can think of to simplify that. Often even the most inexperienced, literal developer will brighten up and say, “Oh! Yeah, we could just…” and suggest not only something that solves the problem, but solves with the least cost to implement, because good programmers are lazy.

Work with Developers

While not up to the level of this hoary fun fact, you can take the title of this piece in multiple ways. Here, I mean two of them.

Work with developers

There are a lot of people who are “consultants” or “strategists” who’ve never built a working website, beyond possibly a simple HTML site in the 90s or configuring a blog in the last decade. That doesn’t mean these people have nothing to contribute; if they’re smart, they’ve observed what’s worked and what hasn’t, have good insights, and can relate to your organization’s problems in a way that helps bridge the gap. However, it’s rare that they have as intuitive a sense of how a potential solution will look in software, what level of effort it will require, or what alternatives might solve the problem as well.

Nothing’s more frustrating than talking through a problem, imagining a solution, and then being told three days later that it isn’t within your budget or, worse, being told at the end of the process that it’s going to require several more development cycles to complete when you’re already out of budget. A strategist or consultant who has built lots of websites similar to the one you’re considering will have that sense and be able to steer you to solutions or at least identify that more research is needed.

None of this, of course, means that just any developer will do. You will want someone who is good at bridging that gap between engineers and people who need their services. If they don’t have experience or expertise in your sector, then be prepared to educate them on the wherefores and whys of what you do and the special challenges you face that other organizations don’t. This, by the way, is a good role for those non-technical strategists or consultants: if they have expertise or experience in your sector, they can help bridge that gap to a developer who doesn’t but is good at working with customers.

Fortunately, most successful web development firms are founded by and staffed with technical types who are filling other roles but who started out as engineers. Likewise most successful independent developers are successful because they’re good at working with people to identify solutions that fit within their budget. This may not be true for agencies who “also” do web development, so if it isn’t obvious, look for technical experience in the resume of anyone proposed as your strategist and ask them about that experience before hiring them.

Work with developers

Planning a site isn’t something that happens only before the site is built. In fact, some argue that writing software basically is writing a design document in detail. [PDF] Once you’re working with someone who has stayed up late sweating out lots of different web development problems, you have someone who can work with you to figure out a way to address your problem that fits your budget, rather than simply taking orders and coming back to you late and over budget.

Additionally, software projects are notorious for having requirements change or receiving feedback that changes your initial assumptions—which then changes the requirements. So when you’re working collaboratively with the developer instead of handing them a specification, you have someone who can help you think through alternatives when those inevitable changes occur.

It takes a team

None of this is to say that web developers are the only people who should be involved in building your site or application: rare is the developer who also understands strategy, design, and user experience equally well as they understand code. But the stereotype of the socially inept engineer who can’t talk with anyone has left technical people out of the planning, and planning a site never stops. So make sure they’re in the room, have a voice, and are working with you to balance your needs, your budget, and their technology.