Your Users Don’t Know Their Requirements Until They See Them

In my previous post, I explored the dilemma of a service organization creating a website or web application: while you’re the one whose goals the site has to support, you aren’t the ultimate user of the site. This isn’t unique to nonprofits and government, but it is harder because there’s no easy metric like sales to let you know how well you’re doing. If you’ve figured this out, congratulations! You’ve gone a long way toward making your site successful by focusing on the end user. But there’s a problem: your users can’t often tell you what they need, and when they tell you what they want, they’re likely to unintentionally lie.

Humans are notoriously bad at predicting what they will respond to. That simple fact has given rise to parts or all of the fields of behaviorism, psychology, user experience, efficiency, human factors design, and psychometrics. It turns out that we often state what we’d like to be true of ourselves and then do something different when forced to act. Economists have a term for this: revealed preference. The upshot is that we’re really bad at doing something like looking at a wireframe or design for a website and accurately predicting how we’d use it and if it will solve our problems.

Agile methodology illustrationSo how do you create a successful website if you can’t test it on yourself and you can’t simply ask users what will work for them? There are of course general principles of user experience, and any good developer should steer you toward them. But those principles are notoriously hard to apply perfectly in every case. Fortunately, the software world has been aware of these problems for some time and has developed several methodologies to help, collectively known as Agile.

Even though details vary, all agile processes share one insight in common: users don’t know what they need until they try working software and their requirements change quickly over time. The goal is to quickly get out the most important features to users and let them begin working with them, and use metrics and usability tests to assess how well that version is working. By doing this over and over (iteratively, in Agile terminology), you can make small changes, measure their impact, and change accordingly until you get the feedback you want.

This is a big mind-shift from how nonprofits are used to operating, but it’s one that can make the difference between a website that you can demonstrate to funders actually fulfills its mission versus one you simply hope gives you good results by the next grant application. I’ll explore some of the challenges in adopting it unique to nonprofits in a future post, but meanwhile, Google and this article can get you started on understanding how selecting a developer who uses an agile methodology will make your project more successful.

This month, be sure to check out my company’s Kickstarter campaign and consider backing us: we’re developing two new productivity tools that will help non-profits.

You Don’t Know Your Requirements, Your Users Do

One mistake a lot of organizations get drawn into is focusing on the organization’s requirements on a website. No, really.

It seems like a strange sentiment, because why else would you build a website or application? The problem is that when you circulate requirements for the website, people focus on their own jobs and what would best represent how they think about their job. What they often don’t do is focus on the organization’s goals.

Your goals and your requirements aren’t synonymous by any means. Requirements are often just a wishlist of features collected piecemeal by people who take a moment to think about what they might want a website to do before going back to their main job.

The problem is, unless you’re building a site for use by your employees, they aren’t really the users of the site. The people the organization is trying to reach are the ultimate users of the site, and its their requirements that should drive the features of your website or application, not yours.

Instead, you should start with a list of your goals generally as an organization, and then the goals you have for the website. Those goals are the ones that usually triggered the decision to invest in a site or application in the first place, such as increased engagement with volunteers, reaching out to a broader base of smaller donors that aren’t feasible by traditional means, or convincing a key demographic that they should support a policy.

The goals for the website should logically support one or more goals of the organization. Goals are the “why”, and requirements are often the “how.” The tricky part is that each person in your organization also has a job that supports your organization’s goals. (Well, they should, right?) The tricky part is that the requirements they have to do their job better don’t usually map very well to what the ultimate users of your site will need to support your organization’s goals. So simply collecting their requirements and turning them into an RFP is a recipe for a site that makes everybody internally satisfied, but goes over with a big thud with the people you’re trying to reach.

This is another reason it’s a good idea to approach web developers with problems rather than solutions. The problem you have is that you have goals and you need to use the web to help you meet them. It’s the web developer’s job to help you figure out how to do that. Your staff should certainly be involved, but they have jobs of their own and shouldn’t be specifying your website for you.

This approach can also prevent some political problems, as specific requirements related to jobs can be conflicting, and wish lists are inevitably larger than budgets. Getting everybody’s input on goals and feedback on proposed solutions is easier than dealing with hurt feelings when a cherished feature doesn’t make the cut.

There are several techniques for getting to the heart of what users need, and they get to the heart of better ways to work with web developers. But by remembering the end users are the ones who hold the key to getting the right requirements for your site, you’ll set yourself up for success, and potentially avoid some internal conflicts.

Link

Here’s an interesting post about that rarest of all creatures, the Unicorn prototype.

I’d say that in the nonprofit world, it’s more a Flying Magic Unicorn Pony, as it essentially never happens in agency contracts with nonprofits. There are several reasons: often deliverables are spelled out in advance, budgets are unbelievably tight, and there’s a lack of trust if an answer isn’t immediately available–”I don’t know” isn’t accepted, by and large. That lack of trust extends to the idea that prototyping time is “wasted” if it’s not to be used in the final product.

What works better is an agile process where an idea can be tried out in a sprint. That requires some preconditions and a mindset that I’ll expand on in a future post.

In the meantime, prototypes can work in nonprofit engagements under very specific circumstances:

  1. The requirement wasn’t in the initial discussions for the engagement, so it’s a legitimate surprise to the agency.
  2. The prototype is incredibly focused to answer a yes/no “will this work with the technologies we’ve selected” or “will this work within the budget we have” question, rather than a “do you like it” or “will users respond to it” question.
  3. The prototype can be done in less than a day’s time.
  4. Resources exist to do the prototype immediately.

Ideally, the prototyping can be done internally to the web team without involving the larger circle of stakeholders.

Given those requirements, it’s not too surprising that it’s a rare-to-mythical beast. However, they can be a powerful tool, so if you can make room in your budget, they can free you from either getting an adequate but less than ideal site due to enforced conservatism or getting a feature that becomes a maintenance nightmare as attempts to fix a fundamentally wrongheaded approach are slapped one on top of the other until the whole thing is written off as a failure.

Link

While much of my advice is for organizations dealing with outside contractors or web development agencies, many organizations choose to actually hire web developers. If you go that route, I suggest reading this excellent post by Laura Thomson on managing engineers. Engineers are like other skilled white collar professionals, but more so in many of the ways that reward manager-as-servant mentalities.

You’ll also need to hire them, and this post at Autodidacticism, while aimed at developers, covers the non-techical things you’ll want to look for in an engineering recruit. Much of the theme of this blog is that both developers and those working with them forget that success depends a lot on non-technical factors that the myth of technology as magic and engineers as robotic, mysterious, yet interchangeable resources obscures.

Present the Problem, Not the Solution

“How hard would it be to make a button…”

I can’t tell you the number of times I’ve heard sentences start with those exact works from clients. Early in my career, usually after the initial delivery of a beta, I’d get that question. I’d listen to the whole thing, put my head down, maybe ask a couple of questions about the details, think for a moment, and then come up with an estimate. Since I’m a pessimist, unlike most programmers, that number would often dissuade the client from proceeding.

After a few rounds of this, they’d complain that we weren’t as responsive as we were in the initial engagement. At first, that puzzled me. Our response time was usually just as fast, and in fact sometimes I was giving the estimate live in the meeting. They’d respond we weren’t as “flexible,” which I took to mean they were getting sticker shock and wanted the rate of feature delivery they got in the main site development even though they didn’t have a fraction of the budget.

Listening past the question

Over time I realized that’s not really what they meant. What they really meant is that we weren’t engaging with the same level of problem-solving we were in the initial engagement. In some ways, that wasn’t fair: in the initial site build, we often had strategic goals, long discussions about what they wanted, what their workflow was like, and how we could adapt the technology to meet their needs and stay within the budget. We were in contact several times a week and got updates on changes at their organization quickly.

Once the main site was done, that level of awareness was gone, so when they’d come out of the blue with a request, my automatic customer-service response was simply to answer the question as asked. Once I figured out they were frustrated with several rounds of that, I learned to listen past the question and instead of asking details, follow up with, “What’s the problem you’re having now that this would solve?”

Overcoming literalism

Unfortunately that’s not an insight that comes easily. Not only does it take experience, it takes a certain mindset. Junior people haven’t learned it yet, people from a sales background want to get an answer back quickly above all, and programmers are notoriously literal-minded and don’t often think about the meaning behind what is being said.

On the organization’s side, it’s also easy to get caught up in “feature fever,” especially as you start to understand the possibilities of web applications and how they can make your life easier. It’s incredibly tempting to envision a magic easy button that solves your problem. The only hangup is that while it looks like the flying magic unicorn pony simply drops elements on a page and they are easy to use and work perfectly, it actually takes a great deal of user experience and engineering skill to make a site function naturally and intuitively.

So the best defense against a good relationship going sour is a good offense: ask your web developer or agency to propose solutions to your problems. Explain to them the way you are doing things now, whether online or offline, let them know where the pain comes in, and ask them if there are any ways they can think of to simplify that. Often even the most inexperienced, literal developer will brighten up and say, “Oh! Yeah, we could just…” and suggest not only something that solves the problem, but solves with the least cost to implement, because good programmers are lazy.

Work with Developers

While not up to the level of this hoary fun fact, you can take the title of this piece in multiple ways. Here, I mean two of them.

Work with developers

There are a lot of people who are “consultants” or “strategists” who’ve never built a working website, beyond possibly a simple HTML site in the 90s or configuring a blog in the last decade. That doesn’t mean these people have nothing to contribute; if they’re smart, they’ve observed what’s worked and what hasn’t, have good insights, and can relate to your organization’s problems in a way that helps bridge the gap. However, it’s rare that they have as intuitive a sense of how a potential solution will look in software, what level of effort it will require, or what alternatives might solve the problem as well.

Nothing’s more frustrating than talking through a problem, imagining a solution, and then being told three days later that it isn’t within your budget or, worse, being told at the end of the process that it’s going to require several more development cycles to complete when you’re already out of budget. A strategist or consultant who has built lots of websites similar to the one you’re considering will have that sense and be able to steer you to solutions or at least identify that more research is needed.

None of this, of course, means that just any developer will do. You will want someone who is good at bridging that gap between engineers and people who need their services. If they don’t have experience or expertise in your sector, then be prepared to educate them on the wherefores and whys of what you do and the special challenges you face that other organizations don’t. This, by the way, is a good role for those non-technical strategists or consultants: if they have expertise or experience in your sector, they can help bridge that gap to a developer who doesn’t but is good at working with customers.

Fortunately, most successful web development firms are founded by and staffed with technical types who are filling other roles but who started out as engineers. Likewise most successful independent developers are successful because they’re good at working with people to identify solutions that fit within their budget. This may not be true for agencies who “also” do web development, so if it isn’t obvious, look for technical experience in the resume of anyone proposed as your strategist and ask them about that experience before hiring them.

Work with developers

Planning a site isn’t something that happens only before the site is built. In fact, some argue that writing software basically is writing a design document in detail. [PDF] Once you’re working with someone who has stayed up late sweating out lots of different web development problems, you have someone who can work with you to figure out a way to address your problem that fits your budget, rather than simply taking orders and coming back to you late and over budget.

Additionally, software projects are notorious for having requirements change or receiving feedback that changes your initial assumptions—which then changes the requirements. So when you’re working collaboratively with the developer instead of handing them a specification, you have someone who can help you think through alternatives when those inevitable changes occur.

It takes a team

None of this is to say that web developers are the only people who should be involved in building your site or application: rare is the developer who also understands strategy, design, and user experience equally well as they understand code. But the stereotype of the socially inept engineer who can’t talk with anyone has left technical people out of the planning, and planning a site never stops. So make sure they’re in the room, have a voice, and are working with you to balance your needs, your budget, and their technology.

$ing£e $ign On

One of the biggest causes of heartburn I’ve seen on projects is when organizations want single sign on.

It should be simple!

On the surface, it’s simple, right? Why remember a bunch of passwords, when you can just tie those systems together and have them use the same login across them. After all, you’re the same person, so why should you have to identify yourself differently to each program you use to do your work?

In practice, especially in the nonprofit world, many applications date from a time before a ubiquitous internet pushed applications from your local computer to a web server. Since most people would ever only need to access that application and a few others, and often identity was tied to a computer rather than a person, there really wasn’t a reason to build in a security nightmare of letting other applications access your users or tell you who someone was.

Additionally, lots of those applications date from the time when everyone tried to be everything to your organization. Some later applications (*cough*Convio*cough*) still think that way, and try to have a “component” to do each thing you want done, rather than being the best of breed in one area and letting other companies handle other aspects of your operation.

Additionally, there’s no good standard for sharing identities—your login—across application. Sure, there are plenty of standards, but no good ones. Some require you to go to a trusted third party website to authenticate, and some are enterprise-y schemas that aren’t implemented the same way across systems.

And guess what? There isn’t a standard that everybody supports. So almost no matter what scheme you try to use, no two popular systems likely use them. Most open source systems allow you to “hook in” to their login and put custom code in its place, but the more custom code that has to be written, the more development time, testing, and room for general bugginess—and it opens up new vectors for hackers.

So what do you do?

Fortunately, there’s a pretty easy solution: if your applications are web-based, every modern browser supports a “remember my password” function. It’s free, it’s already there, and you probably use it at home. I logged into hundreds of websites regularly when I was consulting, and I just had the browser remember the passwords. Sure, there’s that extra step of going through a login screen, but it turns out most shared identity systems actually make you go through a username screen, just without a password. Since the browser remembers it for you, you’ve lost zero time.

If for some reason that doesn’t work for you, then look into password managers. They’re great applications, and if you use Mac OS X, it’s built into the operating system. (Look for “Keychain”—it even syncs over iCloud!) Here’s a review of five different systems, most of which work cross-platform.

Yeah, but I want it! (And a flying magic unicorn pony.)

Be prepared to add 20%, minimum, to the cost of your project.

That’s a lot of features you’re giving up to save yourself a few seconds of using a password manager each day. Assuming your project is $120,000 and you have 20 users, each of whom (at an extreme) spends 30 seconds per workday dealing with password managers, that’s paying $20,000 to save you the equivalent of one person doing a week’s work. Unless your employees cost you $500 an hour, almost anything you do will get you more value for your budget.

Save the money, and just train yourself to click “Yes, remember this password,” when that dialog pops up.

Hello World: I’m here to help you understand web developers

I’ve been a web developer for a long time. An eternity in the web world: 15 years. I’ve also spent all but a year of that time consulting, usually for nonprofits and government. I’ve worked in-house and in an agency. I’ve probably worked with a hundred or so organizations in various capacities, from basic development to strategic advice to multi-tier applications. I’ve been around the block, in other words.

And in that time I’ve seen most of the mistakes web developers and web agencies can make, but I’ve also seen a lot of cases where a lack of understanding between the two has cost budget, delayed or killed projects, and left everyone with a bitter taste in their mouths.

So why am I qualified to bridge this gap? I was actually trained in international relations and understand the environment of nonprofits and government as well as business and web development. And since so much of the conflict is needless, I want to put myself in a position to be a trusted advisor without selling anything beyond my advice.

If you’re about to engage in a redesign, create a project site, or want to figure out how to use the web effectively to get your message out and engage people, the content here will help you avoid learning these lessons the hard way. And if you want personalized help or for me to come and talk to your organization, let me know.