The Heretech

PM's latest annoying responsibility

The more social we get, the more we expose about ourselves. That's a fact of life that antedates social media, the Internet, or computers. From this perspective, social media are just a new way to  reveal ourselves, sometimes by choice, other times not. 

For the PM in charge of product requirements or release checklists, working on a social media product requires standing your ground on security and privacy issues. I won't beat on the unfortunate launch of Google Buzz further, except to make one last point. To avoid the sort of morass in which Google finds itself, someone in PM needs to have both the authority to insist on whatever external testing, use case analysis, red team testing—some way, any way, to avoid a class action lawsuit.

Unfortunately, very few PMs are experts in security and privacy. Historically, compliance and risk management don't overshadow the tech industry the way they figure prominently in pharmaceuticals and other tightly-regulated verticals. The section 508 requirements of the Americans With Disabilities Act, an important box to check before releasing a new product, is the closest many PMs get to anything resembling regulation.

That situation may change. Security specialists in tech companies deal with technical issues like, "How vulnerable are we to cross-site attacks?" Normally, they are not the people who think through the capabilities of the product to see if there is some way that product might compromise security or privacy. Even if they volunteered for the job, it's not feasible to bring them into the product development cycle early enough to make a difference. Nor will they be able to stay engaged to monitor how the features inevitably change over the course of development.

Therefore, it's unavoidable that security, risk management, privacy, and related issues naturally fall into the laps of PMs. Social media products raise questions about who can pry into your personal information, or how these details might become public without your permission or knowledge. Other products create different risks. For example, the Strange Case Of The Voyeuristic Laptops at a Philadelphia-area high school raises the question, is the company that produced the software used to spy on students at all liable for the abuse of the technology? Does circumventing an operating system's security features create other points of legal exposure?

I've yet to meet a PM who loves dealing with these kinds of issues. Who wants to be the person who, in a release meeting, presents the 15 potential security problems that need to be addressed before the product ships? Particularly if, just before you, someone gave a brilliant demo of how incredibly cool the product will be, if only it could get into the hands of customers. And, of course, success is defined as the absence of problems, not positive gains that everyone can celebrate. ("Woohoo! Three product releases without a lawsuit!") Sadly, I don't see a way of avoiding this responsibility.

When communities smother your product

Any PM who has worked with customers extensively learns how to deal with the hard cases. There are different species of difficult customers, such those who exaggerate every problem to the level of a showstopper, or the ones who think there's only answer to every implementation question.

Product teams often worry that social media will just open the door to a larger, louder group of problem customers. That's not the typical result, for a variety of reasons (healthy communities police their own members, teams sometimes exaggerate how unreasonable customers are, etc.).

What product teams should worry about is the potentially insidious influence of the customers who love them. That phenomenon may be part of the explanation why Second Life, once touted as the vanguard of virtual world technology, seems to be getting a lot less attention than it used to. (Certainly, the public attention to Second Life pales in comparison to Facebook, Twitter, and other social media.

Second Life: The new "vast wasteland"?
Although I never felt a strong compulsion to use Second Life, I try to keep an open mind. Maybe there's more valuable activity happening in Linden Lab's virtual world than I've experienced, or read about, or heard from organizations who have experimented with establishing a presence in Second Life.

That's why I read "What Ever Happened To Second Life?" at PC Pro, editor Barry Collins' experiment in jumping back into Second Life. The executive summary: hard to find anyone at all, except in the carefully-delineated porn enclaves.

That conclusion inspired many outraged Second Life afficionados to give Collins a piece of their minds. If only Collins had dug deeper into Second Life, they argued, he would have found a thriving community busy doing useful things, such as teaching. And, obviously, Second Life has considerable potential as a creative outlet, particularly for people who want to create 3D, moving representations of things normally depicted in 2D, static terms.

Collins' critics may be right. Beyond the small slice of Second Life that's immediately visible, there are interesting things happening in Second Life. Some of the people who responded to Collins cited live concerts, college courses, NASA's 3D museum of the space program, and the "steampunk" community of New Babbage. Depending on your tastes, at least one of those Second Life offerings should sound interesting. (Truth in advertising: I haven't checked them out personally, so I can't tell you how good the experiences really are.)

The user experience of yesterday, now for today and tomorrow!
Unfortunately, Collins is also right. The initial experience of Second Life is at various times bewildering, frustrating, and disappointing. When I tried out the client application about a year ago, it was still how I remembered it, slow and buggy. Where MMORPG companies like Blizzard and Paragon have honed their UIs to the point of maximum usability, accommodating both first-time and veteran users, Second Life's UI is a pain to figure out and use.

Even after you've soldiered past these problems, Second Life gives you very few clues about what kinds of content might be out there, somewhere in the Lindenverse. Therefore, the claim that one of Collins' critics made, "You get out...What you put in," overlooks the fundamental flaw of  Second Life: new users have no idea what they could get out. Or even what's already there.

There's no question that Second Life has a lot of very happy users. The aficionados are right: if you put sufficient effort into Internet searches about Second Life content, or search within Second Life long enough, you'll probably find something interesting. Unfortunately, the investment in time is probably more than most people are willing to invest (if they even considered Googling for content about Second Life outside of Second Life). Second Life may work for the people who responded to Collins, but that doesn't make Second Life poised for growth.

Customers who love you to death
Gaetano Mosca noted the tendency of an elite group to form in any organization--what he called the Iron Law of Oligarchy. The elite wields some combination of power and influence, more or less of each depending on the setting. In the US Senate, senior senators have power over things like committee appointments. The parents who really call the shots at PTA meetings may not sit on the PTA board at all, but have considerable influence over the faculty, staff, and other parents. (And then there's this comic but terrifying story of a bare-knuckles political battle among department store Santas...)

The same Iron Law holds true of user communities. Over time, a subset of customers emerge who participate regularly in user group meetings, discussion forums, the comments sections of blogs, groups in social media channels, and other channels of face-to-face and electronic communication. Because vendors are interested  in feedback, this group of notables get increasing attention from product managers, product marketers, and the like. Unless the company takes deliberate steps to mitigate the Iron Law of Oligarchy, a small and often unrepresentative sample of users will wield disproportionate influence over the vendor's thinking about products and services. 

Second Life is an extreme case of how you can develop a very happy group of customers, and still fail miserably at reaching a wider audience. Some businesses are comfortable with that outcome, as long as the customer base stays loyal, and the business stays profitable. Most would be terrified to discover that their best customers are, in subtle ways, holding them back. I can't say for sure that the Second Life notables are the reason why the UI is still klunky, and the useful content is hard to find, but I definitely have my suspicions.

It's odd, in an age of social media that pull in millions of new users every month, to see one of the original members of the social media glitterati fade into the background. Stripped of the most dorky or pornographic elements of Second Life, there's definite potential there to attract an audience of people who may occasionally want to tour the ruins of Rome, or talk with band members after a concert. While the Second Life diehards might know if these were more than just possibilities, the larger world of people who might be Second Life customers do not. And Collins isn't the only person to have had trouble finding signs of life in Second Life.

[Cross-posted at The Forrester product management blog.]

Jeff Lash says, drop the shotgun!

Jeff Lash has a great post on why product managers should resist the urge to stuff as many features as possible into a single release. You might call this the blivet model of product development, but I prefer the shotgun metaphor: Load the shotgun with as many features as you can, fire it at the customer, and hope that you hit the target.

The widget test of non-value

I almost wish that this excellent post from Tyner Blain wasn't titled "Agile Prioritization: Which Widget?" If you left out the word Agile from the title,  you'd still have a good cautionary tale about how you prioritize features and capabilities. More importantly, something phrased as a development project ("Build the following widgets...") doesn't necessarily convey what a feature does does, how it provides value, and why the team should build it in the first place.

On the other hand, to give Agile its props, user stories have moved the quality of requirements forward. By their nature, user stories force people to make at least some minimal effort towards explaining how someone will use a piece of technology. Depending on the quality of the user story,  plus other supporting references (personas, etc.), the user story can explain rather easily why the technology has value.

Of course, you can write bad user stories, or fail to build the conceptual bridge between the human activity it describes and the technology that might support it. If everything in your backlog (or other requirements content, if you're not using that particular Agile medium) has titles like ERP widget, or even File upload widget, you're probably not building that bridge.

How many personas?

Mark Lauden Crosley poses a great question: how many personas are enough? Or too many?

Since I have a fancy degree in political science, my own answer naturally comes from that background. Social scientists have been developing personas for as long as there's been something, self-consciously, called social science. Old Uncle Max (my endearing nickname for Max Weber) had one of the first, and one of the most useful definitions. Unfortunately, it's hidden behind a wall of academic verbiage. And to make matters harder, it's translated from the original German:

It is important to realize that in the sociological field as elsewhere, averages, and hence average types, can be formulated with a relative degree of precision only where they are concerned with differences of degree in respect to action which remains qualitatively the same.

In other words, personas (or, as Old Uncle Max called them, ideal types) are the product of two things. First, of course, there's the role that you're putting under the magnifying glass to study. That's what Crosley refers to in his post on persona development:

On the surface, it may appear that two customers in, say, legal services and construction management have little in common. But perhaps they have similar job descriptions and a similar set of problems in the area of financial planning and accounting, and perhaps these problems are especially relevant to the product you're creating.

You might have someone who behaves in the fashion of a Middle Manager, or Technical Professional, or Persnickety Auditor, in different types of organizations. That's definitely true, and it's the basis for a lot of creative market development, in which people look for problems in new markets that are similar to the ones they've helped address for their current customers.

However, there's another part of persona development, the question you're asking. In the rest of the same passage from Weber's "Definitions of Sociology and of Social Action," he goes on to say (to paraphrase), "Are you studying economic behavior, religious belief, or political organizations? The differences among two different people might not matter at all in one context, and a lot in another."

I faced this question recently when I was working on the PM job/department profiler. Are product managers and technical product managers different enough to create two different profiles for them? If we were looking at just their salaries, or their positions in the organizational chart, then the differences are pretty negligible. If, on the other hand, you're interested in what tasks they perform, or what skills they need to perform these tasks, there's a marked difference between them.

There are practical considerations, too. Again, here's Crosley:

So here's my answer: One persona is almost certainly too few for a commercial product/service, while more than a dozen (especially 37!) is too many. Four to six seems like a sweet spot.

You'll definitely need to economize somehow. That's probably a good rule to apply anyway, since decently smart people can tie themselves in knots, thinking about all the differences that might be relevant. But what's really an essential difference?

My only disagreement with Crosley is, you might not always be able to prune the list down to 3, 4, or even 10 personas. In some contexts, such as B2B technology adoption, you might have all kinds of characters playing significant parts. For example, while the individual salespeople might not be champions of a new SFA tool, they'll certainly be important influencers. And we haven't even yet identified who are the important champions, implementers, decision-makers...At a certain point, you have to just deal with however many personas really matter, even if the research takes longer than you had hoped.

Workshop on "social product management" is done!

Had some great conversations with the workshop attendees on how PM teams are incorporating social media into their customer and market intelligence (including, but not limited to, requirements). Thanks to all who attended.

Interestingly, we spent a good deal of time talking about the type of questions people ask. In trying to make sense of both social media and PM deliverables, I made a distinction between problem-centric questions, which center on the customer, and product-centric questions, which (as the name implies) focus on the vendor's products and services.

In the technology industry, people are far more used to posing product-centric questions. Gradually, companies are learning the importance of the problem-related questions. However, it's easy to slip from one into the other. It reminds me of how I used to struggle with guitar fingering. As a novice player, I would concentrate for a while on keeping my fingers straight on the frets. Inevitably, as I started to think about other things (strumming, rhythm, etc.), I'd start to roll my fingers slightly to the side, which made it harder to hit the notes I was trying to reach.

Problem-centric questions are a lot like good fingering: it takes time, experience, and discipline to reach the point when you shake free of the bad habits. Obviously, there's a time and a place for product-centric questions, but not every time.

[Cross-posted at The Forrester product management blog.]

PMs know that requirements are flawed

The workshop next week, "Using Social Media To Make Smarter Product Decisions," was a good occasion for me to jump into the data we've been collecting about requirements in the technology industry. It's one thing to claim that there are problems that need fixing, and that social media can help fix them. It's another to put that claim to the test.

The requirements survey has produced a lot of interesting results that will see the light of day in a research document to be published later this quarter. The questions we asked build a composite picture of both the content of requirements and the process of creating them. For example, we asked, ""How much time do you dedicate to requirements work at each stage of the development cycle?" Doing a weighted average of the responses, here's the answer:

This picture should be familiar to any PM: lots of requirements work at the beginning of the project; less and less as the project proceeds. Which begs the question, how effectively can the organization make mid-course corrections? When does the "after action report" kind of analysis happen, when PMs look back on the work they just did to figure out what they could do better the next time?

While this spike of requirements attention at the beginning of a project makes sense in bureaucratic terms, it doesn't make sense in research terms. You might pretend that the requirements you build at the beginning of a project are everything that the development team needs to proceed, but at best, that's a comforting fiction. You always need to fill in gaps, provide more detailed explanations, and validate the results.

If requirements are the information that technology teams need to avoid making costly mistakes, then requirements gathering and analysis should be an ongoing activity. While that's certainly true in some PM teams that I've met during my time as a Forrester analyst, in the aggregate, these peaks and valleys of activity still define the requirements landscape for many PMs.

There are plenty of other signposts to deficiencies in product requirements. For example, there's a big mismatch between the skills needed to do requirements, and the opportunities to put these skills to work. While the rest of the story will come later, I thought this one result was interesting enough to now, instead of waiting for the final report.

[Cross-posted at The Forrester product management blog.]

PM is more than features

Jeff Lash makes an excellent point:

If you want to be a good product manager, realize that your job is much more than prioritizing features. Sure, a product manager needs to understand what features need to be added to a product to meet customer needs, though just focusing on collecting and prioritizing features is an extremely narrow view of product management.

Product managers need to have a much broader view, seeing and understanding everything from the underlying customer needs to the business model to the product roadmap to the go-to-market strategy.

While Jeff says that many PMs fall into this trap, under a different set of circumstances, other PMs find themselves spending far too little time on feature-related work. Sales enablement, marketing support, and project management are the usual distractions from feature work.

Balance is difficult, particularly if the organization doesn't fully understand what PM is, or does, or should be doing. When that happens, whatever becomes urgent elbows aside whatever else might be important.

I'm increasingly convinced, therefore, that PMs should craft an elevator pitch that reflects the need for this balance.  Something on the order of, "I'm the person who makes sure that technology contributes to our business, and business allows the development and adoption of that technology."

Personas of interest

Christopher Cummings brings up an always-interesting topic: why people don't use personas as much, or as well, as they might. While persona certainly has a specific meaning in Agile, the same kind of deliverable—a profile of an archetypal user or stakeholder—exists in marketing and sales, too. It may go by different names, and people may develop and use these personas with more or less rigor.

Cummings cites three main pitfalls with personas:

• Dealing with objections about their usefulness. This seems to be the easiest object to handle. Even if someone doesn't officially build personas, they still develop a mental sketch of the average user. You might not like the conceit of writing a fictional profile, but that's just a question of style.
• Picking the wrong personas. People can get a bit sloppy about which persona they choose for a particular user story, use case, or requirements doc. For example, reporting features in a CRM system are really for sales managers more than account managers or telesales representatives, so choose wisely when you prioritize and design those reports.
• Dressing a feature in a persona costume. Naturally, product people like to talk about products, so it's hard for them to stay in persona-mode for too long. Unfortunately, the

The other pitfall I raise may be just an amplification of the "wrong persona" problem, but it's an important one:  When do you stop identifying ever finer distinctions  within roles?

For example, is the role developer enough, or should you make a distinction between development manager and engineer who's an individual contributor? Or should you go further, and define some very senior individual contributor role, such as architect, that's different from the rank-and-file developer?

It's a tough question to answer, since it depends  both on the role and the question you're posing. I've been surprised, for example, to see how different enterprise architects and application developers can be, when citing the sources of information they use to answer technology purchase and adoption questions. Therefore, from a marketing perspective, you need two different strategies to reach these different roles. (A quick, obvious example: Application developers really don't like calls from salespeople.)

In other contexts, these distinctions might lack a real difference. Would you design an IDE differently for rank-and-file developers versus enterprise architects? There might be some differences, particularly when it came to project management details. (Developers might be interested strictly in items related to their own to-do lists, while architects might want a broader view of the project.) However, the degree to which you tailor the product to fit these roles might be far smaller than the amount of custom-tailored marketing you need to do.

Which leads to another potential risk, duplication of effort in persona development. Just because a product manager didn't find justification for major differences in product design doesn't mean that the research into these roles is useless for product marketers. If I were a product marketer and someone were to tell me, "We didn't find any reason to change the product, but here's a lot of information about application developers and enterprise architects that might be useful to you," I'd be glad for the help. Not only would my colleague have saved me a lot of basic research work, but I might spot some marketing-related details in the information already provided.

Persona development requires a lot of patience to tease out the relevant details. However, the work is definitely worthwhile, since it can prevent a lot of costly mistakes. And really, there's no alternative, other than extremely unproductive arguments between people who have not spent time researching John Q. User, which one of them has the better understanding of this archetype.

[Cross-posted at The Forrester product management blog.]

Are PMs ready to focus on research?

Several forces are pushing PMs in the same direction: both product managers and product marketers are under increasing pressure to be good researchers. Here are a quick summary of but a few of those industry changes, and how they put greater stress on the research part of the job:

• Demanding customers. Even before the recession, customers wanted better explanations of a technology's value, and they were increasingly impatient about receiving that value post-purchase and post-implementation. Who's going to do the research about what these roles want, and how products and services can deliver them?
• Innovation. Tech companies realized the cost of making inventions that people don't want to use, or can't figure out how to use, or are not economically viable for the vendor to produce. From ideation to adoption, therefore, PMs are asked to get the information needed to decide whether to continue working on an innovation, and what direction it should take.
• Social media. There's a lot of user-generated content that's potentially relevant to making product decisions, but someone has to make sense of it. If you're a PM for a BPM product, you might get a strong read through different social media outlets that mobile support is important. But for which users? And what use cases? Are these requests coming from your part of the market?
• Agile.  Once product teams get into the pace of shorter iterations, they pose regular questions to help them make good prioritization and design decisions. PMs therefore need to be constantly on deck, ready to take a swing at each of these questions as they come in.

Normally, when I present a list like this one, I see a lot of nodding in the audience. The good news for both product managers and product marketers is, Congratulations, your organization sees the research you do as a strategic asset. For many PMs, the bad news is, You're going to have to give up some other tasks.

Of course, many are eager to do so. The biggest complain among PMs we've surveyed is, We don't have enough time to tackle everything on our to-do list. However, from talking to a lot of PM group leaders, as well as having being one myself, I know how hard it can be to detach PMs from one set of tasks to focus on another.

For some PMs, it's hard to give up on the excitement and glamor of many of the very same tasks that they grumble are distracting them from their core responsibilities. Sales calls are unpredictable and time-consuming...But for people who are interested in what people want to do with the company's technology, talking to these people hardly ever seems like a waste of time. Marketing support, such as staffing the booth at trade shows, carves out weeks from the schedule...But they also can be fun, and they're a way to peek over the shoulders of partners and competitors.

Some PMs have a harder time than others of letting go. One person who worked for me got addicted to questions from the field, the dozens of e-mails that flowed in, every day, and started with the same stock phrase, "I have a customer who..." We needed him to put more effort into product requirements, but it was hard to get him to focus. He hated to let any question go unanswered, or trust someone else to do as good a job of answering it.

Other PMs have a hard time letting go of bug meetings. If you see yourself as the voice of the customer, then Q.E.D., you want to be sure that the customer gets strong representation in decisions about bugs. If you never  do a better job of collecting customer insights and communicating them to the team, you'll continue to sit through hours and hours of bug discussions.

PM managers may worry that, as the demand for research goes up, their team has the necessary skill and talent. They should also worry about the force of habit, which as Thomas Carlyle argued, may be "the deepest law of human nature." PM managers need to think about the psychic rewards that research priorities can provide, as replacements for some old sources of job gratification.

Use case first, then applications

I really like following the evolving product offerings from companies like Zoho and Box.net. Since they're small, SaaS companies that are solely in the business of collaboration, customers reward them directly for making products that fit the way people work with unstructured data. Rather than just try to replicate the features of Microsoft Office, the way that OpenOffice did, the Zohos and Box.nets of this market start with basic use cases, and think about what's important to add.

Box.net's iPhone support is another case in point. Given the popularity of the iPhone, Box.net might have created a check box feature of limited usefulness, beyond marketing. Or they might have put some real effort in a feature, only to design something that doesn't reflect how people want to work with their files and folders through a smartphone. Beginning with the API is a good first step. Not only do you give your partner a way of adding value, but it gives you time to figure out exactly what the right use case is. It's there, somewhere, but you might rush into development, only to get the killer feature of your killer app wrong.

Workshop on social media for development teams

In October, I'm doing a workshop in the Foster City office about social media for product teams. How can social media fill in the gaps left by traditional requirements? Which social media outlets should you use to answer particular questions? What skills and investments are required? What's the tangible business benefit?

A big part of the workshop is hands-on experience with the questions your team faces. Bring an example of a burning question that the product team needs to answer (e.g., What's the impact of dropping this feature? Is our product a good fit for similar business problems in other markets?), and we'll explore how to use social media to find substantive, useful answers quickly.

For more details, click here. Also, in a couple of weeks, colleague Laura Ramos is doing a workshop, "Making B2B Marketing Work," that chock full of useful content.

[Cross-posted at The Forrester product management blog.]

Survey links: Up, and to the right

For your convenience, I've put links to the Agile adoption and product requirements surveys to in a box in the upper right quadrant of this blog.

Best practices for requirements

We're doing a major research project on the state of product requirements in the technology industry.  To make this project as successful as possible, we need your participation in either or both aspects of it, the survey and the examples. (Whichever works for you.)

What's in it for you? When the final research document is done, we'll give you a copy. We also promise to keep any information you provide confidential.

The survey
We just launched the survey at the following link:

http://deploy.ztelligence.com/start/survey/survey_taking.jsp?PIN=15TQ9CMTFTDDZ

If you have approximately 20 minutes to spare, we're interested in hearing how you collect, write, review, and circulate requirements in your organization. Also, please feel free to forward this link to anyone you think would be interested in participating in this survey.

The examples
We've been building a library of examples of requirements documents. (And thanks to everyone who has sent us their samples already.) We don't need to see the actual words, just the structure of the document. In other words, we want to see your template, not the actual contents of a completed requirements document. Fake or redacted data is good, too, if you use a requirements application.

We're being deliberately vague about what a "requirements document" is, since we don't want to lead the witness. Your examples will tell us what you think the fundamental requirements documents are. What do the key stakeholders and consumers in your organization need?

If you're willing, please send your examples directly to me, [email protected].

[Cross-posted at The Forrester product management blog.]

Immersion therapy

Friend-of-the-blog Adam Bullied has a great article on "industry immersion" in the latest issue of Pragmatic Marketing's magazine. The follow-up article should be, "Getting your company to give you the time needed, and reward you for doing a good job of it."

The misuse of social media

The "inbound social media" research grew to massive proportions, then underwent mitosis to become three separate documents, because it's difficult to encapsulate the discipline of using social media. Like any new field, social media are fraught with both opportunity and risk.

In the particular application that I was investigating, social media as a new source of requirements data, people can commit the same mistakes with this new source of information as they have with the old ones. For example, important players in the product development process often make decisions based on the customer with whom they last spoke. An equally common temptation is to listen to customers who echo what you want to hear, and disregard the rest.

In the best possible circumstance, you get a single person who appears representative of a larger user population, gives you credible insights, and even articulates their needs clearly. The obvious problem is that you're only listening to  one person, who will never be 100% representative of the larger population. That's especially bad if the population contains distinct segments of people with different needs, habits, and skills, and you have to build something that fits the bill for more than one of these cohorts.

If you're not taking advantage of the opportunity to use social media to collect more data than you could through traditional, labor-intensive methods  (for example, customer meetings), you're missing a big part of the reason for paying attention to social media in the first place.

And if you don't believe me, take a look at this item from The Guardian about how British businesspeople went gaga over a single 15 year-old intern's analysis of how teenagers use social media. To be fair, both traditional and social media contributed to the buzz around the document. But that's the point: you can repeat all the mistakes of old media when using new media.

[Cross-posted at The Forrester product management blog.]

Picking the right social media for requirements

All three parts of the series on social media as a new source of requirements data are now published. The first shows how product teams can use social media to create reach more accurate conclusions than traditional sources of requirements (customer visits, enhancement requests, customer advisory boards, etc.) alone can provide.

The second document distills the lessons learned from attempts to use social media in this "inbound role" into a methodology, PLOT (persona, location, options, test). Since the choice of which social media can best answer particular kinds of questions isn't immediately obvious, I devoted the third document to that topic alone.

During the research for this series, it became glaringly obvious that there is a major dividing line in the type of information that product teams collect and analyze:

• Product-centric, which related directly to the company's products and services. How many people have voted for a particular feature is a good example.
• Problem-centric, which depicts the customer's world. The patterns in how organizations adopt particular kinds of solutions, for example, is vitally important for product teams, even though the best form of that information makes no reference to the vendor, or even the supporting technology, whatsoever.

Shown here is a graphic that summarizes the kinds of requirements information that falls into these two categories. To borrow a phrase from William James, the breadth of social media options appears, at first glance, as a "blooming, buzzing confusion." This distinction between product-centric and problem-centric issues is the first of several filters needed to select the right social media outlet to answer a particular question.

This distinction is also critical for determining whether the company's current investment in social media can be at all useful for the product team. Already in many technology companies, Marketing's enthusiasm for social media inspires skepticism in Development. In the worst cases, it's almost as if someone were to propose to John Calvin, "The papacy has some interesting ideas about transubstantiation. Why don't we look into them?

This product/problem distinction helps decide whether the existing social media channels will be of any use to the product team. In some cases, they certainly will be. In others, either the company will need to build additional social media tools, or the product team will need to look outside the company's web presence altogether.

[Cross-posted at The Forrester product management blog.]

Why use cases matter: Memex

Suppose you're a company like Memex, in the business of providing software to police departments. Given the nature of the work, almost any electronic assistance might be preferable to the alternative. Would you rather do a quick search on all information about a suspect whom you're interviewing, or dig through file cabinets and desk drawers for information that might not even be there?

The temptation of lazy design
Law enforcement's need for electronic assistance might tempt software developers to be lazy. As anyone who has built an application for a specialized purpose (tracking donations, handling equipment requests, pooling notes about patients, etc.) knows, it's easy to create a bare-bones UI and the underlying database. It's a lot harder to make it usable, and to build in the application logic that the real world demands. And then there are the inevitable customizations...

A quick analogy: I decided to put in a drip watering system for some plants in the back yard. Someone could have handed me a box full of random hoses, PVC pipe, and connectors, and left me to figure out how to make use of it all. While some of the hoses might have been good for someone else's garden, they don't match the layout of my plants. If I don't have the right tools--for example, a wrench to tighten some of the couplings--basic aspects of this household project will be a lot more challenging.

Instead, I bought a kit. It wasn't perfect for my needs, but it got me started a lot faster and more successfully than the approach I was tempted to take, buy the components that, in my earnest newbie ignorance, I thought would work together.

The benefits of starting with the use case
Let's leave the peonies and get back to the felonies. Again, you might throw together a UI that lets you enter incident reports, search on related crimes, and report on the closure rate of investigations. However, making the application usable takes real work, including paying careful attention to how people in law enforcement actually work.

That's what made me very interested in finding out all I could about Memex's applications. The UK-based company hired Peter Ship, a former investigator in the murder unit of London's Metropolitan Police, to advise them on their product design. The company also worked with local police departments to provide further information and validation of how to fit the technology to police work.

Whenever you take this kind of use case work seriously, you wind up emphasizing features that you might not have been expecting to be as important as turn out to be. For example, the decision to design the application around the needs of an inexperienced investigator, potentially the weakest link in the chain leading to prosecution and conviction, prompted Memex to invest in a "Validate With Advice" capability. Based on the police department's own guidelines for how to handle particular situations, the system makes important suggestions to the investigator. For example, if the incident report falls into a category of crimes (smash-and-grab thefts, domestic violence, etc.) that the department is giving special attention, the Memex application can tell the investigator how to prioritize this investigation, and what next steps to take.

This particular example also shows how use case analysis can help development organizations make tough decisions. To make the Validate With Advice feature work, you need a rules engine that's flexible enough to accommodate a lot of departmental guidelines, and simple enough so that the non-technical users who best understand these rules can define them in the application. On top of that, to get people started, the application needs some canned rules that resemble the sort of guidelines that most departments will enforce. That triple combo of capable rules engine, interface for business users, and pre-defined rules is a big development effort, frought with potential pitfalls--exactly the kind of project that many development teams would try desperately to avoid.

Room for creativity and coolness
Sometimes, people in the tech industry grumble that a too-slavish dedication to use cases reduces their creativity. However, Memex's design team shows how the use case can be the springboard to very useful features with a high coolness factor.

Many investigators use a link chart to depict a web of relationships among the facts of a crime. Witness A saw Suspect B in location C. However, we know that 10 minutes earlier, Suspect B was across town using an ATM. So how did he get to C in time to commit the crime.

Memex's tools build this kind of link chart on the fly. Autochart is pretty cool to see on its own, but it's especially cool to see how it works in conjunction with search. An investigator who has doubts about Witness A might search across repositories to find more information about that person. As pertinent facts emerge--for example, all the other times this person provided an eyewitness account of a crime--that the investigator might add to the link chart. Over time, the picture that emerges isn't a machine-generated set of pointers between bits of data, but a carefully-constructed narrative, based on the facts and the investigator's judgement.

Someone who hasn't taken use case analysis seriously might still create a link chart, but it's more likely than not to be a half-assed attempt, designed more for demos than actual use. The connection between the link chart and search is exactly the sort of capability that would be missing in the marketware version of the feature.

I have a gut feeling that people building vertical-specific applications like Memex's might be doing a better job of use case analysis than many "horizontal" technology built for a general audience.  No matter how horizontal or vertical your product might be, the Memex case shows how very, very specific use cases can lead development teams to resist lazy design and make the right design choices.

Why use cases should drive technology design

During a briefing from Microsoft's xRM team, the question of how to integrate structured and unstructured data arose. If xRM (the Dynamics platform) is good at the structured stuff, and SharePoint is good at unstructured content, what's the right way to bridge the two?

Back in my Oracle days, we faced exactly the same question. At a technology level, there's no obvious answer. Bring together two development teams (the structured and unstructured specialists), and you'll first get a lot of technical-level discussions. How should security work? What API changes might be needed? How will metadata span the two kinds of information?

Unfortunately, there's no immediately obvious answer to these questions. In fact, the options are so broad, and the risk of technological quagmires so great, that the endeavor might easily grind to a halt. People ponder the options, argue over which one is best, go back and ponder some more...

At Microsoft, a clear use case accelerated this discussion. The Public Sector group developed a constituent services application for local governments that combines both SharePoint and xRM. Notice how specific the application, the Citizen Service Platform, is. Not all government, but local government. Not several government functions, but constituent services. 

Undoubtedly, the Citizen Service Platform team made it clear which kinds of integrations were important, and which weren't. It's not too hard to generalize the use case to other situations. After all, we're talking about the same CRM platform that property managers and church pastors use, for different but similar purposes.

For people who are already sold on use cases as the right foundation for designing products, this might seem like a "Well, duh!" sort of observation. However, many product teams go full speed ahead with only the haziest use case, or to be honest, none at all. If their product decision-making process is too slow, or too error-prone, they might reconsider the value of a very specific use case.

[Cross-posted at The Forrester product management blog.]

Podcast follow-up: A movie recommendation

Here's the movie, Witnesses, that I recommended in this week's podcast. For those of you who haven't listened to this week's episode of The Heretech podcast, I cite Witnesses as a great way of showing how hard it can be to understand people with whom you have the briefest of contacts. On top of that, the lack of information matters a great deal.

The movie is about the conflict between Croats and Serbs, but the moral applies to a  lot of situations.  For example, an enhancement request is a small piece of information that, because of its incompleteness, can be wholly misleading.

"Good enough" isn't good enough

I'm a big fan of the latest version of Microsoft Office. Rather than just keep plodding forward with yet another set of new features, the product team tried to re-design Office to make it more useful. A good chunk of that effort went into making the capabilities of Office more visible, since most of the features that accreted over several releases were hidden away in various menus and dialogs.

That's only a small part, however, of the very interesting story behind the Office re-design. I strongly recommend taking the time to watch "The Story Of The Ribbon," a presentation by Jensen Harris, the group program manager for the Microsoft Office user experience team. It's also a good history of Microsoft Office interface design, which led Harris' team to conclude that the most common reaction to the UI, "Good enough," actually wasn't good enough.

Harris' team spent a lot of time with users before forging ahead with the re-design. However, the Microsoft Office team wanted to learn how people used Office, and also what they wanted from it. In other words, it was a much different requirements-gathering effort than the typical "Tell us what features you want" meeting, or other mechanisms (e-mail, the bug database, voting on features, etc.) that convey only bits and pieces of the overall picture.

Ultimately, the overall picture is what makes the new Office way, way better to use. As the results show, building a good product isn't a question of having a creative or inventive team. Nor is a stream of customer requests going to tell you everything you need to know about making them happy with your product.  You need both, and the right mental framework into which all these pieces fit.

[Many thanks to Peter Merholz for the link to this presentation. Thanks also to Daniela Busse, one of the members of the Office design team, for a separate explanation of the thinking behind these changes.]

What's the Dev team's interest in social media?

Most aficianados of social media emphasize the customer-facing applications of these technologies. By now, we've all heard interesting stories about how Marketing used blogs to get the message out, Sales used forum postings to help qualify leads, and Support used Twitter to respond to users wrestling with technical problems.

Exciting, new-frontierish stuff, to be sure, but you hear far less about Development's social media strategy. What about the "inbound" applications of social media?

That question was my inspiration for what turned into a three-part series on "inbound social media." The first research document appeared today (Forrester subscription required to read the whole enchilada). The second and third parts are coming shortly.

A lot of development teams are skeptical about their company's investment in social media. Frankly, they don't see what's in it for them. Worse, it threatens to be a distraction from their mission to execute, execute, execute.

But, as I found, social media fills an important informational gap, as long as you're willing to make the commitment to taking it seriously. You may have already heard my rants about traditional sources of product requirements, and why they stink on ice. (And believe you me, I speak from firsthand experience on that topic, as well as the vicarious experience of many PMs I've interviewed in this job.) If not, here's a sample.

The first document in this series is really a summary of why Dev teams should take social media seriously, and how they'll have to adapt to use it effectively. For example, inbound social media are a mechanism for answering questions, some of which will be ongoing, and some of which will be ad hoc. Most development teams (broadly defined to include QA, Doc, and PM) don't necessarily have the right organization and habits to juggle both of these types of inquiry. Driven by release schedules, teams focus on the questions relevant to the next release--not necessarily the bigger questions that span releases. Social media can address both kinds of questions, but teams will have to make some adjustments to find these answers.

Many technology companies will have to change the way they make product decisions. I used to annoy the heck out of stakeholders by saying, "If we've reached a question that we can settle empirically, then let's shut up and find the answer." Not everyone wants to sacrifice their dearly-held opinions, or their freedom of action within the organization, on the altar of the scientific method. But that wrenching change may be necessary for these teams to survive and prosper, given the challenges of convincing customers to buy and adopt new technologies.

[Cross-posted at The Forrester product management blog.]

Traditional requirements stink on ice, and then some

I, The Customer
I've already talked at length about how the assumption of the unified, rational actor called "customer" is a myth. That notion is just as chimerical and comical in product requirements as it is in economics and political science. Customers, like firms and governments, are neither unified nor rational.

Any point that important is worth making again, particularly as part of my recent litany of problems with requirements, as traditionally conceived. Rather than repeat what I already said, I present you with a pungent example, ripped from the pages of Two-Fisted Tales Of Product Management.

This PM For HIre
At one of the tech vendors who employed me as a product manager, we had a Very Important Customer whose implementation of our software had run aground. Our main contact people in the customer's organization were complaining that we had delivered a product that bore little relationship to what we sold them. The primary medium for communicating their dissatisfaction was a string of enhancement requests that were as baffling as they were urgent.

Despite several long conference calls, we couldn't figure out what the real issue was. We actually had the features they needed, with a couple of notable exceptions that were already in the roadmap. What, then, was the source of their dissatisfaction? No amount of patient explanation about how the product already worked, or what was planned for the next release, assuaged them. They needed a particular set of changes, and they needed them now.

The Customer Always Rings Twice
Unfortunately, the person who had actually sold them the product, and who had met with the customer regularly, couldn't provide any further insights. Someone from the product team needed to visit them. Fingers crossed, I got on a plane.

My first day of meetings with "the customer" were a major step forward, at least in understanding what was going on. Here's the thumbnail sketch:

• We had worked with the head of the implementation team--let's call him Floyd--on an earlier project for the same customer. The need was clear, and the deadlines were firm. For a variety of reasons, Floyd's team built a custom application on top of the Vendor X technology, to meet the needs of this particular project. Floyd and my company, Vendor X, emerged with a golden aura around us.

• Having heard about this success, another part of the customer organization wanted to use Vendor X's technology to solve a different set of problems. The implementation would span the entire organization, and (in theory) the project fit the out-of-the-box functionality rather neatly.

• Unfortunately, Floyd had tasted what it was like to build custom code on top of Vendor X's technology, and he liked it. Even though Vendor X's engineers had invested thousands of hours building an application on top of this platform, it was their vision, not Floyd's, that went into the design. Therefore, in Floyd's mind, his vision of how the application should behave overshadowed what the software already did.

• This vendor/customer snarl might have been easier to untangle, had bureaucratic arrangements been different. Unfortunately, Floyd's implementation team reported into a different part of the org chart than the people funding and project managing this initiative. The business analysts and IT people lived in completely different organizations, housed in different facilities. The 20 to 30 minute drive between their buildings felt more like the gulf between the Earth and Saturn.

• The business analysts were struggling to steer their conversation with us back on a productive vector, but they kept having to fight Floyd for the wheel. Given Floyd's prior experience with the technology, the business analysts sometimes second-guessed themselves. In the process, the line between requirements and implementation disappeared.

The Big Creep
This customer's "requirements," therefore, were actually a boiling pot containing many legitimate concerns, mixed with that organization's conflicts and confusions. The ultimately poisonous combination was impossible for either side in this discussion to digest. Meanwhile, feature creep continued, as both the Floyd's them and the business analysts kept throwing new ideas into the pot, hoping that they'd stumble onto some recipe for success.

Eventually, the altogether too many cooks in this customer's kitchen grabbed for their knives. After the survivors mopped up the blood, they were too weakened by the experience to continue with the project. They looked at other technological alternatives, which didn't progress any farther than we did, since the problem wasn't the technology.

The Customer Has A Thousand Eyes
This cautionary tale may be a bit extreme, but it's hardly unprecedented. Conversations with "customers" often dramatize the quasi-feudal conflicts within their organizations, as they did in this case.

However, requirements conversations may also be deceptively placid. As long as the Support and Consulting people at Vendor X spoke only to Floyd, his requests for product changes were heated, but laregley manageable. The grand mal seizures started, tellingly, as soon as the business analysts entered the conversation with us.

I've certainly faced other situations, as a product manager, in which there were great risks in taking at face value any "customer requirements" from a seemingly calm, rational customer. In a few of those cases, we in Development were dealing with the wrong people. And that's when the wackiness ensued.

I Am A Fugitive From A Dev Team
If I were teaching a class on product requirements, day one would focus on the question, "Who's the person who's telling you what's important in your product?" That person might be reputable, reasonable, and sincere. Or, he might be Floyd. Alternately, Floyd might be lurking nearby, waiting to pounce on the discussion. In the worst possible scenario, the "customer" turns out to be a confederacy of Floyds.

Sadly, too many newly-minted product managers wander into these dark alleys far too easily, with the best of intentions. There must be better ways to learn about the mean streets of customer requirements than the infamous school of hard knocks.

[By the way, if you're not as much of a film noir fan as I am, you might not have recognized the paraphrased film titles that I used as headings in this post. Here's an overview of this genre.]

[Cross-posted at The Forrester product management blog.]

Traditional requirements stink on ice, continued

First, let me belatedly acknowledge Luke Hohmann, who ranted eloquently about traditional requirements at his P-Camp session several days before I started this current screed. His observation was Hemingwayesque in its pithiness: "Requirements suck." Mine is more Faulkneresque, using an idiom ("stink on ice"), undoubtedly with a colorful origin that no one remembers.

The second reason why traditional requirements stink on ice is, ultimately, a question of perspective:

• Development teams build technology for people who are wholly unlike themselves.

• No team has the resources to live side-by-side with the people for whom they're building the technology. The users don't have the time or inclination, too.

• Most development teams rarely interact with people on a different floor in the same building, so living side-by-side with the target users wouldn't be an automatic mechanism for osmotic transmission of insight.

• Therefore, development teams generally view requirements in their own terms, and not the world as the end user sees it.

By necessity and habit, development teams think and speak in terms like features, projects, and releases. Occasionally, users may use this language to make themselves understood to you, but it's not the terms that define their world. They want to do their jobs more quickly and successfully, and  then go home and think about something other than inserting a table of figures into a Word document.

Traditional requirements both reflect and reinforce this language barrier. Obviously, at the end of the day, you do need to go back to your cubicle and work on features. However, at some point in the process of designing, building, and releasing technology for people unlike you, you need to speak and understand their language, not yours.

To illustrate, here's an archetypical exchange between an ISV and a customer:

Customer: "We've been asking for this feature for a while. When will we get it?"

Vendor: "Very soon now. Meanwhile, what do you think of this feature? We think it's a great idea."

Customer: "Sure, it looks interesting."

Vendor: "Cool, because we're hoping to get it into the next release. We thought you might like it."

While there are many, many things going wrong in this conversation, for our purposes, the chief problem is linguistic. Both parties are guilty of speaking the vendor's preferred language, without making it clear how it translates into the customer's. Asking where the ion flux regulator appears in the product roadmap doesn't clarify why it's important to have one in the first place. Nor does it help understand whether the etheric vortex stabilizer helps solve the same problem for the customer, or anything at all.

Here's when some Agile aficianados are jumping out of their chairs to say, "Oooh! Oooh! Let me tell you about personas and user stories!" I largely agree that these Agile-approved requirements media can go a long way towards understanding the customer. However, they're not complete solutions for two reasons:

• The medium often constrains the understanding. For people whose Agile approach involves a lot of Stickie notes or index cards, it should not be a shock to hear that minute tiles of information, as useful as they are, do not always automagically transform into a beautiful, accurate, or complete portrait of the target user. At least in practice, that is.

• The sources of information still deserve some skepticism. You might drop the jumbo Word and Excel documents for personas and user stories, but you still may be extracting information from an unrepresentative sampling of users.

Social media are not a panacea for this language problem, or any other congenital defect that afflicts traditional requirements. Nonetheless, there are ways to use social media to help crack the language barrier.

Mandatory caveat: As in other situations, it's possible to use social media badly here. For example, giving users the ability to vote on feature ideas is a wonderful thing. However, keep in mind that this new channel of communication you are allowing them to speak your language more clearly, without necessarily improving your ability to speak theirs. In fact, some vendors' parochial use of social media resembles what many Americans do when confronted with someone who doesn't speak English: "Maybe you people will understand us IF WE JUST SPEAK MORE LOUDLY."

[Cross-posted at The Forrester product management blog.]

Traditional requirements stink on ice

I've had to put blogging on the back burner for the last week because one research document, covering how product managers can use social media (blogs, Wikis, forums, etc.) as a new source of product requirements, underwent mutation, and then mitosis. Now, it's three separate documents, each of which demands all the empirical and stylistic discipline that Forrester demands. In short, I've been busy.

However, now that two of the three documents are drafted and in the capable hands of my research director, I need to vent. The target of my ire isn't the tribble-like spawning of new documents, which at the end of the day, is the right call, and has made the author much happier with the final product.

No, what's really churning in my guts is the inevitable outcome of talking to people about product requirements, writing about how to improve them, and re-visiting the topic during the editing process. Let's be honest: usually, they stink on ice.

I like that idiom because it evokes a truly epic scale of odiousness.If, in the depth of winter, when everything liquid or semi-solid freezes, and the great slab of ice on which you're standing leeches all the warmth and life out of you, slowly killing each of your senses in turn you're smelling something foul, it must be really foul.

No single post can hope to capture the complete odiousness of traditional requirements. I'll space this philippic across at least a couple of post, but let's start with the most obvious problem, and  the destruction that it wreaks on sensible product decisions, not to mention the working relationship between product management and development.

Wandering in the informational desert
What are the most frequently used sources of requirements in the technology industry? According to our surveys, the results are anything but surprising to us veterans: (1) customer meetings, either face-to-face or over the phone; (2) bug database entries; (3) enhancement requests transmitted through e-mail, often via mailing lists. Customer meetings are also the most trusted source of insight, because they provide far more information than a vague "I have a customer" e-mail from someone in Sales, or a bug report that conveys more frustration than information.

Of course, as "useful" as customer meetings are, they're hardly frequent enough to create a statistically significant sample. People should be skeptical, when the N is small, of how representative the customers are of the larger market, and how reliable the conclusions based on this information can be. Scheduling a few more conference calls, or getting leave to visit another customer on-site, will not provide enough empirical fuel to launch to slip the surly bonds of skepticism.

All of which leads to statements we've all heard before:

"He strongly

• holds the opinion of the last customer to whom he spoke."

• "This customer is full of spurious requests. How much can we trust their judgment?"

• "I've spoken to as many customers as you have. What makes you a greater expert on customers than me?"

• "We don't have time to keep going back to these people to see what they really want."

And so on, truly ad nauseum.

Therefore, one of the virtues of social media as a new source of insight into product decisions is the sample size. Would you rather base the next 12 months of development on an hour's conversation with five key customers, or information collected from hundreds of customers posting on discussion forums and blogs?

Certainly, there are lots of potential ways to misuse social media in this fashion. Aggregate information may be miles wide, but inches deep. Collected sloppily, the sample may be just as skewed as the handful of customers with whom you normally deal.

Still, it's hardly a courageous leap to say that social media provide some useful information that can bear on product decisions. At the very least, these other sources of information can supplement the traditional stuff.

When we return, be prepared to hold your nose as we wade through the other problems with traditional requirements. 

[Cross-posted at The Forrester product management blog.]

Space Marines, attack!

A segment of the technology industry reaches a point where the different species of applications are well-known and well-established. As long as no new species emerge, vendors settle into a features arms race, hoping to impress potential customers an ever-increasing number of options and capabilities. Meanwhile, many customers look at their limited time, limited budgets, and limited interest levels, and wonder why on earth these vendors think that added complexity is always a good thing.

Sound familiar? It should--and perhaps not for the reasons you think. I'm actually talking about the computer gaming industry, which in many respects leads their more serious brethren among software companies. Take a moment to consider the challenges that game vendors have faced and overcome:

• High quality levels. Oddly, people get far angrier over bugs in a $50 Wii game than they will about a much higher bug count in a $500,000 ERP system.
• Customization. The challenge of maintaining quality increases another order of magnitude the moment you add customization capabilities. On the other hand, the demand for customization (called modding in the gaming world) 
• Demanding channel. Once you set a release date, you're highly penalized for missing it. Forget staying on good terms with retail outlets and media outlets that count on you making your date. High turnover. The shelf life of the average computer game is measured in
• The Internet in the product. The gaming industry have its version of software as a service (SaaS), the massively-multiplayer online role playing games (MMORPGs) like World of Warcraft. Even best-selling on-premise games like Call of Duty need a strong online component.
• The Internet in the business model. An increasing number of gaming companies are shifting to an online distribution model (for example, Steam). Online distribution also makes smaller components--both add-ons and independent products--more economically viable.
• Social media. Game vendors invest heavily in creating and maintaining their communities, such as the XBox Live, to the mutual benefit of customer and vendor alike. These communities take real investment, however, including both the implementing technology and the people needed to monitor the community to maintain a good customer experience.

Note that the list above describes changes that affect the computer gaming market more than computer games. Standards for the actual products, such as quality levels and customization, haven't changed that much. You can still categorize games into well-understood genres, like first person shooters, with little changed in the last few years. (The notable exceptions, of course, are the MMORPGs.)

Which leads us back to the "more is better" dynamic. Despite more impressive graphics and an ever-increasing number of game options and capabilities, many types of computer games have plateaued in sales, or even started to drop.

MMORPGs are one source of distraction. Economic upheaval doesn't exactly help sales. However, there's more to the story, in which the protagonist (the computer gaming vendor) reveals his fatal flaw (addiction to complexity).

Some gaming vendors have spotted the problem, and taken steps to fix it. The developers behind Warhammer Online, for example, recognized that many gamers spend time "soloing," spending a lot of time playing on their own, instead of teaming up with other players to meet game objectives. Warhammer Online, therefore, doesn't distract the solo player with information about team options that he doesn't care about.

This interview with the designers of the just-released Dawn of War 2 is one of the most interesting discussions of this issue. You don't need to be a gamer to appreciate comments like these:

To us, it felt obvious that we were falling behind for a reason. We decided we needed to simplify the entry point but retain the depth at the back end. And we need to work on the pacing. We need to work on ways to draw the player in...

If you substitute the word user for player, and ERP system for game, you'll see why the computer game industry has many important "lessons learned" for people building other kinds of software.

[Cross-posted at the Forrester blog for product managers.]

Tell me lies

Christopher Cummings' latest entertaining and insightful post is worth reading:

Think about the last time a salesperson at a retail establishment asked what you were looking for. How honest was your answer? And how complete was it?

Exactly. Product teams often don't realize how misleading information coming straight from customers can be:

• They see the conversation as part of an ongoing negotiation with you, the vendor. This bigger context shapes what they say, and how they say it.
• You're dealing with middlemen. Sure, members of the IT department has a wish list, but on many issues, they're really fronting for business users who want something from you. How well they're representing this Silent Majority is anyone's guess.
• They're an unrepresentative sample. Let's be honest: the customers to whom you speak, or with whom you'd rather speak, are not the same as all your customers. You lean towards the people who are easy to work with, who understand you clearly, and who can speak the language of releases and features that you understand. That's a pretty skewed sample, and it's even further from the population of all people who might be your customer.

In other words, the traditional sources of product requirements are inadequate. Even the one that seems on the surface to be the most trustworthy, the face-to-face meeting, can be incredibly misleading.

And you often have yourself, Dear Vendor, to blame. You choose to be deceived. You get into these conversations with your own agenda--the boffo feature for which you're seeking validation, the workaround that you want them to use, the argument you just had with a co-worker about a design decision that you'd like to win, if only this customer would agree with you. You want the customer to tell you lies.

There are ways out of this trap, which is why I wrote that document about the Web 2.0 sources of product requirements that I keep talking about (and will be published soon, honest!). However, you have to want to face the hard truths of what you don't know, you might not initially understand, and in some cases, you may not want to hear.

Calling all PMs: Sample requirements documents

What we're doing
The research into a research piece on best practices for requirements documents is underway. As anyone who has ever been responsible for requirements knows, there are two sides to this question:

• The requirements process. The overt part of requirements is the process of writing them. The subtler part of the process is figuring out the information needed for the audience. Development (including QA and documentation) might be the primary audience, but they're hardly the only ones.
• The work product. In other word, the actual requirements document. Or, in many cases, documents. For example, while there might be a template for feature requirements, it may link to related or supporting documents (personas, customer feedback, etc.)

How you can help
As with any type of research, the larger and more representative the sample, the better. Which means that you, Dear Reader, can help make this research project all that it can be. In this case, we need your help with that second aspect of requirements, the work product.

If you're willing to share your template for requirements, please send it (or them) along. We're definitely not interested in the content--just the template. We promise to keep the source of any archived requirements documents confidential.

• Samples of what you might have that we'd love to see include the following:
• The Word document for feature requirements.
• The data model for requirements maintained in a fancy-schmancy requirements tool.
• The standard format for use cases and user stories.
• The mock-ups used to depict requirements, if you don't keep traditional requirements.

You can send this information directly to me, [email protected]. If you're not sure what to send, just ask.

Plus, if you're willing to be interviewed for the requirements process part of the story, please let me know,

What's in it for you
As always, we'll share the final report with you, once it's published.

Agile, social media, and requirements

Speaking of social media, one of the two research documents now in the editing queue looks at using social media as a source of product requirements. Using Forrester's POST* methodology as a starting point, how can product managers harness the enormous amount of potentially useful information transmitted in the clear through blogs, forums, Wikis, and similar technologies?

The other document in editing is the "Agile company" piece, covering the results of the survey and interviews we conducted to understand how Agile development changes technology companies. To foreshadow the results, I had to divide Agile adoption into two stages. To date, Agile aficianados have focused on the first, Agile within the development team. Clearly, for the story of Agile adoption, that's only Chapter One.

* In this approach, the steps for analyzing social media involve people, objectives, strategy, and technology (POST).

[Cross-posted at The Forrester product management blog.]

Your attention is required

One of the hot topics during my visits with Forrester clients this week was requirements. How do other people do them? How do they change when development teams go Agile? Why is it so damn hard to re-use the information written for the development team for other purposes, such as training the salespeople? And so on.

I'll be starting some research on requirements in a month or so. Meanwhile, as busy as my writing agenda is, I can't help but ponder the topic. To complete the distraction, I was also reading some of Tyner Blain's recent posts on the topic, such as this interesting discussion about user stories and use cases.

To a large degree, requirements have to conform to the peculiarities of an organization. While there might be some Platonic ideal of requirements, manifested incompletely in this imperfect world of ours, there's no escaping the regular back-and-forth between the authors of requirements and their readership. Development teams have their own preferences for information, both quantity and format. You'd be a very bad product manager (and probably not employed for too long) if you were to ignore that reality.

Still, requirements design isn't as simple as adding or removing pieces into a requirements data model until all parties involved are happy with the results--particularly since one kind of requirements information doesn't really seem like it has a data model at all.

Measurable facts
Suppose you were to write requirements for a salesforce automation system. One of the target users for this flavor of CRM is a salesperson. Assuming you haven't worked in Sales yourself, how do you find out what sort of technology this person needs?

One way to find out is to grab every piece of hard data available about the job. What's the average income of a salesperson? What are the tasks that person performs? What obstacles do they encounter along the way? How strong or weak are the computer skills of that person? What kinds of software and hardware do they use?

After collecting this Grey's Anatomy of the sales profession, you can pinpoint where the CRM technology  can help the archetypical salesperson. According to our research, salespeople spend 72% of their time away from their laptops, which makes mobile support a top priority, etc. etc.

If that approach seems a little bloodless...Well, it is. Not every important aspect of human behavior reveals itself through this sort of information. After poring over Department of Labor statistics, you might also need to go to the movies.

Observational insights
Statistics may tell you how people spend their time. Observation, on the other hand, reveals many of their attitudes, inclinations, hopes, dreams, enthusiasms, and complaints that you might otherwise miss.

The trick, of course, is knowing how to observe people well, and whether you're observing the right person. If you want to know what life looks like from inside the head of a salesperson, you might rent Glengarry Glen Ross, or Death of a Salesman:

You don't understand: Willy was a salesman. And for a salesman, there is no rock bottom to the life. He don't put a bolt to a nut, he don't tell you the law or give you medicine. He's a man way out there in the blue, riding on a smile and a shoeshine. And when they start not smiling back―that's an earthquake. And then you get yourself a couple of spots on your hat, and you're finished.

After Salesperson Movie Night, you might look a little differently at the CRM feature list. Something that makes it easier for salespeople to keep their customers feeling as though they're getting special treatment might have much greater appeal, even though that task might consume only 10% of that person's time, and may not even figure all that clearly in the scientifically-collected work profile that you've collected.

This type of information has its own perils, however. David Mamet and Arthur Miller might be fantastic playwrights, but there's always the risk that even the best artists don't do their subjects justice. We judge these two plays on their own merits as dramas, not as social-ethnographic profiles of salespeople. Similarly, you might spend a quality day shadowing a salesperson, only to find out that she's not typical of her profession. For example, she could be the very best, or the very worst. Either extreme might not tell you what the average salesperson really needs.

Plus, what do you do when you have to write the requirements? Use cases don't exactly capture the "smile and a shoeshine" aspect of Sales. Nor do carefully-maintained functional requirements. While people often make fun of personas, or lack the time to pay careful attention to them, they do have their place (when done well) in transmitting some important insights to anyone--developers, support technicians, product marketers, you name it--who has contact with that user.

So, what then is the ideal form of requirements? Maybe the triune division of business, user, and functional requirements makes the most sense. Or maybe these are just expressions of another level of distinction behind them: knowing facts about people, versus understanding them at a deeper level.

[Cross-posted at the Forrester PM blog.]

Proving the value of better requirements

Ever since I did the research on tools for product managers, I've been diddling with an ROI calculator for improved product requirements, which can come from two sources: (1) hiring another product manager; and (2) implementing a specialized requirements tool.

Here's the final product. Really, this is a way of demonstrating the value of product management. Product managers and PM tools are just the tangible manifestations of that business function.

[Cross-posted at the Forrester PM blog.]