I apologize to everyone I promised this post to weeks, if not months, ago. The irony here is that I approached this in the least lean way possible and employed no customer development in the process, beyond gauging initial demand. This is to my detriment, and as you’ll see, to my readers’ as well; a lesson in how difficult it is to give up the endemic desire for completion among those who have only seen the results of creation and suffer from having taste. That violates the fundamental tenets* I’ll describe here.

Like a startup, this should be a living thing, a shark, that has to constantly move forward or it dies.

And I think what we got on our hands is a dead shark. As Mark Cuban points out, no one remembers where you came from. The same goes for writing.

While I dawdled, Ash Maurya wrote a superior introduction to these concepts, here. See also the “definitive resources” on Customer Development. The steady flow of new and better material on the subject conspires against me.

Introduction

I’ve heard the Lean Startup methodology likened to a framework. Its central precept is that a startup should develop its customers as it develops its product – that is, both concurrently and in a structured, concerted fashion. If you’re taking advantage of a framework like Rails for the latter, you shouldn’t be using the business equivalent of compiled Java for the former (or more accurately, cf. Waterfall, Agile). It’s about mitigating risk at every decision point. It’s a floor wax and a dessert topping. Even use it for your blog.

I’d further liken the Lean Startup to an entrepreneurial diet. As one trims the fat in Lean Manufacturing (skeptical as I am of Taylorism’s descendents) by reducing batch sizes and inventory, so the Lean Startup pares down and focuses its product builds – but don’t confuse lean with cheap.

More aptly, a diet is an ostensibly easy prescription – eat less, eat healthier – which, to execute successfully, requires rigid discipline and instruction. Then you have to be certain that you’re not following Atkins to a bypass.

Key players (even if they don’t think so themselves)

Steve Blank – Defined and developed most of these ideas in his book, The Four Steps to Epiphany. Have a look at his compelling case study on why and how he extrapolated the Customer Development methodology.

Eric Ries – Cofounder of IMVU and leading Lean Startup evangelist. He coined those terms that Steve Blank didn’t. He blogs at Startup Lessons Learned.

Sean Ellis – Extraordinary startup marketer, championing the concepts I’ll talk about here. Some of his posts on the subject. Notably, he suggests that “you’ve achieved product/market fit when 40% of your customers say they would be very disappointed if they didn’t have your product.”

Marc Andreessen – Netscape cofounder and proponent of product/market fit.

Paul Graham – Y Combinator founder. Strong advocate of making something people want and rapidly iterating it to ramen profitability.

David Heinemeier Hansson and Jason Fried – Creators of the Ruby on Rails framework, pushing the idea of “Getting Real.” DHH, the Paul Bocuse of startups, adheres to the idea that “less is better” (a great read) and the immensely important notion of “renegotiating requirements,” what he considers the secret to high productivity, at about 45:50 in his RailsConf ‘09 keynote (only tangentially related to Rails, and unusually light on the Lenny Bruce).

Nivi – The mononymous patron of VentureHacks. Promotes many of these ideas.

Dave McClure – A pirate or something. Favorite of milliners.

John Boyd – A fighter pilot who explained why American planes with inferior specs were able to attain air superiority. The OODA (observe, orient, decide, and act) loop emerged from his studies.

Andrew Chen – See his post on Minimal Desirable Product. Clever fellow.

Ash Maurya – For his attempt at a textbook application of these ideas.

Everyone in the Lean Startup Circle – I haven’t yet posted here. I need to get out of the building. Maybe I’ll post about this post and post about that here.

Sean Murphy – I don’t read him regularly, but I hear that I should.

Wil Schroter – As far as I know, not a conscious practitioner of CD, yet he implements it better than anyone else I know, never leaving his startups to chance. Arguably, insufficiently recognized for this because he doesn’t speak the language of Hacker News.

Tim Ferriss – Admittedly, a stretch.

Ben Yoskovitz – Writes the Instigator Blog.

Neil Patel – Cofounder of KISSmetrics and blogger at QuickSprout.com.

Steve Jobs – He does Customer Development.

A personal note: he gave me a brilliantly false perception of entrepreneurship, one that I cherish. I think it was a Time Magazine cover reading “Charisma?” And my reality distortion field (the original story, here) needs calibration. When it works, I’m told it’s more Rohypnol than acid.

The proper definition

You can find this on Wikipedia:

1. Get out of the building
2. Theory of market types
3. Finding a market for the product as specified
4. Phases of product & company growth
5. Learning and iterating vs. linear execution
6. Premature Execution

I’m not following the list, so read the book for that. I’m conflating concepts.

Customer development

Eric Ries’ introduction:

Many of us are not accustomed to thinking about markets or customers in a disciplined way. We know some products succeed and others fail, but the reasons are complex and the unpredictable. We’re easily convinced by the argument that all we need to do is “build it and they will come.” And when they don’t come, well, we just try, try, again.

He cites it as one of the three pillars of the Lean Startup:

1. The use of platforms enabled by open source and free software.
2. The application of agile development methodologies.
3. Ferocious customer-centric rapid iteration.

It is not a focus group:

Customer Development is about Testing the Founder’s Hypothesis
Any idiot can get outside the building and ask customers what they want, compile a feature list and hand it to engineering. Gathering feature requests from customers is not what marketing should be doing in a startup. And it’s certainly not Customer Development.

In a startup the role of Customer Development is to:

1. test the founders hypothesis about the customer problem
2. test if the product concept and minimum feature set solve that problem

This is a big idea and worth repeating.  Customer Development is about testing the founder’s hypothesis about what constitutes product/market fit with the minimum feature set. Thereby answering the questions, “Does this product/service as spec’d solve a problem or a need customers have?” Is our solution compelling enough that they want to buy it or use it today?  You know you have achieved product/market fit when you start getting orders (or users, eyeballs or whatever your criteria for success was in your business model.)

From Wikipedia, via Steve Blank’s book:

Customer Development is a risk reduction methodology for early stage startups. Its premise is that startups are not smaller versions of large companies. Instead these early stage ventures require new tools and techniques. Key tenets are: constant contact with potential customers, continual product iteration by shipping as early as possible for product refinement based on customer feedback. Cash burn is kept low until the onset of adoption at which point additional funding can be sought to refine the model and to scale a proven model. Customer Development has four steps; Customer Discovery focuses on understanding customer problems and needs, Customer Validation on developing a sales model that can be replicated, Customer Creation on creating and driving end user demand, and Company Building on transitioning the organization from one designed for learning and discovery to a well oiled machine engineered for execution. Customer Development works in parallel with Agile Development Methodologies to create products.

The purpose is to escape the traditional startup incubation cycle, where you transfer your grand vision to a slide deck, raise some money, spend nine months in a garage gestating your product Version Infinity, like a schizophrenic Kevin Costner following the voices in his head. Usually, that means you’re actually crazy.

Steve Blank explains this in The Leading Cause of Startup Death – Part 1: The Product Development Diagram.

The most important part of customer development? Getting out of the building. You must engage your customers directly and frequently. As Steve Blank says, “in a startup no facts exist inside the building, only opinions.” Facts live outside of the building.

I’m glossing over the most important point because I’m personally executing this inadequately. But the lessons we have learned from this technique are invaluable and more important, impossible to replicate by any other means. Once you have that insight, it becomes apparent which of your competitors has done their legwork.

More

If you have the time, listen to Eric Ries discuss these topics. I’d highly recommend listening to both, as different parts are fleshed out. The metrics-related aspects are really worth hearing.

· http://venturehacks.com/articles/lean-startup

· http://ecorner.stanford.edu/authorMaterialInfo.html?mid=2273

Steve Blank, video: http://www.ustream.tv/recorded/1450170?

Playing at business

Ries identifies the “startup dollhouse fallacy,” the belief that a startup is a large company in miniature. The pejorative connotation is important, because the temptation to play dress-up and measure your company’s worth by quantity of employees and processes is strong. Maybe this is the James Bond Villain fallacy. In the same piece, he introduces a counterintuitive idea, that for a startup, $30,000 in revenue can be better than $1m. Read it to find out how and why.

The true competitive edge of startups is that they’re more efficient at customer learning than their larger competitors and more flexible in their thinking (“What carries you up will also bring you down”). Entrepreneurs are prone to burying this advantage in an effort to emulate late-stage firms. All great large companies have divisions, we need divisions. All great companies have hierarchies, have office hours, resemble a Dilbert cartoon or Dunder-Mifflin-Wernham-Hogg. As Ries says, a “great startup is more than just a miniature version of a great large company. All of its process should be focused on innovating and learning.” Be taphophobic when it comes to this strength.

A startup can’t physically outwork a larger team, no matter how many nights the team stays up. Those late nights should be put to use making your concept a moving target, while your larger competitor puts icing on the version of your idea you’ve long left behind. This is the pivot, a crucial tactical maneuver for the lean startup practitioner.

I recall an angel investor once saying that, in 90% of the companies he funds, the model they end up making money with wasn’t mentioned in the business plan. What he learns from reading plans is all of the ways a company won’t be making money.

Minimum Viable Product (MVP) and the feedback loop

You’ve no doubt been told to “release early, release often, iterate quickly.” What are you iterating towards? How do you make decisions about what to change? You can derive these answers from a series of Minimal Viable Products incorporating concurrent customer feedback in the form of metrics and Customer Development.

According to Eric Ries,

…the minimum viable product is that version of a new product which allows a team to collect the maximum amount of validated learning about customers with the least effort

This is the canonical definition, and the link is his.

Inc Magazine illustrates the idea with an example of failing fast using a MVP:

After making a version of the tool that had the bare minimum of features … [he] began marketing … through his social network on LinkedIn and Twitter, directing interested buyers to a PayPal site. Although he was able to attract a steady trickle of paying customers at a level of $2 a month, he couldn’t find a way to tap into a surge of new customers or any willing to pay a higher price. No matter how many new features he added or how many A/B tests he ran, he couldn’t find a way to generate enough income to support a business. So he decided to pull the plug. “I found out that most programmers don’t like to test their software as intensely as I do,” he says. “But I don’t consider this a failure. I successfully discovered that this market was too small to support a business, and I gained this information very cheaply.”

Ries responds to the article, addressing a common misconception about the MVP concept:

…MVP is most powerful when it is used as part of an overall strategy of learning and discovery. And this is the most confusing, because MVP does not pay off under this strategy if we are attempting to build a minimal product. For that, release early, release often will suffice. But if our aspiration is to change the world, we need something more.

…

Building an MVP can help mitigate that risk. But it’s not enough. What if customers hate the MVP? Does that mean your product vision is fundamentally flawed, or just that your initial product sucks? There is no way to know for sure. That’s why entrepreneurship in a lean startup is really a series of MVP’s, each designed to answer a specific question (hypothesis). Being systematic about these hypotheses is what customer development is all about.

Building on the MVP is the customer feedback loop, combined with continuous integration on the development side. Rapid development emerges naturally from following John Boyd’s OODA loop.

A concern many entrepreneurs have is that a Version 0 release poorly represents their abilities. This fear can be couched as a concern for losing return customers, and is especially trying for those who went to school in Boston. They’re probably right, and it probably doesn’t matter (I can present evidence for this assertion if there’s interest).

We’re poor arbiters of what makes a product or a feature, much less predicting a successful one: look at Google, Twitter, PayPal, and Posterous. When you abandon “the extra features that make it hard for new customers to discover what’s really valuable about the new, simplified solution,” you leave the future open to your customer’s interpretation of your vision, letting air into the vacuum in which your product is likely built. Consider that, on exams, prompted responses generate better answers than completely open-ended questions. Let your customers question your hypotheses in the most direct manner possible.

For an example of a MVP in action, see Ash Maurya’s “From Minimum Viable Product to Landing Pages” or Tim Ferriss’ approach to selecting a book title, at ~2:26m in (the relevant Hacker News discussion).

What is a release?

The idea of a release disappears when you actively implement customer feedback, to the point where the notion of version releases becomes a quaint relic, like rotary phones, churning your own butter or real estate prices appreciating. In the traditional model, it takes a year to implement a shutdown menu in Vista, with Vista-worthy results.

Zynga runs hundreds of experiments a day. Ries’ IMVU pushed new releases fifty times a day. Flickr makes multiple daily changes. Google, the granddaddy of split testing, modifies even its home page continuously.

There is no reason to avoid this deployment strategy in “mission-critical” software.

Critically, the smaller the batch size, the easier it is to keep scope under control. You can quickly generate actual results and improve morale, while improving feedback between team members and customers. In software, there is no such thing as a “small change” (maybe the most important lesson a non-technical manager can learn), so cut the batch size beyond what seems reasonable.

How do you integrate the feedback loop into an agile development environment? Agile doesn’t tell you. Instead, study this deck. Skip to slide 24 or 36 if you want to avoid the intro, or review a summary, Customer Development Engineering, instead.

Accept that the problem and solution are unknown. Your challenge lies in discovering them. Drill down to the root cause with “5 Why’s.” Use unit testing to bulletproof what you do build so that you can spend your time making improvements instead of fixing defects. A unit of progress should be seen as “validated learning about customers” rather than “a line of working code.” Break your iteration cycles around this metric.

Metrics and finding product/market fit

The panacea of metrics. You can’t manage what you can’t measure.

The problem with metrics is that they don’t tell you why, only what (and, aside, possibly lead to neglecting brand value – if you can afford to care about that). You can optimize a blue button over a red one, but at what point does that matter? What if you’re optimizing like crazy in the wrong direction? There’s rarely a bonus for running into a wall the fastest.

Not to discount the importance of metrics when they’re appropriate. Some of the best insights on the subject come from Dave McClure (“Startup Metrics for Pirates”) and Eric Ries (“Vanity Metrics vs. Actionable Metrics”). A good summary of these ideas, wrapped around a case study, can be found on the Ben Yoskovitz’ blog. The issue, as with pushing sales and marketing, press, or premature scalaculation is when you hit the accelerator before finding product/market fit. It’s premature optimization, easily masked by a myopic reliance on measurement.

Solving problems, not building features

Lazily (or classically?), I’m borrowing from an email exchange with a colleague in the early stages of building his product. He’s a firm believer in rapid, scientific experimentation – that at a 90% failure rate, 10 attempts still yield a success. My response to this plan:

Outside of a technical one, what problem does it solve? What pain?

Every time I hear someone talk about a solution, I want to replace it with problem. I don’t know enough about the space to judge the idea on its merits, so I’m only questioning the way you proposed it: as an answer, not the question. This is where … “developing your customers the way you develop your product” comes in. The scientific method for product development is awesome, and you can use metrics to determine whether one feature or widget is better than another, but that only tells you the ‘what.’ The ‘why’ comes from customers. And it may be where the science metaphor breaks down, because the only absolute truth is what your customers will give you money for.

…

I completely agree with the idea of conducting frequent product development experiments. But my point is that you’re missing half of the equation, which is experimenting with customer development to discover product/market fit (something that I personally struggle with).

I would liken the one without the other to throwing darts (your product iterations) in a darkened room (your idea) at thirty dartboards (subtly differentiated markets). But you have lots of darts. Or you can take those same darts, your finite time and energy, and throw them at the one dartboard you’ve spent part of this time and energy on groping around the room to find.

…the problems you listed don’t describe the needs of customers, but rather the technical failings of existing systems. … Nivi at VentureHacks also collected some interesting, brief case studies. And this presentation is well worth a listen [I still agree with myself here, do check it out].

…

I think revenue – or more abstractly, a purpose beyond ideation – is the underdeveloped part of most startups. Generating ideas tends to be easy for entrepreneurs.

Conclusion

Cool story, bro.

Oh, totally aside, here’s an awesome story from Steve Blank about enterprise product pricing. It has to be profoundly nerdy that I find this more amusing than an infestation of koala bears. Incidentally, is Mitch Hedberg supposed to sound like Bob Dylan?

Please, don’t hesitate to point out factual or grammatical errors. All comments are welcome. You can email me directly: yury @ this domain.

One more thing

Vitaliy made a good point to me just now: I’d like to take my own advice here and ask anyone who would be interested in trying a beta release of our new desktop/webapp for our text message paging system, Recess, to please reach out to us. That’d be the Customer Development thing to do

Thanks for reading!

*Edit: Did I really have this in here for months? I was going to pun on tenants, but I’m embarrassed enough as it is.