Author: Ries, Eric
-
loc 107 - There is a mythmaking industry hard at work to sell us that story, but I have come to believe that the story is false, the product of selection bias and after-the-fact rationalization.
-
loc 119 - Entrepreneurship is a kind of management. No, you didn’t read that wrong. We have wildly divergent associations with these two words, entrepreneurship and management. Lately, it seems that one is cool, innovative, and exciting and the other is dull, serious, and bland. It is time to look past these preconceptions.
-
loc 127 - The engineering challenge before them is immense: creating virtual worlds, user-generated content, an online commerce engine, micropayments, and—last but not least—the three-dimensional avatar technology that can run on anyone’s PC. I’m in this second story, too. I’m a cofounder and chief technology officer of this company, which is called IMVU. At this point in our careers, my cofounders and I are determined to make new mistakes. We do everything wrong: instead of spending years perfecting our technology, we build a minimum viable product, an early product that is terrible, full of bugs and crash-your-computer-yes-really
-
loc 138 - Traditional business thinking says that this approach shouldn’t work, but it does, and you don’t have to take my word for it. As you’ll see throughout this book, the approach we pioneered at IMVU has become the basis for a new movement of entrepreneurs around the world.
-
loc 142 - Despite the volumes written on business strategy, the key attributes of business leaders, and ways to identify the next big thing, innovators still struggle to bring their ideas to life. This was the frustration that led us to try a radical new approach at IMVU, one characterized by an extremely fast cycle time, a focus on what customers want (without asking them), and a scientific approach to making decisions.
-
loc 145 - My Note: Way of change
-
loc 148 - Throughout my career, I kept having the experience of working incredibly hard on products that ultimately failed in the marketplace. At first, largely because of my background, I viewed these as technical problems that required technical solutions: better architecture, a better engineering process, better discipline, focus, or product vision.
-
loc 163 - studied lean manufacturing, a process that originated in Japan with the Toyota Production System, a completely new way of thinking about the manufacturing of physical goods.
-
loc 214 - Planning and forecasting are only accurate when based on a long, stable operating history and a relatively static environment. Startups have neither.
-
loc 219 - Most people think of process and management as boring and dull, whereas startups are dynamic and exciting. But what is actually exciting is to see startups succeed and change the world.
-
loc 275 - It taught the world the difference between value-creating activities and waste and showed how to build quality into products from the inside out.
-
loc 280 - A comprehensive theory of entrepreneurship should address all the functions of an early-stage venture: vision and concept, product development, marketing and sales, scaling up, partnerships and distribution, and structure and organizational design.
-
loc 297 - The Lean Startup asks people to start measuring their productivity differently. Because startups often accidentally build something nobody wants, it doesn’t matter much if they do it on time and on budget. The goal of a startup is to figure out the right thing to build—the thing customers want and will pay for—as quickly as possible.
-
loc 305 - An internal combustion automobile is powered by two important and very different feedback loops.
-
loc 309 - Startups have a similar engine that I call the engine of growth. The markets and customers for startups are diverse: a toy company, a consulting firm, and a manufacturing plant may not seem like they have much in common, but, as we’ll see, they operate with the same engine of growth.
-
loc 323 - Unfortunately, too many startup business plans look more like they are planning to launch a rocket ship than drive a car. They prescribe the steps to take and the results to expect in excruciating detail, and as in planning to launch a rocket, they are set up in such a way that even tiny errors in assumptions can lead to catastrophic outcomes.
-
loc 331 - The Lean Startup method, in contrast, is designed to teach you how to drive a startup. Instead of making complex plans that are based on a lot of assumptions, you can make constant adjustments with a steering wheel called the Build-Measure-Learn feedback loop. Through this process of steering, we can learn when and if it’s time to make a sharp turn called a pivot or whether we should persevere along our current path. Once we have an engine that’s revved up, the Lean Startup offers methods to scale and grow the business with maximum acceleration.
-
loc 405 - We often lose sight of the fact that a startup is not just about a product, a technological breakthrough, or even a brilliant idea. A startup is greater than the sum of its parts; it is an acutely human enterprise.
-
loc 421 - Most tools from general management are not designed to flourish in the harsh soil of extreme uncertainty in which startups thrive. The future is unpredictable, customers face a growing array of alternatives, and the pace of change is ever increasing. Yet most startups—in garages and enterprises alike—still are managed by using standard forecasts, product milestones, and detailed business plans.
-
loc 487 - Out of a hundred good ideas, you’ve got to sell your idea. So you build up a society of politicians and salespeople. When you have five hundred tests you’re running, then everybody’s ideas can run. And then you create entrepreneurs who run and learn and can retest and relearn as opposed to a society of politicians. So we’re trying to drive that throughout our organization, using examples which have nothing to do with high tech, like the website example. Every business today has a website. You don’t have to be high tech to use fast-cycle testing.”
-
loc 519 - Leadership requires creating conditions that enable employees to do the kinds of experimentation that entrepreneurship requires. For example, changes in TurboTax enabled the Intuit team to develop five hundred experiments per tax season. Before that, marketers with great ideas couldn’t have done those tests even if they’d wanted to, because they didn’t have a system in place through which to change the website rapidly. Intuit invested in systems that increased the speed at which tests could be built, deployed, and analyzed.
-
loc 535 - Unfortunately, “learning” is the oldest excuse in the book for a failure of execution. It’s what managers fall back on when they fail to achieve the results we promised. Entrepreneurs, under pressure to succeed, are wildly creative when it comes to demonstrating what we have learned. We can all tell a good story when our job, career, or reputation depends on
-
loc 563 - Like most communication networks, IM is thought to follow Metcalfe’s law: the value of a network as a whole is proportional to the square of the number of participants. In
-
loc 688 - In other words, which of our efforts are value-creating and which are wasteful? This question is at the heart of the lean manufacturing revolution; it is the first question any lean manufacturing adherent is trained to ask. Learning to see waste and then systematically eliminate it has allowed lean companies such as Toyota to dominate entire industries.
-
loc 693 - The answer came to me slowly over the subsequent years. Lean thinking defines value as providing benefit to the customer; anything else is waste.
-
loc 698 - Would it have been possible to learn the same things with less effort? Clearly, the answer is yes. For one thing, think of all the debate and prioritization of effort that went into features that customers would never discover. If we had shipped sooner, we could have avoided that waste. Also
-
loc 699 - Clearly, the answer is yes. For
-
loc 726 - The next few months are where the true story of IMVU begins, not with our brilliant assumptions and strategies and whiteboard gamesmanship but with the hard work of discovering what customers really wanted and adjusting our product and strategy to meet those desires. We adopted the view that our job was to find a synthesis between our vision and what customers would accept; it wasn’t to capitulate to what customers thought they wanted or to tell customers what they ought to want.
-
loc 735 - Aligned with a superior strategy, our product development efforts became magically more productive—not because we were working harder but because we were working smarter, aligned with our customers’ real needs. Positive changes in metrics became the quantitative validation that our learning was real. This was critically important because we could show our stakeholders—employees, investors, and ourselves—that we were making genuine progress, not deluding ourselves. It is also the right way to think about productivity in a startup: not in terms of how much stuff we are building but in terms of how much validated learning we’re getting for our efforts.
-
loc 748 - Even after ditching the IM add-on strategy, it still took months to understand why it hadn’t worked. After our pivot and many failed experiments, we finally figured out this insight: customers wanted to use IMVU to make new friends online.
-
loc 758 - Despite IMVU’s early success, our gross numbers were still pretty small. Unfortunately, because of the traditional way businesses are evaluated, this is a dangerous situation. The irony is that it is often easier to raise money or acquire other resources when you have zero revenue, zero customers, and zero traction than when you have a small amount. Zero invites imagination, but small numbers invite questions about whether large numbers will ever materialize.
-
loc 777 - Thus, we can mitigate the waste that happens because of the audacity of zero with validated learning. What we needed to demonstrate was that our product development efforts were leading us toward massive success without giving in to the temptation to fall back on vanity metrics and “success theater”—the work we do to make ourselves look successful.
-
loc 799 - The question is not “Can this product be built?” In the modern economy, almost any product that can be imagined can be built. The more pertinent questions are “Should this product be built?” and “Can we build a sustainable business around this set of products and services?” To answer those questions, we need a method for systematically breaking down a business plan into its component parts and testing each part empirically.
-
loc 815 - FROM ALCHEMY TO SCIENCE The Lean Startup methodology reconceives a startup’s efforts as experiments that test its strategy to see which parts are brilliant and which are crazy. A true experiment follows the scientific method. It begins with a clear hypothesis that makes predictions about what is supposed to happen. It then tests those predictions empirically. Just as scientific experimentation is informed by theory, startup experimentation is guided by the startup’s vision. The goal of every startup experiment is to discover how to build a sustainable business around that vision.
-
loc 834 - If Zappos had relied on existing market research or conducted a survey, it could have asked what customers thought they wanted. By building a product instead, albeit a simple one, the company learned much more: 1. It had more accurate data about customer demand because it was observing real customer behavior, not asking hypothetical questions. 2. It put itself in a position to interact with real customers and learn about their needs. For example, the business plan might call for discounted pricing, but how are customer perceptions of the product affected by the discounting strategy? 3. It allowed itself to be surprised when customers behaved in unexpected ways, revealing information Zappos might not have known to ask about. For example, what if customers returned the shoes?
-
loc 877 - The Lean Startup model offers a way to test these hypotheses rigorously, immediately, and thoroughly. Strategic planning takes months to complete; these experiments could begin immediately. By starting small, Caroline could prevent a tremendous amount of waste down the road without compromising her overall vision. Here’s what it might look like if Caroline were to treat her project as an experiment.
-
loc 883 - The value hypothesis tests whether a product or service really delivers value to customers once they are using it. What’s a good indicator that employees find donating their time valuable?
-
loc 889 - For the growth hypothesis, which tests how new customers will discover a product or service, we can do a similar analysis.
-
loc 899 - concierge minimum viable product
-
loc 903 - For example, if the growth model requires that a certain percentage of participants share their experiences with colleagues and encourage their participation, the degree to which that takes place can be tested even with a very small sample of people.
-
loc 919 - In the Lean Startup model, an experiment is more than just a theoretical inquiry; it is also a first product.
-
loc 930 - Do consumers recognize that they have the problem you are trying to solve? 2. If there was a solution, would they buy it? 3. Would they buy it from us? 4. Can we build a solution for that problem?”
-
loc 951 - The Kodak Gallery team built a simple prototype of the event album. It lacked many features—so many, in fact, that the team was reluctant to show it to customers. However, even at that early stage, allowing customers to use the prototype helped the team refute their hypotheses. First, creating an album was not as easy as the team had predicted; none of the early customers were able to create one. Further, customers complained that the early product version lacked essential features.
-
loc 967 - “Success is not delivering a feature; success is learning how to solve the customer’s problem.”
-
loc 1006 - My suggestion was drawn straight from the principles of this chapter: treat the CFPB as an experiment, identify the elements of the plan that are assumptions rather than facts, and figure out ways to test them. Using these insights, we could build a minimum viable product and have the agency up and running—on a micro scale—long before the official plan was set in motion. The number one
-
loc 1049 - Their challenge is to overcome the prevailing management thinking that puts its faith in well-researched plans. Remember, planning is a tool that only works in the presence of a long and stable operating history. And yet, do any of us feel that the world around us is getting more and more stable every day? Changing such a mind-set is hard but critical to startup success. My hope
-
loc 1049 - Their challenge is to overcome the prevailing management thinking that puts its faith in well-researched plans. Remember, planning is a tool that only works in the presence of a long and stable operating history. And yet, do any of us feel that the world around us is getting more and more stable every day? Changing such a mind-set is hard but critical to startup success. My hope is that this book will help managers and entrepreneurs make this change.
-
loc 1071 - As in lean manufacturing, learning where and when to invest energy results in saving time and money.
-
loc 1073 - I call the riskiest elements of a startup’s plan, the parts on which everything depends, leap-of-faith assumptions.
-
loc 1073 - I call the riskiest elements of a startup’s plan, the parts on which everything depends, leap-of-faith assumptions. The two most important assumptions are the value hypothesis and the growth hypothesis. These give rise to tuning variables that control a startup’s engine of growth. Each iteration of a startup is an attempt to rev this engine to see if it will turn. Once it is running, the process repeats, shifting into higher and higher gears.
-
loc 1077 - Once clear on these leap-of-faith assumptions, the first step is to enter the Build phase as quickly as possible with a minimum viable product (MVP). The MVP is that version of the product that enables a full turn of the Build-Measure-Learn loop with a minimum amount of effort and the least amount of development time.
-
loc 1083 - When we enter the Measure phase, the biggest challenge will be determining whether the product development efforts are leading to real progress. Remember, if we’re building something that nobody wants, it doesn’t much matter if we’re doing it on time and on budget.
-
loc 1085 - innovation accounting, a quantitative approach that allows us to see whether our engine-tuning efforts are bearing fruit. It also allows us to create learning milestones, which are an alternative to traditional business and product milestones.
-
loc 1090 - Finally, and most important, there’s the pivot. Upon completing the Build-Measure-Learn loop, we confront the most difficult question any entrepreneur faces: whether to pivot the original strategy or persevere. If we’ve discovered that one of our hypotheses is false, it is time to make a major change to a new strategic hypothesis.
-
loc 1107 - By all accounts, what impressed investors the most were two facts about Facebook’s early growth. The first fact was the raw amount of time Facebook’s active users spent on the site. More than half of the users came back to the site every single day.
-
loc 1123 - Many entrepreneurs are attempting to build the next Facebook, yet when they try to apply the lessons of Facebook and other famous startup success stories, they quickly get confused. Is the lesson of Facebook that startups should not charge customers money in the early days? Or is it that startups should never spend money on marketing? These questions cannot be answered in the abstract; there are an almost infinite number of counterexamples for any technique.
-
loc 1129 - Every business plan begins with a set of assumptions. It lays out a strategy that takes those assumptions as a given and proceeds to show how to achieve the company’s vision. Because the assumptions haven’t been proved to be true (they are assumptions, after all) and in fact are often erroneous, the goal of a startup’s early efforts should be to test them as quickly as possible.
-
loc 1133 - The first challenge for an entrepreneur is to build an organization that can test these assumptions systematically. The second challenge, as in all entrepreneurial situations, is to perform that rigorous testing without losing sight of the company’s overall vision.
-
loc 1134 - The second challenge, as in all entrepreneurial situations, is to perform that rigorous testing without losing sight of the company’s overall vision.
-
loc 1140 - They are called leaps of faith precisely because the success of the entire venture rests on them. If they are true, tremendous opportunity awaits. If they are false, the startup risks total failure.
-
loc 1146 - Previous technology X was used to win market Y because of attribute Z. We have a new technology X2 that will enable us to win market Y2 because we too have attribute Z.
-
loc 1146 - Previous technology X was used to win market Y because of attribute Z. We have a new technology X2 that will enable us to win market Y2 because we too have attribute Z. The problem with analogies like this is that they obscure the true leap of faith. That is their goal: to make the business seem less risky. They are used to persuade investors, employees, or partners to sign on. Most entrepreneurs would cringe to see their leap of faith written this way:
-
loc 1163 - There is nothing intrinsically wrong with basing strategy on comparisons to other companies and industries. In fact, that approach can help you discover assumptions that are not really leaps of faith.
-
loc 1172 - “Out of these analogs and antilogs come a series of unique, unanswered questions. Those are leaps of faith that I, as an entrepreneur, am taking if I go through with this business venture. They are going to make or break my business. In the iPod business, one of those leaps of faith was that people would pay for music.” Of course that leap of faith turned out to be correct.
-
loc 1186 - As we saw in the Facebook story, two leaps of faith stand above all others: the value creation hypothesis and the growth hypothesis. The first step in understanding a new product or service is to figure out if it is fundamentally value-creating or value-destroying. I use the language of economics in referring to value rather than profit,
-
loc 1205 - genchi gembutsu, which is one of the most important phrases in the lean manufacturing vocabulary. In English, it is usually translated as a directive to “go and see for yourself”
-
loc 1220 - It is common to think of selling to consumers as easier than selling to enterprises, because customers lack the complexity of multiple departments and different people playing different roles in the purchasing process.
-
loc 1228 - The results were impressive, boosting the Sienna’s market share dramatically. The 2004 model’s sales were 60 percent higher than those in 2003. Of course, a product like the Sienna is a classic sustaining innovation, the kind that the world’s best-managed established companies, such as Toyota, excel at.
-
loc 1241 - The first step in this process is to confirm that your leap-of-faith questions are based in reality, that the customer has a significant problem worth solving.8
-
loc 1255 - The goal of such early contact with customers is not to gain definitive answers. Instead, it is to clarify at a basic, coarse level that we understand our potential customer and what problems they have. With that understanding, we can craft a customer archetype, a brief document that seeks to humanize the proposed target customer.
-
loc 1264 - model. No amount of design can anticipate the many complexities of bringing a product to life in the real world.
-
loc 1265 - In fact, a new breed of designers is developing brand-new techniques under the banner of Lean User Experience (Lean UX). They recognize that the customer archetype is a hypothesis, not a fact. The customer profile should be considered provisional until the strategy has shown via validated learning that we can serve this type of customer in a sustainable way.
-
loc 1273 - Other entrepreneurs can fall victim to analysis paralysis, endlessly refining their plans. In this case, talking to customers, reading research reports, and whiteboard strategizing are all equally unhelpful.
-
loc 1277 - too much analysis is dangerous but none can lead to failure, how do entrepreneurs know when to stop analyzing and start building? The answer is a concept called the minimum viable product, the subject of Chapter
-
loc 1334 - As one manager put it to me recently, “I know for me, the MVP feels a little dangerous—in a good way—since I have always been such a perfectionist.” Minimum viable products range in complexity from
-
loc 1334 - As one manager put it to me recently, “I know for me, the MVP feels a little dangerous—in
-
loc 1334 - As one manager put it to me recently, “I know for me, the MVP feels a little dangerous—in a good way—since I have always been such a perfectionist.”
-
loc 1416 - That one early adopter got the concierge treatment. Instead of interacting with the FotT product via impersonal software, she got a personal visit each week from the CEO of the company. He and the VP of product would review what was on sale at her preferred grocery store and carefully select recipes on the basis of her preferences, going so far as to learn her favorite recipes for items she regularly cooked for her family.
-
loc 1433 - Before long, they had built a substantial service offering, first in the Austin area and eventually nationwide. But along the way, their product development team was always focused on scaling something that was working rather than trying to invent something that might work in the future. As a result, their development efforts involved far less waste than is typical for a venture of this kind.
-
loc 1438 - time. In a concierge MVP, this personalized service is not the product but a learning activity designed to test the leap-of-faith assumptions in the company’s growth model.
-
loc 1480 - “As we refined the product, we would bring in six to twelve people weekly to react to mockups, prototypes, or simulations that we were working on. It was a mix of existing users and people who never saw the product before. We had our engineers join for many of these sessions, both so that they could make modifications in real time, but also so we could all experience the pain of a user not knowing what to do.”
-
loc 1492 - Instead, they used Wizard of Oz testing to fake it. In a Wizard of Oz test, customers believe they are interacting with the actual product, but behind the scenes human beings are doing the work. Like the concierge MVP, this approach is incredibly inefficient.
-
loc 1507 - Most modern business and engineering philosophies focus on producing high-quality experiences for customers as a primary principle; it is the foundation of Six Sigma, lean manufacturing, design thinking, extreme programming, and the software craftsmanship movement.
-
loc 1514 - Even a “low-quality” MVP can act in service of building a great high-quality product. Yes, MVPs sometimes are perceived as low-quality by customers. If so, we should use this as an opportunity to learn what attributes customers care about. This is infinitely better than mere speculation or whiteboard strategizing, because it provides a solid empirical foundation on which to build future products.
-
loc 1536 - Customers don’t care how much time something takes to build. They care only if it serves their needs. Our customers preferred the quick teleportation feature because it allowed them to get where they wanted to go as fast as possible. In retrospect, this makes sense. Wouldn’t we all like to get wherever we’re going in an instant? No lines, no hours on a plane or sitting on the tarmac, no connections, no cabs or subways. Beam me up, Scotty. Our expensive “real-world” approach was beaten handily by a cool fantasy-world feature that cost much less but that our customers preferred. So which version
-
loc 1545 - Thus, the Lean Startup method is not opposed to building high-quality products, but only in service of the goal of winning over customers. We must be willing to set aside our traditional professional standards to start the process of validated learning as soon as possible.
-
loc 1551 - As you consider building your own minimum viable product, let this simple rule suffice: remove any feature, process, or effort that does not contribute directly to the learning you seek.
-
loc 1553 - Building an MVP is not without risks, both real and imagined. Both can derail a startup effort unless they are understood ahead of time. The most common speed bumps are legal issues, fears about competitors, branding risks, and the impact on morale.
-
loc 1577 - Many startups plan to invest in building a great brand, and an MVP can seem like a dangerous branding risk. Similarly, entrepreneurs in existing organizations often are constrained by the fear of damaging the parent company’s established brand. In either of these cases, there is an easy solution: launch the MVP under a different brand name.
-
loc 1586 - Visionaries are especially afraid of a false negative: that customers will reject a flawed MVP that is too small or too limited.
-
loc 1592 - The solution to this dilemma is a commitment to iteration. You have to commit to a locked-in agreement—ahead of time—that no matter what comes of testing the MVP, you will not give up hope. Successful entrepreneurs do not give up at the first sign of trouble, nor do they persevere the plane right into the ground. Instead, they possess a unique combination of perseverance and flexibility. The MVP is just the first step on a journey of learning.
-
loc 1605 - The solution to this problem resides at the heart of the Lean Startup model. We all need a disciplined, systematic approach to figuring out if we’re making progress and discovering if we’re actually achieving validated learning. I call this system innovation accounting, an alternative to traditional accounting designed specifically for startups.
-
loc 1612 - A startup’s job is to (1) rigorously measure where it is right now, confronting the hard truths that assessment reveals, and then (2) devise experiments to learn how to move the real numbers closer to the ideal reflected in the business plan.
-
loc 1635 - This is the kind of storytelling that takes place at most startup board meetings. Most milestones are built the same way: hit a certain product milestone, maybe talk to a few customers, and see if the numbers go up. Unfortunately, this is not a good indicator of whether a startup is making progress. How do we know that the changes we’ve made are related to the results we’re seeing? More important, how do we know that we are drawing the right lessons from those changes? To answer these kinds of questions, startups have a strong need for a new kind of accounting geared specifically to disruptive innovation. That’s what innovation accounting is.
-
loc 1641 - Innovation accounting enables startups to prove objectively that they are learning how to grow a sustainable business. Innovation accounting begins by turning the leap-of-faith assumptions discussed in Chapter 5 into a quantitative financial model. Every business plan has some kind of model associated with it, even if it’s written on the back of a napkin. That model provides assumptions about what the business will look like at a successful point in the future.
-
loc 1659 - HOW INNOVATION ACCOUNTING WORKS—THREE LEARNING MILESTONES Innovation accounting works in three steps: first, use a minimum viable product to establish real data on where the company is right now. Without a clear-eyed picture of your current status—no matter how far from the goal you may be—you cannot begin to track your progress. Second, startups must attempt to tune the engine from the baseline toward the ideal. This may take many attempts. After the startup has made all the micro changes and product optimizations it can to move its baseline toward the ideal, the company reaches a decision point. That is the third step: pivot or persevere.
-
loc 1668 - The sign of a successful pivot is that these engine-tuning activities are more productive after the pivot than before.
-
loc 1674 - A smoke test measures only one thing: whether customers are interested in trying a product. By itself, this is insufficient to validate an entire growth model. Nonetheless, it can be very useful to get feedback on this assumption before committing more money and other resources to the product.
-
loc 1680 - When one is choosing among the many assumptions in a business plan, it makes sense to test the riskiest assumptions first. If you can’t find a way to mitigate these risks toward the ideal that is required for a sustainable business, there is no point in testing the others. For example, a media business that is selling advertising has
-
loc 1711 - We tracked the “funnel metrics” behaviors that were critical to our engine of growth: customer registration, the download of our application, trial, repeat usage, and purchase.
-
loc 1716 - Five dollars bought us a hundred clicks—every day. From a marketing point of view this was not very significant, but for learning it was priceless. Every single day we were able to measure our product’s performance with a brand new set of customers. Also, each time we revised the product, we got a brand new report card on how we were doing the very next
-
loc 1716 - Five dollars bought us a hundred clicks—every day. From a marketing point of view this was not very significant, but for learning it was priceless. Every single day we were able to measure our product’s performance with a brand new set of customers. Also, each time we revised the product, we got a
-
loc 1716 - Five dollars bought us a hundred clicks—every day. From a marketing point of view this was not very significant, but for learning it was priceless. Every single day we were able to measure our product’s performance with a brand new set of customers. Also, each time we revised the product, we got a brand new report card on how we were doing the very next day.
-
loc 1723 - Day in and day out we were performing random trials. Each day was a new experiment. Each day’s customers were independent of those of the day before. Most important, even though our gross numbers were growing, it became clear that our funnel metrics were not changing.
-
loc 1730 - To read the graph, you need to understand something called cohort analysis. This is one of the most important tools of startup analytics. Although it sounds complex, it is based on a simple premise.
-
loc 1731 - Instead of looking at cumulative totals or gross numbers such as total revenue and total number of customers, one looks at the performance of each group of customers that comes into contact with the product independently. Each group is called a cohort. The graph shows the conversion rates to IMVU of new customers who joined in each indicated month. Each conversion rate shows the percentage of customer who registered in that month who subsequently went on to take the indicated action.
-
loc 1738 - Customer flows govern the interaction of customers with a company’s products. They allow us to understand a business quantitatively and have much more predictive power than do traditional gross metrics.
-
loc 1752 - Before this failure, in the company’s earliest days, it was easy
-
loc 1752 - Before this failure, in the company’s earliest days, it was easy to talk to potential customers and come away convinced we were on the right track.
-
loc 1765 - This pattern would repeat time and again, from the days when we were making less than a thousand dollars in revenue per month all the way up to the time we were making millions. In fact, this is the sign of a successful pivot: the new experiments you run are overall more productive than the experiments you were running before.
-
loc 1767 - This is the pattern: poor quantitative results force us to declare failure and create the motivation, context, and space for more qualitative research. These investigations produce new ideas—new hypotheses—to be tested, leading to a possible pivot. Each pivot unlocks new opportunities for further experimentation, and the cycle repeats. Each time we repeat this simple rhythm: establish the baseline, tune the engine, and make a decision to pivot or persevere.
-
loc 1776 - However, these tools for product improvement do not work the same way for startups. If you are building the wrong thing, optimizing the product or its marketing will not yield significant results. A startup has to measure progress against a high bar: evidence that a sustainable business can be built around its products or services.
-
loc 1782 - When I meet with those teams, there are always improvements to be made and I recommend them, but invariably the real problem is not a lack of development talent, energy, or effort. Cycle after cycle, the team is working hard, but the business is not seeing results. Managers trained in a traditional model draw the logical conclusion: our team is not working hard, not working effectively, or not working efficiently.
-
loc 1796 - Learning milestones prevent this negative spiral by emphasizing a more likely possibility: the company is executing—with discipline!—a plan that does not make sense. The innovation accounting framework makes it clear when the company is stuck and needs to change direction.
-
loc 1807 - Energy invested in success theater is energy that could have been used to help build a sustainable business. I call the traditional numbers used to judge startups “vanity metrics,” and innovation accounting requires us to avoid the temptation to use them.
-
loc 1809 - VANITY METRICS: A WORD OF CAUTION
-
loc 1820 - But think back to the same data presented in a cohort style. IMVU is adding new customers, but it is not improving the yield on each new group. The engine is turning, but the efforts to tune the engine are not bearing much fruit. From the traditional graph alone, you cannot tell whether IMVU is on pace to build a sustainable business; you certainly can’t tell anything about the efficacy of the entrepreneurial team behind it.
-
loc 1824 - Innovation accounting will not work if a startup is being misled by these kinds of vanity metrics: gross number of customers and so on. The alternative is the kind of metrics we use to judge our business and our learning milestones, what I call actionable metrics.
-
loc 1865 - Instead of writing a specification for a new feature that described it in technical terms, Farb would write a story that described the feature from the point of view of the customer.
-
loc 1879 - However, I sensed that Farb and his team were left with lingering doubts about the company’s overall progress. Was the increase in their numbers actually caused by their development efforts? Or could it be due to other factors, such as mentions of Grockit in the press? When I met the team, I asked them this simple question: How do you know that the prioritization decisions that Farb is making actually make sense? Their answer: “That’s not our department. Farb makes the decisions; we execute them.”
-
loc 1898 - Agile is an efficient system of development from the point of view of the developers. It allows them to stay focused on creating features and technical designs. An attempt to introduce the need to learn into that process could undermine productivity.
-
loc 1918 - Compared to a lot of startups, the Grockit team had a huge advantage: they were tremendously disciplined. A disciplined team may apply the wrong methodology but can shift gears quickly once it discovers its error. Most important, a disciplined team can experiment with its own working style and draw meaningful conclusions.
-
loc 1921 - Grockit changed the metrics they used to evaluate success in two ways. Instead of looking at gross metrics, Grockit switched to cohort-based metrics, and instead of looking for cause-and-effect relationships after the fact, Grockit would launch each
-
loc 1921 - Grockit changed the metrics they used to evaluate success in two ways. Instead of looking at gross metrics, Grockit switched to cohort-based metrics, and instead of looking for cause-and-effect relationships after the fact, Grockit would launch each new feature as a true split-test experiment.
-
loc 1926 - If you wanted to test a catalog design, you could send a new version of it to 50 percent of the customers and send the old standard catalog to the other 50 percent. To assure a scientific result, both catalogs would contain identical products; the only difference would be the changes to the design. To figure out if the new design was effective, all you would have to do was keep track of the sales figures for both groups of customers. (This technique is sometimes called A/B testing after the practice of assigning letter names to each variation.)
-
loc 1941 - The questioning inspired the team to seek a deeper understanding of what customers really wanted. They brainstormed new ideas for product experiments that might have more impact. In fact, many of these ideas were not new. They had simply been overlooked because the company was focused on building social tools. As a result, Grockit tested an intensive solo-studying mode, complete with quests and gamelike levels, so that students could have the choice of studying by themselves or with others.
-
loc 1951 - Validated was defined as “knowing whether the story was a good idea to have been done in the first place.” This
-
loc 1971 - Engineering may finish a big batch of work, followed by extensive testing and validation. As engineers look for ways to increase their productivity, they start to realize that if they include the validation exercise from the beginning, the whole team can be more productive.
-
loc 1975 - Under the old system, he or she would just build it and find out later what it was for. In the new system, that behavior is clearly counterproductive: without a clear hypothesis, how can a story ever be validated?
-
loc 1980 - A solid process lays the foundation for a healthy culture, one where ideas are evaluated by merit and not by job title.
-
loc 1984 - lazy registration, to see if it was worth the heavy investment they were making in ongoing support. They were confident in this feature because lazy registration is considered one of the design best practices for online services. In this system, customers do not have to register for the service up front. Instead, they immediately begin using the service and are asked to register only after they have had a chance to experience the service’s benefit.
-
loc 1998 - To their surprise, this cohort’s behavior was exactly the same as that of the lazy registration group: they had the same rate of registration, activation, and subsequent retention. In other words, the extra effort of lazy registration was a complete waste even though it was considered an industry best practice.
-
loc 2013 - These examples from Grockit demonstrate each of the three A’s of metrics: actionable, accessible, and auditable.
-
loc 2022 - Vanity metrics wreak havoc because they prey on a weakness of the human mind. In
-
loc 2031 - Actionable metrics are the antidote to this problem. When cause and effect is clearly understood, people are better able to learn from their actions. Human beings are innately talented learners when given a clear and objective assessment.
-
loc 2052 - Every day their system automatically generated a document containing the latest data for every single one of their split-test experiments and other leap-of-faith metrics.
-
loc 2056 - Instead of housing the analytics or data in a separate system, our reporting data and its infrastructure were considered part of the product itself and were owned by the product development team. The reports were available on our website, accessible to anyone with an employee account.
-
loc 2060 - When people needed evidence to support something they had learned, they would bring a printout with them to the relevant meeting, confident that everyone they showed it to would understand its meaning.
-
loc 2065 - The employees at IMVU would brandish one-page reports to demonstrate what they had learned to settle arguments, but the process often wasn’t so smooth. Most of the time, when a manager, developer, or team was confronted with results that would kill a pet project, the loser of the argument would challenge the veracity of the data.
-
loc 2073 - The solution? First, remember that “Metrics are people, too.” We need to be able to test the data by hand, in the messy real world, by talking to customers. This is the only way to be able to check if the reports contain true facts.
-
loc 2077 - Second, those building reports must make sure the mechanisms that generate the reports are not too complex.
-
loc 2082 - the story is always structured the same way. First, we see the plucky protagonist having an epiphany, hatching a great new idea. We learn about his or her character and personality, how he or she came to be in the right place at the right time, and how he or she took the dramatic leap to start a business. Then the photo montage begins.
-
loc 2089 - Only 5 percent of entrepreneurship is the big idea, the business model, the whiteboard strategizing, and the splitting up of the spoils. The other 95 percent is the gritty work that is measured by innovation accounting: product prioritization decisions, deciding which customers to target or listen to, and having the courage to subject a grand vision to constant testing and feedback.
-
loc 2102 - This is not true. There is no way to remove the human element—vision, intuition, judgment—from the practice of entrepreneurship, nor would that be desirable.
-
loc 2105 - Companies that cannot bring themselves to pivot to a new direction on the basis of feedback from the marketplace can get stuck in the land of the living dead, neither growing enough nor dying, consuming resources and commitment from employees and other stakeholders but not moving ahead.
-
loc 2110 - The heart of the scientific method is the realization that although human judgment may be faulty, we can improve our judgment by subjecting our theories to repeated testing.
-
loc 2158 - The goal of creating learning milestones is not to make the decision easy; it is to make sure that there is relevant data in the room when it comes time to decide.
-
loc 2161 - In Silicon Valley, we call this experience getting stuck in the land of the living dead. It happens when a company has achieved a modicum of success—just enough to stay alive—but is not living up to the expectations of its founders and investors. Such companies are a terrible drain of human energy. Out of loyalty, the employees and founders don’t want to give in; they feel that
-
loc 2161 - In Silicon Valley, we call this experience getting stuck in the land of the living dead. It happens when a company has achieved a modicum of success—just enough to stay alive—but is not living up to the expectations of its founders and investors. Such companies are a terrible drain of human energy. Out of loyalty, the employees and founders don’t want to give in; they feel that success might be just around the corner.
-
loc 2169 - identified his leap-of-faith questions explicitly at
-
loc 2169 - identified his leap-of-faith questions explicitly
-
loc 2217 - In this pivot, David did what I call a customer segment pivot, keeping the functionality of the product the same but changing the audience focus. He
-
loc 2219 - In other words, David went from being a business-to-consumer (B2C) company to being a business-to-business (B2B) company.
-
loc 2234 - David decided to reduce staff and pivot again, this time attempting what I call a platform pivot. Instead of selling an application to one customer at a time, David envisioned a new growth model inspired by Google’s AdWords platform.
-
loc 2279 - If the cuts result in a slowdown to this feedback loop, all they have accomplished is to help the startup go out of business more slowly.
-
loc 2332 - In my experience, less than a few weeks between meetings is too often and more than a few months is too infrequent. However, each startup needs to find its own pace.
-
loc 2461 - Pivots come in different flavors. The word pivot sometimes is used incorrectly as a synonym for change. A pivot is a special kind of change designed to test a new fundamental hypothesis about the product, business model, and engine of growth.
-
loc 2463 - Zoom-in Pivot In this case, what previously was considered a single feature in a product becomes the whole product.
-
loc 2466 - Zoom-out Pivot In the reverse situation, sometimes a single feature is insufficient to support a whole product. In this type of pivot, what was considered the whole product becomes a single feature of a much larger product.
-
loc 2495 - As we’ll see in Chapter 10, there are three primary engines of growth that power startups: the
-
loc 2495 - As we’ll see in Chapter 10, there are three primary engines of growth that power startups: the viral, sticky, and paid growth models. In
-
loc 2499 - In traditional sales terminology, the mechanism by which a company delivers its product to customers is called the sales channel or distribution channel.
-
loc 2502 - A channel pivot is a recognition that the same basic solution could be delivered through a different channel with greater effectiveness.
-
loc 2508 - Technology pivots are much more common in established businesses. In other words, they are a sustaining innovation, an incremental improvement designed to appeal to and retain an existing customer base.
-
loc 2517 - What is generally less well known are the pivots that were required to discover those strategies. Companies have a strong incentive to align their PR stories around the heroic founder and make it seem that their success was the inevitable result of a good idea.
-
loc 2520 - It’s extremely difficult to know if the analogy has been drawn properly. Have we copied the essential features or just superficial ones? Will what worked in that industry work in ours? Will what has worked in the past work today? A pivot is better understood as a new strategic hypothesis that will require a new minimum viable product to test.
-
loc 2524 - Those familiar with the technology life cycle ideas of theorists such as Geoffrey Moore know certain later-stage pivots by the names he has given them: the Chasm, the Tornado, the Bowling Alley.
-
loc 2528 - Modern managers cannot have escaped the deluge of recent books calling on them to adapt, change, reinvent, or upend their existing businesses. Many of the works in this category are long on exhortations and short on specifics.
-
loc 2531 - It is what makes the companies that follow Lean Startup resilient in the face of mistakes: if we take a wrong turn, we have the tools we need to realize it and the agility to find another path.
-
loc 2538 - We have learned to steer when moving slowly. Now we must learn to race. Laying a solid foundation is only the first step toward our real destination: acceleration.
-
loc 2543 - Product releases incur overhead, and so from an efficiency point of view, releasing often leaves less time to devote to building the product. However, waiting too long to release can lead to the ultimate waste: making something that nobody wants.
-
loc 2546 - Spend too much and you waste precious time that could have been spent learning. Spend too little and you may fail to take advantage of early success and cede market leadership to a fast follower.
-
loc 2551 - Startups need organizational structures that combat the extreme uncertainty that is a startup’s chief enemy.
-
loc 2553 - The critical first question for any lean transformation is: which activities create value and which are a form of waste?
-
loc 2556 - Recall from Chapter 3 that value in a startup is not the creation of stuff, but rather validated learning about how to build a sustainable business. What
-
loc 2560 - In Part Three, we will develop techniques that allow Lean Startups to grow without sacrificing the speed and agility that are the lifeblood of every startup.
-
loc 2564 - Just as lean manufacturing has pursued a just-in-time approach to building products, reducing the need for in-process inventory, Lean Startups practice just-in-time scalability, conducting product experiments without making massive up-front investments in planning and design.
-
loc 2572 - Chapter 11 shows how to build an adaptive organization by investing in the right amount of process to keep teams nimble as they grow. We will see how techniques from the tool kit of lean manufacturing, such as the Five Whys, help startup teams grow without becoming bureaucratic or dysfunctional.
-
loc 2577 - In fact, we’ll see that an advantage of a successful startup’s rapid growth is that the company can keep its entrepreneurial DNA even as it matures.
-
loc 2582 - I have included an epilogue called “Waste Not” in which I consider some of the broader implications of the success of the Lean Startup movement, place it in historical context (including cautionary lessons from past movements), and make suggestions for its future direction.
-
loc 2592 - The father won the race, and not just because he is an adult. It happened because the one envelope at a time approach is a faster way of getting the job done even though it seems inefficient. This has been confirmed in many studies, including one that was recorded on video.
-
loc 2592 - The father won the race, and not just because he is an adult. It happened because the one envelope at a time approach is a faster way of getting the job done even though it seems inefficient. This has been confirmed in many studies, including one that was recorded on video.1 The one envelope at a time approach is called “single-piece flow”
-
loc 2596 - the “batch size” refers to how much work moves from one stage to the next at a time.
-
loc 2602 - It seems more efficient to repeat the same task over and over, in part because we expect that we will get better at this simple task the more we do it.
-
loc 2604 - Even if the amount of time that each process took was exactly the same, the small batch production approach still would be superior, and for even more counterintuitive reasons.
-
loc 2609 - All these issues are visible in a process as simple as stuffing envelopes, but they are of real and much greater consequence in the work of every company, large or small.
-
loc 2613 - Lean manufacturers discovered the benefits of small batches decades ago. In the post–World War II economy, Japanese carmakers such as Toyota could not compete with huge American factories that used the latest mass production techniques.
-
loc 2622 - Instead of buying large specialized machines that could produce thousands of parts at a time, Toyota used smaller general-purpose machines that could produce a wide variety of parts in small batches.
-
loc 2626 - Shigeo Shingo created the concept of SMED (Single-Minute Exchange of Die) in order to enable a smaller batch size of work in early Toyota factories.
-
loc 2632 - Thus, Toyota could serve its smaller, more fragmented markets and still compete with the mass producers. Over time, that capability allowed Toyota to move successfully into larger and larger markets until it became the world’s largest automaker in 2008.
-
loc 2634 - The biggest advantage of working in small batches is that quality problems can be identified much sooner.
-
loc 2638 - andon cord can interrupt this careful flow as the line is halted repeatedly. However, the benefits of finding and fixing problems faster outweigh this cost.
-
loc 2644 - Toyota discovered that small batches made their factories more efficient. In contrast, in the Lean Startup the goal is not to produce more stuff efficiently. It is to—as quickly as possible—learn how to build a sustainable business. Think back to the example
-
loc 2664 - At IMVU, we attempted to design, develop, and ship our new features one at a time, taking advantage of the power of small batches. Here’s what it looked like.
-
loc 2665 - Instead of working in separate departments, engineers and designers would work together side by side on one feature at a time. Whenever
-
loc 2668 - For tiny changes, the whole process might be repeated several times per day. In fact, in the aggregate, IMVU makes about fifty changes to its product (on average) every single day.
-
loc 2674 - Analogously to the Toyota andon cord, IMVU used an elaborate set of defense mechanisms that prevented engineers from accidentally breaking something important.
-
loc 2678 - Going back to our business-to-hobby example of the missing checkout button, let’s make the problem a little more interesting.
-
loc 2681 - This class of problems is hard to detect solely with automation but is still catastrophic from a business point of view. At IMVU, our immune system is programmed to detect these business consequences and automatically invoke our equivalent of the andon cord.
-
loc 2690 - At IMVU, we called this continuous deployment, and even in the fast-moving world of software development it is still considered controversial.
-
loc 2770 - Small batches pose a challenge to managers steeped in traditional notions of productivity and progress, because they believe that functional specialization is more efficient for expert workers.
-
loc 2775 - From the point of view of individual efficiency, working in large batches makes sense. It also has other benefits: it promotes skill building, makes it easier to hold individual contributors accountable, and, most important, allows experts to work without interruption. At least that’s the theory. Unfortunately, reality seldom works out that way.
-
loc 2787 - When I suggested that he try switching the work process from large-batch to single-piece flow, he refused—because that would be inefficient! So strong is the instinct to work in large batches, that even when a large-batch system is malfunctioning, we have a tendency to blame ourselves.
-
loc 2789 - Large batches tend to grow over time. Because moving the batch forward often results in additional work, rework, delays, and interruptions, everyone has an incentive to do work in ever-larger batches, trying to minimize this overhead. This is called the large-batch death spiral because, unlike in manufacturing, there are no physical limits on the maximum size of a batch.
-
loc 2806 - Delivering smaller batches every four hours reduces the total workload for the pharmacy and ensures that the right meds are at the right place when needed.
-
loc 2818 - In traditional mass production, the way to avoid stockouts—not having the product the customer wants—is to keep a large inventory of spares just in case.
-
loc 2828 - The ideal goal is to achieve small batches all the way down to single-piece flow along the entire supply chain. Each step in the line pulls the parts it needs from the previous step. This is the famous Toyota just-in-time production method.
-
loc 2831 - [called work-in-progress (WIP) inventory]
-
loc 2833 - Startups struggle to see their work-in-progress inventory. When factories have excess WIP, it literally piles up on the factory floor. Because most startup work is intangible, it’s not nearly as visible. For example, all the work that goes into designing the minimum viable product is—until the moment that product is shipped—just WIP inventory. Incomplete designs, not-yet-validated assumptions, and most business plans are WIP.
-
loc 2836 - Almost every Lean Startup technique we’ve discussed so far works its magic in two ways: by converting push methods to pull and reducing batch size. Both have the net effect of reducing WIP.
-
loc 2841 - This assumes that customers could tell us what products to build and that this would act as the pull signal to product development to make them.
-
loc 2845 - development process in a Lean Startup is that it is responding to pull requests in the form of experiments that need to be run.
-
loc 2846 - As soon as we formulate a hypothesis that we want to test, the product development team should be engineered to design and run this experiment as quickly as possible, using the smallest batch size that will get the job done.
-
loc 2888 - The Toyota Production System is probably the most advanced system of management in the world, but even more impressive is the fact that Toyota has built the most advanced learning organization in history. It has demonstrated an ability to unleash the creativity of its employees, achieve consistent growth, and produce innovative new products relentlessly over the course of nearly a century.
-
loc 2924 - Sustainable growth is characterized by one simple rule: New customers come from the actions of past customers.
-
loc 2927 - Word of mouth. Embedded in most products is a natural level of growth that is caused by satisfied customers’ enthusiasm for the product. For example, when I bought my first TiVo DVR, I couldn’t stop telling my friends and family about it. Pretty soon, my entire family was using one.
-
loc 2930 - As a side effect of product usage.
-
loc 2935 - As long as the cost of acquiring a new customer (the so-called marginal cost) is less than the revenue that customer generates (the marginal revenue), the excess (the marginal profit) can be used to acquire more customers. The more marginal profit, the faster the growth.
-
loc 2945 - fact, one of the most expensive forms of potential waste for a startup is spending time arguing about how to prioritize new development once it has a product on the market. At any time, the company could invest its energy in finding new customers, servicing existing customers better, improving overall quality, or driving down costs. In my experience, the discussions about these kinds of priority decisions can consume a substantial fraction of the company’s time.
-
loc 2952 - Startups have to focus on the big experiments that lead to validated learning. The engines of growth framework helps them stay focused on the metrics that matter.
-
loc 2968 - Therefore, companies using the sticky engine of growth track their attrition rate or churn rate very carefully. The churn rate is defined as the fraction of customers in any period who fail to remain engaged
-
loc 2968 - Therefore, companies using the sticky engine of growth track their attrition rate or churn rate very carefully. The churn rate is defined as the fraction of customers in any period who fail to remain engaged with the company’s product.
-
loc 2975 - Even the actionable metrics they were using, such as the activation rate and revenue per customer, weren’t very helpful because in the sticky engine of growth, these variables have little impact on growth.
-
loc 2987 - Alternatively, the company could do something more direct such as messaging them about limited-time sales or special offers. Either way, its focus needs to be on improving customer retention.
-
loc 2994 - Growth happens automatically as a side effect of customers using the product. Viruses are not optional.
-
loc 2999 - They added to the bottom of every single e-mail the message “P.S. Get your free e-mail at Hotmail” along with a clickable link. Within weeks, that small product change produced massive results.
-
loc 3005 - Every sales pitch is an opportunity not only to sell Tupperware products but also to persuade other customers to become Tupperware representatives.
-
loc 3008 - It is called the viral loop, and its speed is determined by a single mathematical term called the viral coefficient.
-
loc 3012 - For a product with a viral coefficient of 0.1, one in every ten customers will recruit one of his or her friends.
-
loc 3015 - By contrast, a viral loop with a coefficient that is greater than 1.0 will grow exponentially, because each person who signs up will bring, on average, more than one other person with him or her.
-
loc 3020 - A consequence of this is that many viral products do not charge customers directly but rely on indirect sources of revenue such as advertising.
-
loc 3034 - Imagine another pair of businesses. The first makes $1 on each customer it signs up; the second makes $100,000 from each customer it signs up. To predict which company will grow faster, you need to know only one additional thing: how much it costs to sign up a new customer.
-
loc 3053 - Once variable costs are deducted, this usually is called the customer lifetime value (LTV). This revenue can be invested in growth by buying advertising.
-
loc 3054 - Suppose an advertisement costs $100 and causes fifty new customers to sign up for the service. This ad has a cost per acquisition (CPA) of $2.00. In this example, if the product has an LTV that is greater than $2, the product will grow.
-
loc 3055 - This ad has a cost per acquisition (CPA) of $2.00. In
-
loc 3077 - Wealthy consumers cost more to reach because they tend to become more profitable customers.
-
loc 3088 - Also, there is no reason why a product cannot have both high margins and high retention. However, in my experience, successful startups usually focus on just one engine of growth, specializing in everything that is required to make it work.
-
loc 3099 - Conversely, in a terrible market, you can have the best product in the world and an absolutely killer team, and it doesn’t matter—you’re going to fail.
-
loc 3101 - When you see a startup that has found a fit with a large market, it’s exhilarating. It leaves no room for doubt. It is Ford’s Model T flying out of the factory as fast as it could be made, Facebook sweeping college campuses practically overnight, or Lotus taking the business world by storm, selling $54 million worth of Lotus 1-2-3 in its first year of operation.
-
loc 3104 - Startups occasionally ask me to help them evaluate whether they have achieved product/market fit. It’s easy to answer: if you are asking, you’re not there yet.
-
loc 3130 - Even with no insight into these two companies’ gross numbers, we can tell that company A is making real progress whereas company B is stuck in the mud. This is true even though company B is growing faster than company A right now.
-
loc 3169 - Having no system at all was not an option for IMVU and is not an option for you. There are so many ways for a startup to fail. I’ve lived through the overarchitecture failure, in which attempting to prevent all the various kinds of problems that could occur wound up delaying the company from putting out any product. I’ve seen companies fail the other way from the so-called Friendster effect, suffering a high-profile technical failure just when customer adoption is going wild.
-
loc 3204 - This is one of the most important discoveries of the lean manufacturing movement: you cannot trade quality for time.
-
loc 3218 - Therefore, shortcuts taken in product quality, design, or infrastructure today may wind up slowing a company down tomorrow. You can see this paradox in action at IMVU.
-
loc 3234 - Figuring this out from a top-down perspective is challenging, because it requires estimating two completely unknown quantities: how much it will cost to build an unknown program against an unknown benefit you might reap.
-
loc 3255 - Repeating “why” five times, like this, can help uncover the root problem and correct it. If this procedure were not carried through, one might simply replace the fuse or the pump shaft.
-
loc 3296 - The Five Whys ties the rate of progress to learning, not just execution. Startup teams should go through the Five Whys whenever they encounter any kind of failure, including technical faults, failures to achieve business results, or unexpected changes in customer behavior.
-
loc 3305 - I call it the Five Blames. Instead of asking why repeatedly in an attempt to understand what went wrong, frustrated teammates start pointing fingers at each other, trying to decide who is at fault.
-
loc 3318 - When blame inevitably arises, the most senior people in the room should repeat this mantra: if a mistake happens, shame on us for making it so easy to make that mistake. In a Five Whys analysis, we want to have a systems-level view as much as possible.
-
loc 3324 - At IMVU we told new hires, “If our production process is so fragile that you can break it on your very first day of work, shame on us for making it so easy to do so.”
-
loc 3332 - For the Five Whys to work properly, there are rules that must be followed. For example, the Five Whys requires an environment of mutual trust and empowerment.
-
loc 3338 - The first rule encourages people to get used to being compassionate about mistakes, especially the mistakes of others. Remember, most mistakes are caused by flawed systems, not bad people. The second rule gets the team started making proportional investments in prevention.
-
loc 3357 - Whenever something goes wrong, ask yourself: How could I prevent myself from being in this situation ever again?
-
loc 3358 - Start Small, Be Specific Once you are ready to begin, I recommend starting with a narrowly targeted class of symptoms. For example, the first time I used the Five Whys successfully, I used it to diagnose problems with one of our internal testing tools that did not affect customers directly.
-
loc 3362 - It’s better to give the team a chance to learn how to do the process first and then expand into higher-stakes areas later.
-
loc 3368 - For example, you might decide that every complaint involving a credit card transaction will be investigated. That’s an easy rule to follow. Don’t pick a rule that is ambiguous.
-
loc 3395 - IGN had the experience of trying to solve all of its “baggage” issues that had been causing wasted time for many years.
-
loc 3399 - Since baggage issues are endemic, they naturally come up as part of the Five Whys analysis and you can take that opportunity to fix them incrementally. If they don’t come up organically, maybe they’re not as big as they seem.
-
loc 3488 - Net Promoter Score2 to evaluate customer satisfaction with its many products. This is a great source of actionable metrics about what customers really think about a product.
-
loc 3511 - As Greg put it, “Organizations have muscle memory,” and it is hard for people to unlearn old habits. Greg was running up against a system, and making individual changes such as arbitrarily changing the release date were no match for it.
-
loc 3549 - employees, changing the culture is not sufficient. As we saw in Chapter 9, lean management requires treating work as a system and then dealing with the batch size and cycle time of the whole process.
-
loc 3556 - To get the batch size down, the QuickBooks team had to invest in new technology. They built a virtualization system that allowed them to run multiple versions of QuickBooks on a customer’s computer.
-
loc 3590 - Internal or external, in my experience startup teams require three structural attributes: scarce but secure resources, independent authority to develop their business, and a personal stake in the outcome.
-
loc 3600 - sensitive to midcourse budgetary changes. It is extremely rare for a stand-alone startup company to lose 10 percent of its cash on hand suddenly. In a large number of cases, this would be a fatal blow, as independent startups are run with little margin for error. Thus, startups are both easier and more demanding to run than traditional divisions: they require much less capital overall, but that capital must be absolutely secure from tampering.
-
loc 3600 - It is extremely rare for a stand-alone startup company to lose 10 percent of its cash on hand suddenly. In a large number of cases, this would be a fatal blow, as independent startups are run with little margin for error. Thus, startups are both easier and more demanding to run than traditional divisions: they require much less capital overall, but that capital must be absolutely secure from tampering.
-
loc 3601 - are run with little margin for error. Thus, startups are both easier and more demanding to run than traditional divisions: they require much less capital overall, but that capital must be absolutely secure from tampering.
-
loc 3612 - Third, entrepreneurs need a personal stake in the outcome of their creations. In stand-alone new ventures, this usually is achieved through stock options or other forms of equity ownership.
-
loc 3618 - “Financial incentives aside, I always felt that because my name was on the door, I had more to lose and more to prove than someone else. That sense of ownership is not insignificant.”
-
loc 3620 - This formula is effective in for-profit companies as well. At Toyota, the manager in charge of developing a new vehicle from start to finish is called the shusa, or chief engineer:
-
loc 3629 - There are no objective criteria by which a team can gauge whether it will win this coveted lottery. Teams have little confidence that they will receive any long-term ownership of their innovations. Thus, teams rarely are motivated to take real risks, instead focusing their energies on projects that are expected to win the approval of senior management.
-
loc 3643 - One problem was that nobody could agree on what the data meant. Many custom reports had been created for the meeting; the data warehouse team was at the meeting too. The more they were asked to explain the details of each row on the spreadsheet, the more evident it became that nobody understood how those numbers had been derived.
-
loc 3737 - The sandbox also promotes rapid iteration. When people have a chance to see a project through from end to end and the work is done in small batches and delivers a clear verdict quickly, they benefit from the power of feedback.
-
loc 3741 - Functional specialists, especially those steeped in waterfall or stage-gate development, have been trained to work in extremely large batches.
-
loc 3762 - Managers of this kind of organization suffer the fate of baseball umpires: criticized when something goes wrong, unappreciated when things are going well.
-
loc 3767 - Every successful product or feature began life in research and development (R&D), eventually became a part of the company’s strategy, was subject to optimization, and in time became old news.
-
loc 3839 - A programmer expects to be coding all day long, for example. That is why many traditional work environments frustrate these experts: the constant interruption of meetings, cross-functional handoffs, and explanations for endless numbers of bosses all act as a drag on efficiency. However, the individual efficiency of these specialists is not the goal in a Lean Startup. Instead, we want to force teams to work cross-functionally to achieve validated learning.
-
loc 3843 - It does not matter how fast we can build. It does not matter how fast we can measure. What matters is how fast we can get through the entire loop.
-
loc 3845 - In my years teaching this system, I have noticed this pattern every time: switching to validated learning feels worse before it feels better. That’s the case because the problems caused by the old system tend to be intangible, whereas the problems of the new system are all too tangible.
-
loc 3864 - In 1911 Taylor wrote: “In the past, the man has been first; in the future, the system must be first.” Taylor’s prediction has come to pass. We are living in the world he imagined. And yet, the revolution that he unleashed has been—in many ways—too successful.
-
loc 3927 - Like Taylor before us, our challenge is to persuade the managers of modern corporations to put the system first. However, Taylorism should act as a cautionary tale, and it is important to learn the lessons of history as we bring these new ideas to a more mainstream audience.
-
loc 3948 - This in no way diminishes the traditional entrepreneurial virtues: the primacy of vision, the willingness to take bold risks, and the courage required in the face of overwhelming odds.
-
loc 3961 - Anytime a team attempts to justify its failures by resorting to learning as an excuse, it is engaged in pseudoscience as well.
-
loc 3961 - Anytime a team attempts to justify its failures by resorting to learning as an excuse, it is engaged in pseudoscience as well. If learning has taken place in one iteration cycle, let us demonstrate it by turning it into validated learning in the next cycle. Only by building a model of customer behavior and then showing our ability to use our product or service to change it over time can we establish real
-
loc 3961 - Anytime a team attempts to justify its failures by resorting to learning as an excuse, it is engaged in pseudoscience as well. If learning has taken place in one iteration cycle, let us demonstrate it by turning it into validated learning in the next cycle. Only by building a model of customer behavior and then showing our ability to use our product or service to change it over time can we establish real facts about the validity of our vision.