Growth Office Hours with Anu Hariharan and Gustaf Alstromer | Transcription

Note: This transcription is split and grouped by topics and subtopics. You can navigate through the Table of Contents on the left. It's interactive. All paragraphs are timed to the original video. Click on the time (e.g., 01:53) to jump to the specific portion of the video.

Introduction

Intro (00:00)

All right, Gustav, probably the best place to start here is let's just differentiate growth from growth hacking and all these buzzwords that people talk about. Sure. So the growth hacking term to me is a marketing term. If you, it's kind of like the new social media manager. If you want to make people believe you're working on growth, you call yourself a growth hacker. And if you actually work on growth, you call yourself a proc manager working on growth or an engineer or a designer. That's the simplest definition. I think a more kind of deeper explanation might be that you can't hack your way to true sustainable growth. You have to invest long term and the term growth hacking somehow give the idea that there's a one small thing you can do and just look for that one thing and then you're done. And that's just not how it works. So maybe the best entry point into this then is to talk about the growth posts and talk about how someone actually might go about investing in growth. So Anu, where would you start? Yeah. You know, before I start on the framework for growth, the reason we actually wrote the post was because a lot of growth state CEOs come to us and ask the number one question which is what does a growth team do and why should we set up a growth team? Because let's be honest, it's a relatively new concept. It didn't exist a decade ago and Facebook was the first to pioneer that. I think as Gustav mentioned, rightfully growth pros don't like the term hacking because it sort of implies a gut driven approach.

Understanding And Implementing Growth Strategies

What is growth, and what is growth hacking? (01:35)

So that's where a growth team comes into play. It is a very data and scientific approach to scaling growth. And right after you achieve that product market fit, which we call the zero to one face, to accelerate growth, you have to use a data driven approach and that's when a growth team comes into play. But before you set up a growth team, the number one step you need to check is whether you have stronger attention because too often most companies form a growth team and then wonder why they're not growing fast. Well, you have a leaky bucket at the bottom. And so the most important step is retention. And so even if you look at the post, the framing is first check whether you have good retention. So you want to check whether your users that you acquired in the long term, there is stable retention, which is parallel to the x-axis and also it's good retention. So for example, if you're a social network and you have less than 10% retention and it's stable, it's meaningless because as a social network, you need to have at least 50 to 60%. So it's more important to also benchmark whether your stable retention is good versus benchmarks or better than benchmarks before you start focusing on growth. So good stuff. When you're at Airbnb, how did you decide on that number? What retention number were you shooting for? Good question. You like to give the impression that you did everything right from the beginning. And when I got to Airbnb, we were just kind of like trying to figure things out. So I don't think that that was the very first thing we looked at. We were just trying to figure out where are the metrics, what are the things, how does the funnel look like. If you have great metrics and measuring time, use of funnels through a product, it's pretty easy to figure out your attention rate. It's pretty easy to figure out where people are dropping off and where the opportunities. When I joined Airbnb, we were just scrambling to get all the different places we had the metrics together. And that's kind of where we started, just making sure that we're tracking our user base and our host and our guest in one single place. And there was one single source of truth that we can all agree on. And from that, it gets easier to start measuring some of these things. And a marketplace, I think today there's a lot of knowledge around how to measure and evaluate the performance of the marketplace. I had not seen a ton of these things before when I joined Airbnb almost over five years ago that I didn't know exactly what to look at. So we just started looking at all the different metrics and trying to figure out where the opportunities were. Now, today Airbnb is quite different than most other products in that travel is a very rare occurrence. You travel once, maybe twice a year, which means that looking at retention on the guest side, you'll have to wait a long time. Or you have to have a long couple of years of historical metrics that are not like that actually in good shape. Very often when you start something, you never have a year's worth of metrics. So that means it's hard to figure that out. For most companies, you just want to try to figure out repeat purchase rate, repeat booking, repeat use of some kind. And that repeat use have to be meaningful. Like you can't just be like, oh, I send them a tuition notification that came back and now I have repeat use. That's not just the act of coming back isn't meaningful unless you do something that gives you value from the product. And so, Anu, when you're putting data together for the posts, what were those metrics that other companies use to discern like, oh, this is a good use case rather than just checking the site? Yeah. I think the most important thing is a sign of the action of usage of your product. So in the case of Airbnb, I mean, you guys tracked bookings. The biggest that actually shows you booked a room where Airbnb not just visited the site. In the case of Uber, it was a trip completed, right? Where you booked the trip and you didn't cancel it, but you actually completed the trip. In the case of Stitch Fix, this is an interesting one because now obviously there's a lot of talk about how Stitch Fix has done amazingly well. But for the first four years, they actually only focused on retention because they didn't focus on growth because they had organic growth, but they realized that they had a lot of opportunity to improve retention. So their sort of North Star metric for the longest time was number of second fixes in the first four months. So what that means is how often did the customer order a second Stitch Fix after the first purchase within the first four months because data had shown that people who did that retained much better than others.

What drives retention- Repeat purchase #1 (06:09)

So they focused on driving that. So the growth team actually was really focused on driving number of second fixes. Wow. And so what caused them to wait four years rather than just going for it from the beginning? Because if you have figured out how your product has to evolve to improve retention, then you're much better at scaling growth because now you know how to retain them better. With a product like Stitch Fix, which is more frequency than travel because it's not once a year, but at the end of the day, it's a parallel. And it's a lot about style and fit. So for the longest time, people still liked walking into a store. One of the reasons why Amazon didn't do extremely well for dresses, they do well for standard clothes. And so Stitch Fix had to get that element right. It was about hitting the right style and the right fit. And if you just scaled pretty quickly, you would have grown quite fast, but you wouldn't have retained a lot of users and those are wasted marketing dollars because you have to go back and re-acquire them. So really that metric and that place of focus on retention is really what helped them scale. And in the process, they were getting a lot of organic growth. And in the last one and a half years, they've actually invested now on growth as well. But only now the, you know, the not-star metric has probably moved to number of first fixes.

Growth Engineering Checklist (07:47)

But for the first four years, it was about repeat. Okay, gotcha. So I think certainly the entrepreneurs in the investment community have gotten more sophisticated over the last five, seven years around how to think about growth and retention. I think probably seven years ago, people were more obsessed with new user growth and just like the people growing top of the funnel. And they have learned that it is not everything and it's actually not like even the most important thing. So I think we've learned that quite a bit. And when I do office services, YC companies, the first two questions I try to figure out is basically what is the kind of ideal or expected behavior? How often do I use this product? Let's say it's some kind of like, let's say it's like some kind of retail product. Like, how often am I expecting to do this? It's an e-commerce product. How often am I buying this thing? If the subscription should be on a regular basis, if it's not a subscription, I still want to figure out how often I do it. And then once you have that number, and you can get this number by kind of thinking out from like, what is that you'll use case? Or you can look at the data and see how does your most typical active users or retained users actually look like? And then from there, and this is exactly what we did there, when we looked at the most kind of like typical retained users and then what is the typical kind of booking window? How long does it take between each booking? And from then you can kind of come up with like, this is the scale. And then after that you figure out, okay, how many people fit the scale? How many people actually are able to get to kind of do it the way that the ideal metrics suggest? And then you kind of figure out your attention rate. So that's kind of how you start off. And you don't get everything right away, but that's kind of a good way to start. Many people don't really know what the expected time between these events should be, but it's very important to figure that early on. So this is kind of an easy possible question for you guys, but just tools in general, like basic things like, what should someone set up in the beginning to start tracking this stuff?

Tools For Metrics (09:39)

So when you start, you're probably saving your user data somewhere. So you're probably saving that some kind of... Hopefully. Some kind of database that isn't too complicated. Okay. And very often when startups go a little bit bigger, it tends to have this data in several different places. It's important that you have one person responsible for the single source of truth data. Yeah. The data that everyone can trust so that when you're arguing about metrics, you're not pulling from different sources. You're pulling from the same source. That's critical. Now when you make, say, funnels or experiments, often you don't have the data storing it in the same place you store user data. So you might have that in, like, let's say, a mixed panel or maybe you have your own events database, but it's very easy to start with something like a mixed panel. So those are the two places where you store your data. Now, there are many different solutions. There's a mixed panel, there's amplitude. It's a whole slew of them that... Optimizely. Optimizely. Optimizely. Many of these companies, but they're lots of different ones you can try out. Segment is one way to try the mod at all at the same time. Like you can use segment. You can test all of them at the same time. But that's usually how people start. They start with something external. They're external ones are pretty good right now. When they go to scale or even be, many of them end up building something on their own. For different reasons. But it's kind of... Let's dig into that. So at Airbnb, at what point did you decide to roll your own analytics around this stuff? I think when you have one or two data engineers that you can spare for this product. So typically 10, 20 engineers, then you have enough that you can say, "I'm going to build this myself."

Using Your Own Tools vs. Thirdies (11:15)

Yeah. So I think with case of Airbnb, we started saying, "Okay, we have our own event database built on clients to send data to that database from different platforms, iOS and Android Web." And now that then you have to kind of start learning how to make sure that data is always like available.

Setting Growth Team Goals (11:27)

So there's a lot of work that goes into just making sure that data is always available for all the backscores and all the things you need the data for and make sure that it's correct. You have to do it with a lot of time on this data. So it's actually quite a lot more work than you think you can do it part time. But there are some benefits to doing it yourself when you get to that stage. How big was the growth team when you guys started building our own tool? So when I started Airbnb, we were three people on the growth team that was excluding performance marketing. There was probably another three or four or five people. And we didn't actually build the first database. It was a separate team. But I think when I joined, we were about 35 engineers and maybe six, three to six months after I joined, we decided that this is the time to start investing internal tools. And over the next year, we basically built everything that we needed internally. And it wasn't perfect. So over the next three years, we made that perfect. But within a year after we started, we had all the components that we needed. And that was basically a segmentation tool that allowed us to show different parts of Airbnb to different users. It was an event tracking database where we can send all the events. So we run queries and look at these usage from different clients. We can do that from our own data. And then we built the experiment framework UI. So it would automatically show us a different-- let's say we run an AB test. It would show us a different metrics in the control group and the experiment group. It would automatically calculate fiscal significance and power and all these different things that you need. We didn't have all that in the beginning. So in the very common way people start is to store the data somewhere. And they have a data scientist kind of in Excel, pulling the data from the different groups and then kind of figure out if that's something is significant or not. So this is an insane amount of work. What are the things you guys did that maybe were too much work, too early? And maybe things you saw Anu putting together the post, like, you know what, if you're an early stage company and even if you're a later stage company, you don't need to do that right now, this is the effective stuff to focus on in the beginning. I think one of the mistakes that people make early is that the role of using data to inform their company is kind of on the founders and the CEO. One of the mistakes I see is that, as my answer to the question, my mistake says that data becomes someone's responsibility who's like a data engineer but no one else's. And then you're not really succeeding because the goal is to use this data to make decisions throughout the entire organization. And then you have to make the data accessible and available. And if you start building all the technology first, it's not going to be a hundred from accessible or available because you have to build dashboards, you have to build all kind of visualization things. And that takes a lot of time. So the number one goal is not to build everything. The number one goal is to make it available, accessible for everyone so everyone can start making decisions and mix panel and an attitude is great to start with. Like you can give everyone a login and everyone have access. Like you can build email reports, all kinds of things like you can build in day one. And just building an email report from your own system, that takes a while to work so that there are a lot of benefits to start with something external. And I would recommend most people that. Yeah. And I think that's probably the single biggest statement. You know, we heard as well when we were interviewing all the growth experts was you need a common source of truth, which is why you use these tools or you try to build something internal if you feel that tools are not helping you build a common source of truth. But the more important thing is engagement from the CEO and alignment on taking action based on data.

Yi-Da takes away the title (14:53)

This is not, you know, and this is a very difficult thing for a CEO to do as well, especially if you're a product and CEO. Right, because there's always intuition and there's always data. And it's not necessary that data always drives all decisions. Sometimes you want to run experiments because, or you want to do build things which are not driven by data. And you know, but I think that the CEO that sort of knows or learns how to balance both it actually gets the most out of it. And to this day, I think the from all the interviews, I'd say the CEO that really stood out in that decision making is actually Zuckerberg at Facebook. Pretty much every growth expert that I had spoken to who either worked at Facebook still works at Facebook are now no longer but used to said that Zuck was very clear when the data came to pose a question. But at the same time, if he wanted a product decision to be pursued, which the data didn't support, he was clear about that as well, saying, look, as some, my gut says this is the right way to try. Let's try it. And in that, if you use scenario B where you're not using data, but you're using, you know, some product intuition that you have, the growth team actually works as defense. So say you make a change and the change probably is slightly more detrimental than you thought. The growth team can actually discover the impact in minutes, not hours, not days. And so a great growth team can alert pretty quickly and course correct if they need B. So if you remember the famous slogan that Facebook had, which is, I think it's like, you know, do think, I forgot the exact thing. Move fast and break things. Move fast and break things. They dropped the break things now because they have 1 billion users. And this is exactly why you can't afford to break things when you have 1 billion users. Well, they break things for some fraction of those users. Yes. So I think that you, as a single most important thing is if you decide to form a growth team, the CEO has to be aligned. It's not the growth team's responsibility. They won't be set up for success if the CEO is not aligned. And so it's extremely important that the CEO endorses the goal of the entire growth team with the company. And when data is surfaced, is asking objectively the right questions and helping different teams make decisions based on the data. Were there moments at Airbnb where you guys were like, we're just going to go with our gut on this one? Oh, many times. Many times. Mostly. And here's how we think about it. So this is a great kind of way to talk about experimentation. So when you start with a product, most of the ideas you have, like, I know what I'm building. I know what I'm building. And you build it and you talk to users, you build more things. And you kind of, you have a good idea. At some point, your product is so complex. So many things on the product that you make a change, a human cannot fully comprehend all the impacts that that change will have. So software and metrics will, because software keeps track of everything. So Airbnb and Facebook are certainly at the stage where if you make a significant change or a small change on the product, then it will have some impact down the line that we cannot comprehend or we do not kind of foresee that that happens. So in order to solve that, you run experiments. So experiments, basically, I divide that user group of Airbnb into two groups, a control and experiment, a launch to change in my experiment group. And I look at all the metrics that I care about and see how they change. So let's say I changed the date picker on the Airbnb website, because I have a new idea how to make it better. Well, I really want to see if search with dates is going up or down. And at that point, you don't really know. Like we run this thing called experiment review at Airbnb, I think Facebook do as well, where you literally are, take the entire team in an inner room, you show a bunch of experiments that you've been working on. Before you tell the results, you ask everyone what to think.

You make data products (18:51)

It turns out that the room is very often divided and they have different opinions, but that's because making practice decisions are really hard. It's really, really hard when you have such a scale and such a complex product to know what's going to perform better or worse with certain metrics. So that's why you need experimentation. Now growth teams tend to attract people who are very black and white in their mindset, because they can use data to hit other people in the head and be like, "No, look at the data, it shows this." So we have to do that. That's not a good approach because it's much more complicated than that. That vision of the company might not end up exactly the place of the experiment. I take booking.com as an example of a product that is probably very successful, but not a product that I'm super proud of to be working for. Because if you experience it, it is incredibly in your face. There's pop up and there's things that goes all over. And it's grateful conversion, but I wonder if it's something that a product is signing a wonderful product. Now you can still do all those things right and still build a really awesome product. I think Facebook and Instagram are an example of something that is very, very optimized, very, very good, but it doesn't feel like you're being sold all the time. So I think you can get to a point where you combine a strong product vision. You have some kind of design guidelines or product guidelines and still use experimentation and validation to make your decision. So I'm 100% believe you can do both. Is there a good example you guys know of that? Where like, I mean, probably Airbnb where you push something forward and say like, this is optimizing the funnel for us. This is better, but I don't know. Something about it is just like a little too hacky or whatever term you might have used where you had to pull back. Now I think that absolutely happens, but I think you have to what people that work in growth tend to do bad is to explain the purpose of what they're doing.

Role of SEO in Growth (20:39)

So a lot of people that work in growth, let's say you work in SEO. The word term SEO sounds bad. Sounds like I'm hacking Google. Let's talk about what is SEO. There are a lot of people on Google, like all of us, that go to Google to look for answers. And if your product does not show up on those first three links on Google, you're not going to be clicked on and you're not actually there. SEO is a way to get your product to be one of those three answers. So people are not hacking people's mind. You're not doing anything kind of abusive by doing SEO. In any way, you're literally trying to help people that are on Google, trying to find answers to questions. And if M.B. offers the best product to people that are looking for vacation rentals in the world, we should be on the top three results on Google. And I think that's a great example. We'll like just have to change the way you talk about it and people will appreciate it a lot more. It's so important to make all this effort into being on top of Google. Another example I can give is actually Facebook was Instagram. Facebook was actually known for its probably on the higher end of aggressive growth tactics in the early days. Because when they were building social network, there was no other social network that had more than 50 million M.A. use. So people thought that they would cap at 100 million. And so they have a paranoid about breaking that paradigm and sort of telling the world that they can be the first global network. And they did it. But the early days, there were lots of email marketing campaigns. And if you remember, if you didn't log into Facebook, you would get an email saying, somebody uploaded a photo of you. Now what are the odds, one login to Facebook to see what that photo is about. But Instagram on the other hand is the team that founded Instagram is very different. The DNA of Instagram is very different. And so they didn't feel as comfortable using all the tactics of Facebook. So there is a big difference in the growth tactics that the Instagram growth team uses versus Facebook's growth team uses even within the same company. So in fact, when you talk to the heads of growth in the respective groups, they would say they said something called heuristics. So there are a set of things that are okay that Facebook can try. But not all of those things are okay in Instagram. So the rule of thumb they generally use within each team is don't ship something you won't become fuddable shipping to everybody. So if Instagram's whole DNA is I don't want to send an email to anyone saying your photo is dagged, then that won't be something that would be experimented on. So it has to also go with the entire flow of the experience that you envision for the product. And as long as you set those heuristics for the team, people know how much to experiment in order to achieve growth.

Set Guidelines but Flex for Expansion (23:34)

And what would you tell a founder who's just going to get started and they want to set their own heuristics? I think that the main thing is look, every founder knows what's best for the product and the vision. They're listening to their users. They should definitely set guidelines, but also be open to adjusting guidelines as they evolve. One of the simple reasons for that is as you expand internationally, you may have to adapt your product in different ways locally that you may have thought you would never do. Right? So for example, even with Instagram, which is heavily among women as users, when they launched in the Middle East, they saw all men and they realized, well, maybe we should change our targeting mechanism. Right? For some reason in the Middle East, we are attracting a lot more men. But one of the important elements of the growth team is to have user groups that do use the research. When the team was on the ground, they realized that all these profiles that had male photos were actually women because people in the Middle East are worried about harassment. So they didn't want to pose their pictures. So imagine if you don't really understand the product or the users, you may not be able to tailor it to local taste. So you may have to make differences in how you onboard users so that it's easier for people to adapt locally. So always start with a set of heuristics, but decide what you're going to flex and not flex for expansion. You may have to adjust certain things, but there are certain things which you may never want to change. So how do you start building out a team as your company is growing that follows these principles where, you know, like Gustav, you weren't the founder of Airbnb, but you're very early. How do you think about scaling out a growth team? So I would say the most important, well, the typical team within growth have eventually all those disciplines, engineering, product, design, data science, and user research. And sometimes there's a specific, let's say, discipline like online marketing, performance marketing, SEO, there might be very specific skill sets that you add to that, but that's typically how each team look like.

Building a Growth Team (25:42)

The very, very beginning and unique engineering. So engineers typically the very first person, a pragmatic manager can't get much done with that engineering. Ideally, either one of those few people are technical enough to run their own data and like be able to run their own queries and analyze data. After that, I would kind of grow out from there and maybe typical early growth team is two engineers in it and product manager. One of those ideally, the product manager is very savvy in understanding and using data to make decisions. And then from that, you kind of add on the different disciplines that you need. The term growth team, the reason most companies succeed, most companies kind of start a growth team is because it's very hard to tell a company that now you're all in charge of growth. I've almost never seen that work. So now you have a couple of different options. You either do that, that doesn't work, or you don't have anyone responsible for growth. That's kind of like how growth team starts because you make one individual or one individual team responsible for growth and say, this is your area of opportunity and this is your metric that you're trying to optimize. And that eventually can teach the rest of the company to kind of slowly break out into different funnels and have similar minds that are similar way that they're working towards solving those problems. So that's kind of often why growth teams or call growth teams when they start. And now you can start them within the product team. You can start an asset or product team. My recommendation would be starting within the product team as a specific part of the funnel. If you start an asset or product team, you're effectively saying that the product team might not be able to learn growth as good as the growth team and therefore you should be a separate organization. I'm not sure if I believe that's a good idea. So my recommendation would be within the product team. And it needs to be have the CEO needs to basically be on board with this idea they have a growth team that is moving a little bit faster, using data a little more than the rest of the team. And the typical people you get on that team are people that are ready to throw away a lot of stuff. Because you run a lot of experiments, they're not going to work. You're going to throw them away when they don't work. People that are willing to try a lot of things, they're moving pretty fast. Like you're going to do a lot of small experiments and it's often the velocity of experimentation that's more important than the quality of the idea early on. Because you don't really know where to look. You just have to have a framework to look everywhere. And then someone who uses data to kind of validate which idea started to work well. Yeah.

Debate of Growth Teams Being Part of Product? (28:33)

I think the single source of debate, at least I heard when interviewing all the growth experts was actually whether they should be part of product or should be separate. Right? Actually, I would say majority of them would agree with you that it's better to be product product. But if they had it their way, they wanted to be independent and report directly to the CEO. And I think the main reason for that, a lot of the expert cited was the tension between the head of product and the head of growth. Because growth is so data driven. And if you have a product lead who is usually not as data driven, but because they come up, come at it from the product design perspective, then that can be tension. Second reason for tension is, you know, a startup is all about growth.

Key Takeaways And Conclusions

Conclusions, conclusion (29:16)

And so it almost feels like the growth team is owning growth. So what are the other teams owning? And I think that tension is the second tension that teams have and this tension and types of experiments or changes that a growth team does versus what a product team does, which is why I would have thought that being part of the product team helps stem that. But you know, I think it really comes down to the communication between the head of product and head of growth, because you could be part of the product team. But if you're not aligned and if the growth head is two levels away from the CEO and that CEO endorsement doesn't exist. It tends to fall apart. And I think the only company that actually has done this successfully being independent is Facebook again. Right. So Facebook is the only one that has still maintained it as a separate growth team. And from what I understand, there are very separate accountability metrics and which is why it works. I think it works. Any structure can work as long as there's accountability for each. So the growth team owns the M.A.U. and D.A.U. Right in Facebook. The product team owns the engagement and retention. So it's damn clear how it's split.

Split Incentives (30:25)

And so I think whether you're part of product or whether you're not, as long as the metrics are clearly defined, then you have better understanding and articulation of who's doing what. Uber actually had two separate teams. They had a product team and a growth team. And at scale, just one and a half years ago, they merged. But even when they merged, you know, they had different subgroups. So Ed Baker, who was the head of growth, took over a head of product and head of growth. But the teams underneath were rider, driver, marketplace and platform and few other teams. So I was actually curious as to why there was rider, driver and marketplace, right? And the interesting thing is, rider team focuses on rider growth. The driver team focuses on driver growth. But someone has to make sure that the supply and demand are matched. That's the marketplace. So they were actually measuring true proficiency matches percentage of times in ETA that the marketplace owned because that was the check of liquidity versus rider and driver growth teams.

Overlap Overlap (31:20)

So I think it's all these and that's worked well for Uber. So these things all all lead to the fact that it's what is most important, whether you sit as part of product or not, is what is that accountability metric for that group? And how does it all roll up into your not star metric? And is it clear? Because if this overlap, it breaks, right? It's just A plus B plus C equal to D, but you need to be able to build your driver to and sort of give each team the metric. I think what can be uncomfortable for a founder is that at some point you realize that in the early days you build a product, you've talked to a lot of users, spent a lot of time in interacting with users. At some point you got to scale and you can quantifiably make a lot of decisions by just testing things. And that I can imagine that being very uncomfortable for a founder that it's just like a complete different way of building product at scale. And that's often why introduction of growth thinking in a startup can be a little bit painful because it just changes the way you do things.

Reflections On Growth

Growing Up (32:17)

You do things in a very different way in the very early on and then you do things quite differently at later stage. All right, guys, this has been great. So we're probably going to have to do around two on this one. But thanks for coming in. Thank you so much. Thank you. Thanks for having us.