Ep 1
===

Aleks: [00:00:00] She combined some of these features into feature constellations or groupings that help unblock a core workflow or a job for a customer. And we found that when you combine these two features, they're willing to pay five x, 10 x what they would for each individual feature. Because you're streamlining the workflow, you're unlocking additional value. 

Jeff: Alex, how you doing? It's great to have you on the show today. I.

Aleks: Thank you so much for having me. I'm doing well. How are you doing?

Jeff: I am doing all right. I'm doing all right. I'm down in sunny Dallas today doing this remote but really stoked we could make the timing work and we've known each other a little bit for, a year and a half now. So really excited to have you on the show and be able to do this.

Aleks: Thank you so much. I'm so glad to hear that it's sunny where you are as it's cold out here in Utah.

Jeff: Yeah I called back home and heard that it was snowing back in Boston where I'm based out of, but which is great 'cause it was like 62 days ago, but but Dallas is, sunny and wonderful so I'm here and got a smile on my face and we're gonna do this thing. 

Aleks: Do it.

Jeff: yeah, I think you present an interesting case, right?

Because you started in psychology, leveraged that into consumer insights [00:01:00] market research led through that really got into like user testing and digital experiences that way, which makes complete sense, but I don't know if we've ever really talked to a lot of people who got into product that way.

But that led you on, a journey that in retrospect makes a ton of sense, right? Doing product for companies that do this kind of research, like Qualtrics SurveyMonkey. But for the past year and a half, you've been the CPO over a Typeform a company we actually use for a lot of our surveys.

But yeah, , it's been an interesting career, but I love this line that you said when we were talking a little bit ago, which was this all kind of put you in a unique place to be able to leverage research to bolster decision making. But without replacing your gut instinct or any of that.

But it also provides the credibility and confidence for that gut instinct, which I love. 'cause there's always this kinda like push and pull on, do you go with gut? Do you go with, deep research, how much research is too much research. And I think this provides a great background for having the knowledge and all that kind of with that.

Yeah. Is that, did I miss anything in your career there?[00:02:00] 

Aleks: That was so eloquently put a hundred percent hit it out of the park. So thank you for that, Jeff.

Jeff: Alright. Yeah, so you've been CPO over at Typeform for, a year and a half at this point, but, you didn't walk into the role to a company that was up into the right and growing, a hundred percent year over year. It was not all daisies when you walked through the door. So maybe let's start out, we're gonna address the situation.

I'm gonna be honest, reader listeners, you're probably gonna wanna get a notebook out and take notes throughout this 'cause the story is pretty. Cool. And we're gonna get into a lot of practical stuff, but you walked in and maybe give us the context of what happened when you kinda joined Typeform right from the start there.

Aleks: So when I joined Typeform it's interesting because the industry as a whole, I feel like was going through quite a bit of change at that point in time. Qualtrics had been purchased by private equity at that point. SurveyMonkey was going through a private equity transition. And so there was, I think a lot of pressure from the market in general for anything that was formed, survey related.

And Typeform was no different. It had [00:03:00] experienced this big boom over Covid and then, started to experience some flattening growth over time after Covid. And of course that was. Compounded by rising competition, a bunch of new players that are generating forms and have a lot of really quick iteration on their product development out in the market.

Free tools like Microsoft Forms, Google Forms, et cetera, that are in the market. And so it really puts extra pressure on tools to have, more comprehensive capability sets be able to solve real world problems for core customers. And I'm not gonna lie, there's also this pressure on price, right?

Because that is another component that is very salient in the purchasing process for customers who are looking for tools like this list

Jeff: It's gotta be tough when at some level you're competing with tools that maybe can't do it as well, but they can do it free or near free. And, you are providing a , very high quality experience to forms there. Beyond that. It must be a little tough where like [00:04:00] anyone can use for, it's a little bit of the Airtable problem, right?

Anyone can use forms. The persona base is just almost everyone in any company. That's so much complexity. I have to imagine that causes of spot where there's a lot of risk of being second guessed because just the market can go any which direction.

Aleks: A hundred percent. And I think what happens in, in organizations that happen to be in markets that are so horizontal, like Typeform is it makes it hard to pick who are you building for, right? And so often what people do is they default to the stack ranked list of capabilities that are the most frequently requested, but there's not really a intentional segmentation around who is buying your tool, what are they buying it for, what are their intentions, what are they trying to accomplish?

And all of those components, when not pulled into the prioritization framework, make it much harder for you to actually hit some of the targets that you've got from a business perspective. So there's a lot of challenges that I think horizontal organizations face that are materially different than [00:05:00] organizations that maybe have a very specific or verticalized offering that targets a very clear ICP which is interesting.

I.

Jeff: Yeah. It seems like the key here is going to be how do you ship the most effective things for the most people that are gonna drive the most value for the business. But there is just such a high chance of shipping a lot of features that just don't move the needles. You can be busy but not productive.

Um, Percent. And you get stuck in these patterns of incrementalization, right? Where a lot of tinkering happens on things that you already have. So you can get those slight margins of improvement across the board, but you're sometimes you get stuck in these places where you're not necessarily innovating or not completing bigger job roles because those put you in collisions with other tools in the industry as well that are maybe outside of the scope of what your tool is currently responsible for delivering to the market.

Aleks: And there's some, I think, extra competition on that end for sure.

Jeff: yeah. And you also run into, I've seen companies kinda where you have that great fit with initial product. You have to do process something a little bit different [00:06:00] going forward. One company I was at, I remember it launched an internal, almost competitor to the primary product. It was literally an internal team.

And they developed all the technology that should have been put into our primary. Product. And they started to rise up to the point where salespeople, like sales teams were literally covering their whiteboards. So the other sales team, from a, internal but competitive product wouldn't see their deals and stuff like that 'cause they'd potentially snipe each other.

And finally, I remember we, we did get acquired from a PE company in that company and one of the first things they did was like, no, nope. Put it together. Let's be smart, come up, put the innovative thing in the prime thing. But I feel like this is a role you walked into that was built to showcase the strengths that, that we've talked about here, about your background.

So I guess, let's stop messing around with the precursor stuff like. Let's dig into the research. , you guys did some really rad stuff around research and how do you distill out what are the product features that you need to build?

How do you hit the most people [00:07:00] with the most impactful stuff, pricing, packaging, like we covered the whole gamut. And we only got to talk for 10 minutes about it, so, I'm stoked. Let's do it.

Aleks: Let's do it. Maybe I can read you into what I walked into a little bit, and that'll set some of the context for some of the decisions that were really helpful for me as I got onboarded to the role and the team, et cetera. And so at this point, so you're experiencing hyper competition. You're experiencing pressure on pricing, you're experiencing you know that flattening of growth, it's softening, right?

It's not necessarily stopped, it's not hurting. Companies still growing, but it's not quite growing as fast as it maybe was before. And it's raising some alarm bells across the organization. The other piece is there had been some interesting leadership turnover, which was creating, outsized impact as it relates to, the morale for the teams and the clarity and alignment from a cross-functional collaboration perspective.

And so that was getting a little bit in the way as well. And so that's why I was brought in to help reignite growth through their customer centric [00:08:00] approach and strategy that, I don't think the team knew quite how much I leveraged research and product development at the time. But I work with an incredible research partner who just, her and I, we have this wonderful synergy that allows us to break problems down without overly aligning to certain rigorous standards. And it exists. All of our research is rigorous, of course, but it's not anchored in, The ways in which people tend to use research that aren't necessarily as open to being adapted to ways that they're not thinking of using it creatively. So there's some blockers from a best practices perspective that would preclude I think, certain researchers from being able to dip into some of these tactics.

But as long as you can put some of those things to the side and be open-minded, are you getting better quality data and information then you would be getting if you weren't leveraging these tools? And if the answer is yes, and they're helping you refine your process and your strategy and some of the other components that you're talking about, 

Jeff: I gotta ask, is this a bit of, you gotta know you, you gotta learn the rules before you learn how [00:09:00] to break 'em kind of thing.

Aleks: Yeah. I think what often happens, and this is what I found across my career, and I'll give you a quick example. For me it's finance, right? Finance is the thing that I close my eyes to if I can avoid paying attention to it. I love the revenue side of things, but the, all of the details of the things that finance teams have to do in order to support organizations, they're super complex, right?

And if you don't understand that level of complexity, you are talking past each other. You're not aligning on a solution that is actually meeting the needs of the business. And that kind of a dynamic, I think happens with research a ton, right? So people don't feel comfortable, knowing what you can use research for, what you can't use research for.

And so they'll talk to their researchers and they end up having a scenario surface where. They just say a very high level objective that they have. And the researcher isn't getting a ton of detail as to what they wanna accomplish. So they give them the best practices approach to how you would answer that question systematically.

[00:10:00] But neither one of them are quite articulating exactly what they need. And with me and Lee, my research partner on my team, because I also have a research background, I can understand the types of things that research can be used for. And I, and she understands a ton about methodology, right? And so she's a methodology rockstar.

She has pulled methods together that I wouldn't even have thought of on my best day as a researcher, right? So thankfully I'm no longer in that field as my core function.

Jeff: I'm sure you hold your own.

Aleks: But between the two of us, I can imagine some magical future in which, research can solve some of these core questions for me. And she the mindset shift that I think she takes on is she doesn't tell me all of the ways that the methodology won't answer that question. She tries to help me figure out how we could structure it to get as close to the answer that I'm looking for as we possibly can.

And I think that makes all the difference, right? Is that mutual collaboration and trying to get a little closer from each of our points of view [00:11:00] to what the ideal outcome is. Knowing all the caveats, knowing all the red flags and the challenges and what we can say, what we can't say, and how far we can actually take this data, right?

So I wouldn't take it to say this is exactly how the product roadmap and the strategy and the outcomes are gonna play out, but I would say we have evidence that this is a better approach than the one we may currently be following. And that allows me to make a lot of progress internally within the organization as well.

Jeff: Yeah, makes sense. And then I think one thing you talked about earlier is this idea of, some level just have to do the work and get the deep customer understanding. It sounds like you have the methodology lined up and the knowhow to do that, but how did that kind of come together?

Like what did that look like to get there? Or am I skipping us ahead of of

Aleks: No, this is great. So I walk in and I would say when you're going into any new organization you wanna meet the people, right? That exist in that organization. And you want, you get as you wanna get, as much context as you can on what's working, what's not working, where they feel blocked, where they feel empowered.[00:12:00] 

Do they have the tools, the processes, the support to do their jobs effectively. Of course they started with the product team and then expanded cross-functionally, and I found that the perceptions externally to the product team were. That the team wasn't innovating necessarily in that there was a lot of incrementalization happening and there wasn't like a cohesive strategy moving the product forward to some ideal end state.

But internally, I was hearing that the product team was getting a ton of very fractured input, right? Because, and it's, I think this is a unique function of potentially a horizontal organization or a product where it can serve so many different use cases. There was no alignment on ICP. Who are we building for?

What are we trying to accomplish? There was no macro company vision that would lead you towards, okay, we're gonna solve these. Core things that was there. Of course there's always a company strategy, but there wasn't one that was palpable and specific enough for the product team to be able to understand, okay, that means I need to go build [00:13:00] this, and this in order to help us achieve that as a business moving forward.

But there was this general sense that something in the product was not evolving in the direction the market was evolving and we needed to figure out what, and we needed to align it to that company strategy so that we could connect those dots. And for me. The first step is I wanna understand what the market feels about Typeform, about forms in general, about the use cases that are the most important right now, right?

Because also these things go through cyclical changes. And I don't know if you remember, I would say 10, 15 years ago, experience management was the thing, right? So everybody was putting in place customer experience programs and while these things are all important, they definitely ebb and flow in terms of people's willingness to invest in tools and processes to unlock the power that those kinds of, tools and processes enable over time, right? So I'm not saying that CX is not a thing today. Of course it is. People use CX tools, et cetera, [00:14:00] but it's not exploding. People know about it. They know which tools are valuable and they have their solutions in place. And so you're probably coming in to replace something as a tool if you're in that CX space more often than you are educating people on what CX is, right?

And so similar themes surface cyclically across the evolution of products in different industries. And so what we ultimately did is I came in and I was like, I need to know what to build and I need to know what to build fast, right? Because I'm coming into a situation that needs some level of collaboration, being able to pull people across the organization together to, towards a unified strategy and not one that we're just gonna iterate our way into.

And I didn't have six months to a year to be able to go do that. And so in that kind of an environment, research is a really helpful tool because it helps me tap into the market and understand what customers need, what they're feeling about Typeform today, et cetera, and be able to pull some of the core elements into my strategy without having to wait [00:15:00] for all of the PMs to do their own research, do their own competitive intel, come back with opportunities or initiative plans, refine those, right?

We just didn't have time to do that from scratch. So I collaborated with my research partner and what we ended up figuring out is there was, from customer success, sales, marketing, product managers, as well as, feedback through our various survey tools, et cetera, we had so many different ideas of features, capabilities, or product requests that have come through all of those different channels that we ultimately ended up aggregating all of those and articulating 'em.

I think we ended up with something like 80, 80 to 86 different features or capabilities and they're not tiny, change the font, right? They were like bigger packages of capabilities that really solved key problems and and we wanted to make sure they were articulated in a way that our customer base could understand 'em.

And then we went to market and we said, Hey, anyone who is looking for forms like [00:16:00] software, we wanna understand who you are, what you're doing, what your budget is why are you using Forms software, what your biggest pain is with form software today. And we put together this kind of max diff research where we had them look at these features and tell us which one is the most important, which one is the least important to them, or most valuable, least valuable.

I can't remember exactly which of those variables we ended up using. And we, we got through, the 80 features and people gave us really good feedback on what would be the most helpful for them versus not. We also got a good read on willingness to pay. So how much were they interested in paying for a combination of features that they had said were valuable which was also really interesting for us.

And then we were able to do the analysis and come up with some really interesting, elements that then helped us drive the roadmap. And honestly, I'm no stranger to using research and product management in the past, but there are some very interesting, unique things that surfaced out of this [00:17:00] research that I would be shocked if people were already thinking about, or leveraging or using in the way that they're planning their roadmaps that could be potentially helpful for others.

Jeff: So when you're doing this research it's, I think people be curious to understand a, like in a max diff, is that, are they selecting just the most important, least important, or is it giving a spread across or like what is the idea behind that and what was it kinda showing? Like how can people use that?

Aleks: the main goal of a MaxDiff is that you're forcing a preference, right? Because if you do research with any subset of customers around your product and you ask 'em, what do you, what would you like and how much would you like to pay for that? The answer is gonna be, I want all the things that I wanna pay as little as humanly possible for them, right?

Jeff: All.

Aleks: Right. You want it all for free. Exactly. And so in that environment, it makes it really challenging to leverage that data to then make decisions, discriminant decisions about I have fixed resources and I have to make sure that the right things get built that deliver real customer [00:18:00] value to the right subset of customers.

For us, what the MaxDiff did is it created a forced choice, right? You had to pick what was the most important, what was least important, and we randomly alternated which options you saw from a feature set. So you went through, I think, 20 of these pairings in total and told us that. And so over time you end up getting a score for how relatively important each of the features are relative to others based on the total aggregated scores that every single feature got across all of the respondents that answered some of these questions.

And so then we got out of that, we created a value score, which is tied to that, and then of course, a willingness to pay score. So we isolated, we were able to isolate how much each individual feature was worth, and that piece was super helpful as we pulled together the strategy because we had. Of some really interesting things surface.

So in a horizontal product like Typeform often when you're looking at some of these features, you're thinking, oh, this would be the most helpful for this subset of the market, right? [00:19:00] People who are doing customer experience or people who are doing lead generation, et cetera. And we had 80 features, so we were thinking we're, some people are gonna want some of these features, other people are gonna want other features and really we're just gonna have to decide who we wanna serve moving forward.

When we did the analysis based on kind of the three major customer groups that were using Typeform, so that would be marketers or, and not always marketers, but people who are doing lead gen like activities, people who are doing talent acquisition like activities and people who are doing feedback, market research, customer experience, types of activities.

We looked at the feature spread across those three groups, and 75% of them wanted the same features. So of those 80 features, 75% were common across the different use cases and different ICPs, 25% were different, which tells me Typeform product has not reached forms level horizontal maturity to address all of these different use cases.

And we still have some core [00:20:00] functionality to build and to evolve and to drive forward. And then that 25% were different, right? So they had unique use cases that certain features would help unlock, et cetera, that would help us, close more of their jobs to be done, gaps that they have within their use case evolution.

And that was something that I don't think I was expecting. I thought we would see much more differentiation across the different capabilities we were surfacing.

Jeff: Especially if you asked me, I would expect probably marketing and cx, for instance, to have pretty disparate wants. But I gotta ask, like those personas, is that something you went into having defined as like these are the sets of groups or was that something that came outta the research as well, or,

Aleks: Yeah, good question. That was something that came out of the research. This piece of research was like, it had 10 objectives in one study. It was really incredible to see it come together because it really dissolved several of our challenges all at the same time. So we kept it pretty open.

We weren't trying to constrain who we were looking for. Our main constraints were you're looking for, or shopping for forms related software and you're [00:21:00] using it for work. Those were the two main things we were constraining. And then we let everything else fall out. What part of the business you were in, what function you were part of, which types of use cases you were gonna use that form software for everything else.

And those were the three that were the majority. Of course, there's smaller groupings of customers that have much more unique and diversified use cases, but these were the three large pockets that we mostly serve.

Jeff: So were you looking for, in that case, what jobs to be done, are you trying to use, or whatever framework you wanna use for assessing that kind of thing? Like what department are you in? Like how did you pull together like where they fit persona wise, before you start to look at feature sets for them?

Aleks: We wanted to first organize by functional group, right? So if you were a marketer or if you were part of the go to market function, you would be bucketed in one way. If you were part of the people team or looking at recruiting, you would be bucketed another way. And if you were doing more feedback, market research type of things, you would be bucketed the third way.

We also wanted to validate that when we looked at those three distinct functions, that they actually had distinct use [00:22:00] cases as well. Because if you are an HR doing lead generation for marketing, then probably there's either something off about our understanding from the research or our categorization did not work effectively in terms of bucketing you in the right space.

And so we did find that distinct separation by different functional groups for the most part, with some unique caveats for startups and early organizations where maybe someone's wearing all three hats at the same time. Doing multiple jobs in any case which happens.

Jeff: So you kinda at this point developed, okay, here's the personas, here's the people who care, and here are the people who care, who have a very. Large overlap of what they all want to accomplish, their goals. Was there anything looking at the problems they're facing, getting there, or what is I guess the current state of the world for them or what is their problems or why can't they accomplish these things yet, or,

Aleks: Absolutely. So one of the things that actually evolved that I would recommend, this was follow up to this piece of research. One of the [00:23:00] variables that I would absolutely include in product planning in the future is it is a different problem to solve when someone has frustration around a marginally well met need versus an unmet need completely, right?

Those are two very different environments. And so always asking like, are you currently getting this need met today? Or is this something that you absolutely have no way of doing today? And so you're even more frustrated is a critical component 'cause it drives urgency, right? So when we have. Needs that people are frustrated by, but they have a solution in place already.

It's harder to get them to take action, even though they may still value your solution and they may still wanna pay for it, still wanna adopt it. Your adoption curve is gonna be longer than if it's an unmet need that they don't have a different workaround for, but. It's interesting. One of the other things that we did to click down is there's a lot of overlap, right?

That 75% overlap that we talked about of commonality between features and it makes sense in some instances, right? Because [00:24:00] some of the core workflow elements are similar. So if you think about lead generation, you are gonna collect information from people, you are gonna wanna send them follow up communications, you're gonna wanna analyze information about them.

Similar for talent acquisition, if you're thinking about feedback use cases, right? You're gonna create a form to capture that information. You're gonna analyze that data, you're gonna wanna follow up, maybe take action on it. So some of the core components of the journeys, even though it's different use cases are pretty similar to each other.

The details are in the. Way that they wanna execute. Each of those stages are slightly different and have different constraints that kind of blend into that 25% differentiation across the feature set. So that's always fun to, to manage. But the other piece that I also would not have necessarily thought of initially until we did this research is though they wanted similar features, the maturity that those features needed to be in order for them to find a true value out of them was different.

For example, there were a couple of features where marketers were happy with a much more. [00:25:00] Low maturity solution then let's say, would need to exist in order to facilitate market research use cases or customer feedback use cases, right? And so that was helpful for us for two reasons. One, I think we got to see the true value differentiation between a low maturity and a high maturity version of that feature.

So how much more beyond if we just deliver that low maturity feature, would we be able to generate in terms of impact from a revenue perspective and in terms of what our customer core customers are expecting. But the other piece was from a prioritization standpoint, right? Because now I can help target.

So in addition to having that 75% overlap, the fact that marketers were much more willing to accept some of these features that were less mature, but because they were meeting an unmet need. They were open to just gimme anything. And I will find value in it, which means our time to value is gonna be faster to be able to give them a solution that meets their needs.

But their willingness to pay was also higher across the board for all of the feature bundles that they evaluated. Because I think marketers have larger [00:26:00] budgets. That's not surprising. But also I think their tech stack is incredibly fragmented relative to some of the other populations that we're talking to which is causing extra frustration, right?

So they've got budgets, they've got pain points, and they need less mature components of some of the things that we were building already. And then it's like, okay, it's very clear that we're gonna prioritize marketers as the core group we're gonna build for. We're still building that 75%. It's gonna be valuable across all three segments, but in the case where we need some unique differentiators, we're gonna put those in the marketer bucket, right?

Because we're gonna solve for their use cases since they're the most gonna benefit from them moving forward.

Jeff: And kinda looking at that, how did you go about assessing, 'cause it's one thing to have the kind of like form around what feature is important and the max diff and that kind of maybe broad one to many assessments, but how do you dive into, I'm a marketer, I need it done this well, but I am a customer researcher or a market researcher.

I need it done [00:27:00] five times better than that. What, is this kinda where it starts to get more one-on-one? Or like, how do you kinda get that extra level of detail to, to be able to start to get to enablement level understanding, or, sorry, creation level understanding.

Aleks: Makes total sense, so you're hitting the nail on the head. This research is super helpful for prioritization of the workflow, but there is no substitute for product development lifecycle processes where a PM talks to customers, understands the detail of exactly what they mean when they say these words.

A designer comes in and has that same conversation with the PM and the customer, and then tries to mock up what that process and what that experience is gonna look like, and they try to get closer and closer to what the customer is expecting when they say these things. This process helps prioritize what goes through that process in my mind, but it does, it's not a replacement for it.

We tried to get closer through the research by having two different descriptions in some places where we knew that our maturity level. To get to that ideal end state [00:28:00] was gonna take longer, right? And so we broke it down into two separate descriptions. One was lower maturity, one was higher maturity, hoping that there was some differentiation in giving us a read on how far do we really need to go down this path to bring value to customers.

Do we truly need to wait until the entire thing is done, or are there some pockets of value we could generate sooner? So that was helpful for that. But honestly, Jeff, there's very little that replaces actually having conversations with customers 'cause their use cases are so unique. And people use words in such different ways to mean different things that you, you almost really need that alignment between them and you.

And looking at the solution that you're proposing and seeing, where does this break down for you? Where is it enabling you and where is it breaking down for you?

Jeff: Sorry. We had um, last week we had on the show a woman named Deep De Manata and she's the VP of product at hunger Rush. And she talked about this, but her solve was that they called it just, you have to make the pizza. And it was 'cause like her and the team actually went and they make software [00:29:00] for like quick serve restaurants and they went and actually will work in the restaurant.

They will go, she did a 12 hour double, which I've done a, early on when I was a teenager, I worked in McDonald's and I can't imagine now working a 12 hour double. That's hard work. But she came away with so many, like what she called paper cut problems that were really.

Adding up to one, a couple big outcomes, but it was not any major thing. But it was all because they got there, they worked with the team, they understood, when you reach out to the customer and you're trying to fill in something on the POS system, the window keeps closing on your arm and you drop things and you have like just little things like that.

If we can make like this workflow, three less button clicks, it's gonna be so much easier for that like that's kinda level that you guys got down to maybe not going out and, making pizzas, but actually understanding at that user level, like what they need.

Aleks: Exactly. A hundred percent. It's really hard. This is the question that surfaces often is how important is domain knowledge in product management? And I feel like for me, over my career, [00:30:00] understanding from a researcher's perspective what I needed out of tools like a Qualtrics, like a SurveyMonkey to create those research projects helped me have a more honed in intuition around it.

And product intuition is a real thing. Actually, Lee, my researcher, ended up doing some research on product intuition. 'cause of course, researchers, we doubt everything, right? So we have hypotheses against all things and we have to validate and prove them wrong and all of those things or right, depending on the context.

And so she really was curious. Do people with higher domain expertise can they predict consumer responses to certain. Investments in product strategies or reactions to concepts better than PMs without that domain expertise. And she found that intuition, that honed intuition that's driven by repeated exposure and domain expertise is a real thing.

Like they, PMs who had that were able to predict what consumers were gonna say about particular areas within their domain twice as well [00:31:00] as PMs without that domain expertise. And so I think it's critical. It can be built, of course, but I think the, how you build it is you make the pizza, right? You get into.

The workflow, the pain points, the challenges that your customers are actually experiencing, and you try to see it from their point of view in detail, right? So it's not enough to just have a 30 minute call where you ask them for feedback or you ask them at a high level what their pain points are. It you need those coworking sessions where you're really trying to unblock them together and find solutions to what is causing them pain.

Jeff: Yeah. I remember, I can't remember for life of me who said this now, but it was so smart when I heard it, which is taste is. The ability to know something's going to work before you launch it. And that I do firmly believe that exists, and I've seen it where you can definitely, if you have enough understanding of who you're making it for, what they need, what their pain points are, you don't always need to go out to the market or what you find from the market might actually be wrong because you're not talking to the right [00:32:00] people, or you're not talking to you're not finding out in the right way, versus if you have that learned experience.

You said as a researcher, you knew a lot of the problems that would come up using some of those tools and what you needed. So you could potentially, because you had internal knowledge, you could do better than trying to go out and siphon through and take secondhand feedback from even the people living it.

I love that she actually went out and did that is such a researcher thing to do though. I gotta go prove that my intuition is right.

Aleks: hundred percent. I love that level of commitment, when we have our personal points of view that we wanna validate or not validate, but just to just that, even that intellectual curiosity to make sure that our hypotheses are right or wrong, right? Because that's a good check for ourselves.

Sometimes we, we carry lots of decisions forward from our inherent biases or our perceptions, or our hypothesis about how things work. And so it's really nice to have people on the team that are willing to double check some of those things rather than just assuming that they're right and letting it

Jeff: Oh, a hundred percent. I love the obsessive quality there of making sure it's right. [00:33:00] And now I do want to get into how you guys then took all this output and go for it. But there's one last thing I wanna make sure we touch on. 'cause it, it was such a neat concept that I had never thought before, but this idea of feature constellations came out

Aleks: Oh yes.

Jeff: And this is just, I have been doing this for 20 years and I've never heard this idea, but it, you said it and it was like a light bulb went off and I was just like. How has, how is this not a bigger thing? This is so obvious in retrospect, but it's not until you hear it.

Aleks: Yeah, it was another interesting thing that surfaced for me from this research, which was , 'cause of course we were planning on having these features stacked, ranked against each other. And of course we were gonna get a value score for each and willingness to pay score for each. And we were gonna be able to model all kinds of financial impacts from that. But the thing that we were lucky enough is to, of course, have an amazing researcher on the team that was able to think ahead and notice that there were certain patterns that were emerging in the data. And so then she ran certain models to get a sense for, wait a minute, some of these features feel like their willingness to [00:34:00] pay and value increase together o versus like separately right at odds with each other.

And so she ran separate. Analysis where she combined some of these features into what we would call feature constellations or groupings that help unblock a core workflow or a job for a customer. And we found that in some places, the individual feature value was fine, right? It was people were willing to pay for it.

But when you combine these two features, they're willing to pay five x, 10 x what they would for each individual feature. Because you're streamlining the workflow, you're unlocking additional value. And I'll give a good example of this, so. Last year, Typeform released video in Typeform, right? So you can create a video question.

You can also create video answers where I can in a form ask you or a bunch of people to submit answers to me. And there's an option where you can submit with a video answer. And so video answers had a decent willingness to pay, but when combined with [00:35:00] AI analysis of themes and sentiment and topics and things like that, all of a sudden those two capabilities together had a much higher willingness to pay.

And you can imagine why that would be the case, right? Because if I'm getting. 50 to a hundred videos without any ability to really extract context and be able to analyze them at scale. I'm left watching 50 to a hundred videos, which makes me feel like that's gonna be extra work. It's still valuable, it's just not quite as valuable as layering and additional capabilities that then help me extract that true value from that video question capability in the form.

And there was a few of these interesting feature constellations that helped us from a prioritization standpoint, right? Because maybe what we would've done is we would've. Released video questions without this capability because maybe it's stack ranked a little bit higher relative to other features in our prioritization.

But now that we understand that some of these features can work together to unlock real customer value, we would prioritize [00:36:00] differently. So even if one of them individually has a lower willingness to pay, we're using that aggregated willingness to pay that's higher based on, what both of those features can bring together for prioritization.

Jeff: I love the example you used there too, because this is something we have seen I have particular expertise on this topic right now because session replay right is the same problem as video answers. And that's what Log Rocket does, right? It's you have instead of 10, 50 or a hundred.

If you're a large company, Typeform size, you have millions and millions of session replays, you're not gonna watch that. Lee's not gonna watch it. Lee sounds like an awesome, just get stuff done person. But even, even Lee's not gonna watch millions of sessions.

But we realized that, and this is kind something, kind of something we understood a while back and we've building two sense is if we could build the tools to understand, right?

Everyone has a heuristic, there's rage, clicks, dead clicks, all that kinda stuff. But it's really noisy is law. False positives. If you [00:37:00] can build, first of all the model to understand what are the few ones that really matter, what are the few ones that really signaled someone having a problem, in a session, not being able to check out or not being able to complete an onboarding flow or complete, maybe a form in Typeform.

Then how do you discern those from the 99%, which is just like. Yeah a link didn't load in the footer for the T's and C's and no one clicks on, which is pretty often. So we started there, but it, then it became, building AI vision models that can actually watch the sessions and tell you where people are getting tripped up, right?

Oh, the user doesn't understand that you want to multi-select here. That's not clear. Or we've run into things where it's in these three form, screen sizes, the CTA button is off the screen or not in an obvious spot, or like all these really deep things that only come from like going beyond event stream analytics data, but actually understanding as if you had, 10,000 leads all watching everything but a magic hive mind of Lee to then aggregate it all together.

So I a hundred [00:38:00] percent understand your thing and can see where that example is. Like I would pay, 20 x for that. Knowing the value you can get out of that versus having to manually watch even a hundred user, answers to understand and then correlate it all together.

Aleks: Yeah, a hundred percent. It's interesting as hindsight 2020 because you'd be like, dove, of course, some of these features like would just cluster together and create additional value propositions. But it was just fortuitous that we structured the research in such a way that we could actually model out and actually estimate what the willingness to pay together would be relative to individually.

Because then it starts to change the kind of conversation you can have with the rest of the organization. Because a lot of this research is helpful not just for informing the product strategy and the product vision and how you're going to execute and move your strategy forward. But the other piece that I think.

Sometimes research is underutilized for, is really in bringing the rest of the organization around along for the journey, right? Because it's one thing, if it's my opinion or my point of [00:39:00] view, even though I have curated intuition across, being a form user survey, user researcher over the course of a career, it's still my opinion in the eyes of many folks within the organization.

But if it's corroborated by research and a point of view that is bringing forward what the market is surfacing, then it changes the game in terms of what kind of goodwill and intention and alignment you can drive within the organization.

Jeff: exactly. It's not just having Right a, I think you should just bring around Lee's report with you and be like, no intuition. I have certified intuition, so you should just trust my opinion. But on top of that, here's all the data backup, but it in all serious note, that really is important to have people understand this is the roadmap, but here's the reason why.

Here's the reason why this is gonna help us win. Because you don't get ahead by having just a product team believe it, or even just a product team and engineering team. The company has to be aligned. Behind, how are we going to get from our current state to this future state that is so much better. And part of that is, is people need to be excited.

[00:40:00] They need to believe, you need to put the Ted lasso sign up and, but they, sign or not, they need to actually do it. Cool. So this is fantastic, right? So much knowledge. I guess one last question. These ideas of feature constellations, is there something people can do now?

Aleks: Knowing, knowing what you know, is there, how can people go into this process and set themselves up for success to be able to identify these kind of constellations If they are there for them? 

I think \ , it depends on the type of insights that they're gathering, right? So , if you're doing market research, just pull apart concepts and remember to do that kind of an analysis to see if there's overlap in both the ICP that is saying multiples of these things are interesting to them.

Aleks: And maybe even, you can do some of this AP priori, right? If we had thought in advance that these things could be combined together at an additional value we had a sense of course that, that they would be but we could have structured specific questions within the survey to be able to elicit, how much some of these features working together would actually solve for core parts of [00:41:00] the workflow.

But just keeping it in mind and making sure that you are you're trying to find opportunities where. Things ladder together. And you're not just doing a blanket prioritization of individual features, right? Because even if something might be stack ranked higher across your different metrics and the goals that you have, know that there's probably something else that would make that feature stronger.

It may be lower on the list, connecting them together has real market value, even if it's harder for you to articulate what that is. So it's worth doing the work to make sure you can uncover as much of what that is as you need in order to be persuasive within the organization. Right?

Because that's the other thing is sometimes the perception of some of these things is a little bit more challenging than the reality. And there's a lot of pressure within product organizations to meet the needs of the customers, to meet the needs of support teams, to meet, make sure marketers have something cool to talk about, make sure sales teams feel like they can sell the right thing, right?

So it's a lot of different customer and sub customer groups that you're [00:42:00] serving. And being able to use things like this to be able to articulate why you're prioritizing certain things over other things, I think can be really helpful. Because I don't think anybody's gonna tell you no don't do that, right?

Even though it's gonna generate 10 x the willingness to pay, don't do that. Do this other thing that's gonna be half of that from a value perspective.

Jeff: And see this is where intuition and research knowledge come together. 'cause you have to know how to look at it to find those constellations and a little bit of. Like these things ladder together to make incrementally, exponentially more value together.