Roman Gun audio
===

Emily Kochanek: [00:00:00] Welcome to the LaunchPod, a product management podcast from LogRocket. Today, our guest is Roman Gunn, Vice President of Product at Zeta Global, a cloud based marketing technology company that focuses on multi channel marketing tools. Before Zeta, Roman's eclectic background includes chief innovation officer at Fevo, senior product manager at Submersive Media, founder and head of product at GivingBell, and so much more.

On today's episode, LogRocket's VP of Marketing, Jeff Wharton, talks with Roman about addressing real customer problems with AI, utilizing a work back plan focused on a design North Star, how Rick Rubin influenced Roman's own first principles thinking, and how to create products that can be elegant enough to solve everyday problems, but flexible enough to also solve the advanced problems.

So, here it is, our conversation with Roman Gunn.

Jeff Wharton: Hey, Roman. Thanks so much for joining. Great to have you on the show.

Roman Gun: It's a [00:01:00] pleasure to be here, Jeff.

Jeff Wharton: you know, I think people I've talked to, I've you have one of the cooler stories about how you actually got in, right? There's always people at jobs, people did this people, but , I think you're the first person I met who. Got in via propaganda can you maybe,

Roman Gun: call it manifestation.

Jeff Wharton: yeah, exactly, there you go, can we talk maybe a little bit about that and just want to lead in with, how did you get to where you are now? What's the story that got Roman from back in the day to now,

Roman Gun: Yeah, so it all started with a resume that I was working on. So I was fortunate enough have some really great designer friends. And one night I said, I want to make a really awesome resume. Then we worked back for what is a resume? And we landed on really it's essentially a piece of propaganda.

So that resume evolved into a giant propaganda poster featuring my silhouette and we just ran with it. And the tagline on it was, who is Roman Gunn? And as we started evolving that, it just was a lot of fun. So we started creating propaganda posters based on various historical time periods, like [00:02:00] Soviet union type propaganda posters, like Gandhi's posters, Uncle Sam wants you posters, pretty much like all like the greatest hits of history featuring my silhouette.

Just. Proclaiming Roman Gunn wants you, essentially.

Were you in PM at that point or, how'd that tie back to the job, 

Roman Gun: So at that point, I was doing quite a bit of freelancing, some freelance copywriting freelance, what's called producer work from the agency side. And really evolved into being a true PM at that point.

Jeff Wharton: so that, segued into some more design roles, it sounds and then , how'd you work from, Design into kind of more what seems to be a more product focused role. Now, 

Roman Gun: it was really marketing focused. And because I was so close with a bunch of great designers that I'd brought into the company so, I started working with them to help them. Design out what was happening. And because I started putting together the requirements, I started being the one that communicated that to the engineers.

And as I was being the one that was communicating with engineers and the designers, I was then communicating [00:03:00] right back to the stakeholders and investors. And I evolved from there.

Jeff Wharton: I can appreciate a good straight line story, but it's always fun to hear that, I designed a propaganda poster and here I am now working at, Zeta global doing all these, generative AI projects and stuff like that. And, the in between was a whole bunch of curvy roads.

It's,

Roman Gun: that's right.

Jeff Wharton: That's, it's the journey that gets us the interesting stories. Right. Now talk about Zeta had the good fortune of having you on the spotlight interview series earlier, and, I was able to look back on that and there's some interesting points.

I think that came up in that you've referenced and other times you've talked about how you run projects. But do we want to kick off with what does the role look like? What are you focused on right now?

Like what's getting you going right now?

Roman Gun: Yeah, I think right now we're really focusing on how do you take AI and distribute it to the people? Like, how do you take this concept of creating agents and letting the everyday person essentially create their own AI assistant to do the jobs that are important to them instead of the [00:04:00] jobs that the product manager has prioritized or the engineer has built or the designer really wanted to get in because it was cool and shiny.

Hey, how do we really get the people, the foundation elements they need to achieve their job?

Jeff Wharton: right. And I really liked the part you talked about where you start with a work back plan. And I think I get what that is. I, so I'm not in product. I've, we always talk about, we run our marketing team here, like a product and we sell to product managers and product teams a lot.

So I've talked to more than my fair share, probably of PM leaders, but can you explain to me a little bit like this idea of a work back plan and what does that mean? And what does that mean in practice too?

Roman Gun: So I love to start with something called a design north star. So essentially work with a great design book to create a end state that we're trying to reach. What is the ultimate thing that we're moving towards? And it doesn't have to be perfect. It doesn't have to use all our standardized components.

It doesn't have to have. All the answers in terms of deep flows, but it has ultimately [00:05:00] where and why we're going. And because of that, it brings the reality of the end state much closer. It starts to one, feel more attainable and feasible. And two it gives a common language and something for people to point towards.

I think that one of the things that I've learned in my career is that. It's unfair to make people imagine at times, you want to give them something to reference and by having a visual aid that they can say, Oh, this is one polished to this has all the elements that I want and three, like, how do we get there?

Now? I think that's the thing that we really try to build. And then based on that we then start creating it. Different phases of how do we build towards that, right? So we start figuring out what are the most essential solutions, what are the solutions that scale into other parts of the product, and essentially, the entire architecture of how it's going to work.

But it all starts with, this is where we're going. Because without that I think a lot of companies fall into what I call the incremental march to nowhere. They just [00:06:00] build and measure and build and measure, but where are you going? It's great that you moved like another mile, but a mile towards what?

So that's what we want to avoid.

Jeff Wharton: Yeah, it makes sense. And, when you're building that work back plan, , what are you working back from? Cause there's always, I think given the nature of products you work on, you could be looking at where you're going to be three years out. Where do you want to be a year out?

There's always this kind of idea of what's critical. And then there's all the add on things that you could add on, starting from. The end state what's the minimum we need to do to launch what we need to launch initially and then go over?

How does that work? How are you looking at all the additional things you can do afterwards?

Roman Gun: Yeah, so we start with the end state and then we figure out what are the most impactful points in getting there. What are the things that the industry demands right now? And what are the biggest opportunities for us to grab? Because I always find it's interesting to balance between the necessity, the function, and the sizzle, right?

You have to deliver both in lockstep because you're gonna be missing out on different biotypes if [00:07:00] you don't focus on both. So I always try to think of both the tactical and the strategic. So the way I talk to PMs about this sometimes, it's like, Look, you have to think about 60 to 70 percent functional epics that people aren't going to necessarily like celebrate or share, but it's going to make their life significantly easier.

They're going to be locked into your ecosystem. And the other 30 to 40 percent is going to be the things that cause people to buy your platform in the first place.

Jeff Wharton: That's fair. And do you have a process for how you understand what those different elements are? You talking to customers? Is it gut feel? Is it a little bit of both or something else?

Roman Gun: Yeah, so you definitely need to work with individual business units you have. You definitely need to work with the people who are in the weeds actually using the platform because those things aren't aligned always. And then you do have to intuit a bit of where things going to go, right? Because it's one thing to build what a competitor has.

It's another thing to say, I want to be better than a competitor. And you can't do that by copying. You have to do that by finding [00:08:00] inspiration. And sometimes that inspiration is It's best found, not within it, within the competitor, it's best found within an entirely different space. Because I think one of the things that I've appreciated working in an agency environment earlier in my career is being able to see how much of the problem sets we try to tackle are similar from industry to industry.

There's a lot of, if you do some, good pattern matching, like 70 percent baseline of problems are the same. Like, how do we like, templatize things and scale them and make them repeatable and we operationalize the folks that are hands on keyboard. How do we keep the strategic users happy? It's same from industry to industry.

The difference is the working solution. So if you think about those different pillars that you have to hit. You have to pattern match against it. And now that you know that the pattern's there, you just look at like the most interesting and compelling and easy solution, even if it doesn't come directly from your field.

Jeff Wharton: Having experiences across all sorts of things. Like we said, the windy road gets

Roman Gun: Yeah.

Jeff Wharton: spot. When I was [00:09:00] reading, our background here, you have five pods over at Zeta.

What are they focused on? It's not just all generative AI, is it?

Roman Gun: No, it's not. At Saga, we actually have before the pod layer level the layers. So we have the intelligence layer, which is where all the pods that I had a bar. Let me also have the experience layer and the data layer. So the data layer is all about how we ingest information, how we make sure that there's great governance around that great security around it, et cetera.

We have the experience , layer for how do you execute on all these things? How do you actually create segments? How do you send your emails? How do you target on a CTV or other display channels or direct mail, et cetera. Then you have the intelligence layer, which is how do you combine all the data that's ingested into something that's meaningful and actionable and in order to do that, we have things like analytics.

We have things like forecasting. We have things like recommendations. We have things like personalization. And we have things like MLOps and generative AI. So those are the pods that we're tackling. And the [00:10:00] next pod that we're going to be standing up is going to be splitting off attribution from analytics into its own dedicated space.

Jeff Wharton: I won't get on my soapbox today, but attribution is one of the ones that I have lots of opinions there. 

Roman Gun: Yeah, especially come this summer when the cookie less world kind of crumbles. We're going to have to figure out how do we really attribute it in a meaningful way and what's the right point of view to have in a world where people do want to feel like they're secure, that their privacy is being respected, but they also still get the relevant type of marketing.

To them. So that's something that we really have to focus on. And how does the marketer really credit the right channel? Because, you have last touch models, you have first touch models, but, all those models are out of touch. So we have to figure out the right way to do it.

Yeah,

Jeff Wharton: 10 plus companies at this point, over 20 years of work in marketing, and no one does it the same. But I don't think any of them were dead on wrong either. So it's just, [00:11:00] it's a passionate topic of mine, but um, I digress.

Roman Gun: but I think this is right and this is going to be the big challenge. I think again, come this summer, we're just like, is this is an opportunity for people to unite on? What is the formula for this? Can we standardize this? Lots of other technologies, whether it's USB C or whatever have you, they've said, Okay, enough of all the wires.

Let's pick a universal standard. Maybe it's time for attribution to do the same.

Jeff Wharton: I would be one of the happiest people on earth. I think if that happened , my background is marketing operations, demand gen growth, and then brand came later. So this is something that has been near and dear to my literal daily life for 20 years.

So, um, 

Roman Gun: We're hiring. So you have like three PMs that you want to come on over for this let's do it. Let's solve the problem.

Jeff Wharton: I would love if you guys sell the product. Let's Let's stay close on this because I want to make sure I want to hear what you guys are working on. But in that even, with those different pods you have, I assume, and especially given other teams as well, people don't operate or people can't possibly operating at the same cadence.

So while you have this kind of work [00:12:00] back plan, Yeah. does it implement or what does it look day to day different kind of between, some of the teams that maybe are doing like analytics is, I'm sure it's been around a while, it's a little more advanced. There's a lot of stuff built there.

Gen AI or, some of that stuff is probably a little bit more new and you're figuring out what you're doing. Do those teams operate differently under you? Do you try and keep them in the same process or how does that work? How do you accommodate both?

Roman Gun: So I embrace freedom. I think that you always have to meet deliverables. You always have to meet towards a business North Star and you always have to be incrementing. 

But you have to do it in a way that makes sense for the team and product line that you're working on. To your point, the big difference between analytics and Gen AI is at once R& D and development.

You're executing and figuring out how to execute all at once. It's interesting because one of the partners that we have that we work with is Langsmith or Lang chain for going and creating our agents. And, when we started on this adventure almost a year ago we were contributing.

To their open source library because we [00:13:00] needed these features, right? So you're like developing and helping the frameworks build in real time as you're building them because it doesn't exist, right? You're literally weighing the groundwork that you need and that others that are going to come after you are going to utilize as well.

So it becomes a very difficult an interesting challenge, which is different than, analytics, which is okay, great. We have all these pipelines. We have all these metrics and dimensions. We're going to be ingesting data from 120 different sources for an individual client. How do you build a data mapping tool to standardize all these things and report on them?

I don't want to say that one is a solved problem, but it's a much more understood problem while the other one you're developing solutions on the fly. Because of that, the process and how you work with a team is going to be a little bit more flexible, right? So for analytics, maybe your PRDs are super concise and you're like, no, this is exactly how we're going to aggregate.

This is the exact universal formula we're going to work on. This is exactly how we standardize whereas with something that's a little bit more recent you're not going to [00:14:00] have that luxury because , one, the technology is evolving. Right underneath your feet. Like we build with the assumption that one, everything that we build has to be modular because there's going to be a better version of it in the next three months that we can plug and play and replace.

And two, we can't think about the tooling that exists today. We have to know where we're going and build understanding that everything is. going to be imperfect 

and it's 

going to evolve. 

Jeff Wharton: Yeah, it's, it's so fine because I can relate really heavily to that. , here we've been continuously building out our analytics platform. And as one of our own key stakeholders is our marketing team. We're big consumers analytics. And so we've worked closely with the PM team here, which is, which has been great.

But there is definitely a cadence of just, we know we need to have these kinds of graphs. We need to be able to query these data points. And it's just. Analytics is a bit of a laundry list at sometimes. We also launched what we call Galileo AI, which is our. AI, basically it's, the function of the way we think about it internally is, it's basically you have a PM, a [00:15:00] junior PM who can watch every single session within, our product.

You can watch every session replay. It analyzes every user path and services patterns and behaviors and, problems to you and to your product team. And that's been the really interesting thing 

because I talked to that product team a lot and they're always, there's always just something new that you're blows your mind about what they're doing.

And the path to get there is often, like you said, we have the work back plan and then how we get there is a little circuitous analytics is just, yep, this week we're going to deliver this, then this. 

Roman Gun: Yeah, analytics is a map.

Gen AI is a little bit more GPS based. We know the destination, this street's going to be closed. That one's going to be under construction and a superhighway just got built right in front of you. Go, go, Go.

Jeff Wharton: Yeah. It's a little bit of a super highway, but also, an odyssey at the same time, you know, about where you're going, but the route to get there is a little circuitous and you got monsters on the way and adventures and all sorts of stuff, but it's going to be, it's going to end up well.

That's super cool to hear. I got to ask you about this. , cause when you were going through and talking about your process Yeah. You referenced a thought [00:16:00] leader who I don't hear talked a lot about in normal books or uh, PM. I'll be honest, it's not talking about a lot of PM, stuff I read Rick Rubin and his whole idea of basically, reduce, get to the essence, get to the core truth Where did that come from?

And how do you actually apply that? What's that look like in practice? I agree with you, by the way. I think it's a great reference, but I didn't expect it. I'm going to be honest.

Roman Gun: That's that lateral thought for you. I think it's It comes back to the essence of what you're solving, right? Some may call it first principles, but you're really just looking at what are you trying to achieve? And what are you trying to achieve if you forget the weight of what's already been?

I think a lot of people, and this is why again, I love the visual North Star that we build and not being constrained by components that we have or processes that we have, because it's about what are you really going after? What is the truth of what you're trying to do? I think a lot of people they start with.

These are all the constraints I have. [00:17:00] And then let me work from there. I think you have to start by treating PMs, engineers and designers as artists and artists need to start with white space. And then as the time draws near, you can start closing in the space and layering in more requirements, laying in more constraints, layering in hard deadline instead of a soft deadline.

And all these things allow for the artist to then really channel their inner muse. Because I do think that it's interesting that people start coming up with solution sets that are really similar in different companies in different parts of the world. All around the same time, it's because the information that we're taking in is universal in a way.

Why not let the people that you hire to be the experts of that wield the universe in their favor?

Jeff Wharton: kind of love what we can extrapolate from there is everyone views, first principles thinking is this, kind of San Francisco startup VC thing. In [00:18:00] reality, it was a rap pioneer. 

Roman Gun: It always comes from counterculture.

Jeff Wharton: I mean, That's the thing where you look at all that kind of stuff.

Like you said the idea of lateral thought is so important, even, micro and macro I avoided my soapbox once on attribution, but to quickly tie back. I, like I said, I grew up through the whole demand gen, UTM codes and, multi touch attribution, and it wasn't until I started to, to.

Get into more leadership positions and thought about brand and all that kind of stuff. When you look at it and you realize, and I think , my push towards brand and, actually being a marketing, executive here who ran a whole team, had to think about more than just payback was, I realized this idea of complex attribution didn't really matter when you got to brand.

There's a couple core things that people will come to you because they're looking for a product. And the big thing they come to is because they've heard of you. And brand is like the overwhelming. Tailwind that can destroy any other attribution metric. And it was just thought just getting outside my little bubble, even just a little bit, I think maybe so much better at thinking about my role [00:19:00] there and curious to see how that's applied and worked with you because I.

I would wager, given you're more kind of design background and some of the other stories is there's some good stuff here.

Roman Gun: Yeah, I love what you just said about expanding out. So the way I talk about it is having different levels of zoom. And I think that you need to have different levels of zoom depending on the stakeholders that you're speaking with because you can have the exact same project. But the way you're going to speak with a junior engineer who really needs specific requirements is different than the way you're going to speak with a designer who wants to be motivated to create something beautiful.

Okay. Which is gonna be different than the executive that you're speaking to that wants to figure out the ROI on a team that you put together and how is this going to impact our market position versus competitor X, right? And you're still building the exact same thing in every scenario, but the point of view that you have.

It is completely different because it speaks to the different elements the macroeconomics of the market of the way that your brand disposition any sort of business headwinds or tailwinds and [00:20:00] the reality of teams. And I think that one of the things that you have to do with teams is you have to be.

Their tailwind, right? Because I think that if you allow the topmost level of zoom interact with the bottomless level of zoom I think things don't always go very well, and I think that's a great PM's job is to essentially be the umbrella or the shield between those layers in a way that everyone gets communicated what they need communicated to them and that you can measure the success in a way that's meaningful to all those people at the day.

Jeff Wharton: I love that. You gotta be your team's own tailwind. I feel like once in a while you meet people who are great putting out like a big view into a tiny soundbite. And I feel like there's a few of those with you. 

Roman Gun: That's what I'm here for.

Jeff Wharton: well, I love talking to people like that.

Cause it's, I think at some point. Being able to do and lead is important, but also, if you can boil things down, like you said, with, the Rick Rubin idea, boil it down to his essence there's just a lot of power in having those kind of couple core, really tight operating principles and be your team's tailwind, I think is something I'm going to, [00:21:00] I'm going to have to add that to my own repertoire 

Roman Gun: Thank you. So I can't credit what I'm about to say, but I think all of this started with me hearing the expression brevity is the essence of elegance. I heard that and it really stuck with me. I don't know where I heard it, but I know I was fairly young when I did and it just latched on. So that's what caused this.

Jeff Wharton: I always tell people I think, I'm probably get skewered if this is wrong, but I think it's a Mark Twain quote about, I would have written you a shorter letter, but I didn't have the time and I always love that kind of stuff. I think you had another one that I read and I don't know if this is, maybe I just picked this out of something you said, or maybe you say this, Frequently 

but you talked about basically like as PMs, you stitch the superhero cape, but the users are the ones who are wearing it. , what do you mean by that? What does that kind of mean, to you? And how does that come out in the kind of end use?

Roman Gun: Yeah. So I think that you can build features that have a linear path and all features should have a very specific use case that they that they [00:22:00] solve for. And everyone can use it universally in the exact same way. But I think what makes a feature powerful. is when you give it enough flexibility to become the tool for a marketer or ticket selling rep or a I don't know, a journalist or anybody to wield into whatever use case that they have.

I think the goal is to build tools that are elegant enough to solve the everyday problem, but flexible enough to solve the advanced problems. And I know some people really hate the terminology of having an advanced or pro user. But that's to say that there's always going to be a variety of use cases.

For instance, this agent ecosystem that we're building. The goal here is, obviously, you can, create an agent that's going to help you write a subject line. You can create an agent that's going to help you write a PR break. But what if you write an agent that helps you automate your marketing experience?

What if you write an agent that helps you de dupe audiences? What if you write an agent that helps you [00:23:00] forecast merchandising? These are all use cases that people have. If you build something that is a building block, something that is a sandbox that other people can build upon, then you have much more compelling experience is built into your platform.

So it's not just the junior person who can get their entire workflow done quickly and consistently. It's the person that now has the impossible deadline that they have to hit that can do it. Those are the types of features that I love to build.

Jeff Wharton: and that kind of brings up just the general something that I have gathered very quickly. you know, I I think the same way I have a lot of opinions and attribution. I will talk about them at length. If people don't stop me. I think AI in the kind of just the explosion we've seen lately.

And this is somewhat that of yours and, I don't disagree with you there. I, that's another danger topic I can go on for a while. But I really liked, you talked about like an AI agent is like a secret agent. It's all about the tools it has access to. So I guess AI in general, like what's your view and how is this best use?

Like what are we going to see in the next couple of years here? Do you [00:24:00] think?

Roman Gun: So I think what we're going to see in the next couple of years is There's going to be two different things we're going to see. We're going to see the human side, and we're going to see the technology side. I think from the human side, we're going to see something interesting. I think that over the last 30 years or so, the term Luddite has meant laggard.

It's meant the person at the end of the adoption cycle of a technology. Just because they didn't want to, for the most part, not necessarily because of ethical concerns or thought around what this, what is this going to mean for my career, anything of that nature. And I think that's shifting. I think that the term Wadai is actually being wielded in a new way now by a generation of folks who are very aware of what this technology is capable of.

I think they're actually wielding it in a way that says, Hey, I understand what this can do to industry. I understand the power here. And I think we need to have conversations around what is permissible and what isn't, and how do we roll this into society? So I think that's a really interesting conversation that we're going to keep experiencing over the next [00:25:00] several years.

And I think you're going to see people do one of a few things. You're going to see the people who just adopt it easy. Great. You're going to see the people who offer up some resistance. And then they start playing with the tooling. And they realize, Oh, wow, this is going to make me way better at what I do.

So let me put this into my toolbox. You're going to have people who play around, understand what you can do and say, Oh, no, I need to reject this because this is going to be no bueno. And then there's going to be the folks that are just going to be the laggards because they're laggards, right?

And I think the both sides . Of the let's call it the spectrum are easy to figure out what's going to happen there. I think the middle is going to be interesting. I think the people that adopt this technology are going to find a way to up level their careers. And I think there's going to be a career reshuffle that happens in the next few years because there's going to be people that utilize this technology and elevate themselves.

Simply because they decided that I'm [00:26:00] going to. And they're just gonna be folks that fall behind because they decide not to and I think that it would behoove everyone to just say, Hey, it's here as Government doesn't always work so fast to regulate, so let's let's do the best that we can with understanding and utilizing the way to help ourselves, and I think we're going to see that play out in, in many different ways over the next few years.

From a technology standpoint I think it's fair to say that OpenAI is the biggest game in town right now but I think it's going to diversify over the next few years. And I think what we're going to see is, it's not going to be around who has the best model to achieve x. It's going to be who's great at stringing together models to achieve x, y, z. Chaining together agents, chaining together models, that orchestration level that's going to be what crowns the king in this space. And I think that's going to be super, super exciting. And I'm very particularly excited for creatives in the coming years. I know. There's still some fun [00:27:00] and discomfort there with a certain subset, of creatives, but I can see some of the best ad work, like of decades coming out shortly because people that traditionally don't have access to tooling are suddenly going to have access to really powerful tools.

And I'm very excited to see what comes out of that.

Jeff Wharton: I feel like what we've seen so far here has been, it's just such a wide spectrum, right? You have people saying we can just generate content and this kind of stuff. And you can just hold on, just put the whole job onto the AI. And that I think has worked medium well at best. Those people are coming up with like very focused use cases or, enabling.

Looking at bigger sets of data or, augmenting human processes. And that's been at least what I've seen is some of the best outcomes has been, how can I brainstorm through a hundred different ideas really fast? And it gives inspiration points or how can I pull themes out of a huge set of data?

That's, I think what I talked about. Our focus here, product wise has been, how do [00:28:00] you basically give this tool to, have a machine watch every user session if are recording 10 million sessions and you need to pull out insights and themes from that kind of stuff.

But also we used it, actually, we used it to help us name the podcast. We had several ideas and gave it the theme and it ran through and most of them were terrible. But it did give us those kind of themes of like, Oh, we didn't think about this way or that way. And dove in it, but it sparked some good conversations that I don't think we would have otherwise had that in the end.

A few of us really sitting together and talking through and bring the human element, got it done and got us the final kind of outcome there. But it was, it sparked us and helped us move, I think, faster than we otherwise would have.

Roman Gun: Absolutely. This is a collaborator. This is a hundred percent, a collaborator and not a replacement. It's just the same way that, you know, people use Photoshop to edit photos, right? It's just something that helps you do the job, it doesn't take away the job.

Jeff Wharton: yeah, I think I heard you referenced it once. You called like the command line for people. I thought that was just so smart. , it doesn't do anything by itself, but can help you do stuff faster or, it's another set of tools that can help [00:29:00] you operate faster or augment.

But to go back real quick, you talked about chaining agents. A series, , and I've not here, but in my own kind of just like personal experimentation with this stuff have, tried things around this and what I've always seen is it's cool. And theoretically should work, right?

You have one tightly bounded workflow, another tightly bounded workflow, another tightly bounded workflow, and just think like iterate iteratively. It should work well. And you're controlling risk, but, you always run this thing of, you know, I think about it similar to how you think about experiments in a business when you're forecasting.

Stacking risk is not just 5 percent and 5 percent and 5 percent and 5%. They compound on each other. So how do you see that evolving where, if you're stacking a bunch of, AI workflows or AI agents, each one of those is kind of exponentially potentially increasing error or something like that.

Roman Gun: So I might be giving away some secret sauce here, but I think a part of it is that every subsequent agent has to have the context of the agent before it. Right now, people are [00:30:00] saying, Hey, Agent A is really good at Task A, Agent B is really good at Task B. So when Task A is done, pass it to Task B.

But it needs to refer back to what happened in Task B to properly understand what to do next, right? It needs to understand that it's a part of an assembly line. And it also needs to have that data passed over in a way that is meaningful and parsable for the LLM. So I think that there's the context that needs to come in and there's also what's that standardized way of actually chaining.

And I don't think that there's a great standard for it yet. And I think those are things that are going to evolve.

Jeff Wharton: That's fair. Yeah, . I think the workflow I've heard people reference. I don't have experience with it, but is can you ask the previous agent how confident it is in the answer?

Um, 

Roman Gun: there's definitely the confidence level perspective for sure. But I think a part of it is if you consider an agent to essentially just be a simulation or any LLM interaction to be a simulation machine that figures out, [00:31:00] hey, I am explicitly this, out of all the things that can be in the universe, I am this.

That this has to understand and be an auditor of the prior this. It needs to like, within its context, understand that your job is to audit the prior step. And that's a problem that fall you get context window issues there and whatnot. But I think these are all very solvable problems.

Jeff Wharton: So if we're looking at where we are in this kind of roadmap or whatever odyssey we're on of AI, it seems like this year is going to be a year where, if last year was chat application, you know, chat interface this year, it looks more like everyone's going all in on agents.

Things can go wrong there, but everyone can make one, which isn't necessarily a good thing. What does that lead us to? , let's close down with what does that mean when everyone can make an agent?

Roman Gun: It means a lot of them are going to be garbage. And we're going to have to get, I mean, really, that's what it means. And I think we're going to have to create a standard for figuring out one good prompting into what makes an agent successful. And I think that, frameworks [00:32:00] around the best way to prompt right now are a dime a dozen.

But I, I don't think that there's a framework for chaining agents. I don't think there's a real framework. For how to get confidence levels out of these things that are accurate and how to make sure that every single one is fine tuned. I think those are all things that have to evolve. And I think, again, the same way that Anytime a new technology comes out, there's going to be lots of people that come into the space that get VC dollars and that die out, and there's going to be one or two that are exceptional and are going to, be the stewards of the space.

We're going to see a lot of that and I think it's really exciting I think that, just seeing I, I forgot what the name of it is, Sora the new video model that, that's coming out. These things are evolving so so quickly uh, that there's always going to be something around the bend.

And that's why I keep saying you have to build with the perspective of, I am trying to achieve X. And I'm going to make it modular so that when the right model for doing XYZ is there, I [00:33:00] can plug that in. So I think now more than ever, you have to be firm on where you're going and why you're going there and you have to be okay with the Details and technical implementations being fuzzy at the moment or imperfect at the moment because they can be there, but they're not going to generate the content quality that you want.

Constantly get asked to, added image generation for brands into the product. And I'm just like, it's not there yet. You can keep asking, but. It's just I don't feel comfortable releasing a product that is going to be suboptimal. And there's great players in the space that are getting closer to being able to do things, not just like, oh, create a wartime photo from 1978 that looks like XYZ, right?

But like, oh, create an actual template for this type of thing and make it actually take in the context of brand assets, take in the context of the user types and all that. It's still not quite there. I will say that Majority is probably the closest. But [00:34:00] yeah, it's an evolving space. Know where you're going, know why you're going.

Increment there and then plug and play the elements that aren't performing today. That's essentially what it is.

Jeff Wharton: it's our version is our current version of uh, people used to build for where Moore's law predicted we were going to be by the time it would hit. So , someone will have to figure out a pathway to predict this advancement and. All that's people can do it, but yeah, no, I think that's a great way to look at it.

Roman, I really appreciate you coming on, man. This is great. I love to talk about this and always happy when I can stick Rick Rubin into a conversation about product management. 

So thank you so much. 

Roman Gun: Absolutely. Jeff, have a good one.