hugo bowne-anderson
Hey there, Sarah, and welcome to the show.

sarah catanzaro
Thank you. I'm glad to be here.

hugo bowne-anderson
I am very excited to have you here. And I think you're aware of this. But you are the first investor I've had on the podcast so far,

sarah catanzaro
I sometimes forget that I'm an investor. But go on. Thank you for the reminder.

hugo bowne-anderson
We always need to remind each other of the boxes that we put each other in, I think, but on your website, for example, it says you're first and foremost a data scientist. And always think of yourself secondarily as an investor. So that's one of the reasons I really want to get in the weeds with a lot of the tools and a lot of the open source tools as well, and the community. And in fact, I remember before the first time we spoke, someone said to me, do you know Sarah Cannizzaro? And I said, Well, I know of her and we've interacted on Twitter. I don't think we've ever spoken. And they said, Oh, that's weird, because she's spoken with everyone in the space, right. And so I think a worldview and a breadth of knowledge, which I'd like to explore here. But before we get to that, as I said to you before, I think of this podcast as some form of I'm very far from a sociologist, but I think I want to explore people in their interests and kind of develop different worldviews of the space with respect to different types of people in the space. And so I'd really like to go on a journey with you. And I told you, I wanted to talk about how you got into the data world, but I even want to go further back. I was looking at your website earlier today, and I want to read something you wrote to you. So forgive my accent, not going to try to sound like you. But you wrote when I was growing up, my favorite pastime was playing with liquid nitrogen. My father who's a molecular biologist would bring me to his lab where I dunked flowers, Pepsi cans and other objects in the liquid nitrogen tank. experimentation and analysis became delightful games that engage my natural inquisitiveness. So this is so beautiful for me, because I think in these conversations, we forget about elements of play and curiosity, and fun and dancing with ideas and riffing and jazz and all of these different elements of what it means to do science. So I'd like to hear how your career kind of took off. But starting with the young girl, Sarah dunking stuff in liquid nitrogen tanks,

sarah catanzaro
well, apropos of nothing, my father is Australian too, so Oh, wow, read it in an Australian accent actually, you know, really brought back those waves of nostalgia. Amazing. Yeah. So my father is a molecular biologist. My mother is a psychiatrist, but also did a lot of clinical research. And so science was just always part of my upbringing, whether it was kind of scientific inquiry and constantly like asking questions about why things work the way that they do, running these mini experiments to better understand why things work the way they do, or creating buildings that have marshmallows so that we could better understand how the architecture and structure would impact the stability. I think my parents just always encouraged that sort of inquisitiveness and experimentation as play. They they encouraged my brother and I to always ask why, and seek to answer those questions in a more rigorous way. But I think I may have mentioned in my bio, too, that one of the things that led me into the defense and intelligence sector was living in New York during 911. And kind of the profound impact that that had, I still remember today, like watching those videos on no TV, watching the planes hit the towers, realizing that this was an intentional terrorist act, being horrified by it, but also just stuck in the why, like stuck in the like, why would anybody commit such a horrific act of violence? And I think most people who witnessed that act, again, it profoundly affected them, but didn't necessarily feel like they had to go and answer that question. But for me go, I ended up spending the first four years of my career really kind of going deep on this question of like, why do people commit horrific acts of violence? Why do terrorists commit atrocities? Why do individuals join terrorist insurgencies and so on so forth? So maybe it's never stopped being you know, bad, like, two year old four year old that will just keep asking why? until like, you lose breath.

hugo bowne-anderson
Absolutely. The 50 Why's of a child is phenomenal. Yeah, I suppose there's some form of inflection point though there where in your story, it goes from Play and fun to something deeply serious. And you do discuss that on your website and the fact that a lot of your family also lived in New York and or Manhattan at the time, so I'm actually very interested in working in and intelligence and in particular quantitative analysis at that point in time, which I think probably was pretty ahead of the curve, including your time at Palantir. Right, like the types of techniques and stuff you did. There was pretty foreshadowed a lot of techniques that are applied elsewhere today, and perhaps even aren't yet right.

sarah catanzaro
Yeah, absolutely. So it's funny because I remember while I was still at the Center for Advanced Defense studies, a friend tried to like recruit me into a consulting firm with which he worked. And I had a couple of interviews where I was trying to kind of convince the the interviewee that, like AI would change the world that that AI would certainly change consulting, and that so many of the businesses and organizations with whom they worked, would end up like integrating automation and AI into their practice. And they thought I was crazy. They're like, Oh, that's cute. There's this like, 22 year old who thinks that like computers are going to change everything. I like to reflect back on that and realize that I was kind of bright like that there was something to my like, craziness of believing that computational methods that machine learning that automation would have a profound impact on how we work, how people behave, how companies achieve their objectives. But yeah, I mean, at the time, there wasn't kind of as much momentum really behind like machine learning and AI and other computational approaches to either studying social phenomenon like terrorism, more achieving, you know, business objectives, like revenue goals. I think, though, just relating this back to what we were discussing before, like I've always been so preventively interested in understanding how things work. And I really saw computational methods as a way to like more objectively answer those questions of why, certainly not the only approach. But I think one of the things that I really appreciate about working in these kinds of computational social scientific settings was that like our work could be informed by psychology and economics. And we're really trying to embrace a more multidisciplinary method. But we did see the what I guess we now call data science as being kind of a really important tool to help understand why things happen not just to predict the what will happen.

hugo bowne-anderson
I liked that my inner physicist which is retired, of course, now wants to say science can't isn't very good at answering why questions it's good at answering how questions and I think in the world of physics, that's right. But maybe in the world of studying well, sociology and why people do things, maybe this type thinking about causal inference there can help us answer a why questions, I suppose. Yeah,

sarah catanzaro
I think sometimes to like the act of science or the insights gleaned through science help us as people answer why questions. So it's not that like data science gives you the answer to that why question. But in studying something quantitatively, the process I think helps you generate insights into why something might be happening, or look at a problem in a unique way. That couple steps later helps you uncover the answer to the why

hugo bowne-anderson
it makes sense. I want to push back slightly on the idea that computational methods are more objective, I think we've seen that any aspect of the data generating process from collection through to cleaning can introduce all types of biases in I'd reframe it slightly as you get systematic biases, right, as opposed to subjective biases. But I do think the types of as we've seen, at scale, the types of biases you can get, don't necessarily make it objective. Is that a pushback you'd dance with?

sarah catanzaro
I think that things can be both objective and biased by you're making these decisions, which get encoded in data. I think perhaps the bias can actually be more easily introspected. But there's certainly I would not to make the case that data scientific methods lack bias. And, you know, certainly I think that there are a lot of subjective decisions that are made during the data, scientific process during experiments. Still, there is something that feels pure, about like quantitative approaches, and working with numbers that maybe it's just the ability to kind of introspect decisions, albeit perhaps not in the deep learning context that

hugo bowne-anderson
oh, yeah, but of course, we have a big push that hopefully we'll be increasing towards causal machine learning. And of course, one of the first things I saw you ask about on Twitter, actually, when I first started seeing you Around was around the woeful adoption of causal inference in our shared discipline. And you were very helpful in providing fantastic proof reading for causal inference report I wrote for O'Reilly as well, which I appreciate.

sarah catanzaro
Yeah, it's funny because I feel like the answer to my question, why isn't causal inference more widely adopted, actually seemed to be like fairly consistent, which was just like, it's hard. I can accept that. But how do we make it easier?

hugo bowne-anderson
Absolutely. And I think we need to this is dangerous. where my mind goes is we need to answer questions about like foundational, modern data, postmodern, what would like dataset stuff, right? Like, I'm not going to unbundle any of these terms yet. And I might even edit out using the word unbundle because that's been, I think I've blocked it on Twitter recently, I'm very interested in your trajectory from a practitioner to the decision to become an investor and what that look like and why.

sarah catanzaro
Yeah, it's funny, because for somebody who has always like embraced science, who like loves, like, objectives and processes and rigor, my approach to my own career has been very well not rigorous to say the least,

hugo bowne-anderson
I'm the same. I'm actually I joke that my career has been data, not data. Yes, we say data instead of data, but it has been like, kind of surrealist brushstrokes. I've noticed the same in a lot of scientists actually, as well. Yeah, totally.

sarah catanzaro
I mean, when I was still in college, so there have been like a couple of things that have kind of influenced my trajectory, I initially thought I was going to go into high banking or consulting, I've been studying statistics. And so it kind of seemed like a natural path to go into finance. But then the economy tanked. And I saw my friends having like, their job offers pulled and everything. So I figured I would just instead focus on something that I was, like, really excited about and passionate about. And, you know, as I had mentioned earlier, 911 had had a real impact on me. And I remained kind of obsessed with this question of understanding why terrorist groups commit these acts at the trustee. So we ended up going into the defense and intelligence sector, but I think like, some were on the, through the depths of the internet, like maybe on Facebook, there was one of those like, answer 10 questions about yourself. And so like, there is a digital record of me saying that age 21 My goal was to become the US Secretary of Defense. And clearly that's

hugo bowne-anderson
not Yeah,

sarah catanzaro
yeah, I know. Yeah. Next up pentagon, exactly. First Sand Hill Road, then pentagon. But I thought I was going to kind of stay in the Defense Intelligence sector. But I think what I discovered at some point was like, I actually loved tech. And I loved like data science almost more than I loved the application to the public sector. But also like, life brings you in weird places at the time when I was at Palantir. still kind of in the defense and intelligence space. I was dating somebody in San Francisco, I wanted to move out to the Bay Area. And Palantir didn't have any open roles. And so I called a friend. They said, like, I know how to do things with data, where should I go work? And he ended up connecting me to Daniel Morell, who is the CEO of matter, mark at the time, matter, Mark collected data on other startups and sold it to investors, including VCs. And I think I actually went into Mater mark to interview for a product role. Although I had been at Palantir. Although I'd been working for a defense contractor in the tech space. Prior to that, I didn't have like real startup experience. And so in looking at like various job listings product seemed to be like, the data focused position. And not many companies were hiring data scientists at the time. I don't even know if the term data science existed at a time what it was this was probably like, 2012 13.

hugo bowne-anderson
Yeah. The term had been floating around for a year or two after I mean, DJ Patel and

sarah catanzaro
calm. Yeah, like 2012. Right. Yeah. And Jeff

hugo bowne-anderson
hammer, Bakker and DJ I think coined it in 2008. But it didn't it was around that time, it started to

sarah catanzaro
know that I always thought it was 2012 for some reason. So that's

hugo bowne-anderson
when it became popularized. That's absolutely right. But it was coined a few years before

sarah catanzaro
Ah, got it. Yeah, well, I mean, I looked at it a job listing I'm like oh product says that they need to like look at data and make decisions that sounds about right. But I went in for my final interview with Mater mark and having moral the CTO of have become He comes into the room and he says, I have good news and bad news. The bad news is that we're not hiring you for a product position we already hired ahead of product. The good news is we want you to build our data team. Great. I've got a job. This sounds awesome. And I think in many ways, the timing actually was very opportune, because I had the opportunity to build a data science and ml team, as well as kind of the analytics function and data engineering function. At a time when you know, ml had just really started to take off. As I'd mentioned, before you matter, Mark was collecting data on other startups and selling it to investors, we had a relatively small team. And so we needed ways to collect data about startups at scale. So we ended up developing like an LP fact extraction pipeline so that we could automatically collect information, news articles, we had Bayesian classifiers to determine company, industry, all sorts of stuff. But again, kind of ended up in venture by accident, one of our customers like reached out and said, like, Hey, have you ever considered doing investing? And I haven't really, like I thought, maybe like, the people with whom I worked, or the people to whom we sold the matter of our product were interesting. And like, maybe it was something I'd consider doing one day, but it not for like another 10 years. But the opportunity seemed exciting. And so I kind of ended up in VC by accident.

hugo bowne-anderson
Cool. And did you go to amplify them?

sarah catanzaro
No. So I spent a year and a half ish at a firm called Canvas, which was like a good learning slash self discovery process. For me, Canvas was like a great place for me to kind of learn the venture ropes. And I think I realized during my time there that I really did love investing. But I was neither good at nor very passionate about investing in any category Canvas is what we call a generalist, firm. So they do consumer marketplace enterprise investing, I was far more passionate about like GPU accelerated databases and Kubernetes as a service. And I really like could not give a bleep about like, next generation wedding registries or like, dog walking marketplaces, not to say that like those things don't really matter. I just don't care.

hugo bowne-anderson
Would you clearly care about data tooling? And yeah, allowing data scientists to leverage the best technology and best products possible?

sarah catanzaro
Yeah, yeah. I think, you know, as a VC to like, you want to care about the things that you invest in, like, we ended up spending hours and hours each week with our portfolio companies. So having our intellectual interests aligned, actually, I think really matters. Like, I'm able to work harder on behalf of my portfolio companies, because I care about what they're doing. So I joined the amplify to really focus on investing in data tools and platforms. Yeah, but it was not my first venture gig. Um, so

hugo bowne-anderson
what is your kind of broad mission for you personally, at amplify, like, what do you want to achieve in this position?

sarah catanzaro
So this is where I get like, very critical of like the VC field writ large. Anybody in venture has two goals is to optimize TCPI and IRR. We're in this to make money.

hugo bowne-anderson
Can you tell us what those acronyms are? For your LPs? Yeah. Can you spell out all the acronyms? We just vomited? Yeah. So T

sarah catanzaro
V. Pei is total value paid in and IRR is internal rate of return? And so like those are metrics that essentially measure how much capital are we generating based on what was invested into amplify,

hugo bowne-anderson
and LPs are limited partners, and they're the people who invest in your funds.

sarah catanzaro
And like our our LPs are incredible there. Many of them come from academic institutions, or pension funds, where they're managing the retirement plans of the people who do really, really important work for the United States and you know, society. But when you hear VCs say that, like, oh, their goal is to like change the way in which people interface with technology or, you know, do this that or the other thing, like, you know, one thing that I do appreciate about venture is that we actually have very clear KPIs and objectives and they are to really just maximize the profitability of potential investments. That does not mean that I don't care about data.

hugo bowne-anderson
Yeah, well, it's a two sided coin, right, like for any company I've worked for our mission is to make money in a sustainable business. But we also have another mission, which may be to educate people or to build human centric products for data scientists or to give distributed systems as a service to people wanting to use the pydata stack. Or I've just listed several jobs I've had, essentially, but it's a two sided coin of making the Benjamins, so to speak, and also perhaps doing something else, I very much appreciate your transparency there as well, though,

sarah catanzaro
yeah, that is very, very true. And I would say that, like we have kind of a broader goal of empowering technical founders. Certainly, I think, like, one of my personal objectives is enable companies to use data in meaningful ways, and effectively to like, make the lives of my former colleagues, former reports, etc, slightly better having been in that position, it's tough. So like, there are other things that really motivate me. But I think particularly as a former data scientist, I'm intrigued by how we try to kind of align the company strategy, funding strategy. And though our personal motivations, what very often are just clear, like revenue or clear, like financial metrics, and I think if we are more transparent, those financial metrics matter, and is that this is an alignment problem that we need to think about, like company mission funds mission in the context of these metrics as well. Frankly, I think we can do better work like it can have a very focusing effect. And in

hugo bowne-anderson
terms of alignment, there's another form of alignment that you and I are very interested in, which is the alignment of tools for working data professionals, data star, like engineer, analysts, whatever right call them data stars, that's great. Insert regex, into professional nomenclature, so that our data stars, building tools to help them do their jobs. And you and I are in this paradox of constantly talking and thinking about tools in a landscape that's so noisy and bloated, but we're doing it in order to help people do their jobs. And something a place I want to go with this is when thinking about building tools for data stars or any industry, we need to think about pattern recognition for what flows actually look like and the potential and what they can, can look like. So where are we at even performing pattern recognition for data professionals more generally, with a view to how we think about building tools?

sarah catanzaro
Yeah, no, it's interesting, because I think like, it really depends what category of data we're talking about. Frankly, I think like the analytics stack has has matured at a much faster pace in recent years than the data science and machine learning stack. It's something that I've been thinking about for a while, including, as I evaluate potential investments in the ML stack. I think there were a couple of things that happened in analytics, that really unlocked opportunity for those building tools for those adopting tools. One of those things was certainly the adoption of cloud data warehouses. And I think the adoption of cloud data warehouses mattered not only in that it was a consistent behavior across several companies and organizations, but also the data warehouse effectively became a system of record for the analytic stack. So that any new vendor standard, right, yeah, standard. Well, it's like both a standard and a system of record. so others could build on top of the the data warehouse, because it was both standard and their system of record. I think there were other standards, though, that emerged too. So for example, like the transition from ETL to ELT, you had companies that were now transforming data within the data warehouse, which often meant that they were dumping more raw data into the data warehouse. Frankly, I think that's created more opportunity for like data testing and monitoring tools, certainly created opportunity for the ELT tools, is now creating opportunity for those that are building metrics layers, but there was this change in behavior, it became standard to transform your data once it was in the Data Warehouse. And then you had like the emergence of clear roles too. And this is something that I give a lot of credit to DBT, which is another amplify portfolio company. They really built kind of a community of analytics engineers that agreed upon like what the expectations were for the role and assert The importance of the role to the the broader organization. So you have now you know, a set of behaviors, you have a system of record. And you have clearly defined roles. And frankly, I think like those three things have really enabled the modern data stack to take off in very powerful ways. That hasn't happened in ML. And I need to constantly remind myself that like, we're in the earliest innings that you know, the analytics ecosystem had decades and decades to evolve. But it can be very frustrating because I don't see that much kind of consolidation or homogeneity in ML around rolls. Definitely not that much around like behaviors and workflows. And what's the system of record? I don't know, you ask some people, they'll say like, it is an experiment, others will say it is your label data, others will say it's your features. Others will say it is a deployed Prediction Service. Like, it's really hard to build great ml tools today, because it isn't clear like what you should integrate with, I think like the lack of like a system of record has compelled many ml companies to build end to end platforms, it isn't clear for whom look serve for whom you exist, I still see a kind of a split between orgs that think, data scientists should work end to end. And those that think data scientists should be paired with ml engineers, I'm starting to see like a little bit more kind of concentration with regards to like the ladder, the pairing of the data scientist in ML engineer. But there's still a tiny bit of dissonance in terms of like, what the expectations are for them all engineer, some companies will say that, like the ML engineer is exists to basically build an application around a deployed Prediction Service. So that data scientist needs to be able to deploy their own models. Others believe that like the ML engineer should be responsible for deployment, so so even if we're getting some clarity on what roles may exist, there's no clarity on what the definition of that role is, or of those roles are. Then when you even think about like practices for best practices for training and deploying models, like some companies will say that you should try to deploy a randomly weighted model so that you can better understand your system requirements. And so like, really, training and deployment are not sequential processes like they ought to be interleaved. Others have like very kind of concrete, or discrete, like phases of training and deployment. So how do you build something great for everybody, when there's just so much heterogeneity in terms of roles, tools, workflows, and behaviors? I think the opportunity is really for a vendor similar to DBT, similar to snowflake similar to 805 Tran to like, have an opinion, and really kind of catalyze the ML community to stand behind that opinion to support a consistent set of best practices. I don't even know that like those best practices need to be best. They just need to be standard.

hugo bowne-anderson
Practice practices. Yeah, exactly. Yeah. Why would it be from a vendor and not an open source solution?

sarah catanzaro
I mean, it could be an open source solution to I don't think it does necessarily need to be a vendor, it could be one ml engineer,

hugo bowne-anderson
I don't know if this is something open source is necessarily well equipped to solve like, at scale organizational challenges, I think open source is and you know, and all listeners, you know, I love open source. So don't at me with some BS about word vendor, blah, blah, blah. But I think open source is phenomenal, has proved to be phenomenon in the data space at solving individuals challenges, but thinking about collaborative tools that prepared for it for all of the things that an organization needs. I'm not convinced. And also, I don't think open source has been fantastic at the deployment story. So you know, my background is in for the most part in a pydata world when it comes to open source. Although I spent a lot of time with the R community. And this pydata came out of science right came out of neuroscientists and geologists and astronomers staring into the sky and building software to figure out what the heck was going on up there. But they weren't necessarily interested in the deployment and production story, and it's not clear how much open source can help this entire stack. So anyway, So those are my reasons for thinking that vendors may

sarah catanzaro
Yeah, I mean, I think like, do you need kind of like cult of personality to? Like, there are certainly examples, I'll be it's open source projects that did eventually become commercial. But within the software development world of these projects that did end up shaping behavior that did end up shaping the way that roles would be defined. They think like about Docker, I think about Kafka. But would those projects have such a profound effect if you didn't have like J craps evangelizing the log centric architecture and the primacy of the log, or Solomon hikes? This is

hugo bowne-anderson
also a big part of the venture capital play at the moment as well, right is to find people doing interesting open source projects and trying to figure out whether this is something that's productizable.

sarah catanzaro
Yeah, I mean, certainly, I think we spend a lot of time thinking about like, is this something that's productized at all, but I think another thing that we do need to consider both as VCs, and like for the future of the data science and ml community is like, is this team is this group of people committed to evangelizing their opinions and like setting a standard will they drag a community kicking and screaming, to their way, knowing that, until there's a way, tools are kind of going to suck, like you need to have the tools should support workflows, tools should support the set of behaviors and humans in humans. And if humans are just all doing different things, it's really hard to build great tools. But if you tell humans what to do, then you can build great tools to help them do the thing that you tell them to do.

hugo bowne-anderson
And if you talk to them about they want to do and try to perform pattern recognition there as well. So this actually, I love your historical tether of the analytics space, I wonder what else we can learn from web development, right? Because we used to have the webmaster right now has always been the joke for the past decade, I wonder if the data scientists will go the way of the webmaster, and front end and back end all of these things. So I wonder if there's something in there that we can use to think about what division of labor will look like in five to 10 years? What ideally would division of labor look like to you in this space?

sarah catanzaro
I think one of the things that's interesting in thinking about like web development is that specialization is kind of a spectrum, you don't meet that many people that are like pure front end or pure back end anymore. Like most front end developers know a little bit of back end, most back end developers know a little bit of front end. And I feel like when we talk about the data scientist in the ML engineer, the ideas that I've heard about those roles tend to be like, rather black and white. We don't think about like data scientists who know a little bit of ml engineering or ml engineers, who know a little bit about like creating new algorithms. And maybe we need to have more conversation about like, what those gray areas will be such that we can build tools that enable that data scientists to do ml engineering ml engineers to do data science, but also most critically, data scientists and ml engineers to communicate and interface more effectively. I've become like more and more of a believer that having that specialization matters, having a data scientist and an engineer on your team is probably a good idea. But I don't think that those roles need to be very, very rigorously defined. And in fact, like, if they are it may have a negative impact.

hugo bowne-anderson
Yeah, I like the idea of fuzzy boundaries and overlap them. I think that's key, particularly in such technical roles, which, as we still figure them out, as well. I mean, I've said this on this podcast before, I think but the amount of times I've seen early stage startups hire their first data scientist without a data engineer, they become a data engineer for 12 to 24 months, usually on the upper end of that range. Are there any other historical examples that could be useful in thinking this through?

sarah catanzaro
Gosh, I'm sure there are I just have not thought about the others as much. I've definitely thought a bit about it's kind of like a DevOps and the container, the development of the modern data stack.

hugo bowne-anderson
This question can go in a different way, actually, because you mentioned DevOps, what can DevOps tell us about, quote unquote, ml ops, and whether there's even a difference that whether we need new things for ml ops, you

sarah catanzaro
need a manifesto? I kind of joke that way. But I think what matters in DevOps is like there's a clear set of practices or clear set of kind of best practices, and not everybody needs to adopt All of those practices all at the same time. But if you talk to 10 people in DevOps, about what good looks like, you're not going to get wildly different responses, I'm sure that you will get somewhat different responses. But if you could like measure the variance in ML ops and the variance in DevOps, my guess is that ML Ops is going to be a much higher.

hugo bowne-anderson
Yeah, absolutely. So I want to drill down into tooling a bit more, it's constantly feels like now there both way too many tools and not enough tools. And why is this the case?

sarah catanzaro
Yeah, I mean, I think it's the case because most of the tools are bad. Or like, the tools aren't bad. Most of the tools are great for a very narrow set of people, and bad for the rest. Because again, like tools ought to serve their users, they ought to help them do their job more effectively. But ml jobs and ml people are like, so different. Right now, they think it's really hard to build great tools. So I think there are a lot of tools.

hugo bowne-anderson
Yeah, even like, if it serves a small amount of people, it often only serves a small amount of their job as well in their pipeline. Right. So yeah, you have to have almost this collection of interoperable connected layers, right?

sarah catanzaro
Well, I think that that is a key word to like, tools ought to inter operate. And you do see that very clearly with you know, the modern data stock. Many of these tools have formal integrations and partnerships. I think part of the challenge that we see in the ML space is that few if any tools have really gotten critical adoption. So if you're thinking about like, who should I integrate with or like, what integrations do we want to develop? That's really hard.

hugo bowne-anderson
Yeah. And you're preaching to the choir on that one, as I'm sure you understand. Also, you and our mutual friend and colleague, Pete Soderling, last year or the year before had a series of blog posts called 25 was the first one, the next one was 20. I think, tools what they do and don't do so but and this was in incredible, gave a lot of value to the community. But the fact that you even had to do a blog, or that that provided value, like listing 25 tools and what they do and don't do that another 20. That says something about the space, right?

sarah catanzaro
Oh, 100%. I mean, if you were to sample like 10 ML tools at random, and go to their website, and then try to write down what each of them did. You'd have no idea what is a direct competitor, what is an indirect competitor. And which of those tools should go partner or develop integrations, like everything is at platform, and I think they're there, it's just so difficult for potential buyers to determine what tools do for a potential partners to identify partnership opportunities, for VCs to navigate conflicts of interest.

hugo bowne-anderson
Let's just think about the buyer for a second, like, if each role has a bunch of steps in their pipeline, and they need a different tool for each one. How often they have to speak with vendors even boggles my mind, right in order to get all their tooling correct. So how do we even how can we help people navigate that part of the space, the messaging

sarah catanzaro
and positioning is clear and reflects what the tool can do. And if they focused on kind of developing a great like self serve onboarding experience, then it's actually fine. Again, if I look at like the analytics tools, in order to move data into an out of the data warehouse, you have an ELT tool, you have a data warehouse, you have a data Transform tool, like DBT, you might have a metrics layer, you might have a reverse ETL tool, you might have a data monitoring tool, you might have a data testing tool, like I'm already at eight. And yet, people in that space are content. Because these tools are easy to adopt. They all like integrate well together. And so does it really matter that you have have eight tools, if it feels like you're using one, you can have best of breed? The contracts are like pretty simple to navigate. You may never need to even talk to a salesperson. I mean, I think there are issues that arise like as companies scale, as you're thinking about observing these end to end pipelines. And so I don't think that like eight is ideal, but does it have to be one probably not like it can probably be like four or five and somewhere between eight pipeline or A stack composed of like 10 best of breed tools, and a stack composed of like one average platform. There's got to be, I think, a happy medium.

hugo bowne-anderson
And when will this happen, though? Because at the moment, I mean, even as you say, navigating websites and figuring out what tools do, then speaking to maybe 10, different monitoring companies, or whatever it is, then trying to navigate which ones you think will survive as well, because a lot of these are early to middle stage startups that we don't even know will be around in three years. So what type of risk you're taking on when doing that? So when do you forsee, I'm asking this is channeling your internal neural networks and machine learning here. But when do you see this leveling out in some way, when it's when there are more obvious choices for buyers? Yeah,

sarah catanzaro
I mean, I don't know. I hoped it would have happened by now. Again, the one thing that gives me

hugo bowne-anderson
isn't it worse than it was a year ago right now? Or am I do I need to put on my rosy colored glasses?

sarah catanzaro
Yeah. But I think like, it's always going to get worse before it gets better. Like you'll have, you know, the Cambrian explosion of the tools and platforms before like some emerges dominant. I think one thing that I had mentioned earlier that I am seeing is like a little bit more companies that are moving to a model where a data scientist works with an ML engineer. So okay, it's one thing. But frankly, I would have expected more best practices to emerge, I wouldn't have expected more ml tools and platforms to get critical adoption. And I think we're almost at a point where like, we don't need more tools. We need big personalities, we need people who believe that there is a right way to do machine learning. And

hugo bowne-anderson
people with strong worldviews. Yes, yeah, yeah.

sarah catanzaro
And are like committed to converting people to their worldview. The benefits of standardization, I think, are can be like very clear, when you have a critical mass of people who are working in a similar way, it is far easier for startups to emerge to serve their needs for open source communities to flourish, for communities in general to succeed, and then for others to start thinking about, like, how do I integrate with this way? So how do we get there at this point? Like, I think it's just personality. Great Tech, strong opinions.

hugo bowne-anderson
Yeah. Great. And so I'm interested if we're doing a bit of prediction what best and worst case scenario of what the ML tooling space looks like, in five years and or 10 years.

sarah catanzaro
I mean, I think the worst case scenario is that we continue on the path that we're on, where there are 25,000 ML stacks today. They're like, no 2 ml team share any workflows or organizational structures in common. I think the worst case scenario is basically like a world without best practices, or a world without standards. Best case scenario, I think, is a world where there has been standardization, there is a dominant design for the ML stack, perhaps like the analytics ecosystem, we go through a phase where there's probably still a bit too many tools. And then there's some consolidation. But we don't end up with like end to end, we still have kind of a more modular ml stack, and alternatives for most of the components. So that even though there's standardization, there's a set of best practices, those can still be adapted to meet, you know, more specific contexts. But without people to kind of fight for that. I don't think that that is something that is just going to happen naturally.

hugo bowne-anderson
That makes sense. I'm also wondering in terms of like these shared patterns, worldviews, processes, workflows, division of labor, is there an argument that it should differ industry to industry or type of company to so for example, I think we've realized that Faang tools, like Google Scale tools, maybe for the long tail of machine learning, most of us don't need Kubernetes, for example, perhaps that's just an example. I don't really think I don't want to dive into a Kubernetes discussion. But is there an argument that these stacks should look different for different disciplines and or types of companies?

sarah catanzaro
Yeah, I mean, that's great to say like, I think it's important to have like alternatives. And it's important to be able to adapt these components to your context. You can have standards, you can have a community that agrees upon a set of best practices, while also acknowledging that Like those standards and best practices are not going to be best for everybody. And so like there will be outliers like Faang companies that adopt a modern data stack centered on you know, snowflake, or

hugo bowne-anderson
But part of the problem is that with our mindshare has been co opted. I'm using slightly negative terms, but let's just get over that has been co opted by the outliers. Right?

sarah catanzaro
Yeah. Well, I think that's very true. I think it's not just our mindshare. I think a lot of the kind of early activity in ML defined by Yeah, tools and platforms came out of was defined by the faang companies because they have data. But you know, now we're starting to see more companies kind of build their own ml tools and platforms. It's kind of like that set right beyond the Fang, or at least like right beyond. I guess it's not Facebook anymore.

hugo bowne-anderson
Mehta, whatever it is, yeah, Google, and Amazon.

sarah catanzaro
So you know, whether it's Uber, Netflix, Airbnb, even some of the kind of like, new generation of unicorn companies, the stripes and Brex's and notions of the world, like I think it's this kind of set of companies that may be able to kind of define what should be best practice where they're at a scale that you know, other earlier stage companies aspire to be. But that scale is within reach. It's not Google scale.

hugo bowne-anderson
Yeah, I think there's also an argument that we don't hear a lot about it because of Google's coms incentives, but they do a lot of what we would call reasonable scale machine learning within Google and Netflix, but they just it isn't the type of stuff they publicize so much, right?

sarah catanzaro
Yeah. The other thing that I found interesting, too, is that we're talking about like personality, and you know, open source communities, open source projects, like, do you look at like the PyData community, like, you've got personalities, and you have people who were like wedded to their project, I don't see kind of the same commitment, even among like to tool builders in the ML ecosystem, people kind of like bounce between, like various projects. And I think you know, that is probably also hurt the evolution of the ML ecosystem.

hugo bowne-anderson
I think you're right, we've used the term modern data stack enough for me to want to drill down into what that actually means to you and to our community.

sarah catanzaro
Yeah, I mean, frankly, I think like what it is somewhat marketing jargon, but like, I guess, the next step beyond standards, best practices and systems of record is marketing brainwash. But I think that the modern data stack really just means these set of tools that are focused primarily unlike descriptive analytics and experimentation that are closely coupled to the Cloud Data Warehouse.

hugo bowne-anderson
That's what I was thinking, I'm glad you mentioned, marketing, brainwash, that dovetails quite neatly with something I'd like to pick your brain about. And this is a challenging conversation. For me. I mean, it comes from a lot of self reflection around my career, where I'm at what I want to do in the future. As you know, historically, I've done a lot of evangelism, I'm now running Developer Relations at outerbounds. I've done a lot of work, as I've been on marketing teams and ran marketing teams. And what we would call in like, is kind of modern marketing, part of the rationale behind that I think a lot of the world, a lot of the professional world has been subsumed into marketing due to the data that's available to marketers, among other reasons. Also, developers generally skeptical of marketing. And what that means is that developer relations and evangelism has emerged in its own right. And as you know, I've worked with I speak with a lot of founders, a lot of investors, I've worked with a lot of startups in order to help their developer relations and marketing efforts. And I do like to kind of reflect on, let's say, the negative externalities, what I do in my work, and I do think there's an argument that marketers such as, quote unquote, my I'm still can't refer to myself as a marketer. I mean, this almost feels like therapy. Now, I love it. But I think there is an argument that there's some form of tragedy of the commons in the tooling space. And in kind of the information, abundant landscape, when I create a lot of content for a portfolio of companies, I do wonder what where I'm creating signal where I'm creating noise. I do think with all the investment coming into the tooling landscape. Now, we're both at a point where there's an argument that we are contributing to a tragedy of the commons in some ways, in terms of having these portfolios, and in particular, a tragedy of the commons for data stars, data practitioners, data scientists, and each company that I've worked with, I feel like this is actually valuable. But this is my own personal opinion, right? And I think you're actually in a similar yet distinct position. So I'm wondering how you think about this aspect of creating signal versus noise and when you're doing good for the space and what the negative externalities might actually be?

sarah catanzaro
Yeah, I mean, I think like as, as I've reflected upon this conversation, like, I still think that like the marketing does more good than harm I think the evangelism does more good than harm, because again, I think we need communities, we need communities that are aligned on a set of standards, a set of best practices, we need communities that like believe in the way. And how do you get that without like marketing, Dev evangelism, etc. So I think like, this is highly necessary. I think where the some of the marketing and messaging has gone wrong, is that there's a lot of like marketing around the vision for tools, and not really like, what tools allow their users to do today,

hugo bowne-anderson
where they unblock you at work, what are you doing that this thing helps you do? So you can give what you need to give to whoever you need to give it to? So what does this tool actually do? That's why your blog post is so fascinating. What does it do? And what doesn't it do? Come on? Yeah, I know what the cloud SaaS product dude. Sorry, I'm gonna stop, stop now. But like, Tell me more, right? No, no, no, no, I know, it's most I know your offerings. multicloud. But what the bleep does it do?

sarah catanzaro
Yeah, I mean, I think like this, though, is an area where VC can have a kind of like, nefarious effect, because we want to hear about like the billion dollar vision, we don't want to hear about the widget, that'll get you to like, you know, 1,000,000 or 5 million arr. But the widget that gets you to 1,000,000, or 5 million ARR is the thing that like practitioners need to know about. They need to know how this tool helps them today, not like the multibillion dollar company that the tool will be in the future. And I think, in many ways, the tool, the widget aligns more clearly, with the standards that emerge, than like the big vision, it's much easier to standardize something specific than it is to like standardize a like 10 year vision.

hugo bowne-anderson
So what I'm hearing is that there's a potential misalignment of incentives, due to the way capital works, is that one way to phrase but I guess,

sarah catanzaro
I mean, I think capital encourages people to think big, but standardization starts small,

hugo bowne-anderson
right? Is that fixable? Is there some way we can meet in the middle? Yeah,

sarah catanzaro
I think it is eminently fixable, you can have a multibillion dollar vision, and start small by building a tool that helps a large enough set of people with like some parts of their work flow. Deriving standardization there

hugo bowne-anderson
really Oh, is one of my favorite examples of that, actually, I think is that something that resonates?

sarah catanzaro
I mean, absolutely. I think they're Twilio certainly is a great example. And stripe as well, actually. Yeah, there are so many companies were like, if we can describe the tool in a pretty similar way, without, you know, hearing each other's response, like, that's probably a good thing. I think, like maybe we just need to be kind of more forthcoming when we're interfacing with our colleagues or, you know, when I'm interfacing with my portfolio companies like, what are you pitching the vision? And when are you solving the problem? And when should we be talking about the vision? And when should we be talking about the problem? This is

hugo bowne-anderson
one of the first conversations you and I had when I was considering joining outerbounds? Yeah, of course, you're an early investor in outerbounds.

sarah catanzaro
Yeah, no, I remember that we were talking about like the vocabulary and how do you put into words, what outerbounds does, so that anybody can understand it, so that people do understand the set of problems that they solve. And frankly, I think like, it's easy for me to like, preach, oh, and go, our portfolio companies, founders, etc, should be specific about the problems that they're solving. Like, it's kind of hard when you're building technical tools, like it can be hard to like, put that feeling of using a tool like medical low into words.

hugo bowne-anderson
Very much so. And it's a work in progress. I am interested in whether there's also a similar never Well, I mentioned the negative externality that I think marketing can have, especially when you have a portfolio, I'm going to say kind of tell a narrative here, which may or may not be right. And I may not even understand all the details of what I'm saying. But I want to get your take on this in my mind. A VC firm succeeds by having a portfolio of companies from which a handful succeed, right? So there's an incentive to have a wide portfolio and you don't need everything to win in order to make money for your firm and your LPs. When that happens, there is an incentive to perhaps try a bunch of things which may not be in all in the favor of the end user and take risks, which can, I suppose, pollute the landscape? I've used a bunch of negative critical words there. But I'm wondering what your take on that is, and how you think about it. Yeah.

sarah catanzaro
So I mean, I think the thing there that we need to take into account too, is that while we don't have an expectation that all of our portfolio companies will be successful, we do also have a pretty strong incentive to make sure that our companies don't lose or like, don't go horribly wrong. So like, not everything is going to be a decacorn. But we try to make sure that things don't go to zero, because when they go to zero, like there could be like a five to $10 million, like, gaping hole in our portfolio. And now we need to account for that, by ensuring that, you know, we have more winners there. So we still well, we know that not everything is going to go to a billion. We're also heavily incentivized to make sure that the remaining things don't go to zero, that like, we've got a couple of hits where like, I don't know anything about baseball, but like we get to like first base or second base, but it doesn't have to be a home run.

hugo bowne-anderson
That makes sense. 

sarah catanzaro
And I think that that incentive forces us to be a bit more prudent about like, where and how we invest. Now, that is something that is like somewhat unique to amplify our approaches to concentrate capital. So we only do like 20 to 30 investments per fund across six check raters, if we were investing in 6075 companies like we wouldn't have the same incentive, because we wouldn't have, you know, five $10 million. In any given one.

hugo bowne-anderson
I'm just I mean, your portfolio is cool, Sarah. I mean, I'm looking at with DBT, and hex, I mean, there's all types of the hit rate, in my opinion, are the bets you've made really strong. Apologies for stroking your ego. But it was I believe that

sarah catanzaro
I love the ego stroking and and you know, I would say that, tell me about it. My burden on outer bounds was pretty good,

hugo bowne-anderson
too. But you ain't seen nothing yet. Yeah,

sarah catanzaro
I will say that like one of the hardest things for me as a specialist investor, though, is like navigating these conflicts of interests, like we work very closely with our portfolio companies. So like, I try my damn best to make sure if I'm not investing in competitors, but even when I have not perfect, but like near perfect visibility into what a company is building and what they want to build in the future, I think so many of these tech stacks are still so nebulous that like sometimes they collide. And that probably does speak to some of the kind of more dangerous tailwinds that we've discussed.

hugo bowne-anderson
Totally. I think that leads nicely to the other types of what we're missing here. I actually, as you know, I had a question with all our focus and conversations on tools, were missing a lot of other conversations. And I want to know what you think the conversations were missing. And I actually just saw you tweeted out the other day, something relevant you wrote, tell me how you're using in blockquotes data. I'm sick of hearing about how you're producing, I love reading things you've written to you for some reason. So tell me how you're using data. I'm sick of hearing about how you're producing data or building data stacks. As one great data scientist one said, The only stacks that matter are those of Benjamins. And you've got this great gift. I think it's from Chappelle Show. It just seemed like it's maybe it's the Wu Tang Chappelle stuff, but he's like raining. I think

sarah catanzaro
that's Cardi B but close enough. Oh, right.

hugo bowne-anderson
Holy moly, I may edit that out, but maybe not. Yeah, but she's got the wig on. Okay, let's forget about the gif. But I'm definitely not gonna edit that out. But yeah, how people using data and what conversations are we missing through all the stuff all the time? We're talking about tools? Yeah.

sarah catanzaro
So one of the tougher conversations that I've had in the past month was with a colleague, who's actually doing like her PhD on ML related problems. And the topic of conversation was, will ml enable us to build better products? Like, is ml even important? Or are most of the problems that people have most of the product opportunities that exist? Not related to intelligence and automation? Like, are they better solved with I don't know, like, stronger wireless connectivity or faster data or whatever it might be?

hugo bowne-anderson
This actually speaks to marketing bullshit as well, like we do live in some form of speculative bubble as to the power of ML and then ML is applicable to every single problem, right? And we need to slice through that. So that's just something that resonated with me there. Yeah.

sarah catanzaro
So I mean, we started kind of like brainstorming About this, like, what are the problems that you face in your day to day and have those problems like which should or could be solved with machine learning. Now as somebody who actually doesn't have her driver's license, like, I'm big on AV, like I would like to, I don't like driving, I did have my driver's license at one point in my life, but like, I would like my car to drive me around, that is something that I really would like to exist. And that does hinge on machine learning. Conversely, there are Uber, Lyft, are the problems that I experienced with those apps related to machine learning, like not really like I just it's more of like a latency and performance and scaling thing. Like I'd like the app to be faster,

hugo bowne-anderson
although the stuff that they do that's really important to them, like pricing, which has impacts on us, of course,

sarah catanzaro
yeah. So this is all to say that, like, there are some problems that machine learning can solve and some problems that machine learning need not solve. And I think like even just having more dialogue around like, what are people using ml for today? And then what types of data are they using? What types of models are they using? I think like, if we start with like, what are the applications and backed down from there, we can have much more rigorous conversations or interesting conversations. In fact, one another amplify portfolio founder texted me today, and said, like, Hey, do you know there's a list of like ML use cases somewhere? And I thought about it? No, like, I don't think that exists.

hugo bowne-anderson
Yeah, there must be something. And if not, we should build it was just saying there's an incredible GitHub repository of ml fails, which of course is the opposite. But I think it's as instructive. I think developing a taxonomy of negative use cases as well. And value modes is incredibly important. I've just seen another tweet from you, which is relevant to this conversation. Has anyone asked sales leaders account executives, or STRS? If they actually want or even care about a CRM built upon the modern data stack? Yeah. Because maybe they're doing fine without it, right? I don't think they are. But that doesn't mean the modern data stack is the solution. Right?

sarah catanzaro
Exactly, exactly. I think like, we have ended up in the same mode that we know is so toxic, where we're pushing technologies, or pushing for, like use cases and applications based on what that technology unlocks, rather than thinking about like, what are the problems that people have? Or the problems that they say that you have? What are the opportunities that we see? And what is the best technology, or best technical approach to addressing those like, there's so much like, first principles thinking that's happening. And we've both been in data long enough to remember 10-15 years ago, like all of the companies that were like, I've got a bunch of data, how can I use this like fancy shmancy AI stuff and great value? And you have to like, shake them and be like, No, tell me about your business problems. And then we can think about how you can apply AI to your data and or do other things with your data to potentially solve those problems. And oh, man, it makes me so frustrated to like, see, companies falling down the same rabbit hole that we went through the 10 years ago.

hugo bowne-anderson
Are there like some game theoretic incentives at play as well, like when one player does in an industry, then it's like, oh, yeah, these so and so's doing this. So we need an AI strategy. And then you hire a bunch of people, it costs you millions of dollars. And it doesn't necessarily deliver value when it isn't delivering value, a lot of time teams will pivot do stuff with different stakeholders, package things to package up to seem like it's delivering value. And it does create a speculative bubble in the industry as a whole. Right? Totally.

sarah catanzaro
Yeah. And look like sometimes the the game theoretic forces, I think, can actually be beneficial. So for example, like, the real estate sector, I think, is probably like more advanced in its adoption of machine learning in many other industries. But I think participants in ML in the real estate sector actually are having conversations around, what are the problems that our businesses have that should be solved with machine learning that can be solved with machine learning, and that can be reliably solved with machine learning. So they're having conversations around problems, not just around technologies. And I think those conversations around problems determining maybe we're not going to use ml for pricing. Maybe instead, we're going to use ml for like lead generation or lead prioritization.

hugo bowne-anderson
I love that you said that because I agree, but also, we saw what happened at Zillow last year, right? Okay,

sarah catanzaro
that I think though, was a lot of finger pointing at data science teams. hims on what was just objectively like, a bad business decision?

hugo bowne-anderson
Yeah. Not only teams but at the algorithm? Yeah, finger pointed at the algorithm, right?

sarah catanzaro
They finger pointed at the algorithm when, again, it was a problem that could not be solved by any algorithm.

hugo bowne-anderson
Well, that's why I love that you framed it in terms of maybe pricing isn't the best thing to use ml for, but lead scoring and these and I actually think we've talked about this before, but lead scoring, ranking algorithms are some of the best use cases of ML, particularly in an information abundance landscape, getting the right information to the right people at the right time, in a certain rank hands down. Yeah,

sarah catanzaro
yeah. Particularly where that's a problem that matters to your business. Exactly. And in real estate, it is. So like identifying the these use cases that are important across various players in an industry and for which the technology is well suited. Instead of just saying like, Okay, we've got data, and we've got this tag, like, what models should we be using with what data sets and then like, problems and priorities are secondary concern? It's a different way of thinking, it's arguably like better to think about what your objectives are, rather than, like, what are the techniques that you use to solve those? But it seems like we've kind of regressed in terms of AI business strategy, if you will. I

hugo bowne-anderson
know, of course, that global security is very dear to your heart. What other industries or disciplines or fields are you most excited about the applications of machine learning?

sarah catanzaro
I told you, I'm waiting for my AV. It is slowly happening. But you know, apparently, like waymo taxis are going to be in existence this year? That's a great question. I mean, I do think that there are opportunities to use ml to build better tools, including for various technical practitioners. And you know, some of the work that was done around like co pilot is really phenomenal in terms of how it changes developer behaviors. I think there are kind of clear alignment of problems, datasets, and technologies in the biotech sector or biopharma sector, where we have massive datasets where the problems are, in fact, almost a scoring and ranking one, for example, with identifying like lead drug candidates. So I see very kind of promising and real applications. I've been all there. But honestly, I think often it's just kind of like those mundane problems. They're not very sexy. They're just things that people experience, you know, all the time and spam detection.

hugo bowne-anderson
Yes.

sarah catanzaro
spam detection. Yeah.

hugo bowne-anderson
I mean, that's something we've done. Right. But that's one of the most beautiful, yeah, beautiful things, I think. I think it's probably not that hard, like, dear sir, I am or whatever, right. But I think these types of things are really obviously important.

sarah catanzaro
Yeah, totally. What are like algorithmic innovations that have like, impacted my behavior in a positive way? I do like autocomplete. Like, I forget idioms all the time. And so it's not necessarily that, like, I need AI to write an entire email for me. But like, it's nice that I don't have to worry about remembering various idioms anymore. If I'm using Google compose translation was one that I think is also really cool. It's awesome being able to like travel the world and know that I can communicate with anybody, because the translation apps have gotten so good. I think that's one that has like, really changed the way in which people interact. And that's super cool. So we need more conversations like this, like, wow, AI enables me to communicate with anybody in the world. Like that was not possible before.

hugo bowne-anderson
Totally. Another example that I really liked that I've discussed on this podcast before with two people with Rachael tatman, who is at Rasa now and Heather Nolis. Who runs she's a principal machine learning engineer AT T Mobile, but conversational AI, chatbots. Right. So not only what they can provide, but how to incorporate them into human workflows. So the fact that if you have a conversational AI, do you want to actually serve important responses solely from that AI or Chatbot? Or do you want a human in the loop in some way? So the Chatbot can tell them kind of the message that they'd want to send and the human can massage it or these types of things, right. So figuring out what that essentially it's how we think about cybernetics, how we think incorporating humans and machines into workflows. And I think that will probably be key in this process as well.

sarah catanzaro
Yeah, absolutely. And thinking about how do we still afford agency that humans when they're interfacing with algorithmic systems,

hugo bowne-anderson
and how do we roll out algorithms? I'm stating this example because I think it's non controversial but in support systems in organizations if you're going to roll out a chatbot to support for customers. One mode that I've seen work really well is first, you roll it out internally so that it's real people doing the support with the Chatbot, helping them figure it out. And once that flow becomes works really well, and you know, it gives responses, which actually make people happy and satisfy the customer, then you can begin to roll it out slowly to customers as well.

sarah catanzaro
Yeah, totally. I mean, I think the cool thing too, that this reminded me of is like, sometimes the interaction between humans and ml systems actually like changes human behavior in really cool ways. To another one of the amplifiers portfolio companies is called runway and runway has been focused on developing like ML driven video editing tools, but also creating kind of this marketplace where they make various models, accessible to creatives. And many people, I think, often like conflate ml with automation. And there's some fear that like ML will, you know, get rid of jobs. And so if you think about ml and design, it's only natural to think like oh, AI is going to like, generate, you know, the next shoe. I remember speaking to somebody at New Balance about how they were using runway. And he said to me, like, our biases, or our ways of like seeing the world are actually so entrenched. And so they'll use these generative models to kind of like, reimagine what a shoe can be. And it's not that they take those designs and just push them forward. But in fact, like, the models can help him as a designer, think about like the shoe in a completely different way, because he's so locked into like his existing kind of notions of like a heel, or a sool and laces. And so to have these models that like don't actually learn these con or know these concepts in the same way, as a human, it provided him with creative fodder. So, yeah, it was like it's not models that are automating work. Instead, they're like amplifying human creativity. I thought that was really cool.

hugo bowne-anderson
Absolutely. I love that idea. Because I'm during lockdown, I started getting back into playing chess. And firstly, like, there's this concept of a centaur human and a machine playing on the same team, like a human informed by machine, which is super cool. But where I wanted to go is I was watching. I don't know what like YouTube chess is full of like some real like, energetic legends, like people who put me to show they make me look like say a very quiet human being. But there are some real chess legends who studied a bunch of games with AlphaGo and AlphaGo. Zero, in which there were moves that AlphaGo zero made that made no sense to the human right that like that, that is a totally oblique bizarre move. And then a few moves later on, it became apparent what space this actually opened up in the game. And it was a game that these people who they're grandmasters and all that but in games they've never seen before, because this one very strange movie opened up a whole new space of creativity and combinatorial explosive options. And I think that's really fascinating.

sarah catanzaro
Yeah, it is cool to think, not just how does AI replace our behavior? But how does it make us behave differently?

hugo bowne-anderson
Is it also important how we, what we believe in them and what we don't so I'm gonna butcher this, but I think the message is going to come through there was a study about trust in robots. And it was, I think it was it had a simulation, human study in a building with a simulation of a fire. And the experiment was a human directing people how to get out of the building. And sorry, that was the control in the experiment was a robot directing people. And even when it was obvious that they were going the wrong direction, that people trusted the robot more than they did the human. When it was a human, they'd be like, Dude, that's where the fire is. And when it was a robot, they were like, okay, the robots probably not wrong. So thinking about more about what we mean, there are cases of people like driving off a pier because Google driving down a one way street and having an accident because Google Maps has told them to right, so thinking about these things that I suppose create a lot more convenience, in some ways, but having more uncertainty baked into it with respect to our relationship to it. Yeah, I

sarah catanzaro
mean, it's funny, I feel like we need to, like learn how to empathize with ml systems, like, we need to learn to understand their shortcomings, we need to learn that the generative model has no concept of what a heel is, or what a soul is. And then some of this, I think, means not just marketing, all of the amazing things that AI can do, but also like, ml bloopers. I love it. Yeah,

hugo bowne-anderson
we do need an ML file blog. Yeah,

sarah catanzaro
yeah. If only to like help us start to think about what should we expect from these systems? How can we adapt our behavior to what they're good at and what we're good at and what they're bad at and what we're bad at?

hugo bowne-anderson
I love that you mentioned The sole and this is potentially an unorthodox question for such a podcast, but we're going there what is becoming more and more reliant on machines due to the human spirit and the human soul?

sarah catanzaro
I don't think it needs to have any bad effect. You know, like we've been working with tools, maybe a good effect for ages. I think the best tools make you think about what you do well, and what you do poorly, the best tools, I think, make humans more human. I think, you know, there's a risk that like, we try to automate away what makes us human, I think there's a risk that we build tools to mimic us instead of to compliment us like I often say, like amplify invests in machine intelligence, not artificial intelligence, because I fundamentally don't believe in artificial intelligence, I don't think we should be building machines, robots, systems, software that just emulates human intelligence. Like we've got plenty of humans, there are way too many humans in fact, like. But I think if we can be self aware about our own limitations and our own strengths, and design tools that kind of compliment us, I think machine should give us more soul?

hugo bowne-anderson
I think so. And I do think part of the challenge is, tools that nourish us, help us manifest ourselves are incredibly important. But I think we do live in a landscape currently, where a lot of the tools, I mean, we think about FAANG companies, for example, and I don't want to get too finger pointing. But I do want to say that the incentives have tools that provide something to us for free, and have advertising revenue as their financial model. They're not necessarily aligned with providing the best tools to the end user. I mean, there's the old saying, if you don't pay for the product, you are the product. I think I like Shoshana Zuboff's on it, which is if you don't pay for the product, you're not the product, you're the rotting carcass from which the product is ripped. And of course, her take is relatively strong. I agree with a lot of it. But in terms of the incentives of who are giving these products, I think that needs to be a broader conversation not only about Google and Facebook, of course, but around who's building what for whom? 

sarah catanzaro
Yeah, I mean, I again, I think like part of this is like self awareness and not building things that exacerbate our weaknesses, including, you know, addictive tendencies and polarization and things of that nature. What are the things that we want to do better? Again, I come back to the translation example, enabling us to communicate with more people. That's great. And I can pay for Duolingo. And I'd probably pay for Google Translate. If I had to, maybe I'd prefer to. Yeah, yeah, exactly. I like it. 

hugo bowne-anderson
Let's wrap up in a second. I do want to pivot back a bit to tooling and because we're talking more about kind of our human condition as a whole. I am wondering, I think a lot about low code, and no code tooling. And who was part of my mission is we'll get a bit personal with me here. But you know, my backgrounds in academic research, where I saw research scientists who had brilliant, brilliant scientists, experimentalists, who had next to no time in their daily life, to even figure out how to do statistics robustly, to do generative modeling to interact with the command line, and and cron jobs and all the things that they started to need to do. And that really made me, made part of my mission, wanting to help what essentially as millions, if not 10s of millions of current scientists and future scientists have tools and workflows to do their jobs better and in a more human way. And I do think, a low code future for a lot of them is important, not for everyone, right? And a no code future could be really useful for maybe 80% of them, maybe 20% of them. I honestly don't I don't want to put a number on that. There's a lot of variance there. But I'm wondering what you feel about. I mean, you work on tools and speak a lot of people who build tools that up for very technical people, right. So what you think about the future of low code and no code tooling?

sarah catanzaro
Yeah. So firstly, I think like, we shouldn't necessarily be like bucketing, like, low code and no code together. I think there are there like theories, major differences between no code and low code tools. And frankly, I think like, there are many circumstances where like, code affords you much more flexibility than like a visual interface would. But code shouldn't be hard. It shouldn't be that challenging to write code, and then build an application. You shouldn't have to think about like environments, you shouldn't have to think about Kubernetes, etc. And so like, I do think that there's a lot of promise in the low code space. But I'm almost reminded to have like, you know, my, what I was talking about with, like these translation apps, like, I use a translation app when I'm traveling, you know, throughout Mexico to communicate with somebody at a pharmacy. Yes, it serves that purpose, but I'm not going to have like a meaningful dialogue with a friend or colleague over Google Translate, I'm going to have to learn Spanish. If I want to do that, when I think about coding the kind of a similar way, I think, you know, there are things that no code platforms might enable us to do, which don't require a high degree of complexity or don't require a high degree of flexibility. If you need more flexibility, the flexibility that code affords then, like, maybe a low code system is better. And then sometimes, like, maybe we don't know what the opposite of low code is high code. But code code code code. Yeah, but yeah, I mean, I think different use cases for different things. My favorite tools, though, are those that enable you to kind of like scale these ladders of abstraction. I think Excel is like, one of the most amazing products of all time, because you can interact with it as a GUI

hugo bowne-anderson
visitor, great if we're talking about getting millions of people to use it as well, like, it is absolutely amazing.

sarah catanzaro
Yeah. Well, like you can interact with Excel is basically just a GUI for viewing data. Like you don't have to write any code in Excel, you can just look at the spreadsheet and get like a sense of like, what data is there totally, you can write relatively simple formulas and use excel in a different way. You can write like a VBA. And like, extend the functionality of excel in amazing ways. And I think it invites people,

hugo bowne-anderson
I'm pretty sure you can embed arbitrary Python code in Excel these days. Maybe like, you may need some of that. I think there's a product that does it. Yeah,

sarah catanzaro
yeah, you can interface with the tool according to your use case and technical ability. And I think like, it's just amazing.

hugo bowne-anderson
Yeah, Sarah, I'd love to know if you have a call to action for our listeners. But before that, I just love to say thank you for a wonderful conversation. And let's say I've asked some difficult questions and self reflection and that type of stuff, and also being really open to having a broad conversation around this stuff.

sarah catanzaro
Wow. It's crazy. So it was super fun. I love all the directions that we went in. I enjoyed the conversation a lot. So thank you for having me on the show.

hugo bowne-anderson
Absolutely. So yeah, what would you call to action? Yeah. What do you want people to do?

sarah catanzaro
Define standards for machine learning and think about problems that ought to be solved and not just technologies that are cool.

hugo bowne-anderson
I could not have said that better myself. Thank you once again, Sarah, of course.

sarah catanzaro
Thank you.


Transcribed by https://otter.ai