Coté (00:00) Well, tonight we accidentally ordered a pizza for pickup in another part of town. You know how it is, you're on a tiny phone and you don't know what's going on. And also, I'm sure this happens to you all the time, Whitney. The site's in Dutch, so you're not quite sure what you're doing. And it turns out I had to drive somewhere to go pick up the pizza. Now it was it was in our old neighborhood, and you know, we live kind of in the suburbs. Coté (00:25) Which has all the benefits Whitney Lee (00:25) Mm-hmm. Coté (00:26) of suburb stuff European style. and but then I went to the neighborhood we used to live in, which is more in in the city of Amsterdam. So it's more lively and it's warm here now. So people are running around in their shorts, you know, they're outside, they're doing stuff, they're talking with each other. It's not quite like that stereotypical like New York street sort of thing, but it's just like crawling with people and on bikes and everything. And so Whitney Lee (00:49) Can't Whitney Lee (00:49) I can't get past Kote. I can't get past the fact that you ordered it. You're still gonna drive to the wrong part of town to get your pizza. Like I think maybe that makes you a good person or something. I'm not sure I'd do that. I would at least call them to be like, hey, I messed Coté (01:03) Well Whitney Lee (01:03) up, I'm not getting the pizza. You can eat it, you know, or whatever you do with unpicked up pizza. Coté (01:05) That's true. That's true. Well, we Coté (01:08) we we could we couldn't figure out how to cancel it without calling them. And I think I think Whitney Lee (01:12) So it's Whitney Lee (01:13) all this this driving was to avoid a phone call. Coté (01:16) Well, it's funny you should so first of all, Coté (01:18) I I I I wasn't the one who ordered the pizza. I was I was Whitney Lee (01:21) Okay. Coté (01:21) the I was the the the cleanup person, so to speak. And Whitney Lee (01:24) I don't think that's relevant, but good to know. Coté (01:26) yeah, and and and then it also didn't occur to me halfway into driving that we probably should have just called them. So anyways, that's how that's how that started happening. But you know, ever ever every every Whitney Lee (01:39) Okay, now I can hear the rest of your story. Thank you for clearing that up. Coté (01:42) every time I go further into the city. Coté (01:45) There's a certain point where Whitney Lee (01:45) Uh-huh. Coté (01:45) you get to the tourist part of Amsterdam and and I don't have this thought. but I go I go kind of into this this this area. I'm like, this place is great. Why don't we live here? Like we used to. And you Whitney Lee (01:56) Mm-hmm. Coté (01:57) know, it just kind of like invigorates you seeing all this action around and stuff. And, you know, like I know I know the reasons, more or less. Kind of, sort of. But it it got me thinking, like, you know, how do you like pick which part of the city you want to live in? Whitney Lee (02:12) What actually I'm thinking lately, especially I live in Austin, Texas, and it's it's wildly hot here right now. This time of year, every year, I'm like, you know, why do I live here? I need to move. And Coté (02:24) You have you have the Coté (02:25) you have the flip thing that everyone in the Netherlands has and and also this part of the world, as as our guest probably knows, equally bad, is like when it comes to late fall to about Whitney Lee (02:35) Uh-huh. Coté (02:36) February, you're like, Why the fuck do people live here at all? Yeah. Whitney Lee (02:39) Why do we Yeah. Yeah. And Whitney Lee (02:42) then yeah, and Texas politics are dicey. Like so there's more and more reason to to get away. And I was, so I have been like scrolling Zillow lately, being like, Where do Coté (02:53) Mm. Whitney Lee (02:54) I like what's next for me? Like, do I wanna I think I wanna go back to the Midwest where I was raised because it's Coté (02:59) Mm-hmm. Whitney Lee (02:59) like slower, people are super friendly. It seems like you know, nowhere is climate change proof, but it seems like better it's better than Texas. I guess that's everywhere. But Whitney Lee (03:09) Being by the Great Lakes where there's like s fresh water, lots of it, that sounds good. Seems like a good idea. Coté (03:13) Mm. Yeah. Yeah. Yeah. Whitney Lee (03:16) and so, but then I'm like, do I want to live in a I city or the am I a city mouse or a country mouse now? I think I'm maybe turning into a country mouse. Or I just I want to be in nature all the time and hang out with deer and owls. And that's who I talk to. And everyone Coté (03:24) Interesting. Mm. Well Whitney Lee (03:31) on Zoom all the time and Claude. Coté (03:34) Yeah, yeah. Coté (03:35) Yeah. You want to be a digital mouse on your desk and a and a country mouse, otherwise. Well how about how about how about well guest Brian, Brian Ross. We worked together for several years and and we've we've we've have is this true? We don't know. We don't we don't keep up extremely, but we keep up with each other, more or less. I d I don't know @BryanRossUK (03:55) Yeah. Coté (03:55) why I'm thinking that and saying it out loud, but let's get back to the point. Whitney Lee (04:01) What's Whitney Lee (04:02) your whole life goate? Coté (04:05) Say saying things out loud that I'm not sure why I am. That's hm Whitney Lee (04:08) Ha yes. Coté (04:12) anyhow, are are you a are you a city mouse or a country mouse? @BryanRossUK (04:16) You know, I'm very much the country mouse these days. you know, it's I enjoy I enjoy peace and quiet. I enjoy getting away from tech and I enjoy more and more like an honest day's work where, @BryanRossUK (04:30) you know, if you're clearing a driveway, you look at the driveway and you go, Well, this is going to be covered in leaves tomorrow, but for right now, the driveway is clear. And it didn't involve any brain cells. And I think that's that's that's where I'm heading these days. Less and less brain work. Coté (04:44) Or or anyone named Claude telling you how to how to clear the the driveway. You know, there @BryanRossUK (04:47) Yeah. Coté (04:48) there's there I d I don't there there's this as Whitney will know, I have this weird niche interest in like UK tech people. And and one of them, for some reason I followed Whitney Lee (05:00) Yeah. Coté (05:03) for a long time this guy, Phil Gilford or something. I don't I have no idea who he is or why I follow his RSS feed, but he lives in the in the Coté (05:11) sa further south than you do in the countryside. And he went through a phase of scything his yard or whatever he calls it, whatever y'all call it over there. But he Coté (05:23) he would have lots of pictures of like, you know, a big scythe like the Grim Reaper. And he would be like, Coté (05:27) I'm learning how to scythe my my grass. And it that's, you know, that's good honest work, I think. @BryanRossUK (05:36) That's it. Like if if you can spend several hours trying to debug a problem with a nineteen eighties Kubota tractor engines fuel system, that's a good day had. Whitney Lee (05:47) What what makes work honest? What's at the core of that statement? @BryanRossUK (05:53) I I think it's the @BryanRossUK (05:58) It's the I don't create anything directly anymore. And Whitney Lee (06:02) Okay. @BryanRossUK (06:03) and I think there's something deep seated and that at the end of the day I like I like something that I can hold, Coté (06:08) Mm. @BryanRossUK (06:09) something that I can show. So I'm I'm all about outcomes and all that good stuff, but they're not necessarily my outcomes. Whereas I think honest work is where I'm just able to do and appreciate kind of within this time frame, I will then get a result. Whitney Lee (06:28) Okay. @BryanRossUK (06:28) but that @BryanRossUK (06:28) being said, the result is probably temporary. But also I kinda like that too. I like the idea that like there's forces bigger that are just controlling, therefore you don't have to worry about it too much, you know? Coté (06:39) Well, I fe I feel like I feel like these notes I've taken about stuff we can talk with, I can just delete them now because this is a good topic. I I was and and and I I I was thinking, let me let me w whatever. I'll fit Whitney's description of what I do here. But I I was I was driving home from picking up that pizza earlier, and I was this is a scenario in my head. I was thinking, like, am I the kind of person in my life right now who I like like to keep a metaphoric canister of gas. Coté (07:09) And like a zippo lighter. And when I'm done with what's going on here, I'm just gonna light it on fire and move on with my life. Like, whether it's like the the work that I'm doing or just like some situation I was dealing with. And I was like, I don't know, but I gotta find where a gas station is. And and I like I I I don't I was thinking that might be, I might be, I might be I I guess I'm gonna be in my fifties soon. And I was thinking like, yeah, that sounds great. Coté (07:36) I may maybe I'm allowed to be cantankerous and just burn the boats. Whitney Lee (07:42) A feature or a bug, but I've always been that type of person. Like, I did this, I got really good at it, and then I decided I was done with it, and I'm never gonna touch it again and do something completely new. And like I I was a professional photographer for decades and I haven't touched a camera, it's collecting dust in my attic. Like I used to play the piano really well, and I'm like, Yeah, did it, done. I went on tour with the band and it's check. Now I'm gonna try tech. I'm gonna go into that career. Whitney Lee (08:08) And at some point, yeah, the I don't know what the light board will look like once it's all melted and it's warps from all the fire. what are you gonna light on fire, Brian, when you're done with it? @BryanRossUK (08:22) Do you know, I think I already kind of have. I made the big Whitney Lee (08:24) Yeah. @BryanRossUK (08:25) step from so I I had a like what I would call a proper job where it was very stressful and I was leading a large team and I was incredibly connected to it all. and I don't think it was good for me long term. Great for development. There was always something new happening, and I and I'd look back on those times with happiness, but I I switched to become a field CTO, which is @BryanRossUK (08:51) A lot more about again, back to being an individual contributor. It's about just helping other people, usually in kind of short bursts, and it's relatively low stress. And I think that was me. I don't I don't think I could ever go back to management and leadership directly. Like don't get me wrong, it might happen, but for right now, I think giving up a lot of that and just dealing with the fringe stuff is kind of more interesting. @BryanRossUK (09:21) not worrying too much about having to do the things that everyone else is doing. And dare I say, like right now, I was having a chat again with one of our one of our ex-colleagues, Coti, about about knowledge and bell curves. And so my thinking is like there there are certain jobs where your knowledge sits in the middle of the bell curve. So you need to turn up nine to five, you need to do the job, and you're kind of paid to do. And for me, I really enjoy the fringes of the bell curve. @BryanRossUK (09:51) So when everyone's looking at one thing, I like to look at something else. or when something says something's great, I'm I'm looking for the flaws in it. Or if something says that something's can't work, I'm the guy that's gonna try and make it happen. So and in in this world of AI, of course, AI is a mathematical principle is looking for the right answer of the bell curve in the middle. And so I think the fringes are like a nice place to play with, right? They're not necessarily getting any direct work done. @BryanRossUK (10:19) But I think they're really valuable if you're not doing something standard. Not quite Coté (10:23) Mm. @BryanRossUK (10:23) sure what I'm going with that, but I think the something there. Whitney Lee (10:27) I like it. I don't know what it is either, but I'm here for it. I wan w for those of us, Brian, who don't know you, will you introduce yourself and t say where you work and what you do? Apparently what you used @BryanRossUK (10:39) Yeah, sure. Whitney Lee (10:40) to do. Yeah, uhhuh. @BryanRossUK (10:41) So I'm Brian Ross. I'm a field CTO at GitLab. been there for just over two years. I the big question, of course, I always get is what exactly is a field CTO and what do they do? the answer for me is that field CTOs are predominantly there to work with customers and we we kind of straddle across sales and customer success and solution architecture. @BryanRossUK (11:09) We've generally done a little bit of everything. So for me in particular, I've spent decades building platforms, running teams. So when I'm looking at problems, I'm able to borrow experiences from lots of different pots. And that helps me s you know solve problems for customers. And again, it's generally starts off with a tech problem and then very quickly turns into a human one. And then we solve those things. Or it might be that I'm recognizing gaps in our product, but again, it's not about what one customer says. It's like, @BryanRossUK (11:38) What's really happening in the industry and what does that mean? And I'm not quite sure why, but I get I get paid to talk as well and have have opinions and write those down, which I find really fun. Whitney Lee (11:52) Wondered so as a so a field CTO, I mean I understand that to be like a higher status position, and so you I imagine, especially like this is a new job for you. So you've only been there two years, so you're you're walking into the company's doors, like the companies that you're serving, not GitLab, but the ones you're you're consulting. Is consulting the right word here? The one you're @BryanRossUK (12:12) I would say so. Whitney Lee (12:13) hanging out with, yeah. you're walking in their doors and you're like, you have a title, but you have nothing else. Like, how do you? Whitney Lee (12:20) How do you win trust? How do you get to know the right things about them? How do you like how do you know their problems and win trust? When you have nothing. Yeah. Uh-huh. @BryanRossUK (12:29) So this might sound a little obvious. I just ask @BryanRossUK (12:32) them. I I think I've got the benefit that I've been in don't let my boyish good looks fool you. I've been in IT for a bit too long. And I Whitney Lee (12:41) Mm-hmm. @BryanRossUK (12:42) think I've kind of earned now the confidence to not know everything. So if I'm working with a with a customer, I will walk in and I'll just ask a bunch of questions. @BryanRossUK (12:53) But I think maybe the difference is I'm not there to to sell them anything. I'm not there necessarily to with any any direction of my own. I honestly just want to understand more about their business, more about them and what they're trying to achieve. And then I'll kind of from there. Whitney Lee (13:09) quite buy it, Brian. I like a like Coté (13:12) W the the the Coté (13:13) boyish good looks part or or is this the other explanation Whitney Lee (13:16) Yeah There's no way you've been in tech longer than two years. Whitney Lee (13:22) By I don't believe that you can say ask so just just ask them what their problems are. I don't think people who are having problems that need solutions have a clear idea of what those problems are or they wouldn't need you. Like the @BryanRossUK (13:35) That's @BryanRossUK (13:36) that's very true. so I must I'm I think the problem is I'm I I don't know I don't have a a script or or a framework or anything fancy that I base it on. For me, it's just again, I've got I've got decades of experience behind me. I've done a lot of stuff. And I I'm generally good at recognizing patterns. And more importantly, I'm I'm fairly good at digging into what the Coté (14:01) Mm. @BryanRossUK (14:02) underlying problem is. Like my ongoing joke. @BryanRossUK (14:05) And it was a customer that came up with this. It was that, you know, the job is chief technical officer, but really it's chief therapy officer. And I've now turned Whitney Lee (14:10) Uh-huh. @BryanRossUK (14:13) that into a brand. So, but that's what it is. It's kind of looking at the psychology of what's going on for that person. They might be in a role that they're kind of they're just in, but I've kind of been in that role and beyond. I've done the becoming a manager thing. I've done the being a manager of manager. I've argued finance and HR. But honestly, I think I. @BryanRossUK (14:34) I have conversations with the curiosity of a child and and thirty years of experience behind me. That's that's really it. Coté (14:42) Well, maybe maybe a way of of rephrasing it. And I think this matches what I've seen with people is is the questions are a tool that you use for the broader thing of diagnosing things, right? And so like if if you go in there, like you're saying, you've got all this experience of stuff that that has happened. And so you start to ask questions to like like a doctor would, I guess. I'm not a doctor. to or or like more analog well, whatever. I'll just say the thing instead of analgizing. But like Coté (15:10) You kind of ask a series of questions to kind of to figure out like I think this might be the issue. And then you ask more questions and you back up. And like it's not so much like you're a manufacturer. How do you manufacture cars? It's it's more like you ask around like the different issues and you try to apply the patterns that you have. and I mean there are moments like you were reminding me of like I we were doing some some planning for a a panel we were gonna have at work. Coté (15:37) And and and you know, sometimes you get asked to do a a a panel and you're like, I'm gonna moderate a panel, that's easy. But then it turns out you're now running the whole thing, which is annoying. I guess. but Whitney Lee (15:48) I've been there. Coté (15:51) it's nice to be paid, I guess. So it's not that annoying. But so, you know, I was planning out whenever I'm moderating something, like I just want people to like introduce themselves really quickly and then get to the questions that I want to ask. Coté (16:04) But someone was reminding me there were some customers in this setting and they were like, you what you should do is ask them what problems they have. And I was like, right, because we're in a commercial situation here and we want to find out what their problems are. And so the whole point being that like that's easy and straightforward to say on this topic what issues you currently have. But when I've observed people like you and other people doing stuff, a lot of what you're doing, well, one to answer on your behalf. Coté (16:30) Or or whatever, like Whitney's question. Also, you've got a whole sales team that like briefs you and tells you what's going on and things like that. So that's always nice to get some some intel there. But then yeah, you just go in there and you're like, So, how's using tickets there? Or you got you got a build automation? What's and then you just kind of see read people's reactions and see how they answer things and like, doing that, huh? And then you just keep diagnosing and asking questions. Whitney Lee (16:55) The way you do it sounds judgy, Kote. @BryanRossUK (16:57) Maybe I'm just very good at asking questions. @BryanRossUK (16:58) Maybe that's what it is. Coté (17:01) That's that's right. Yeah, well that's also I am not in this role because that is that is just that's one of those things I I wanna I wanna douse in gasoline and light on fire. It's just like oof, I don't wanna have that meeting. Just Whitney Lee (17:16) That d do do pe are you invited into the situations they want you there or are you kind of I think it's I think the whole situation changes depending on whether you're kind of muscling your way in or whether you're the there Coté (17:29) Mm. Whitney Lee (17:30) the the people are like, please help me. @BryanRossUK (17:34) So the I am generally I am in I'm being invited usually by the account team. So Whitney Lee (17:42) Mm-hmm. @BryanRossUK (17:43) they've recognized something in the account where they need some help. So it might be they want some help to sell some more software, or it might be that we've sold some software, but we need to increase adoption because we're worried they're not getting value. Or it might be that we've been told they're thinking about leaving us. And so I'll Whitney Lee (17:59) Mm-hmm. @BryanRossUK (17:59) generally get some catapulted into those those situations. @BryanRossUK (18:03) So the customer doesn't generally know it's coming. but it is it is I appear, and then my job is to add value very quickly. And then if I'd again I'm I'm generally good at asking questions, they generally want me to hang around and try and help, which is good. That being said, if they say no, no, we don't need any help from you, then I'm more than happy to to go help some other customers. I've got the benefit that I'm covering a huge area. @BryanRossUK (18:34) and my services are not chargeable to the customer. So my job from a GitLab perspective is to ensure I'm also making some return for them. So it's adding value where I can. Whitney Lee (18:45) Do you Whitney Lee (18:45) have any stories about something that surprised you? You ask a lot of questions and you're like, my god, this is this is what's happening? @BryanRossUK (18:58) I I think the most surprising thing for me is, especially in the IT world, how blinkered some people can be to what's going on. I had one customer, not actually at GitLab, it was at a previous role that I was doing, very similar to Field CTO. And I went to see this customer for first time. And obviously, I need to introduce myself and explain what it as I do. @BryanRossUK (19:22) And I explained I was going to be doing this kind of consultancy and kind of helping along to make things happen. And he immediately blew up and told me that he doesn't need any consultancy. And if that's what we're doing, we may as well cancel this meeting. And in fact, we should cancel all the meetings. because he just needs things done. Now, someone's having a bad day. Now, Whitney Lee (19:41) Wow. Uhhuh. I I think a bad life at that point, yeah. Coté (19:43) Someone someone's having a bad day. @BryanRossUK (19:49) looking over at my account team, I've @BryanRossUK (19:51) I've never really seen anyone that scared before outside of a life or death situation. Whitney Lee (19:55) Ha ha ha. @BryanRossUK (19:56) I think they were kind of worrying that maybe bringing me in was a bad idea. but I then again I've I've had my share of angry men. And so I asked, you meant you said done. What does that look like? Do you mean do you do you mean you need something finished to to like for one internal customer, or do you need all of it finished for a large body of people? @BryanRossUK (20:21) Because I'm guessing what's happened here is that you made a bunch of promises to finance about nine months ago, and they're now asking you for outcomes, and you're not quite sure what to tell them. And immediately the conversation changed. And then we then he wanted to set up a monthly cadence call, one to one, to talk through what we're doing, and he most definitely wanted that consultancy. So but I'm always surprised at how quick people are to @BryanRossUK (20:49) To miss the point of what's going on in their org. Like this one particular gentleman obviously has got this problem around done, but he doesn't have the tools to articulate that to his team. So he's just banging the table effectively. And there's a lot of that that still goes on in IT. So that's been my biggest surprise is that in large enterprise environments, you still have a number of these characters that don't have sometimes the skills to run large bodies of people. @BryanRossUK (21:18) 'Cause that's not a technical job, that's a that's a manager job, right? And they're a different set of skills. Coté (21:23) So in in in in the in Whitney Lee (21:24) Reading you go. Coté (21:25) these large organizations, you must have got exposed to this a lot, but I've been thinking recently, like, should I just accept that the phrase DevOps engineer means someone who works on the build pipeline and that's it? @BryanRossUK (21:39) job titles. I generally Whitney Lee (21:41) Yeah. @BryanRossUK (21:41) don't worry about job titles. So we were sysadmins, and then we were operations engineering, then we were engineers, then we were DevOps, then we're DevSecOps, then we're platform engineers, now we're context engineers, ancient engineers. Like these will continue to change, but I think the thing that doesn't change are the skills that are required. Sometimes a little slightly different mix, but I tend not to pay too much attention to job titles. It's a @BryanRossUK (22:09) It's a vehicle largely for the organization to try and explain what they're trying to achieve. Coté (22:15) Well that that seems like the answer is yes to to that thing. Is is essentially the the what the job the the what the the role does doesn't really ever change. There's just like a slight like a a different name you might apply to it and maybe like slightly different tools. But if we were to take someone working on like a I don't know, it's nineteen eighty two and we have to keep Exxon's mainframe running, it's kind of similar to just like, I don't know, I gotta do this SRE context ops stuff. Coté (22:45) like similar situation that you're gonna do in in processing. You just you just use YAML instead of I don't know how to make a joke about mainframe stuff. but I I don't know. It's it's kind of the same ops thing. @BryanRossUK (22:57) I I think it's subtly different. I think when we change job titles, what we're trying to do an org level is actually change some of the culture. But we're not quite sure how to do that because it's really hard. So we just change job titles instead. So if we think about the difference between a classical sysadmin who might have been working on a mainframe, or you know, Linux boxes, Unix boxes, whatever they are, versus like a DevOps engineer, like DevOps was meant to be bringing @BryanRossUK (23:25) Some understanding of what the end user was trying to achieve, whereas Coté (23:29) Mm. @BryanRossUK (23:30) a sysadmin was very much responsible for keeping that thing running. And I had this myself in past roles where a sysadmin team, the operations team, they they're all they really cared about was keeping that box alive. And they were proud that this Linux box had hundreds of days of uptime. But they they did that by constricting what developers could ever possibly do. @BryanRossUK (23:54) If you want to do anything on that box or change anything, the bureaucracy required to do so is immense. But as long as the box is up, they're happy. DevOps was different. It was about understanding some of the outcome. And I think platform engineering was different again. It was saying, well, actually, your job's not to please one set of developers. Your job is to build something like a product that's going to be valuable to lots of people. But of course, building a product, even though it might use all the same. @BryanRossUK (24:24) SDLC tools that you've been doing in all of those environments, the mindset's different behind b building a product versus helping a team versus just keeping your box alive. So I think it's again, I think we use these titles sometimes to try and influence the culture that we're trying to to embody. Coté (24:41) So like like in that you another related thing is so you know, we used to work at Tonzu together. Now now the Tonzu mindset is you buy the fully integrated opinionated thing and your life is better. Right. And I think, you know, now now that you're out of of Tonzu land, maybe you still think th think the same, but you probably have a little more leniency on the the opinionated platform thing, right? And and the way you think about assembling it. Coté (25:09) At least that's that's a possibility there. So I'm I'm curious, like if if you're away from the you buy the one integrated platform and that's not sort of like, you know, you wake up in the morning and you pour a cup of that to like sip on until at night you have your non caffeinated tea version and then go to bed. I don't know what's up with my analogies tonight. but like like what I don't know, what what's what's your take on all that now? Like is it cool to just like cobble together? I'm trying to be non judgmental. Coté (25:38) assemble together and integrate a bunch of parts yourself? Or is it is it is it a better idea to have like the fully integrated platform? Like what what are people doing out there and what what do you think? @BryanRossUK (25:48) So first and foremost, I think I'm I'm quite proud that I'm not very corporate. I was never very corporate at Tanzu and certainly wasn't at GitLab. Because GitLab's the same, right? The idea is that you've got one piece of software that replaces a whole bunch of others. But we know that the reality, especially in large companies, is more complex. Tool choice isn't just about whether it's good at the job, it's about who owns that tool, who knows the vendor, what are the relationships like, what are the supports, what are the legal agreements, where's the data being held? @BryanRossUK (26:17) Like it's it's a complicated mix. I maybe I'm just an old man, but I think the right answer for IT is always somewhere in the middle. Like, is what you're doing working? And the problem is in IT we generally look at somebody else and try and copy what they're doing. So Coté (26:36) Mm. @BryanRossUK (26:37) to take it slightly away from things like you know, so first of all, I love PaaS and I still do. Always did, always will. It's a really great model for certain environments. @BryanRossUK (26:47) I had this conversation recently about how teams should organize. And so right now it's like hands up who's doing microservices, and all the hands in the room go up, and that's fantastic. And I go, great, great. Why? Why are you doing microservices? Do you remember the purpose of them? Because it's not a technology choice. Microservices was a was was a a new way that we invented after doing things other ways to try and help developers work in parallel. @BryanRossUK (27:17) So, if you've got a relatively small team, microservices might not be a good idea. In fact, a monorepo, a monolith app might be a good idea. Again, I'm old enough to remember how much we managed to achieve by running Linux, MySQL, and PHP on a single box. Like we could do a lot of that, and it was it was very easy to maintain for certain values. And it was very easy to scale as well for certain values. And so I do wonder that. @BryanRossUK (27:47) We always jump to the brand new tech and assume that all old tech is bad. And that's generally not the case. It was not the case with agile versus waterfall. It's not the case with microservices versus monoliths. It's not the same case with public cloud versus private cloud. It's not the case with PaaS versus IaaS or CAS. The world is complex. So Whitney Lee (28:09) Well when @BryanRossUK (28:12) sorry, you go. Whitney Lee (28:12) I totally agree with you, but when when I talk to people who are in that space or I I have a close friend in that space, the it gets frustrated that they it seems like people want the answer to be the new tech. Like that there's some pressure somewhere to use that new tech. AI is the Whitney Lee (28:33) the example, but I imagine it's been happening with cloud and and all of the things. Even when it's not the right tool for the job, it still needs to be shoved in there. How do you well, I guess what do you think about that in general? Do do you find that to be true? And then when you encounter it as a field CTO and you're supposed to be, we'll say counseling people as your field therap chief therapy officer, how do you address that or get people on the the more simple Whitney Lee (29:00) path as opposed to using the latest, greatest thing. @BryanRossUK (29:04) So first of it's hard agree. I think there's a number of organizations, less so these days, but the number of organizations previously that were copying what Facebook were doing, or copying what Google are doing, or copying. And and I think it might be be I might think it might be Scottish. I think you can get away with a lot with this accent. And so my question is usually great, what what do you have in common with Facebook or Google? Like what is it about your business? @BryanRossUK (29:33) That matches what they're doing, which would make all this make sense. Because otherwise, that would just be batshit crazy, right? So and and I'll be honest, my language with them is pro is usually in that kind of style. It's it's pretty harsh because sometimes we need to do that to to break people out of that of the cycle that they're in. And usually they already know. Again, I would say that you know, Kubernetes, obviously, everyone has been using Kubernetes. I @BryanRossUK (30:01) I've not been a huge fan of it because it's pretty complicated. I believe it's great for building a platform because it's like the Rosetta Stone. We can we can finally have a way of describing infrastructure without actually having a direct link. But giving that to developers feels like a huge step backwards. Is it right for everybody? Probably not. Is it right for some? Most definitely. But it it depends, which is the most annoying consultancy phrase in the world. But @BryanRossUK (30:29) That's the large part. I think the detail is in working out what's working for you and being confident enough to make those decisions and measuring your outcomes. Whitney Lee (30:39) Here's another thing I don't understand about your job. And that is, I imagine you walk in, it's similar to what you just said about like a a change of job title often means a company is craving a change of culture somehow. Right. I imagine, and you can tell me for sure, that when you walk into a place, probably the problem is culture related, largely culture related and not software related. Whitney Lee (31:05) Yet your job is to sell software, to get the software, the sales people out of a situation or whatever. How do you how do you reconcile that? Do you just like s if it's a people problem, you do keep it purely at the people level? @BryanRossUK (31:20) So I think this is the difference with a field CTO. So my job isn't necessarily to sell the software, and my job's not necessarily to keep them in GitLab. Sure, that'd be a great outcome, but my view is much more long term than that. So like where we have a a customer that might be making a choice to leave us because they're going to consolidate all their tools. And luck of the dice, they've chosen not to use ours. Now, if what they're doing is sensible. @BryanRossUK (31:49) I'll kind of tell them that's a sensible approach because that's the truth. But what I will say to them is you should bear in mind and keep in touch with us because you will fall out with your new vendor at some stage. Like everybody falls out with a vendor eventually, or they want to make a change, or there's a big feature thing that that happens. Like nothing in IT is going to be permanent. So again, it's it's about finding the right outcome for the customer. And I think that's maybe. @BryanRossUK (32:16) Somewhat unique to a field CTO roles because we've got all of those outcomes in our mind. It's not just about this one thing. It's it's usually bigger. And again, for me, the only outcome I care about is solving the problem that I see in front of me. And I'll do that however best suits the customer. And sometimes that Whitney Lee (32:36) It's @BryanRossUK (32:36) makes me unpopular with my own account team. But that's that's the job, right? It's to look a little bit further ahead and be a little bit more @BryanRossUK (32:45) executive about it? I don't know. That but but bigger thinking. Whitney Lee (32:48) It's Whitney Lee (32:49) is it is the problem ever the other way around where the people situation is just incredible, great culture, great organization, but you really need this one tool that like or maybe several pieces of software that will actually solve everything. @BryanRossUK (33:11) I don't think so. I I think the only thing I the only caveat I'd say is that I've I've been in many organizations that have a phenomenal culture, as in it's a it's a great place to be. It's a company that I'd happily work in, but sometimes there is something in there that isn't functioning well. in fact, many moons ago, I took over a team, phenomenal team, and they were incredible engineers. @BryanRossUK (33:39) And they they did all the right things. They they they were friends. but they took great pride in fixing things themselves. So even Whitney Lee (33:49) Mm. @BryanRossUK (33:49) though we're paying lots of money for vendor contract for support, they would never call the vendor. And they took pride Whitney Lee (33:55) Hm. @BryanRossUK (33:56) in fixing it. And when I took over that team, again, I was not popular because the first thing I did was to contact the vendor and say, How many support tickets we raised? And we're like, We don't even know who you are. Right. @BryanRossUK (34:08) How do I get better support from you? And they were like, well, unfortunately, probably raise more support tickets. And I was like, all right, cool. What I'm gonna do then is I'm gonna automate the process so that whenever we have a problem, I'm gonna raise a support ticket. What information do you want in that ticket? And they're like, How many tickets? I'm like, that's that's a problem for us to solve later. But that's the so it was a phenomenal culture from a human perspective, but there was still an element of brokenness in there. And so I I've never walked into an environment where @BryanRossUK (34:38) The culture is perfect. There is usually something maybe on the business side that maybe doesn't quite click. Or it might be, again, I've come across teams that are phenomenal, but they they have a fear of spending money. They they want to, they see open source software as being free, which to a large extent it is. But if you spend all of your time maintaining and building and upgrading and checking, is that really adding value? Or should you be doing something else like, @BryanRossUK (35:08) building something awesome on top of that software. Those are the types of challenges, Whitney Lee (35:12) It it @BryanRossUK (35:13) I think. Whitney Lee (35:13) culture can be great at a certain size, but then scaling it to remain great is its own problem too. @BryanRossUK (35:21) Yeah. yes. Running a small team is very different from running a big team. and when you get to a large company size, that's like instilling culture and changing it is like it's it's like steering a tanker. I think the the big mistake I usually see is is companies trying too hard to steer quickly. It's gonna be slow. Coté (35:47) So so r related to th things you get surprised by, one of the things, I don't know, it's it sound it sounds like a backwards way of patting myself on the back, but I think I think maybe we all encounter this is like what what what is the law of like once you have a metric, it's crap. Like Betterge's metrics law or something? Like what once once you put a metric in place, it ceases to be a good metric because people game it. Whitney Lee (36:12) Good heart's law. Coté (36:14) Yeah, there you go. That guy. I Coté (36:15) I made a whole page to catalog all these laws because I I can never remember what they are. Whitney Lee (36:20) One already Whitney Lee (36:20) exists, Kote. It's called Hacker Laws. It's on GitHub. Yep. Coté (36:22) There there you go. Yeah, yeah, yeah. Coté (36:24) Yeah. And so you got that. And and like so what's what's weird about that is like, well, doesn't everyone know that? So then therefore they don't do it? And and like, you know, there's other things like you go talk to someone and you're like, have you heard of Conway's Law? And they're like, No. And you're like, Well, okay. But but it's it's it it seems like the metric one, everyone would know that and operate accordingly, but I'm guessing Coté (36:51) you probably encounter people who don't act accordingly. And so maybe is it just inescapable? You need to have metrics, but you should probably change them every six months because they're going to become terrible. Like how how do you deal with that? @BryanRossUK (37:05) Metrics are really hard. Even if you do know what you're doing, I think they're hard. Because I the very act of especially when you're working with engineers. So engineers are generally speaking a certain mindset. If you give them a metric, they're gonna make it good. Or they're gonna try to make it good. And you can use that for for evil purposes, like giving developers a red, amber, green check on their on their @BryanRossUK (37:34) On their pipeline. If they're amber, you'll find they'll turn green eventually, usually by themselves. Like subtle things can change behaviors. I I think the key thing with metrics is to really think about why you need that metric and is it the right one. and understand that all metrics sh need some kind of balance. So at the moment, for example, I've got a lot of organizations. @BryanRossUK (38:04) Screaming that they need to move faster. And I'm all for it. As long as we can maintain quality. So that's Whitney Lee (38:12) Ha ha. @BryanRossUK (38:12) the balancing metric to speed, right? whatever metric it is you want to improve, like there's gonna be need to be something to make sure it doesn't go out control. But also, speed is is that actually the metric that you want? Because is it is it @BryanRossUK (38:30) Why why do we want to move faster? Like these are the types of questions I I get really interested in. It's like, right, so you want to move faster as a team. Great. Great. Why? Like if if you're able to move quicker, what would it mean for the business? What would it mean for the team? Is it that you have a huge backlog of things that you need to work through? Great. In which case, why don't we look at the backlog and think about our burn down rather than just moving faster? Because if we're moving really quickly and we're breaking things at the same rate we're fixing them. @BryanRossUK (39:00) Well, that backlog's never going to increase. That's just busy work at that point, right? Or likewise, if it's about you want to get features out to customers faster because you want to lead the market, phenomenal idea. But that's not the same as developing code quickly, because you're going to have an entire process to get from A to B. And from personal experience, developing the software is not the slow part. Coté (39:26) You know, it it se it seems like so the the the speed thing is another interesting topic where, you know, I've I've I've been around for a while and it's always like we gotta we gotta ship faster. That's always a thing for various reasons. And yeah, I I guess I guess you do get to the point as I don't know who you is here, someone like me where you're like, I don't know, I don't think you actually do need to ship faster. Like what you're fine. Like the the the the the Coté (39:54) The and it not and the way I think about it a lot is like I have a pharmacy down the street and like they don't need to change. They're doing a great job. I don't I you know they I don't need their software to be better. I don't need anything. It's great. Now that that said, they did put one of those drug ATM machines out there, and that's pretty awesome. But still, I don't know if the pharmacy industry needs to change that much. So and and then if I look at banking. Coté (40:18) It's not so much that I want banking to change, is like I would like transferring money in the US to be as easy as transferring money in in Europe. Other than that, I'm cool, right? It's just these little things that I don't think they need to ship very much. They should just fix this one thing. And so it does start me thinking, it seems like outside of tech, you might want to be able to respond quickly to something, but you don't really need to ship features that quickly. Coté (40:47) Like what what are you gonna do every three months? It it seems like it's something you don't actually need. @BryanRossUK (40:54) So I think this is about the business really understanding what it is they're trying to achieve and communicating that all the way through. there was a great example in the UK from years ago, and I will no doubt get this story wrong, but it was about the UK Postal Service, and they wanted to improve their customer satisfaction statistics. So they spent a huge amount of time and effort building out new logistical centers, really modernizing all of their technology, modernizing their delivery fleet, did all of this work and then @BryanRossUK (41:23) Put the surveys back out, numbers didn't change. So then they dug in a little bit deeper into why certain towns or certain areas have got high scores, even though when they look at the data, it it doesn't reflect the customer satisfaction. Like parcel delivery can be slower, but customer satisfaction is through the roof. And it was obvious. It was the local posty. If the posty that's delivering your mail was pleasant and nice to you and you had a good relationship with them. @BryanRossUK (41:53) You were happy with the service. You were happy to wait effectively because the service was a joy to use. Speed is not always everything. I think that's incredibly true in IT now because we we need user interfaces that are pleasant to use. Like I think we we left behind many pieces of old, clunky bits of software and good riddance to them. Like they were maybe good, but @BryanRossUK (42:22) using them was just awful. And I'm kind of glad we're past that now. does that make sense? Coté (42:24) Hmm. Yeah, yeah. I I mean I mean, you know, Coté (42:28) c counter what I'm saying was saying, there is a lot of software that I would like to ship features more frequently in. And but but it you like like you the way you were phrasing it does it I think in my mind what I would substitute is is do you wanna be faster or would you like to get more stuff done? Which is a silly way of putting it. But like you've got a big pile of stuff that's never getting done and you is like you never ship like Coté (42:53) You just want to get through your backlog. And like it seems like it's fast because you have a lot of stuff to release, but really it's just like like that example you're using of like I want to be done. Just like, well, yeah, like you do want to be done. You're just circling around the schedule over and over again. Whitney Lee (43:08) And I think we're not mentioning like the fear of a competitor, if you're not shipping the features, then if you don't have the pharmacy ATM, then the the pharmacy next door does have the pharmacy ATM and that thing's cool and it saves a lot of time and it doesn't smell weird like it does when you go inside. And so and so you switch pharmacy vendors. Coté (43:31) That's true. Coté (43:31) That's true. Yeah. Yeah. Th the you know, this this exactly this case study maybe actually gets to a lot of the points you're making, Brian, because here in the Netherlands, I wouldn't say the pharmacy business is very competitive. Like it's not really it's not like it's not like back in America where like there will literally potentially be three different pharmacies on this at the same intersection that you could go to. And like they're pretty much exactly the same. Coté (44:00) And and it's pretty easy to transfer things around. I don't I don't know why, but like it does feel like in the US there's competition in pharmacies. Whereas here it's kind of like I just the the major feature of the pharmacy here is it's a three minute walk from my house. And and like there's nothing else going for any pharmacy. Like they're they're exactly the same. I don't know. Whitney Lee (44:22) In the US Whitney Lee (44:23) it feels like competition everywhere all the time, all at once. Yeah. Coté (44:25) Yeah, yeah. And and then Coté (44:27) and then so maybe to that end, that's one reason another reason the pharmacy doesn't have to change. I I I don't know. It d it I got nothing else to say on the topic. @BryanRossUK (44:38) I think it's about you just need to be clear about what you're trying to achieve. Like the definition of done, right? It's like, what is that, what does that look like for you? I I was just thinking of the like the opposite of a time where people were asking me to slow down, but I really didn't want to. Coté (44:55) Mm. @BryanRossUK (44:56) and so, for example, in this one, it was, you know, I'll date myself. It was about how quickly we could build VMs. And I remember @BryanRossUK (45:06) Someone again from an ops team coming to me and saying, Well, people don't need VMs that quickly. And for me, I was pretty young and I I would I didn't have all of the business knowledge, but intrinsically I knew that the the way we were delivering things was really slow and that slowed everything else down. But also the way we were doing it was not very repeatable. It was fine, it worked, but it wasn't @BryanRossUK (45:34) Joyful, dare I say, and I wanted to make it better. And that's always been me in IT. I've always wanted to make the process better. But the outcome from that was so was that. Whitney Lee (45:38) Those those joyful VM provisionings. Whitney Lee (45:44) Those joyful VM provisionings that we all experience. Yeah. @BryanRossUK (45:48) And the difference in the in the org was was phenomenal because all of a sudden you could get VMs on demand without contacting anyone. You could provision them, you could configure them, and when you were finished with them, you could decommission them. The and the thing that I was most interested in is all of obviously all of my bosses at the time wanted metrics on how many VMs are being provisioned. Which I had to give them, but that's not the metric that I was interested in. @BryanRossUK (46:18) The metric I was interested in is how many VMs are being decommissioned. Because provisioning just tells us we're doing more of the same thing that we were doing before, just faster. Decommissioning tells us that teams are operating in a different way. I wanted them to get to a point where they're building VMs, doing a release, and then scrapping those VMs. Dare I say pivotal-esque repaving, right? and and that's what we achieved from that. But @BryanRossUK (46:48) So I think and again the back the Postal Service, they they maybe didn't achieve their goal, but having better operations and delivering parcels faster was a good outcome in itself. It's just not the one they were aiming at. So I think for all businesses it comes back to thinking carefully about what it is you want to achieve and why, and then articulating that in a way that people can do their best work to to lead in that direction. Coté (47:15) Mm. So as as you mentioned, part part of part of your professional life, I don't know if this is in your personal life, is like, I don't know, publishing and speaking and things like that. And so you you you like so many people have a Substack newsletter. You you write, I I don't know what cadence you have, but you write frequently enough on it. Now you've also like written stuff even recently in the news stack. We used to do webinars together, all sorts of stuff like that. And so so narrow down a little bit, like when you write something for Substack. Coté (47:45) How is that different than when you like write something for the new stack? Two stacks? Like what how do you cause you don't write the same thing for each? @BryanRossUK (47:54) That's very true. so generally speaking, the the writing that I do in my Substack, which is chief therapy officer, unsurprisingly, is is my own personal ranting. And that's the way that I approach that. When I'm being published elsewhere for in the news stack or other places, like a most of that is for work. @BryanRossUK (48:21) and so for the work stuff, there will generally be a more direct link to what my company is doing and what's happening in that space. But I I do avoid any form of direct writing, if that makes sense. So I will never really be writing about a GitLab feature or a specific thing. I'm writing Coté (48:43) Mm. Right, right, right, right. @BryanRossUK (48:44) about the problems that someone's having. And then inference, of course, is that. @BryanRossUK (48:48) If my name's there, then there's probably a GitLab solution for that. but no, my personal writing, I think, is is my opportunity just to just to rant more than anything else. or just get just understand myself. I think sometimes just writing things down can be really useful. Like recently I was talking about just this feeling of exhaustion, in which I've never had in my role before. And I worked out that I think it was just AI. It's the fear of being left behind, it's a whole bunch of other Coté (49:17) Mm. @BryanRossUK (49:17) things. @BryanRossUK (49:19) you know, that's not really directly to do with work, but it is something that I'm talking to hundreds of people about because when I go and see big enterprises and I'm chatting to graduates or chatting to people that are secure in their careers, or those like me that they're kind of, you know, that the the almost the peaks of where they're at, they've all got these concerns and they're all slightly different. So yeah, that's my general Whitney Lee (49:40) What @BryanRossUK (49:41) split. Whitney Lee (49:42) what are you learning about AI adoption and and approaches? I guess both from a perspective of talking to so many different orgs about how they're doing it and also from a perspective of your long career seeing thirty years worth of of tech transformations. @BryanRossUK (50:03) I think that hours and hours of discussion. I think it is so big that you can spend a long time. I @BryanRossUK (50:14) I'm learning a lot just from my own use. @BryanRossUK (50:22) And and building come building some confidence in where I sit and where the computer sits. Whitney Lee (50:29) Mm. @BryanRossUK (50:30) I have been reading a lot of philosophy books rather than tech books, which has been interesting. and I've not been reading any of Twitter around here's my new harness, here's my new loop, here's my new graph. @BryanRossUK (50:49) I'm I'm not getting too far into that. I'm I'm reading the stories of sw work at home mums in America that have deployed open claw on Mac minis. I find that Whitney Lee (51:03) There you go. @BryanRossUK (51:05) I find people outside of tech now coming into tech much more interesting than us doing things a little bit faster than we were before. I'm interested in removing toil and also as well, I I think the the current @BryanRossUK (51:21) Current storm of news on Twitter about how much work I can achieve in one day by running eight agents in parallel and then running them overnight. Again, I'm I'm old enough to remember how much time and agony we spent explaining to managers the importance of context switching for developers and how they should have meeting free Fridays and how we need to work out that we have meetings in the morning and then the late afternoon. So they've got un @BryanRossUK (51:49) You know, dedicated time for programming. And now the the same people that we're trying to help are bragging about how distracted they are. Again, it's natural. We're finding so we're finding out new things. There's a whole bunch of people burning out as a result of that, which isn't too surprising. But again, it's all brand new. There's so much to unpick. but again, coming back to this bell curve, I've realized that I'm loving the fringes. Whitney Lee (52:14) Yeah. @BryanRossUK (52:14) It's not @BryanRossUK (52:15) the the the the main core, the main stay, it's it's all the fringe stuff that I really I'm interested in. Whitney Lee (52:22) Despite that, can you like what issues like since you see lots of companies, what issues are companies going through at from the Belkar level? Like what common @BryanRossUK (52:33) So @BryanRossUK (52:35) I I think the first is everyone's trying to work out where it fits in their org. And Whitney Lee (52:40) Mm-hmm. @BryanRossUK (52:44) we where to start is usually a problem. we are looking heavily at the moment at adoption rather than outcomes. And again, I think that's probably okay right now, because we're still exploring, right? @BryanRossUK (53:00) But very quickly we're gonna have to start thinking about what the value looks like. So everyone's talking about 10x developers and coming back to our point about what does faster mean? Like, how do you get 10x the value? Is it 10x? and if it's not, then maybe we just need like 3x developers, but happier ones. I don't know. So I think there's a lot in there. the other problems that we're seeing now is huge amounts of tooling, which is very reminiscent of the DevOps world. @BryanRossUK (53:30) So we spent, I don't know, a good couple of years building a whole bunch of new awesome tools, and they were phenomenal. And it changed sysadmin forever. Fabulous. We are still unpicking that though, because every organization I go into, they've got separate teams with separate tools. They're all using Jenkins somewhere. And they've they've not got one Jenkins box, they've all got five or six. And they promised the next one's going to be their clean one, right? And I know this because I did Whitney Lee (53:56) Ha ha. @BryanRossUK (53:57) the same thing. @BryanRossUK (53:58) So I think there's gonna have to be a consolidation at some stage. But again, it's it's difficult to know when to do that. the the big thing is there's no experts and and there's no there's no right answers right now and everything's changing. So I think the best advice that I've got for for my customers is to stay flexible. Don't tie yourself into any one frontier model. don't tie yourself into any one single way working. @BryanRossUK (54:28) And do your best to remember that you you kinda want to get the AI knowledge disseminated in the organization. At the pro my big problem at the moment is that in all the orgs I see, and I don't know how to fix this yet, is that we have a a new focus on regenerating rather than building and reusing. So we used to build tools and then share them and reuse. And now, even for me, I'm @BryanRossUK (54:57) And skills are they're in the right kind of direction, but the packaging of a skill is not brilliant just now. But that's the big thing. I I I don't have great ways of getting the knowledge from one person to another. One of the things that we are doing, and again, not to sell GitLab, but it's it's where the product kind of helps, is that I think AI might be a bit like spreadsheets. So everyone here hopefully has used a spreadsheet. Everyone here @BryanRossUK (55:27) probably is comfortable using a pivot table. I'm a little ropey when it comes to creating pivot tables. And I'm doubtful that anyone here is writing Visual Basic to make macros inside of spreadsheets. And I think AI might be the same. We're we're trying to get everyone to be AI experts and that might not be the way. We might instead want to find a way that we can have experts building great tools. @BryanRossUK (55:56) Making them shareable so that people can use them directly and get value out of them, and having them tinkerable so these power users can take them, edit them, and move forward with them. And one of the best uses I've seen of that with in our own products has been we have a bunch of agents built in the product, as you'd expect, but you can you can copy and paste them and edit them. So with GitLab, because we're a a multi-global company with people all over the world. @BryanRossUK (56:23) The biggest use I see is people just adding a line at the bottom of each agent saying, Can you give me the responses in German or French or Spanish? A tiny change for them, but it they didn't have to re-engineer the whole thing to make it happen. But I think that's gonna be the big challenge is how do we get how do we share the knowledge across a business? yeah. Whitney Lee (56:42) So and and Whitney Lee (56:44) what you're describing is similar to a platform engineering pattern, is it not? Yeah. @BryanRossUK (56:49) I I'm I I'm very biased. I've been platform engineering for too long now. But yes, I I think that whatever you want to call the next version of platform engineering, I've heard agent engineering being thrown around a bit, or agent platforms. I think platforms are probably the way. So a user has something they're trying to do in an SDLC. @BryanRossUK (57:15) The platform should probably make some kind of decision about which model is best for that task. And that choice of model should probably be decided by the finance team and the data governance team, not developers. That's that's not a popular opinion right now, but I think it's gonna become that way. Because models are a data protection problem first and foremost, and they're gonna become a finance problem very quickly. So Whitney Lee (57:43) Mm-hmm. @BryanRossUK (57:43) which model you use. @BryanRossUK (57:45) don't think you actually care as long as you get a good output. But that's a tuning problem. But yeah, building these into platforms and making them usable, having some level of abstraction, some level of pluggability is I think going to be the way forward. Whitney Lee (58:00) And a way to communicate so that not everyone's learning the same lessons, like which which models best for which task. We don't have to all figure that individual individually. We can perpetuate that knowledge somehow. then I work at Datadog, and Datadog has what they call a agentic engineering evaluation platform. So they have a an eval platform that works for developers, individual coding agents. Whitney Lee (58:27) That helps evaluate whether those coding agents are creating code to data dog standards. And then similar to the platform engineering model, we have engineers who own services who who submit evals who say what coding toward the service or related to the service should look like. And then that eval score and recommendations get sent back to the developer as they use those services, as their coding agent does. @BryanRossUK (58:54) Just to go on a bit of a side rant as well, I this is one of the things I in IT at the moment that we're forgetting that a lot of what we're doing with AI is actually old news. Like Daris, I Whitney Lee (59:04) Yeah. @BryanRossUK (59:04) like the you mentioned Conway's Law, Cotia. One of my favorite things about Conway's Law is it was obviously popularized in IT relatively mo relatively recently, but it goes all the way back to like 1960s farming in America, because they couldn't work Whitney Lee (59:19) Yeah. @BryanRossUK (59:19) out why some farmers wouldn't adopt new ways of working or new machinery. @BryanRossUK (59:24) That's where it originally spawned from. Sorry, no, that's I've Whitney Lee (59:26) So for the @BryanRossUK (59:28) got my I've got my laws wrong. That's not no Conway's law was Melvin. Whitney Lee (59:31) Yeah. @BryanRossUK (59:32) I was thinking of diffusion of innovation, which is similar. Whitney Lee (59:36) Let's it I was let's let's state these laws. Conway's law is like the the shape of the teams is gonna the sh shape of your software is gonna mirror the shape of your organization. Yes. @BryanRossUK (59:46) Yes. Whitney Lee (59:47) Okay. And what's your law? The d distribution? @BryanRossUK (59:48) And sorry, diffusion @BryanRossUK (59:49) of innovation is how groups of people go about adopting new things. So Whitney Lee (59:56) Mm. @BryanRossUK (59:57) you will have a bunch of people, and it's it's a bell curve, unsurprisingly, you will have a bunch of people when it comes to new tech, they're gonna try it, even if it's terrible, because they just love the idea of trying new tech. And at the @BryanRossUK (01:00:07) other end, you're gonna have people that will avoid new tech or new ways of working, even though it's been demonstrated by everyone that it is better by all imaginable metrics. @BryanRossUK (01:00:18) And then in the middle, you have other groups. But the the key thing about this this kind of this law is it's the way you approach these different groups is different. So for example, your early adopters, you just need to make it available. But the same thing that helps them is not going to be the same thing that helps like early adopters. They just need to know that they're not the only ones and it has some kind of value. They're kind of late majority, they need to have @BryanRossUK (01:00:47) Some evidence that it's going to be Whitney Lee (01:00:49) Mm-hmm. @BryanRossUK (01:00:49) good for them and that there's no risks. You know, so that's but the the key thing for Etho is that it's it's an old, old rule that still applies. And AI's in the same kind of space just now. I've got lots of people talking about the difference between models, agents, harnesses, loops, and now graphs. @BryanRossUK (01:01:12) And what we're really trying to achieve with all of these things is wrapping non-determinism in some level of determinism to achieve a goal. Dare I say, that's sounds a bit like CI to me because developers were the most non-deterministic thing we've ever worked with. So, Whitney Lee (01:01:31) Mm. @BryanRossUK (01:01:31) you know, this idea of building some code and then you keep going until the code is right, passes some tests, and then continue going until you @BryanRossUK (01:01:40) can prove that it works in a larger set, integration testing, staging and prod, making so security gates, like all these things are roughly the same. I think what we're gonna need to see is a mix of these old new worlds come together, but for the time being, it's all about the new. So we'll see how it goes. Coté (01:01:57) So y Whitney Lee (01:01:58) I do think it's Whitney Lee (01:01:58) a no a nice mental model to have to just think of a coding agent as a person and should be Coté (01:02:05) Mm. Whitney Lee (01:02:06) secured as such. Like it should have limited access to like it should only have access to the the context that it should have access to and nothing more. Like just like it's just another human in the system. It technically it's not a human, but it'll mm it has higher impact for better or worse than maybe a single human does. Whitney Lee (01:02:25) But for all intents and purposes, in terms of security, you can treat it as one. @BryanRossUK (01:02:31) A question you guys, actually, which is one that I've been pondering and I'm not quite sure. I need to go check some old IT books. but I'm I think part of the problem we've got with testing just now, with humans, which were non-deterministic, and agents, which are non-deterministic, is I think the nature of the mistakes that humans make are different from the mistakes that agents make. And I'm not sure that the testing we do at the moment. @BryanRossUK (01:02:59) is gonna be great for agents. Or more specifically, I think we've spent a lot of time developing tests that match the mistakes that humans make. I'm wondering if we need new tests that match what agents do instead. like humans tend to make s Whitney Lee (01:03:15) Do have an example? Yeah. @BryanRossUK (01:03:17) smaller errors, like they're mistakes, Whitney Lee (01:03:20) Mm. @BryanRossUK (01:03:20) whereas agent stuff seems to be more a systemic problem in their thinking. Coté (01:03:26) Mm. Whitney Lee (01:03:27) But so when I was a kid and I went to like a restaurant and got like a paper placemat and it had a maze on it, I used to think that a fun way to fill in the maze was to take all the dead ends and color them in till it's not and then do it throughout the whole maze and then the only thing that was left was the possible solution. And I feel like with humans, you just want to make sure they stay in the maze. I don't know, with with AI, you have to think of everything it's not. Whitney Lee (01:03:56) and account for that 'cause it's gonna just explore every single little edge and like a human's just not going to get it that degree of wrong that an AI is going to try or kind of try the the like the edges of the edges of the edges that an AI might do. So Coté (01:04:13) Mm. Whitney Lee (01:04:13) i I don't know if that answers your question or is related, but I just keep thinking of a mace and coloring in all the wrong ways and with if Coté (01:04:23) Yeah, it Coté (01:04:23) it it se it seems like @BryanRossUK (01:04:24) I can't define it yet, but @BryanRossUK (01:04:25) I know there's that I know there's a difference there. When I'm when Whitney Lee (01:04:27) Yeah. @BryanRossUK (01:04:28) I'm arguing with with Claude, other models are available, but when I'm arguing with Claude, it's a very different argument that I've had with any engineer that I've worked with in any of my teams. And that's Whitney Lee (01:04:41) Yeah. @BryanRossUK (01:04:42) I think that's that's gonna become valuable. I just don't know what it looks like yet. Coté (01:04:48) Well, if it you know, you know, ba based based on what you were saying, Whitney, like I think and Whitney Lee (01:04:48) Well, it's hard to k Uhhuh. Coté (01:04:52) like my experience is AI doesn't make mistakes, it just does the wrong thing, which is which is different than like what what humans do, right? And so like you're always telling it like, no, no, no, that's not what I want. You gotta do this thing. And even even when it's like checking itself, sometimes there's misunderstandings, but it'll be like, I was doing the wrong thing. I'm gonna go back and do the right thing, right? Like if you have it like Coté (01:05:17) Look at your finances and all this stuff. It's never like, I put in a square bracket when it should have been a curly bracket. Or like, you know, I I I do the did the tertiary question mark colon thing wrong. It's always like, I just did the wrong thing and now I need to do something different, which is w what do you call that kind of QAing? That's f not functional. It's the other type of thing. You're seeing if it actually does, as you like to say, Brian, the business outcome that that you want. @BryanRossUK (01:05:45) There's another problem built in as well. So, one of my top tips for anyone that's into management and they're doing spreadsheets for the first time is that, and it was a friend of mine that that gave me this golden method for spreadsheets. And he's like, you just make the font really small and put in lots of pastel shades. And the sad thing is it works because if your spreadsheet looks nice and is well organized. @BryanRossUK (01:06:14) And there's lots of numbers there with all the detail. Humans have a terrible bias where they'll say, well, looks good to me. Like that there's a that you start with the assumption that it's right. Whereas if you have a spreadsheet that is unformatted, the cells are terrible, they've not used formulas properly, there's a there's a feeling that's something wrong here. And that's where that's the problem with AI, right? It's like we're now generating reports from AI and things like that. And it's like, @BryanRossUK (01:06:44) That looks amazing. It's probably right. Like my favorite is that just now we got lots of people taking raw data into a model and then asking it to score the outcome on like one to ten. And AI is phenomenal. Break it down into categories. Well, for this category you got six out of ten. For this one you get eight out of ten. that's great. You've got this toll number. Phenomenal. Great. Apartments all made up. @BryanRossUK (01:07:09) There's no there's no math behind it. And I th but as a human, it's a really nice looking report. We should definitely send that to management. Coté (01:07:16) Well, you know, I think we've done a pretty good job here of only making up ten to twenty five percent of things, which is which is probably more successful than a lot of AIs that are out there. It's it's it's it's been good talking with you about all of this stuff ranging around here. I I had a few more questions, but maybe at some point I'll ask you them later, which would been good stuff. Whitney Lee (01:07:37) This has been wildly fun. Whitney Lee (01:07:39) Yeah. Coté (01:07:40) Yeah, yeah. Coté (01:07:40) So so you I mentioned you have a substack letter, you said the name of it. Where w w do you have any pages on the World Wide Web people could go look at if they're interested in more therapy? @BryanRossUK (01:07:51) yes, if they just go to Brian Ross dot me, they will get to me. and that's got links to all my things. But yeah, the main the main place is my substack, which is chief therapy officer dot coduk. Coté (01:08:03) So Whitney Lee (01:08:03) And Whitney Lee (01:08:04) Brian is spelled with a Y. Yeah. @BryanRossUK (01:08:06) It is yes, apologies for @BryanRossUK (01:08:07) that. Coté (01:08:08) Is that do do you like that? @BryanRossUK (01:08:11) I now that I'm older, I do. When I was younger, I I did have questions from my parents, but it's it's worked out alright. The best the best Coté (01:08:18) all right. Yeah, yeah. I @BryanRossUK (01:08:20) thing really is having two first names because that that confuses everyone. So I get called Ross a lot of the time. And so Coté (01:08:26) Yeah, yeah, yeah. @BryanRossUK (01:08:28) much like your collection of chairs in the Netherlands, what I've got is a collection of emails with my name spelt wrong. my favorite Coté (01:08:35) Yeah. @BryanRossUK (01:08:36) so far is Byron Rose. And if I ever start writing Coté (01:08:38) Byron Rose. Wow. @BryanRossUK (01:08:39) books, that's I'm going with. Coté (01:08:41) Right. I think, I think, I think that was that was one of Scotland's poet laureates in the 1800s. Yeah, it's like I I I have, you know, one of my one of my old older friends is this guy, Barton George, and he he he lives that problem all the time. well you've also listeners lived through a thrilling problem of another episode of Software Defined Interviews. This is episode number, let me scroll to find it, 124. Coté (01:09:08) So if you want to see the show notes for this episode, links to the dot me accounts and the sub stacks that I know you're you're you're craving, you go to software to find Whitney Lee (01:09:15) And hacker laws. Coté (01:09:17) yes, and and and hacker laws. I I have even more laws that are very my own pet peeves and things. But Whitney Lee (01:09:24) okay. Coté (01:09:25) anyways, yes, yes, it's true. you go to software defined interviews.com slash one two four and and you can find all of that. Now I've got to go read up on my laws. Coté (01:09:36) Over over in Hackerland. We'll see everyone next time. Whitney Lee (01:09:40) Bye. @BryanRossUK (01:09:41) Thank you. Cheers. Coté (01:09:42) I'll stop the recording. That's how I should end all episode audio. I'll stop the recording. @BryanRossUK (01:09:47) I'll @BryanRossUK (01:09:47) stop recording.