[00:00:00] Whitney Lee: Coté, you speak at conferences often, yes?

[00:00:05] Coté: Yes

[00:00:05] Whitney Lee: Do you do, like, live demos?

[00:00:08] Coté: Ooh, no.

[00:00:10] Whitney Lee: Never?

[00:00:10] Coté: No, 'cause I don't, I don't really do anything in the terminal anymore, and you know, so I think... Did I ever do a live demo? No, I don't, I don't know

[00:00:20] Whitney Lee: do you do any, like, risky thing on stage? Like something you don't know how it's gonna turn out?

[00:00:27] Coté: Uh, sometimes. I mean, I guess, I guess the risky things I will do are things that I haven't, like, rehearsed or said before. Like, like for example, one time, you know, I did, uh, I did some video series that was like the business bullshit dictionary, where I was like, "When someone says parking lot, here's what they mean," and here's, here's like what-- If you work for an American manager, here's a very important thing to listen to.

[00:00:53] Uh, the, you know. And so I, I had a talk I was giving, and I, and I thought I needed to-- This is always the problem. You think the talk's gonna be too short. And I thought I needed to fill time, so I was like, "I'm gonna interject these into here between slides." And I didn't actually time it out, and it was way too long, so I had to, uh...

[00:01:11] I don't know. So I'll do stuff like that where I'm like, even if I've given this talk before, like I'm gonna do something different and just see what happens. And that's-- I guess I would say I know that's risky 'cause oftentimes while I'm doing it and listening to myself, I'll be like, "This is stupid." But

[00:01:30] Whitney Lee: Uh, d- when, h- when have you seen someone do something risky?

[00:01:34] Coté: Uh, I guess, I, I mean, to over- overthink it, I guess it's always risky when you do something. People just succeed at it sometimes. But I think I don't know if it's risky. I mean, there's, there's all the answers of like, "I'm gonna use this thing that depends on conference Wi-Fi that goes to whatever." And like, I mean, the riskiest is like if you're not using a public cloud and you've got to like tail scale or VPN into something of your own that's, you know, on the other side of the world, that's just not wise.

[00:02:05] Uh, but there was... I, I remember a couple years ago, Adam Jacobs, you know, of Chef and System Initiative fame and other stuff, he's one of these people, for better or worse, who you can tell worked on his presentation on the plane over. Uh, and, and like not only had he- does he do that, but he had decided that he wasn't gonna give his presentation, so he just like went through a Miro board of ideas that he had.

[00:02:33] And normally, as, as when he was doing it, I was like, "Look at this joker with his like Miro board stuff." And, uh, but I think he did all right. I, I think, I think, I mean, I think that was a big risk of like, I just got the Miro board out, and I just like went mind map, Post-It note crazy, and, and let's see what happens.

[00:02:54] And I think he, he figured out, 'cause the, you know, you do choose your own adventure stuff, but, you know, a little, little behind the curtains, very structured, right? Like you, you know, you've mapped it out and you know what's gonna happen. Whereas with that, I think, I think the dude just like, just like put a bunch of shit in a Miro board and like glided around it, and it worked.

[00:03:14] Whitney Lee: I, don't enjoy when people don't prep because I feel like my time is very valuable and I want them to treat my time as valuable. As a presenter, I do a lot of prep because I want to respect my audience members' time.

[00:03:26] Coté: Yeah, yeah

[00:03:26] Whitney Lee: side, I do enjoy very much when people don't use slides at all. Like even, it doesn't

[00:03:32] Coté: Sure

[00:03:32] Whitney Lee: demo.

[00:03:33] It could just be like, "Hey, now I'm on the website of the, you know, I'm in the documentation highlighting certain words," or, "I've, now I've jumped into the Git repo and I'm going

[00:03:42] Coté: Yeah, yeah, yeah. Yeah, I'm, I'm sure,

[00:03:44] Whitney Lee: board example.

[00:03:45] Coté: I'm, I'm sure, I'm sure you've seen th- this, lots of people do, but you know, our friend Josh Long, he's, he does that great where even his bio slide is actually just a terminal window where you see a markdown file that he's, he's like lest or whatever. But I guess the other, the, the final like risky thing that comes to mind is like, I don't know if this is a bit like showy or arrogant, but you know, when you've got to do like a, like an interview or a panel and you've spent like f- I don't know, 20 hours on prep going over like things with, and like the questions they're gonna ask and everything, and like I think that's risky in the sense that it could be hella boring, right?

[00:04:23] Like, if, if people just go through like, you're like, "I call on this person, then, uh, then it's C, then it's D," and the follow-up, and it's just like, it's like not good

[00:04:43] Yeah, yeah. And then, and then, you know, as we demonstrate on this podcast, instantly you wanna just like go off into the weeds about something that's more interesting than whatever you prepped for

[00:04:56] Whitney Lee: Um, guest today is Lin Sun, and Lin, um, is a KubeCon co-chair, which means that she gives a keynote every KubeCon, and this most recent KubeCon, Lin, you took a big risk. Will you introduce yourself, please, and will you talk about your KubeCon demo?

[00:05:16] Lin Sun: Hi, Whitney. Hi, Coté. Thank you so much for having me. I'm very excited to talk to both of you. As Whitney mentioned, I was one of the most recent KubeCon co-chairs and, uh, um, in my Amsterdam KubeCon keynote, I sort of took a interesting approach. Uh, I wanted my keynote to be memorable so I would started thinking about how can I make it memorable? Uh, 'cause I know there are so many, uh, KubeCon co-chairs out there, right? Every year we have, uh, like two or three co-chairs every year and, uh, you know, I, I wa- my, my goal is I want the keynote to be memorable. So, so I started thinking about live demo is a important puzzle of making it memorable. I

[00:06:13] Whitney Lee: And,

[00:06:13] Lin Sun: like that's

[00:06:14] Whitney Lee: and just to say here too, if for those who are unfamiliar, this is the main stage of KubeCon with like something like 10,000 people in

[00:06:23] Coté: Yeah,

[00:06:24] Whitney Lee: So,

[00:06:25] Coté: big

[00:06:26] Whitney Lee: e- even just a regular simple live demo feels like a lot because they're, the stakes are really high and the li- the conference Wi-Fi is especially not going to be great with that many people in the room using it. Okay, go on. What did you do, Lin?

[00:06:41] Lin Sun: Yeah. Yeah, in Amsterdam, that's a, a great point. We had 13, I think 13,000 people. Um, so I decided, first of all, I want a live demo. Uh, second of all, I want to show something AI related because that's what's hot. Uh, third of all, uh, I want to show something that has a physical object, uh, that people can relate with. So that's why I ended, uh, choosing a flying object which is, is a drone. And I, if you know me, I always love to interact with the audience. Uh, so I like to use camera video as part of my demo because I think people love to see themself on stage, and that's one way for me to invite the audience to be on the stage with me. Uh, I think people definitely, you know, if you are sleeping and then the moment you saw yourself on the big screen, you better, you better be smiling and, you know, have your eyes wide open. So that's the, the demo effect. Um, so it was a very risky demo. Honestly, uh, I had to, you know, not only fight with Wi-Fi, but also fight with, uh, not reliable connectivity between my flying object to my laptop, and also transfer the stream.

[00:08:09] And, uh, right in the middle of the demo, it actually failed multiple times, so that was pretty risky. The first time it failed, um, it was because, um, Wi-Fi, because I was connected to the conference Wi-Fi. So I immediately was able to debugging that and fall back to the Ethernet cable, which I was lucky 'cause I was the keynote speaker, so they reserved a Ethernet cable for me ahead of time, so I was able to fall back that. Um, the second time it failed was, um, the un- unfortunately, the, the communication between the drone to my laptop was, uh, was not very reliable when the Wi-Fi is very congested. And what's make this demo really challenging is because I had to kind of switch in between Ethernet to, uh, the Wi-Fi to connect to my flying object.

[00:09:08] And I have to kind of remember when I'm not connecting to the drone, I have to flip back to the Ethernet

[00:09:17] Whitney Lee: you

[00:09:17] can get 

[00:09:18] Lin Sun: the reliable connectivity.

[00:09:20] Whitney Lee: So your demo was a drone that flies, that takes video of the audience and feeds that video back to the computer, which you then use AI to analyze the video. Is that correct?

[00:09:32] Lin Sun: Yeah, that's kind of the goal, but I kind of, I didn't achieve 100% of the goal because of the congested Wi-Fi at the conference. So the idea was, um, be able to control, um, the flying drone from my application running in Kubernetes and be able to, you know, fly to a certain, uh, height and

[00:09:59] Whitney Lee: Uh-huh.

[00:10:00] Lin Sun: to kind of turn the, drone to one way to take a picture and then turn it to the other way, the opposite way, and take another picture and kind of have the audience to have like a different gesture so we can transform the both of the picture to, uh, to a AI model which is running my laptop, uh, it's a coin model to kind of analyze the engagement of the audience. So that was the gist of the demo, uh, along with like, um, AI agents running in K Agent to help s- troubleshooting routing problems along with MCP servers, along with, uh, agentgateway, along with Istio service mesh providing visibility mutual TLS along with using like premises OTel and Kubernetes of course. So a lot of objects, uh, but it, it's kind of fun and thankfully I had a fallback plan which is super, super important. Uh, I spent, literally spent like one or two minutes. The first hiccup I was able to debug right away, but the second hic- hiccup, um, I spent maybe a minute on stage debugging and I gave up to the fallback plan which was using the camera on my phone. I think people gets the idea 'cause at least people was able to see I was able to fly in the drone. I was, uh I was just not getting the stream from the drone back to my laptop, um

[00:11:35] Whitney Lee: I think people are afraid to fail, and you technically failed, but you al- uh, in my mind it was a, a smashing success because that was months ago now. I remember I was there. I remember nothing, no other, not a single other presentation from that day. I like, like you, um, watching you debug and then watching like, watching the drone rise a little bit and everyone's like, "What?" making noise like they're so excited, and then it went back down and we're all disappointed like we're rooting for you. Like it's intrigue. It's, is, is she gonna make it? The fact that you were able to do something was cool and still, again, like so much more interesting and engaging, and I appreciated.

[00:12:16] I'd rather people, as an audience member, I'd rather see a risk and a failure and like someone try for something big than see something that was totally safe and forgettable.

[00:12:26] Lin Sun: Yeah, I think that's why it's memorable because people were not sure what's going to happen next, and

[00:12:34] Whitney Lee: Uh-huh. So yeah, it was fantastic. I, I love that you attempted something risky even though you failed. And, um, I, I don't know if... Yes, and I d- I do the same, and I fail plenty too at live, but I feel like it's, it's worth it

[00:12:55] Coté: I, I think it's also hard to like, like, like, like a, a, a conference like KubeCon, I don't know if it's odd or ironic or whatever, but like it's hard to come up with a keynote that's good. Like, like, 'cause, 'cause, you know, you don't really have that much time, and then also it seem... Well, I, I don't know if this is true, but I'll, I'll, I'll think out loud about it.

[00:13:16] It seems like you also have so many topics and, like, projects and communities that there's kind of a, there can be a disjointness of, like, all the different things you have in the keynote. And then you also have people probably who, like, don't-- sometimes are, like, not really sure what's going on 'cause they, you know, are giving the talk about, like, whatever and, and then, uh, and then I, I don't know.

[00:13:43] The, the, then, I mean, it seems like you can have, like, a, a thought leadership-y kind of thing. Like, like, uh, like Abby Bangser always gives a talk that's like, "Here's how we should be thinking, everyone." But yeah, like, it's y- it's difficult to choose,

[00:13:57] Whitney Lee: do

[00:13:57] Coté: a- an event

[00:13:58] Whitney Lee: how do you capture all these, all these different areas of expertise? Or how do you capture the fact that half the people there, it's usually their first KubeCon, right? At any given KubeCon. Yeah. That like maybe half the audience are beginners to Kubernetes. Um, how can you keep...

[00:14:13] Lin Sun: 50%. Yeah

[00:14:14] Whitney Lee: Yeah. How do you keep the experts interested alongside wh- while keeping the beginners, like, able to stay engaged and un- and follow what's going on? It's a hard needle to thread. Hmm

[00:14:26] Lin Sun: hard. I agree. Um, and I'm with you same, uh, Whitney. Like, I've been to so many conference talks, but at the end of the day, there's, uh, very few I can remember. Uh,

[00:14:39] Coté: Uh-huh

[00:14:39] Lin Sun: especially if they are outside of my domain of real expertise, right? If they are right in the project I'm working on, then, you know, I can relate with them.

[00:14:50] I can remember what they said. But if it's outside of what m- I'm immediately working on, it's very hard to remember,

[00:14:59] Whitney Lee: Yeah

[00:14:59] Lin Sun: six months from the event exactly what people said. So to me, I think that's not very interesting to Whitney's point. 50% of the audience is first-timer to KubeCon. They're relatively new, and I do think the best way to resonate with people is seeing something live, doing something, because that's what they are doing every day.

[00:15:23] Whitney Lee: Uh-huh.

[00:15:24] Lin Sun: moment you show a Kubernetes cluster, you show something, they... It's going to resonate in their head, "Oh, that's my daily workflow too," you

[00:15:35] Whitney Lee: Yeah

[00:15:35] Lin Sun: PR, oh, they have-- she has an agent submitting a PR to help her create a HTTP route. Oh, I'm doing that too. So

[00:15:45] Whitney Lee: Mm-hmm.

[00:15:45] Lin Sun: relate. That's what makes people, uh, remember your talk, I

[00:15:50] Whitney Lee: Oh.

[00:15:50] Lin Sun: is to be relatable.

[00:15:51] Whitney Lee: the, the daily drone flights I make with my Kubernetes cluster.

[00:15:58] Lin Sun: Not necessarily make slide, but like different pieces can be relatable.

[00:16:02] Whitney Lee: Yeah, absolutely.

[00:16:04] Lin Sun: The draw is the wow factor, which you need it too for a good demo. I think you need a wow factor, you need to be relatable, and you need, uh, the engagement. You also need surprises. Like in my demo, I planned it a couple surprises. Um, I think one of the surprises was, uh, when the analysis of the audience engagements, uh, was running, I don't know when it's going to finish, and the moment it finished, it played some music and read out, um, um, by some AI voice, you know, the engagement audience, and everybody just claps without me telling people to clap.

[00:16:46] That was my goal too. Whenever I go on stage, you know, I feel like if you tell people to clap, if people clap because you said, "Thank you so much for watching," or, "Thank you so much for being here. Thank you so much," you know, "for your time," I think that's not a good clap. You want people to clap because you really,

[00:17:08] Coté: it's genuine

[00:17:09] Lin Sun: their heart, right?

[00:17:09] So,

[00:17:10] Whitney Lee: Yeah

[00:17:11] Coté: Yeah, and, and, and I mean, even worse is when someone's like, "Oh, come on, this is the part where you clap." And it's just like, that's fantastic.

[00:17:19] Whitney Lee: it's like they're abusing the power. There's a little bit of power of being on stage, and

[00:17:22] Coté: Yeah, yeah

[00:17:23] Whitney Lee: use that power to tell me what to do. I

[00:17:25] Coté: Yeah

[00:17:26] Whitney Lee: that might be a, a peeve. Uh-huh.

[00:17:28] Coté: Well, what, what do you, what do you think when, when you're, when you're, uh, working on slides for like a giant keynote thing, how do you think differently about the slides than if you were in like, here's a room of 20 people and, and, uh, and I'm using slides?

[00:17:43] Lin Sun: Uh, I think slides are going to be boring regardless, uh, unless the slides is just kind of remind you what you're gonna say,

[00:17:53] Coté: Mm-hmm.

[00:17:54] Lin Sun: have a picture ins- instead. Uh, what I find out is, uh, it's harder for people to look at your slides and listen to you, unless your slides is just a picture, right? So if you have slides that has bullets, most of slides do have bullets,

[00:18:13] Whitney Lee: Yeah

[00:18:13] Lin Sun: read the slides, and I'm not a great multitasker.

[00:18:16] I suspect a lot of people are not either. So they either read the slides or just listen to you. I mean, you have to pick w- how you want people's attention to focus on. You probably want people listen to you, which I think, uh, that w- else they can just read slides without you, right?

[00:18:33] Coté: Yeah, yeah. Yeah, well, you know, that, that, that is, that is a, a risky thing that I don't know if I've really done. I must have at some point. But, like, it, it would, it would be fun to, like, not, not have slides, but more only use a slide every now and then when there's actually something to show. Like, maybe, maybe that's the way to put it, is, like, there are times when the slide is good.

[00:18:56] Like, you know, I think a chart is good. Like, if you had 20 years of, like, data, sure. Right? Like, like,

[00:19:05] Lin Sun: I agree

[00:19:05] Coté: but, but you're reminding me, like, I mean, I don't know, I, I suspect a lot of speakers do this, but a lot of the times the reason I have a slide is so I can remember what I wanted to say. And, and it's kind of like I might as well just have, like, a little notebook that, like, only I can see.

[00:19:20] But I feel like I've only seen a couple of times, not at a tech conference ever, where the screen is black or off, and then at some point when they need a visual, it turns on, and then they turn it off again. And, like, that seems like... I don't know if that's one of those things where, like, only if you speak every now and then you think that would be cool, but, like, an audience would be like, "This is weird."

[00:19:41] But that, that would be a fun thing to do.

[00:19:44] Lin Sun: Yeah. I, I agree with you. If you have numbers or charts or trends, uh, like performance benchmark, I think that's when slides can be really useful. Uh, I would say Steve Jobs is probably one of my, uh, favorite speakers. If you watch him give a keynotes on, like, product announcement of Apple, you will

[00:20:04] Whitney Lee: Yeah, yeah.

[00:20:05] Lin Sun: his slides is so simple.

[00:20:07] It's mostly just a number or logo, and he constantly showcase, uh, demos of, uh, you know, Apple products, uh, uh, as part of his keynote

[00:20:18] Whitney Lee: I like, I like to, to use architecture diagrams, but the secret is not the whole diagram all at once. Like whatever piece I'm talking about, then reveal the next piece, then reveal the next piece, so it only unfolds as I'm explaining what, uh, how it works.

[00:20:34] Coté: Yeah

[00:20:35] Yeah, And then lots of drawings. You gotta have lots of drawings.

[00:20:38] Whitney Lee: yeah, sometimes I do.

[00:20:40] Lin Sun: I

[00:20:40] Whitney Lee: Yeah

[00:20:40] Lin Sun: live, uh, whiteboard works really well too, Whitney, because people don't know what's next, right? So it's kind of int- intriguing. While in a presentation format, if you just show the whole diagram, people is going to be busy looking at it. They're not going to listen to you, you as a speaker.

[00:21:00] Whitney Lee: Exactly. I've done it before with my iPad where I draw

[00:21:04] Coté: What, what, what did you think of that?

[00:21:07] Whitney Lee: the problem was I wanted to flip back and forth between my iPad and my terminal, and I was m-

[00:21:11] Coté: Oh

[00:21:12] Whitney Lee: to do that, and it would take, like, five seconds each time, so it was a little awkward.

[00:21:16] Coté: Yeah.

[00:21:16] Whitney Lee: a lot of little jokes, but I

[00:21:18] Coté: G-

[00:21:18] Whitney Lee: that worked.

[00:21:19] Coté: be- because, you know, I'm, I'm sure we've all seen there's, there's always, there's always the person who does their presentation purely on an iPad whiteboard. And like, I think, I think maybe what you were just saying is driven by the constraint of like, if you have an iPad open to whatever you Procreate or whatever you're using, it's really awkward and time-consuming to switch to something else.

[00:21:41] So if you wanna do that, you kind of have to be like, "I'm only doing this." But then that means, I don't know. I don't know how, how I feel about that kind of presentation 'cause it's... It almost, I- it's, it can be weird. Like, it's different, like, if you're having a conversation and you're drawing something out, but I don't know.

[00:21:59] Who, what, what, what do you think about the only, like, whiteboarding, drawing out presentation?

[00:22:05] Lin Sun: I think it could be very interesting. I was listening to, uh, a talk by the, uh, MIT professor on how to speech. Uh, well, you know, because in a teaching environment, it's kind of like a whiteboard, right? Except that you have

[00:22:23] Coté: Hmm

[00:22:25] Lin Sun: the, the actual blackboard to draw things. I, I think it can be

[00:22:30] Whitney Lee: Yeah

[00:22:30] Lin Sun: very interesting in that context because it can be dynamic and engaging, right?

[00:22:37] 'Cause you could constantly asking questions and poll

[00:22:40] Coté: Yeah, definitely that. Yeah

[00:22:42] Lin Sun: that to determine your content. I, I think at the end, the, the engagement is what people can remember, right? Because, uh... So I, I do think it can win, but, uh, most conferences you don't have that, um, like, environment to do such a whiteboard or blackboard type of drawing, unfortunately. So that's why I do think, uh, you know, demo is probably the next tier of, uh, you know, winning that engagement.

[00:23:11] Whitney Lee: Yeah

[00:23:12] have you planned out what you're going to do for KubeCon coming up in Salt Lake City in November? You set a really high bar for yourself. Yeah

[00:23:21] Lin Sun: completely. It's funny, uh, so I do want to... So I remember in Amsterdam, somebody come to me, uh, at our Solo booth after my talk, or maybe the next day after my talk, and said something to me that resonate with me. He said, uh, "It was brave what you showed, but please don't give up and don't, uh, you know, give up the idea completely and keep trying.

[00:23:46] You will succeed." So, uh, I thought about that in Amsterdam, so I started to ask a CNCF, uh, staff member, you know, can I maybe upgrade to a different model of drone, and hopefully it has better connectivity between... 'Cause the one I had was very education-ish, and, uh, it doesn't

[00:24:09] Whitney Lee: Hmm

[00:24:09] Lin Sun: very well with huge audience.

[00:24:11] I think it works with, like, a 10,000, um, 1,000 people, but not so well with, like, 10,000 people, which I didn't plan for that. Um, so the, the one I have, uh, that was planning, I'm planning to use for Salt Lake City is a upgraded version. Hopefully it's better, but I haven't had time to build for

[00:24:33] Whitney Lee: Ah.

[00:24:34] Lin Sun: So I did get approval for that. Uh, so already thinking ahead of, like, buying insurance for that flying

[00:24:42] Coté: Sure. Yeah

[00:24:43] Lin Sun: They did it for Amsterdam for me, which is super nice and supportive. So I do need to think about what to build because I don't want to be the exact same thing, right? So,

[00:24:54] Whitney Lee: Uh-huh.

[00:24:55] Lin Sun: to think about what to build, um, and also spend a lot of time to testing, uh, make sure that stream, uh, works well, 'cause that's going to be the key and

[00:25:06] Whitney Lee: So,

[00:25:06] Lin Sun: moments if I can get it working.

[00:25:08] Whitney Lee: so actually you could do the exact same demo, exact same presentation, but it'll work this time. That's the goal

[00:25:14] Lin Sun: Yeah, but I mean, I want to have the, the camera or streaming probably the same if I can get it working. But the other pieces, I want to make it more interesting, because if it's the, uh, same people like you or other people, I don't want you feel like you're seeing the same thing. So I

[00:25:34] Whitney Lee: Okay

[00:25:34] Lin Sun: make the other ones, uh, the other surrounding portion, um, definitely different.

[00:25:39] Uh, I'm not sure how much different would be, so yeah.

[00:25:43] Coté: You could, you could,

[00:25:44] Lin Sun: some surprises.

[00:25:45] Coté: you could have like two or three drones this time and be like, "It, it went so, it, it went, it went so kind of well last time that I decided I'd do more." And

[00:25:54] Lin Sun: think I can handle more than one. Handling one is very challenging

[00:25:58] Coté: yeah

[00:26:03] Whitney Lee: Awesome keynote presentation. What else are the responsibilities for being a co-chair, a program co-chair at KubeCon?

[00:26:09] Lin Sun: Um, that's a great question. So most of it, all our time are spent out, um, building the conference schedule. So right now, I think you probably already see on social media, the call for paper for Amsterdam just, uh, went out. So, and so before the call for paper went out, we have to sit together, kind of decide, you know, are these the right tracks we want people to submit?

[00:26:38] Does the track description make sense? So we actually had to make some changes to the track. So one of the changes we made to the track was the AI track. So

[00:26:47] Coté: Uh-huh

[00:26:48] Lin Sun: last time, uh, we added agentic AI and, uh, inference, um, infrastructure AI to the same track. We

[00:26:57] Coté: Uh-huh.

[00:26:58] Lin Sun: a huge mistake because that track is like the single most popular track

[00:27:03] Coté: Uh-huh

[00:27:04] Lin Sun: submissions and, um, we didn't have, uh, um, enough reviewers from the program committee due to the huge number of the submissions come to that track. So this time, uh, for Bas- for Barcelona, we're break- uh, we're breaking out that track into two tracks. So that's just an example to showcase how we are kind of put our thinking onto launching the CFP. And once the CFP is out and, uh, both of you and all the other amazing, uh, speakers, uh, do your submissions, uh, it's fed into a program committee to review the submission.

[00:27:45] Whitney, uh, uh, Coté, I don't know if you've ever been a program committee member. So typically, they're given like two weeks, uh, to review all the submissions, uh, that's assigned to them. And once all the program committee members finish their reviews, uh, the track chair is going to put the recommendation for the track. And then, uh, we got a gigantic spreadsheets as a co-chair, and for each of the track, we're responsible for making the final decision. so as we are making the final decision, you know, select the best, uh, talks and the best speakers, uh, and with the best, uh, diversity member, uh, numbers in mind, we have to do the final tweaks, and that's how you see the conference schedule put out, um, I think

[00:28:36] Whitney Lee: So

[00:28:36] Lin Sun: or three weeks ago for Salt Lake City.

[00:28:39] Whitney Lee: So about how many people submit to any given KubeCon?

[00:28:42] Lin Sun: Uh, it's typically, I would say, uh, the acceptance rates is close to single digit. So normally we accept about 200-ish, uh, talks and workshops and, uh, the, uh, the submissions is probably somewhere between 2,000 to 3,000, uh,

[00:29:02] Whitney Lee: Wow.

[00:29:03] Lin Sun: North America or Europe KubeCon

[00:29:06] Whitney Lee: And then how many of those were for the a- the one AI track this time around?

[00:29:10] Lin Sun: Uh, I, I don't know if I remember the number correctly. I would say it's somewhere between 5 to 600, so it's definitely a lot of them. You probably helped contribute one or two

[00:29:28] Coté: Wait, wait, I, I, I assume like every other conference you have like a lot of, uh, whether completely or assisted, like AI written submissions. Like what, what do you, what do y'all do with that?

[00:29:37] Whitney Lee: yeah

[00:29:38] Lin Sun: Oh yeah, it's, uh, it's very, uh, uh, you know, uh, sad to see people not putting a lot of efforts into their submissions for a prestigious events like KubeCon. sometimes, uh, when we spot, uh, people completely use without, you know, just massage, uh, a little bit. Like for instance, we saw a submission, um, that has the benefit section that has like, "Oh, this is what Claude tells you," and people didn't even mask out that th- this is what Claude

[00:30:14] Coté: Oh, right, right

[00:30:15] Lin Sun: that.

[00:30:15] I forgot the exact one. But one of the program review committee members said, "Look, the first line of the benefit section, this whole thing might be entirely AI generated." And that particular member not putting a good score, you know, on that submission and it's, his or her comment is reflected to the track chair and co-chair, which could lead us to rate down that submission. So I would say if you want your talk to get in, you have to be very, very careful. It's okay to use AI to massage it. I think Whitney wrote a great blog on how to use AI

[00:30:53] Coté: Hmm

[00:30:53] Lin Sun: help your thinking, so that's all good. But just not 100% rely on AI because people can spot out if you have any unique or interesting perspective than the generic stuff produced from LLM.

[00:31:10] Coté: And so, like, do y'all, uh, do, how do, uh, do y'all use AI to process all this stuff?

[00:31:15] Lin Sun: Uh, that's a great question. We started to use AI, but we feel like, uh, we should be thinking about improving our process more. Uh, interestingly, I'm also working on the Agentic AI Foundation, which they have a lot of conferences. They also have like Agent Con, MCP Con in Europe, uh, in Japan, and also in, um, North America, which I'm also a speaker for all of them. And I sit on the program committee for Agent Con, MCP Con North America. And what I find out that their process, uh, is a little bit more advanced than the KubeCon process. So we're looking at maybe, uh, borrow some of their process too. one thing they had in their process is whenever there is a submission, uh, in Sessionizer, there is a AI agent, uh, do the initial screen to see if this talk has been presented before, which is super helpful, because right now we have to use human to do it. And also try to think out if the speaker is a speaker with a lot of credential, which we often have to manually do research for KubeCon. And also try to tell, uh, if the submission looks interesting from the AI agent's perspective. What I like about that agent is it doesn't produce like a, a few paragraphs of It

[00:32:44] Coté: It's brief

[00:32:45] Lin Sun: simple line, which, you know, you wouldn't want to spend time to read it. Yeah, I love that perspective. So we're looking at, uh, using that. The other thing we're looking... we're already using for Salt Lake City is, uh, using AI to help us, uh, do analysis of whether this, uh, proposed, uh, agenda is, uh, the right fit for the track.

[00:33:12] Is

[00:33:12] Whitney Lee: Oh, yeah, yeah

[00:33:13] Lin Sun: for the track?

[00:33:14] Coté: Mm-hmm.

[00:33:15] Lin Sun: And use the AI to help us to think through, is this the right, um, the tracks for the conference? You know, how should we be proposing new tracks or adjust some of the existing tracks? So we've been using that co- sort of like a, another tier of an- analysis after we've done

[00:33:35] Coté: Yeah. I, I could, I could see that would be really helpful, 'cause that's... I don't know. It's, it's, it's like, it's a lot of subjective, it's a lot of taste. And, and like, and, and like AI, you can tell it to give me like, you know, g- give me 10 different taste takes. Uh, which it's hard to get humans to like have that one sentence.

[00:33:54] But you can kind of, even if like eight of them are junk, you like get two of them that are kind of interesting to, to look at. Which, yeah, the, you know, I... Some conferences do this, but the, uh, like DevOps in, in Belgium, they had, they had a, a post recently where, you know, you could tell it was our friend Claude who made the charts, which is fine.

[00:34:14] But, but, but, but the point being they like, they were just like, "Look at all the charts about the CFPs," just everything, all of the interesting data. And like, I think for people like us, it's probably especially interesting. But it was, it was interesting to see, 'cause I don't think it was just selected talks, it was like all the CFPs that they get.

[00:34:34] And it was fascinating to see like not only of the classic stuff like a word cloud, but just like the topics and, and the companies that submit and all this stuff and the rankings and, I don't know. It's, it's interesting to see that kind of data, which I think, you know, if you're a conference organizer, you don't have any time,

[00:34:52] Whitney Lee: Yeah

[00:34:53] Coté: let alone to make a bunch of charts.

[00:34:55] So it's nice to like automate a bunch of that stuff

[00:34:58] Whitney Lee: I,

[00:34:59] Lin Sun: yeah

[00:34:59] Whitney Lee: when I, when I did program committee work, it was before AI was so popular and it, uh, and accessible. And one thing I ran into is I would like get a submission... I looked at everything linearly in the ca- order it came, and I might look at a submission and be like, "Oh, that's really cool. I like that," and like score it well.

[00:35:17] And then later down the line I would get another one for the exact same topic or the exact same idea, or another one for a very similar idea, and it's like, "Well, I shouldn't... Do, do I need to choose one of these? Like, are these sp- you know, specifically pitted against each other?" Like I think I would've liked just to use AI just to categorize like, okay, I'm on the networking track, I wanna, I want to look at all the ones that have to do with networking and observability at the same time, you know?

[00:35:42] And all the ones that have to do with eBPF at the same time and, or whatever.

[00:35:47] Lin Sun: Yeah, totally. I feel like when I was a program committee member, I tend to do that the same. It's like

[00:35:54] Whitney Lee: Mm-hmm

[00:35:54] Lin Sun: trying to finish my task, but not being strategic of review, reviewing these groups together. Yeah, so we-- uh, one thing we try to offset is, uh, try to hire a bunch of, uh, people on the same track to do the program committee work, right?

[00:36:11] When you have a sample of five with-- When you have a sample of one or two reviewers, uh, five is going to be a lot more closer to, uh, to, to a reasonable number for the reviews for the same submission.

[00:36:25] Coté: Yeah, I, I don't, I don't know if there's any good way to have, uh... Or if, if there's any way to have a good experience being, like, a CFP reviewer. In, in, in my experience it's always difficult and, and a, and a lot of work. And, uh, not, not least of which is, is like-- And I, I don't, I don't mean this purely in a complain-y way, but just dealing with your fellow program committee people.

[00:36:47] Like it's-- 'Cause you get in-- At least you seem to get in this weird situation where you're, like, defending someone else's... Defending's the wrong word. You're explaining why someone else's talk that you barely know about is a good choice over another talk that you also barely know about, and so it's just weird.

[00:37:04] It's a s- strange form of advocacy to, to, to be doing

[00:37:09] Lin Sun: Yeah, I agree. And this is also why it's important to build your speaker credentials out, right? People like Whitney, uh, you have great, uh, credentials. So when we look as a conference co-chair, like we, when we look at the speaker, you know, is this a good speaker? Well, no. That's going to help us to draw people attending the conference.

[00:37:30] We're gonna have to prioritize those people first, uh, 'cause they already having a proven record of, uh, you know, people love their talks.

[00:37:43] Whitney Lee: One thing I noticed when I applied to this most recent one was that I actually had to check a button that said, "My submission is not AI slop," something along those lines

[00:37:53] Lin Sun: Noonie added for Amsterdam, I think.

[00:37:56] Whitney Lee: Yeah. Do you feel, do you have, do you feel like that actually makes a difference in people using less AI slop?

[00:38:02] Lin Sun: I don't think so.

[00:38:03] Whitney Lee: No?

[00:38:04] Lin Sun: hopefully, I think people are still using them, but hopefully, uh, this is where we are hoping, you know, if we have an agent to help would be nice. Uh, unfortunately, CNCF, I believe, has a rule you can't discriminate people because their languages are not good. So

[00:38:24] Whitney Lee: Uh-huh

[00:38:25] Lin Sun: polish is definitely allowed, so we can't, uh, run like a GPT Zero

[00:38:31] Whitney Lee: Uh

[00:38:32] Coté: Yeah. Yeah, there, there's, you know, you know, Anthropic had that, uh, watermarking text thing last week, and,

[00:38:39] Whitney Lee: Yeah

[00:38:40] Coté: and it was, I was, I w- I guess I was kinda surprised by the reaction a lot of people had to it. May- a lot. I mean, I'm thinking of a pool of 10 people who, like, write about this stuff. So, but you know, there, there was, there was some upsettedness about, like, uh, one, I don't, I don't want an AI to take ownership from my writing if I help...

[00:39:01] If, if, if I was using it for stuff. And then two, there was almost like this, uh, it wouldn't be a false positive, but it would be, let's call it a too much positive, where kinda like you were saying, like, "Well, of course I used AI because, uh, I need a copy editor, and I don't know how to spell stuff," and blah, blah, blah.

[00:39:19] And so you could imagine you get in a situation where if there is this weird... I still am not convinced that something like this would actually work. But, you know, if there is some weird textual watermarking, it's kinda like, "Yeah, I used it 'cause I can't spell, and I don't know how semicolons work, and now I'm screwed."

[00:39:37] Like, it's... I don't know. So I imagine it'll be, it'll get difficult over time to figure out, like, how... Uh, should we just reject CFPs that are written by AI or, or not?

[00:39:52] Whitney Lee: very easy to work around. I'll just retype exactly what

[00:39:55] Coté: Yeah, yeah.

[00:39:56] Whitney Lee: over here

[00:39:57] Coté: Yeah, yeah.

[00:39:59] Whitney Lee: to make it even faster.

[00:40:00] Coté: Yeah, or, or, or like, you know, I mean, to you, you brought up like Whitney's, uh, how to use AI to help point or, or post, and it is like there, there is, there's a lot of AI usage in there. And, and like the, the final product, maybe it's not like character by character written by AI, but it's kind of like, eh, I mean, it was used quite, you know, to, to help, help sort things and ideas and stuff.

[00:40:24] So I, it, it does... Like I could see the, uh, a time when like, I mean, you could have like 100%, uh, whatever that means, 100% character by character written by AI because there was like someone sitting there like talking on a dog walk, and then they load it back up and they're like, "I don't like that fifth sentence.

[00:40:42] Can you redo that?" And like you could actually spend the same amount of time. I don't know. It seems like it'll be a difficult situation

[00:40:49] Whitney Lee: not that I've done this, but I could see a world

[00:40:52] Coté: But a friend has.

[00:40:55] Whitney Lee: where I trained it to like, to know my voice, and then also I've had like 10 turns worth of like lots of deep conversation of

[00:41:02] Coté: Yeah, yeah

[00:41:03] Whitney Lee: and how I wanna say it, and do have something that's very AI, but also couldn't possibly exist without,

[00:41:10] Coté: Yeah.

[00:41:10] Whitney Lee: my

[00:41:11] Coté: And, but on the other hand then, as you were saying, it's very obvious when people are being lazy.

[00:41:17] Whitney Lee: Yes, true It, it does like, it does like send shivers down my spine when I see something that's just like purely has all the AI tells in It

[00:41:29] Coté: Yeah.

[00:41:30] Whitney Lee: icky.

[00:41:30] Coté: Yeah. When AI's doing, uh, is doing the load-bearing work

[00:41:34] Whitney Lee: Yeah Yeah. So Lin, we've been really focused on KubeCon this whole time, and you do so much other stuff too. You mentioned the Agentic AI Foundation, you're there. Uh, you're, you used to be a technical oversight committee in the is also a big deal. You've written a book, uh,

[00:41:54] Lin Sun: Yeah, I

[00:41:54] Whitney Lee: recently,

[00:41:55] Lin Sun: Yeah,

[00:41:56] Whitney Lee: uh, "AI Agents on Kubernetes."

[00:41:58] You just hold it, held it up. Um, which of, which of those topics do you wanna dig into? Or what am I even missing? You just mentioned sitting on some other, uh, program committee. Oh, Agentic AI too. Yeah

[00:42:12] Lin Sun: uh, y- it's funny, part of my job at Solo, I'm a head of open source at Solo, part of my job is kind of like outfacing, right? To kind of, uh, make sure people know the open source projects, uh, we are working on, we're leading, we're creating. So I thought the job is super fun, you know, gives me opportunities to also do volunteer work, like you said.

[00:42:36] Uh, serve on the technical oversight committee of CNCF, and also I recently become a ambassador for the new Agentic AI Foundation, which is super, uh, interesting. Uh, yeah, so I, I feel like I've been kind of having, you know, the best portion of my tech career. I've been in this industry for 25 years and, uh, 20 years in open source.

[00:43:04] I feel like open source is definitely, you know, the best portion of my technical career. So

[00:43:10] Coté: Uh-huh

[00:43:11] Lin Sun: I, I just love, uh, to help educating people on some of the open source projects, uh, that we created at Solo and we open sourced it. And what's really cool is also it's, uh, not only working in, uh, the, the rising pack, right?

[00:43:31] So starting with my early career with Istio when service mesh was hot, now with, uh, AI agents, MCP, you know, uh, AI gateway is hot. You know, I feel very blessing. I've been always working on the intersection of hot technology and open source.

[00:43:52] Whitney Lee: Mm-hmm. Do you-- I have a, uh, I have a spicy question. So on the CNCF Technical Oversight Committee, my understanding is you help shepherd projects through the process of, like, initial application, become sandbox projects, like, uh, from sandbox to incubating, and then from incubating to graduated. Um, my, my spicy question is: Is the system set up so really truly the best projects rise to the top, or is-- are there more politics involved?

[00:44:24] Lin Sun: That's a great question. This definitely, uh, bias could potentially be considered. Um, it's not intentionally biased. We place on the TOC or CNCF, but if you're looking at the process, uh, it really depends on how much the project is willing to spend the effort to moving forward to go through the process, whether it's

[00:44:51] Coté: Mm-hmm.

[00:44:52] Lin Sun: accepted as a sandbox, whether it's being, um, becoming a incubation or graduation project.

[00:44:58] Because if the project is willing to put a lot of effort and working with the TOC, um, the, chance is going to be a lot higher. And then the second question is how long the project has been in the queue, because there's only a fixed number of TOC member and their workload, um, capacity is not gonna be hugely high per year, right?

[00:45:25] And then there's also a churn, of the TOC member every year. Half of the member, you know, they, they have to either renew or they maybe they decided they're not, they're not gonna serve. So that costs productivity within the TOC potentially as people rolling off. so I think those are two of the factor.

[00:45:48] The third factor is, um, the TOC member. Typically, we assign one or two TOC member to review the application, whether it's incubation or graduation. Um, and depends on how that TOC member is going to review and vouch for the application.

[00:46:06] Coté: Hmm.

[00:46:07] Lin Sun: there's a human involved, there could be a bias, uh, involved. I think the TOC actually looking at automating some of these, um, uh, using AI, so that should help a lot. But whenever there's a human decides and a vote, uh, involved, there could potentially be bias involved too, so.

[00:46:28] Coté: So, uh, what do you, what do you think, uh, I mean, given all the open source stuff, like, uh, uh, like whether you're like solo or other places, like what, what do you... What's good to do as a commercial company to like be successful with open source? And, and by successful, I mean like you take full strategic advantage of it, not just like, you know, the benevolence of being awesome.

[00:46:52] Like, like, like,

[00:46:53] Lin Sun: question.

[00:46:54] Coté: like, for, for example, I think Red Hat has it figured out, and if, if they don't have... And I'm just speculating here, like if they don't have it figured out now, they figured it out for long enough to be set. I, I mean, I think they do have it figured out. But like, then there's other companies who, you know, you talk with people from those companies and they're kind of like, "Why are we not...

[00:47:17] Why doesn't anyone realize that we're great?" Or like, like, "We do all this open source work and people use..." I, I don't know, like, uh, what do you have to do as a business to make it work in, in a, in a commercial situation?

[00:47:32] Lin Sun: Yeah, this is a great question. Uh, you are absolutely right, right? Red Hat is one of the few company has figured out the game with open source. Uh, you know, people... So I'm also a manager at Solo, and people reports to me constantly ask me, "Hey, is working in open source make my job, uh, less strategic for the company," right?

[00:47:53] So that's the ques- question people ask, uh, 'cause a lot of people wants to work on the critical paths of, uh,

[00:48:00] Whitney Lee: Right

[00:48:01] Lin Sun: the business growth, um, of your company, right? So open source doesn't directly tie into a revenue. So how do we justify us as, as somebody sitting on the payroll of Solo and, uh, have our company continue paying for our salary?

[00:48:22] So one way, uh, I tend to look at about this is how much potential revenue our open source have brought our company, right? So for instance, recently I read a report, uh, generated by our go-to-marketing team where they talk about this particular book, which, uh, we spent a few months writing the book, and it actually generate a good poten- a good potential revenue, uh, that could be transferred to the real revenue.

[00:48:54] So it's actually a important, uh, strategy part of our company's growth,

[00:48:59] Coté: It, it sound, it, it sounds like I was terrible at math, but you got, uh, you got the numerator and the denominator. The denominator's on the bottom, right? And, and the bigger you make the bottom one, like even if you only have like less of the, the, the pi- you got a bigger... It's just your pie slice is better.

[00:49:17] And so maybe, maybe that's one way of, of looking at it, is if we make a giant pie, even if we can't eat the whole pie, it's a big slice, uh, uh, or pizza or whatever

[00:49:27] Lin Sun: Uh, right. That's, that's the way to think about it. And, uh, so for the, the... So I gave the example of the book. Uh, so conference speaking, um, uh, writing technical blogs, workshop, a similar venue as the book. The other example I would give is I have people who maintains Istio, for instance, uh, work for me on my team.

[00:49:47] And, uh, and I do know sometimes we close out deals, uh, related to whether it's Istio support or enterprise version of Istio, um, but based on Solo's distribution. Um, and they pick us, um, as, uh, as a vendor because we have this much maintainers on the project and

[00:50:11] Coté: Right, right

[00:50:12] Lin Sun: much leadership on the project because we help create, uh, Istio ambience in the community, right?

[00:50:19] That leadership, I think, uh, it, a lot of time is what's actually winning the deal. So-

[00:50:26] Coté: And, and do you, do you think, do you think that when, I don't know whoever it is, there's probably some sort of VP or C or director in their title, but when they... Like, like, like, l- let's say I'm gonna fictionalize this, but let's say the, you know, the Solo people sit down one year and they're like, "We're gonna do open source for this reason."

[00:50:45] The, like the one you just described. Like, do you, do you actually see... And you must talk with a lot of people in the, the, the CNCF community. Like, are they strategically like doing open source, or is it more like they just happen to do open source and then they take advantage of it?

[00:51:00] Lin Sun: That's a great question. I think every single company needs to be very strategic about open source, or else, uh, this open source is just a cost to your business.

[00:51:09] Coté: Mm.

[00:51:10] Lin Sun: but I do agree with you. Uh, the small company, uh, probably are a little bit more strategic in a sense because with small company, uh, it's more focused on cost efficiency.

[00:51:22] Whitney Lee: Rare.

[00:51:23] Lin Sun: company, uh, sometimes... I used to work at big companies too. I think, uh, the strategic, uh, it needs to be strategic, but because of the politics and because, um, the overall budget spending is not as, you

[00:51:40] Coté: I--

[00:51:40] Lin Sun: overseen as of a s-

[00:51:42] Coté: That's, that,

[00:51:43] Lin Sun: so it's harder to tell

[00:51:44] Coté: that, that's, that's interesting because I could see in a, in a bigger company, you have more money, so it's not so much a question of should we be spending this money. I mean, I know that's kind of ridiculous. But, but the, the, the bigger threat would be like, "You should not be spending this money.

[00:52:00] We should be spending this money in our division." So you're kinda like squabbling over budget between different groups, and then that's a whole other... That seems like an area where open source needs a whole other s- set of justifications that like, uh, that, that, that would make it difficult. I-- So like, uh, uh, also in the open source world, but then in your agentic stuff, like what, like what do you do with...

[00:52:24] I mean, you can put scripts in like skills, but really it's just a bunch of markdown. And like, is that, is that like copyright or is that open source? Like what do you, what do you do if like the thing that you're shipping is just a zip file with documents in it? Like that seems like a weird area for open source to be operating in.

[00:52:46] Lin Sun: Are you talking about like agent skills,

[00:52:48] Coté: Yeah, yeah. Yeah, like you have an agent skill. Like, like let's, let's say we, let's say we take Whitney's blog, and of course, we're not gonna do this ourselves. We're gonna ask an agent to convert it to a skill. And, and all it really is, is like it's literally a bunch of markdown files. And, and like, like I guess you, you could open source that, but it's, it's...

[00:53:09] It also gets complicated because then I could... Well, I wouldn't do this. But then, but then someone could take that skill and point an AI agent at it and be like, "I wanna do that." And then like, is that forking it or is that not forking it? Like, seems like you get into weird things if you're, if, if what you're talking about is a document and not like source code, uh, or y- like compiled code.

[00:53:34] Lin Sun: I, think you are thinking about the Agentic Foundation incorrectly and I mean that to be bland. So the Agentic Foundation hosts, uh, five projects so far. Uh, so the first project was Model Context Protocol,

[00:53:47] Whitney Lee: Yeah, yeah

[00:53:48] Lin Sun: standards, right? The second project and third one is Goose,

[00:53:52] Whitney Lee: Yeah

[00:53:53] Lin Sun: Agents.md, right?

[00:53:54] Agents.md is kind of also a spec. Goose is a very, uh, popular open source MCP client which complements, uh, the MCP spec really well. Um, the fourth project is agentgateway, which is kind of like a AI infrastructure security governance, uh, control, um, providing that for AI agents and MCP and large language model. Um, the fifth project, which by the way, just accepted by the foundation, is the Agent to Agent Protocol started by Google. So if you're looking at these projects, they are either standards or popular implementations of some of the standards or AI agent infrastructure. So none of them are like scripts or MD... or like the

[00:54:41] Whitney Lee: Skills

[00:54:41] Lin Sun: were mentioning about. So they are like setting up the plumbing for you to be successful building your agents or MCP and, uh, building your agentic infrastructure

[00:54:54] Coté: Yeah, you know, I, I think... Is, is this true? I, I think, I... Is Goose the only, like, application in that portfolio and the rest are standards? 'Cause Goose is an actual applica- like a, it's an exec- it's a whole thing.

[00:55:08] Lin Sun: so far, yes. Uh,

[00:55:10] Coté: Yeah, It's, it's

[00:55:11] Lin Sun: example 'cause it implements like the MCP clients

[00:55:15] Coté: Yeah. It's, it's also interesting 'cause, like, I forget what they call it in Goose, but they... I mean, Goose came out, like, so I think before, like, Skills and maybe even MCP stuff was out, I forget. But they have, they have kind of like their own preexisting way of doing that stuff.

[00:55:31] Lin Sun: Recipes,

[00:55:31] Coté: Yeah, there you go. And, and so it is...

[00:55:33] I, I mean, I, uh, Goose is always fascinating, but it, it's an interesting... It's, it's not a standard, but, but it, it's a, it's a good implementation point. I'm, I'm trying not to say reference implementation because it's an actual thing, not just a, an example. But I don't know. Is that, like, is that the role it serves in there?

[00:55:53] Lin Sun: I think so, 'cause I'm, I've been here reading a lot of about how Goose, um, I think the company behind Goose is Block, right?

[00:56:02] Whitney Lee: Yeah, yeah

[00:56:03] yeah

[00:56:03] Lin Sun: Anthropic was partnership on this because Goose was so kind to be able to put MCP implementation out there

[00:56:12] Whitney Lee: Yeah

[00:56:13] Lin Sun: uh, MCP. Yeah

[00:56:16] Whitney Lee: Uh, does the Agentic AI Foundation have opinions about where you run inference and models? Does it want you to run in Kubernetes or does it, is it no opinion there?

[00:56:27] Lin Sun: That's great. Uh, I don't know if it has a strong opinion at the moment, uh, but certainly coming from the Cloud Native Computing Foundation, uh, and the ecosystem, that's where we want people to run, which I think makes sense unless we reinvent, uh, Kubernetes. However, there's a new project I started to contribute, and I didn't put enough time yet, but really exciting.

[00:56:51] It's called Agent Substrate. It's kind of rethinking of for running AI agents. It's basically proposed that Kubernetes is not good enough to running AI agents because the nature of AI agents is like short time, uh, like

[00:57:11] Whitney Lee: Mm-hmm.

[00:57:11] Lin Sun: that you want to run it, uh, when you need it. When you

[00:57:15] Whitney Lee: Mm-hmm.

[00:57:15] Lin Sun: need it, you'd rather not consume any resources, uh, in your cluster. So Agent Substrate introduce, uh, an interesting scheduling and, uh, AI agents management system that allows you potentially run like 20x more pods, more agents, uh, in your Kubernetes cluster as pods, um, to be able to manage your life cycle of AI agents. So a very interesting project, open source, uh, by Google, I think, uh, towards the end of May this year.

[00:57:48] Whitney Lee: So like serv- supercharged serverless? Is that

[00:57:52] Lin Sun: of like a serverless, but it's actually, uh, it's designed to be more efficient, uh, than serverless because it doesn't use PaaS as the abstraction, uh, lifecycle management for your AI agents. 'Cause I think with serverless, you still use PaaS under the cover, right? To kind of, uh, uh, running your serverless job fast.

[00:58:15] Uh, but this one, uh, it realized the cost of spinning PaaS is quite expensive and puts a lot of pressure on the API server. So it allows you to reuse the PaaS, um, by swapping different AI agents and be able to quickly resume and, uh, uh, suspend your AI agents

[00:58:40] Whitney Lee: Fascinating. I have more questions, but it's all getting in the weeds technically. Because I, I think I'm a little, um, mixed up about, or maybe there's... Well, whatever, about like when, what inference versus model serving and what the different needs are there in terms of... So you're speaking of agents, so presumably you're talking about inference when you're talking about, uh, agent substrate, for example.

[00:59:04] Is that correct?

[00:59:06] Lin Sun: So yeah, so typically when you think about AI agents, you're thinking about AI agents typically have a goal, right? To do

[00:59:14] Whitney Lee: Mm-hmm.

[00:59:15] Lin Sun: uh, to interpret, uh, the s- instructions from the user and do the work. And for the AI agents to do the work, it needs to, uh, with the help of calling large language model to be able to determine, you know, what's the right step, what are the tools I need potentially use, um, to, to do the g- particular, uh, goal, right?

[00:59:39] So that's when inference comes into the picture, right? Because

[00:59:43] Whitney Lee: Uh-huh

[00:59:44] Lin Sun: agents, you are only like the consumer of large language model because you need the large language model to do the reasoning for your agents. But the inference is on the serving side. So

[00:59:55] Whitney Lee: Okay

[00:59:56] Lin Sun: think about inference, I'm thinking about the provider, like Anthropic,

[01:00:01] Whitney Lee: uh-huh, like

[01:00:02] Lin Sun: OpenAI, right?

[01:00:04] They need, as the request come in to calling their model, uh, whether it's OpenAI or Anthropic, how am I going to serving this request to be more efficient?

[01:00:17] Whitney Lee: Yeah

[01:00:18] Lin Sun: going to route to the right endpoint, uh, the right GPU or TPU to be able to serve in this particular request? That's when inference comes in. So I think right now a lot of people are not running inferences, but as the tokens become more expensive, people will probably looking at maybe I can use some of the open source model, like Qwen model has been getting pretty be- uh, better, and there's also like TiDB model, a lot of open source model people talk about, and they could potentially host those models in their, uh, in-house, uh, with their own hardware. That's when they could potentially use, uh, inference, uh, to inference the request and this is also where inference gateway could come into the picture to make the request more efficient, to have the, uh, governance onto those requests.

[01:01:12] Whitney Lee: Is there a case for hosting models on Kubernetes?

[01:01:15] Lin Sun: Uh, yes, uh, there are projects that helps you to host models on Kubernetes

[01:01:22] Whitney Lee: that's not the question. Just because you can, doesn't mean you should

[01:01:25] Lin Sun: Yeah, I, I think it definitely makes sense because, uh, the tokens are so expensive, right? Right now I'm pretty much exclusively, uh, running public clouds, uh, running public, uh, uh, endpoints, and I'm also running my local, but what if I want to do it, um... Are you asking the question of, uh, should you be hosting your models directly on your hardware without Kubernetes? Is that

[01:01:53] Whitney Lee: I, uh, I feel like there are a lot of options of where and how to host it, and it just feels like when I think of Kubernetes, I think of ephemeral pods or smaller workloads, and when I think of running a, hosting a model, I think of huge, huge beefy instances. And so I, I wonder if the two still go together

[01:02:11] Lin Sun: Yeah, I would say definitely go together because if you look at a company like CoreWeave, uh, they're running Kubernetes. I think they gave a talk of, uh, uh, in KubeCon, I can't remember which KubeCon, but they gave a talk about running Kubernetes as their infrastructure layer, and what they provide is, uh, like GPUs,

[01:02:31] Whitney Lee: Mm-hmm.

[01:02:31] Lin Sun: to use.

[01:02:32] Uh, so, uh, I do think Kubernetes, a lot of things we invented for Kubernetes, right? To be able to have the resiliency, the failover, um, the scheduling, efficient scheduling can continue to be used.

[01:02:47] Whitney Lee: Yeah

[01:02:48] Lin Sun: Kubernetes is as efficient as we want it to be, there's always room for innovation. Um, but, uh, I think Kubernetes, uh, is definitely on the right track for hosting

[01:03:00] Coté: Well, you know, uh, what listeners are on the r- right track for is the end of the show. They, they, they've been careening towards it, doing a good job inferring when it was gonna happen, and here we are. Which, which is to say thanks for being on. It was fun talking with you

[01:03:15] Whitney Lee: Thank you, Lin

[01:03:15] Lin Sun: Oh, thank you so much for having

[01:03:17] Coté: And do you have a, do you have a homepage on the worldwide web or anywhere else that you would send people?

[01:03:22] M- where could they get that fine book of yours?

[01:03:25] Lin Sun: Uh, oh, okay. So they can go to solo.io, um, to download our book. It's right on our homepage. There's a pop-up they can

[01:03:33] Whitney Lee: It's called AI Agents in Kubernetes

[01:03:37] Lin Sun: And we're, uh, actually refreshing this book, uh, for Salt Lake City as

[01:03:41] Whitney Lee: Already?

[01:03:42] Lin Sun: the pace is moving so fast we, we have to kind of refresh it every, every year. Um, and, uh, I also very active on LinkedIn, so I would love people to, uh, reach out on LinkedIn if they have any questions.

[01:03:58] Coté: All right. Well, we'll put, we'll put links to those in the show notes and, uh, which will be available at, uh, softwaredefinedtalk.com/127. Uh, and, and you can, you can click on that and click on other stuff. Did I say talk?

[01:04:12] Whitney Lee: Yes, you

[01:04:13] Coté: Good, good job.

[01:04:14] Whitney Lee: Yeah.

[01:04:16] Coté: Softwaredefinedinterviews.com/127. If you, if you go to the other URL, you'll get something from like 12 years ago probably which will not have links about AI or, or Lin.

[01:04:29] So, uh, with that, we'll see everyone next time. Bye-bye.

[01:04:32] Bye

[01:04:33] Lin Sun: Thanks