00:01.43 James ah Welcome back, everyone, to Merge Conflict, like your weekly developer podcast. Did you do like that dramatic pause there, Frank, or you saw it? I was like, I know we're two seconds in. 00:06.94 Frank Well. Because you make fun of me every time because I have no idea when that ah podcast actually starts. So i I have to physically look at you. So when you play games with delays like that, my heart just races. I'm like, are we going to record or not? Is he frozen or not? Is he doing something with the video again as he likes to do? Hi, everyone. Welcome to Merge Conflict. 00:29.73 James ah You're one of our co-hosts, Mr. Frank Krueger, and I'm the other co-host, James Montemagno. 00:33.09 Frank but 00:34.73 James How's it going, buddy? 00:36.28 Frank Oh, it's going good. I've spent the day yelling at an AI. I think that's what we've been reduced to as software developers. It's just... uh resisting writing really really really bad language at the ai as it messes up something what you consider to be fundamental and basic and simple i like it when i put a lot of effort into like perfecting some code and then i ask it to make a small change and it rewrites all my code it's always so delightful 01:02.67 Frank and And then you get a bill and then it's like, you're out of credits for the month. You're like, great, great, great use of my time and money. Okay, I'm gonna go pet the cat now. 01:11.69 James Well, I kind of want to talk about this a little bit because how I've been working. With. Models. Has changed. And then also with agents. 01:22.12 James specifically like kind of like long-running agent processes has changed And I feel as though we did a podcast maybe like three months ago. And since that is all ah old and out of date and busted now, I figure we should revisit it. 01:33.83 James um So we. 01:34.08 Frank Okay. 01:34.56 James We did a podcast Frank. 01:37.47 Frank Hmm. 01:37.17 James Um, about how you should never change the default reasoning. 01:42.02 Frank Oh yeah. Yeah, it's... Yeah, I still have mixed feelings on it because I think the advice is different for every im model now, but I'm curious to hear what you have to say. 01:52.11 James Yeah, you know, you see all these benchmarks and they're like, oh, we were doing like, you know, Opus on like this thing and then that thing on this thing and this level on this thing. And I was like. Well, how do you even really compare? 02:02.64 James I feel like that's the hardest part is these. 02:03.57 Frank Yeah. 02:05.31 James these comparisons, like how long did it take? 02:06.22 Frank You can't. 02:07.74 James How many tokens did it do? Like what's these, you know, you know, there's outcomes, all these other things that you do. So I think the benchmarking is really hard for. a a a normal developer or a normal person to understand you see all these like lines and charts and these things are going i'm like i don't even know what this says is it better is it worse i'm just give me a model that does it right and I should just... 02:22.72 Frank Yeah. 02:26.36 James probably use The default, that was our guidance, was use the default. And then are you using the default, Frank? 02:32.93 Frank Yeah, default or lower. i'm I'm still kind of sticking with that because there's there's nothing worse than being on high and watching it just kind of think in circles because that's all they ever seem to do is think in circles. And then it'll still output the wrong code. 02:46.68 Frank So my philosophy is basically this, put it on medium or low thinking. Get the... good or bad code quicker and then i can quickly check like is that good or bad nope try again. And that Turns out to be faster than putting thinking to high, like putting me in the loop. I find It's superior to... um 03:06.94 Frank high thinking. Now, I'm using these things, I believe, as the kids would say, wrong. I'm doing it wrong. I'm holding it wrong. like I think you're supposed to not read code, not care what it's actually doing, go get a coffee, come back after you spend, I don't know, 10 grand or so, and then that's when you find out if it worked or didn't work. I believe that's the correct way to use these AIs, according to the kids, but I do it differently. I i watch over them like a mean... 03:36.24 Frank nanny or something. 03:38.07 James I'm well, that's where I'm going to talk about how I've changed a little bit, but that's majority of what I'm doing is I'm usually sitting there. And I'm normally taking a look at it and I'm. trying to understand like the questions and the output that it has. But I have found... 03:51.70 James Um, something a little bit different in two aspects. First, I want to talk about Planning. I know I want to talk about planning. 03:59.22 Frank Oh god. Oh no. 04:00.91 James ah That, that, that's. 04:00.64 Frank Oh no. I have opinions. 04:02.30 James that That's one area or or I would say research. OK, so like. ah Research planning. And then there is the. Second part, which is. little baby models. 04:12.99 James And ah this is where. 04:13.54 Frank Hmm. 04:15.96 James Uh-uh. In both instances, I think there is high value. in changing the reasoning levels of it. So example one. research, planning, et cetera. 04:28.52 James Now normally, I'm doing a normal plan. Like I need to implement this feature and it's. Pretty high level. Good example is on Tiny Clips. I noticed when the editor window opened. 04:39.65 James And I took another screenshot. It would close that editor window and open a new editor window. So multi-window wasn't enabled. So I was like, hey, do a little bit of research first. Go figure out, go look at docs, and then let's put a little plan together. 04:51.49 James Just normal mode, normal medium, you know, soul, reason you know, maybe Tara reasoning, figure it out, implement, done. 04:52.16 Frank Mm hmm. 04:58.31 James 50 cents, I'm good to go, right? 04:59.82 Frank Yeah. 05:00.37 James However, And then... was doing this really big a cook. And I was doing it with. Um, Fable 5-1. 05:11.93 Frank okay 05:12.23 James And. I said, I want you to do Really deep research. on performance profiling. of recording video on windows with these apis analyze my apis take a look at this sandbox it and then i kind of gave it some guard rails but i put it into uh extra high mode which is scary because it's like ah fables already expensive give me extra high money however 05:34.52 Frank Hmm. Yeah. 05:39.22 James What I wanted to do. um was see if I could get really, really good rationale. And what I ended up finding is that it would... Uh, Go in. 05:50.67 James like really do really really crazy deep research and analysis and look at apis and do this stuff so this was like something that i was like this is a high highly valuable feature performance implementation. 06:01.55 Frank Yeah. 06:02.08 James Like if I was to spend. $10, $15, $20, $30. on making the general performance of the video recording and capturing better. worth my time instead of fighting because i've already fought with it for two months and it's it's okay but i was like do this thing 06:13.99 Frank Mm hmm. 06:19.24 James So I let it cook and it was like letting it cook, letting it grind. And it put together and I had to put together a research document for me. canonical. research of all these APIs. And I think it's in the repo and the tiny clips repo. 06:29.98 James And it's like really detailed. And it was like beautiful. I don't know. Maybe it was like 15, 20 bucks or something like that. So pretty expensive, pretty expensive. 06:36.18 Frank Mhm. 06:36.77 James However, 06:36.58 Frank Mm hmm. 06:37.89 James Then I was able to switch it into. a little model, where it doesn't need to think. because the thinking's already done, and then just implement the puppy. 06:44.40 Frank it's better if it doesn't think 06:46.44 James in in fact yeah in fact yeah exactly so kind of going back to this rationale so i thought that that was 06:46.58 Frank Yeah. 06:52.10 James interesting because you know that's one example now granted i do not use much Fable 5-1. or astra or anything i'm mostly on terra so even this same logic would apply to like terra right maybe put that on max mode which is a middle model of the gpt56 just saying hey listen let's go and cook this up and that would have been a lot cheaper uh in general for for most things and and then Let me get some really detailed part. 07:18.34 James Not that I would do it every time. This is what I really wanted to get into. 07:21.12 Frank Okay. 07:20.68 James Not that I would do it every time, but for a really meaningful, deep. API research reporting analysis. go and then do the thing. And what was cool about that. 07:32.57 James is it wrote guidelines on how to do performance profiling. So when Tara, or I think it was Tara that went to go implement it. It didn't need to do tons of research. It had everything. And then it was able to do all this performance profiling automatically for me in like real time. It was like really cool. So that was like a really neat. 07:51.79 James ah implementation of saying, let's turn this puppy up. So I don't know if you. I know you're... This isn't really, it is planning, but it's not planning. But like, that's the type of planning I'm into now. For example, another example really quick would be. 08:03.04 James I'm building the .NET Conf keynote. potential demo. 08:05.56 Frank Hmm. 08:06.37 James this pickleball app and I was like, you know what I need to do right now is I need to do some deep research. On. 08:13.29 Frank of it 08:13.83 James orchestrating the entirety of this doc, not spec driven development, but like legitimately let's plan this puppy out. So I i took soul, put it in extra high mode. I said, let's cook on a plan together. Ask me every little detail, do this. What MCP servers do you need? What's this? What's that? And it put together 18 documents for me. 08:32.54 James step by step, let's implement this puppy. 08:32.83 Frank Yeah. 08:34.16 James Let's cook this puppy. Let's do it. Right. And it was really, really neat because it, it even understood. you know, it did the additional research to like say, okay, what about. authentication what about mobile authentication what about this what about this what about databases what about this and it was like okay let me look in like Where do you want to run this for development? Do you want to run and WSL? Do you want to do this thing? I'm like. I was like, oh, this is like really neat. So it really put together a super fleshed out plan. 08:59.48 James Um... By doing that deeper research that I don't get by, by just like. going and running, right? 09:06.52 Frank Yeah. ah I'm with you. I actually do a lot of the same. um So where where I was saying about coding, I put it to medium or low. I do use high for not exactly planning the way you're doing it, but more like brainstorming and figuring things out. 09:22.09 James Hmm. 09:23.74 Frank I tend to. i I tend to do the kind of planning in my head a little bit. So when it comes down to like, there's a feature I want or something, I usually say like, I should use those like real me skills or whatever. 09:35.72 Frank But I'm basically like, ask me 20 questions. Like you you are not allowed to do anything until you've asked me at least 20 questions and then we'll go on from there. 09:38.49 James Yeah. 09:44.74 Frank um But that is when I use the high. reasoning, because that's when I wanted to be smart. And I think you're making a good point here. You're using an expensive, smart thinking model to generate an artifact, and that artifact can then guide the little models. 10:00.82 Frank I think that that's a pretty good procedure. I do get a little bit nervous, though, because I watch a lot of... YouTube vibe coding AI experts. And my God, they create a lot of documents, James. 10:12.07 Frank Like for the smallest feature, they'll create like the plan, the requirements, the design doc, the this, the that. 10:18.39 James wow 10:19.73 Frank And it's just like doc after doc after doc. If there's one thing a language model likes to do. It's right. They love to output Markdown. You ask them to output Markdown, they will output some Markdown. 10:28.81 James Thank you. 10:32.77 Frank And I i find that fine because you're whittling the problem down and you're guiding maybe the dumber AIs. Where I've run into personal problems with it is it'll generate this big dock. And then nestled in that dock is something I actually don't agree with. But I'm not going to read. 10:50.02 Frank 50 ,000 words. That's like a novel and it's just not going to happen. So you generate this doc and then you hand it over to Luna or whatever, a smaller model. 11:00.75 Frank You don't know what little hidden things are in there that are going to come to bite you. in the bot later. Now maybe 90 % is fine. It's fine. You know, it's better than not giving lu Luna guidance. It's better than spending $10 ,000 on Astra to do something crazy. um So i'm i'm still I'm with you. 11:18.59 Frank But I've... I still get nervous with all these docs because i've I've had it happen over and over and again where they just hide some stupid little thing that I don't agree with at all in that doc. I didn't read the doc. 11:31.10 Frank No one's going to read these stocks. They're spitting out words like they're free. They're not free. But um and then. something down the road. I'm like, why'd you do that? They're like, well, in the doc, it said to do that. 11:42.92 Frank You got me. you got me brother i didn't read the docs or tfm so i'm with you but with some caveats 11:48.60 James Now that 11:51.62 James I think it's true. You know, we talked about spec kit and we talked about these other things and someone was asking me when I was. I was taking screenshots and sharing the, the, the keynote, that the demo app that I'm working on. And they're like, Oh, what are you doing? Like spectrum and development? I said, no, I just, I did create a bunch of like. 12:05.79 James General. documents about like what like what should be in the api and like what should be in the mobile app and like what should be the orchestration and like what are all the different features basically i was almost kind of like prd docs but they were a little bit more detailed like you said they definitely had like 12:19.25 Frank Yeah. 12:21.73 James spec numbers and things like that and you're right like i'm not going to read any of these docs so i assume that they're correct because there's like a lot of words so that does become ah a problem that being said 12:29.09 Frank Yeah. 12:32.24 James Yeah. Like the real world example where it is smaller and tighter, like the video. um recording stuff i actually did read that doc because it was It was like. 12:39.66 Frank Mm hmm. Yeah. 12:42.71 James smaller and tighter. And i was like genuinely I was genuinely interested. I was like, oh, I am interested in this topic. You know, can I do this? 12:48.77 Frank Right. 12:49.70 James Now, that being said. Most. of these design spec docs, like, you know, even in spec kit, it's like, oh, you you don't read the doc. Someone else reads the doc and review the docs and do the docs. 12:59.96 James And that that that other thing is an AI, right? 13:00.75 Frank Mhm. 13:02.80 James So you can't. 13:02.85 Frank Yeah. 13:03.83 James Feed it all in there. There has to be some. taste at the end of the day, you know, and and doing that. 13:07.84 Frank Yeah. And this is where I think our our perennial topic of... um Greenfield versus what you disgustingly call Brownfield software development. AIs continue to rock. 13:20.90 Frank Round field. When there is already something there and they just need to debug it and fix it or add a feature or something, I trust them. That's the times I don't actually read code. you know or I'm just like, here's a nice code base. Go implement this thing. I beta test it. I'm the tester. It's all fine. It's when you're starting from scratch. That's when it can start putting the weird stuff into all these docs that you won't read. 13:44.85 Frank And then the best advice with these docs is... just When you're vaguely done with that session, just delete the doc. Clear the memories also. You'd be surprised. Go read the memories that these agents are writing. You will be surprised at what nonsense they think is important. like Your one little criticism of them will become like... 14:05.40 Frank 20 paragraphs in a document about passive aggressively trying to satisfy your needs. So I think. The problem with the docs is we don't read them. The second problem is they get out of date. So the moment the feature's done, just delete the doc. just and And go into the memories and delete your memories too. I promise you, life will be better. Just delete those stupid memories. What a stupid feature. I wish I could just turn it off. 14:30.35 James Yeah, it is. It is good. I think that. um Like. If there is anything. You could extract. Like what you could do is you could extract, for example. um very specific pieces of details and maybe you ask the ai to extract some detailed information that's like important when working in this file and then created a custom instruction for working in that file. 14:44.69 Frank Yeah. 14:51.15 James you know what i mean instead of doing like this big here's a random doc like no only care about this thing in general i'll say this like this instance 14:51.47 Frank Yeah. 14:57.34 Frank Mm hmm. 14:59.52 James I do more rare, right? I'm not necessarily always pumping it up. and turning it in on this. However, There is another. change. That I think is more. 15:09.12 Frank Oh. 15:10.38 James realistic that I think you could be doing right now today, Frank, and every single listener. 15:14.76 Frank Okay. 15:15.07 James is that 15:15.30 Frank I'm ready. 15:16.43 James Everyone on Twitter is so obsessed with just using the most biggest expensive model and putting it on the the highest reasoning. hum and like building things that I don't. 15:23.38 Frank Yeah. 15:25.61 James you know, don't matter, right? Like. game clones and this and that it's cool it's like what these things can do fantastic i love it burning the tokens for fun let's do it let's build some real software but 15:28.48 Frank Yes. 15:31.38 Frank Yeah. Mm hmm. 15:35.37 James The, and I do it myself, so I'm guilty on it. Like I literally built. bananify .online, which is one of my favorite websites where you can bananify your website. It's an extension. 15:43.00 Frank ah 15:43.39 James There's a VS Code extension. Visual Studio 2026 is coming soon. 15:46.08 Frank Of course there is. 15:46.52 James Banana, banana fight out online. It's great. 15:48.66 Frank Great. 15:48.72 James You can preview it at Banan. Bye. Nah, nah. fi .online. 15:53.11 Frank I think these AIs are enabling your bad domain purchasing habit. They're just enablers. 15:59.35 James I did. It definitely did. um Okay, so. Frank Grigger. The... Thing that I would call out here is. 16:05.98 Frank Oh. 16:07.14 James There is... Specifically... these little smaller models. That. are really capable, especially more capable than models we had six months ago, a year ago. 16:19.56 James that are extremely impressive. And. Their default reasoning is pretty low. 16:26.80 Frank Hmm. 16:26.82 James However, Their cost is negligible. I'm specifically talking about Two models. that I would call out. 16:33.59 Frank I'm excited. 16:33.34 James which is going to be GPT-56 Luna, or I like to call it. Baby Luna. 16:38.29 Frank Okay. 16:39.54 James Oh, baby Luna. 16:39.45 Frank Baby Luna. 16:40.50 James Baby Luna. 16:40.62 Frank Okay. Little baby doesn't know. 16:41.58 James And then also. MAI Code 1 .1 Flash. 16:45.53 Frank Okay. you've You've been using it? 16:46.46 James No. 16:47.24 Frank I know we talked about it a while ago. I... I'm always scared to click on the my because I do have that. um FOMO thing. I'm like, what if I put all this time into this prompt and then let it think forever and then it generates garbage? 17:01.52 Frank Well, I always be wondering, would Astra have generated garbage? Would Fable have generated garbage? So that's why it's the FOMO thing about using the smaller models, which I find funny because on the podcast, just like a few... 17:07.34 James ye Hmm. 17:13.57 Frank three months ago i'm like hey babe i'm running local models on my own machine and i'm loving it Because I kind of was. And I stopped doing that a bit. And I'm i'm starting to miss it a little. Because I have been. 17:25.82 Frank brainwashed by big AI and using the biggest most powerful model because basically FOMO. And there's no other reason. because And I want to go on record right now. I had Astra burning burning credits, doing good things. and It completely made a mess of what I asked it to do. So even these like... AGI models, you know um They still suck at certain things. and They're still terrible at certain things. so um 17:54.74 Frank I'm excited to hear why you think. May my. And baby Luna. are good for you. 18:01.86 James Yeah. I think. What I've noticed, I've talked to the MAI team. A lot. um because they you know they really do a lot of training and stuff on the harnesses and work really deep integration of a bunch of great things Out there. 18:15.10 James is that you know they they specifically say and i've talked to them like were these really targeted like oh i have a bug fit i have a bug i need to do this like it's really good at like targeted things as well maybe not only scaffolding the entirety of this pickleball application but there's that but what i found is that these little models Or like very fast, like extremely fast. 18:35.81 James And. 18:35.94 Frank Yeah 18:36.72 James the big models it's not that they're slow it's just that they're They're like they're, you know, swimming in a big ocean of possibilities and they love to. you know it's It's kind of like I've always said with the Claude models, but now I think the GPT models are really... 18:50.05 James Especially Astra is starting to. poke around everywhere, right? And Sol's a little bit better as far as speed goes and exploration. 18:53.38 Frank Yeah. 18:57.17 James um Astro is a much more of a, feels a much more, a little bit exploratory, kind of like Opus did and Sodom before that. But. What I found... is that if I take... 19:08.78 James Either baby Luna or MAI. Code 1 .1 Flash. And I just. Turn. the reasoning to the highest possible level. 19:16.26 Frank Oh interesting. 19:16.64 James The highest. possible level. 19:19.35 Frank Okay. 19:19.49 James What I found is that the results. are pretty much... Exactly the same. 19:26.13 Frank Oh wow. 19:25.87 James As if I was to pick Tara or soul on the default level. And I did this in a very pointed one. so That example of Tiny Clips. That specifically. 19:36.31 James Um, And you can do this. ah which it couldn't. Do multiple. things like you couldn't do multiple windows this needed to maybe change like two or three files basically the app cs for tracking windows and needed to do something in the other window I said, this is a great fix. 19:52.95 James So I did it with... Like soul. Tara. And then I did. uh baby luna and then an mai 1 .1 flash the difference was that i did default default Super high. like I think it's max and then like. extra high or whatever on 1 .1. 20:09.83 James And at the end of the day. They all within 99 % had the same exact results. They all fixed the thing. 20:17.17 Frank Yeah. Yeah. 20:18.08 James They all fix the thing. they all also updated my changelog like i told it to do in my hooks And then... Um, I think that... I forgot which one now, but. 20:30.66 James Three out of four also like made sure to implement the test harness. like updates required in the smoke test for it. 20:38.60 Frank Mm hmm. 20:38.40 James So. What I found... And then speed wise. Is that. mai and baby luna super fast however the most important part We're talking about. 90 to 95 % savings. 20:51.97 James And baby Luna. Versus... 20:55.02 Frank Yeah. 20:55.04 James The other ones and like M .A .I. one I want to be balloon, I think are the same prices are very similar prices, but we are talking. ah crazy Which means you can even bump up. 21:04.63 Frank In order of magnitude. 21:04.67 James If you want. 21:05.95 Frank Sorry to interrupt. 21:06.09 James Oh yeah. 21:06.66 Frank Like it's an order of magnitude. 21:06.87 James Oh. 21:08.28 Frank Yeah. 21:08.26 James Oh, work yeah. 21:08.32 Frank In in dollars. 21:10.16 James we're We're talking like I'm talking. Like. a few dollars to like $1 .50 to like 15 cents, 10 cents. 21:17.25 Frank Yeah. 21:17.67 James You know what I mean? 21:18.83 Frank Yeah. 21:18.78 James I'm also doing this thing for this GitHub Copilot Day keynote. where I'm generating this website for Seattle, like coffee shops and stuff like that. And I've been testing nonstop, mostly for timing for the keynote. I've tried Astra. I've sigh tried Opus. I've tried all these things. 21:33.71 James Baby Luna will implement. almost the identical website. I'm not saying it's identical, but 21:39.13 Frank Mm hmm. 21:39.07 James a beautiful website. in under five minutes with full CI CD pipeline, do the testing, do a whole thing. 25 cents. 21:47.26 Frank Yeah. 21:47.41 James the initial prompt to Astra, just like it reading as like, okay, I'm going to implement it. You're already at a dollar. Do you know what I mean? Like, I'm just saying. 21:54.67 Frank Yeah. I do. 21:56.17 James Like, now, it's like it was like, let's go, right? Like, that that initial hit. 21:59.45 Frank Yeah. 22:01.04 James of that input token cost comparatively. So what I'm thinking is. I'm also okay. If. The Max. highest reasoning level on these little models does even take the same amount of time or a little bit longer because the cost differential. is bananas. 22:20.50 Frank Yeah. ah My whole argument against the Max was time. So you're you're compensating for that time by using just a much faster model. So that makes perfect sense to me. Honestly, you're inspiring me a lot because I miss my old local model because it's... 22:37.12 Frank It sounds terrible. It's stupidity was a feature. And I'm noticing this with other models. And I don't mean to insult baby Luna or me, but like, um you know, the model I use the most these days is the stupid one behind the Google search. 22:52.47 Frank I'm using the, I don't know which, I'm sure it's a Gemini. I don't know which Gemini it is, but it's a stupid one. And you can tell it's a stupid one because it's super fast and they're basically giving it to you for free. I haven't noticed any real limits on it. And because sometimes it says really stupid things. 23:10.71 Frank um but it's a very quick model that's always right there always at my fingertips and i'm still googling things left and right so good on you google for integrating the ai into like the place i'm always searching for something But be like. I've gotten very used to it as a small model. Like I know what to expect from it and what not to expect from it. I don't expect... 23:31.29 Frank astral levels of things on it. 23:33.27 James yeah 23:34.12 Frank I don't even know what that means. Astro really hasn't impressed me that much. I'm just using it as the example. Um. So, yeah, anyway, that's my way of saying, like, I'm actually using small models all the time. 23:45.27 Frank But, again, it's the stupid FOMO thing, why I haven't trusted it to actually work on my source code or things I really care about. Like, in a conversation, it's fine. But I haven't trusted it. 23:53.34 James yeah 23:54.42 Frank yet um so i really appreciate you and i think you're inspiring me to Try. Try them out. try Try the little guys out. I mean, I use Git. It's fine. If it's fast, I can just, you know, get revert real quick and then try a bigger model and I need to be more willing to fail. in that regard 24:13.66 James yeah I think so. I mean, I've been doing this because I've been doing this and I'm like, just. Do what works for you. Do what works for you. Throw away, throw away, throw away. It doesn't matter. 24:19.98 Frank Yeah. 24:21.00 James now especially my testing obviously i'm lucky i have lots of tokens that i can in credits to burn through but i think it's like valid testing because i can have this conversation so thank you microsoft but I think it is really valuable. 24:29.30 Frank Yeah. 24:31.82 James And I think this has led me to. Try to figure out how much. And how important it is for me to. Do it. be doing the thing that i have been doing historically and the thing that you're doing historically which is watching the agent do stuff. 24:49.32 James Even if you're doing one thing. And what I found. Frank. Is that... In that initial... stage where i' it's asking me a lot of questions. 25:01.81 James I want to be there. by a sign? And then. i'm like go go off into the future and like go and do stuff 25:07.15 Frank Yeah 25:09.48 James And. ah The GitHub. research team just released something called Hydrofusion. And. 25:16.34 Frank Okay. 25:17.53 James It's a... It's it's not. a model it's an advanced runtime model orchestration that you. You pick. As a model. So instead of picking Terra or Sol or Opus, you pick Hydra Fusion. And Hydra stands for something like Hydra. 25:33.79 James something, meaning. 25:34.87 Frank It's an acronym you mean? 25:36.08 James AI. 25:35.68 Frank I mean Hydra is the monster you cut its head off in Anu. two new heads grow or whatever. It's like... 25:41.51 James Yeah, it 25:41.88 Frank Old Greek myth. 25:43.58 James it It means... Something, though, like, oh, here it's a. Hydra. is a hybrid dynamic routing architecture. You're gonna love this. and This is from... 25:53.95 Frank Sounds like an acronym. 25:56.09 James It's from I'm pretty sure this is from. the github team uh put it here if you type in hy yeah hybrid dynamic routing architecture. I think this is from a bunch of people from GitHub. So basically... 26:07.94 James It's all about. You have this. pool of models that you can pick from. And today... Developers have to pick like you pick a model like what we've been talking about pick a reasoning level. What if you didn't have to pick? 26:19.52 James any of that. Um, In general. 26:22.25 Frank Yeah. 26:23.29 James So... The interesting part is. you could say oh we have that right we have that with auto figure out my prompt and then go pick a model that kind of makes the most sense So Hydra Fusion, which I'll put a link here to the blog post. 26:37.45 James for you is a little bit different. And I'll tell you why this has changed my workflow a little bit, because I'm using this as the default. It's only available in the CLI right now, but it is specifically for all intents and purposes changing how. I think about working. 26:51.20 James on features and projects and things like this and and doing things. Because what it does is with each request, Hydrofusion Hmm. makes a decision. of one of three execution patterns. And there might be more in the future. 27:05.66 James so what it does is it goes and the user request comes in and basically you have task routing now in this case the task browsing which is the hydra scoring which we're talking about It basically looks at different reasoning and like tools available and it looks at some policies and it's like, okay, looking at this prompt, I have these three different options that I can go with without for an orchestration pattern. And. 27:29.64 James What this does... is it will pick single so single is kind of like auto like based on this this is a simple one like maybe can you build this can you build and run this application You don't need advanced orchestration for that. Just pick a model. Then you have this other one, which is cascade. 27:44.63 James And this is the one that I actually find a lot, which is. basically an efficient model. will draft a solution. And then there's a quality gate. where another model decides whether to accept it or escalate to a stronger model that then reviews it and puts it back in and then implements it. So in that case, you might have like MAI. 28:03.71 James code 1 .1 flash and then you might have opus and you might come back on it and then there's a third one which is really fascinating which is a critique This is one model drafts a result. an independent read-only critic. 28:15.46 James from a different model family reviews it. kind of like a rubber duck. And then the drafting model revises it once more and then it gets implemented. So it goes through and it picks one of these. 28:23.61 Frank It's not what rubber duck is. 28:25.93 James And automatically, right? And then it will go and execute. So it has a shared pool where it'll go through and figure out how to. solve, review, and then go through this. Now, the interesting part about this. 28:38.49 James is that they did all this like. terminal bench testing on it. And what they found is that. the quality is pretty much Opus 5 level on the same tasks but at 70 % lower costs. 28:50.76 James Because of how they're doing this and be able to efficiently kind of like we're talking about these models and the smaller ones and the bigger ones. but you don't pick any of the models you're doing you pick hydrofusion it does it i've been building and deploying full applications. full features, full bug fixes exclusively. with Hydrofusion. And the cost differential is... 29:10.18 James bananas and the code quality is super duper good the problem is there's not a lot of 29:11.30 Frank Oh. 29:14.55 James things in the UI yet, because this is in ah ah is is a... What is it? A research prop. project, research. 29:20.21 Frank Sure. 29:21.09 James thing i don't know it's research research preview So. 29:22.01 Frank Yeah. 29:25.33 James What is cool about this is that it totally works awesome, but pretty much it's like i'm drafting i'm reviewing it doesn't output anything it just says that word and then at the end it's all like Bleh. 29:36.47 James And it's like, here's everything. So what I found myself doing. 29:39.14 Frank No watching. Yeah. 29:40.65 James is spinning up a bunch of cli stuff and just hydrofusioning non-stop and then Coming back to it, like. later, because who cares? like And so what I found myself doing more and more. 29:53.56 James It was like. not caring. until the end because this is doing the planning the researching they're critiquing all the things that i'd be doing manually just let the agent cook And then come back to it. 30:04.97 Frank Yeah. 30:05.47 James The difference is if I go into autopilot mode. and I pick a model, it's that one model and it's it's maybe doing some of those things. If the model and if... the harness decides to do that. 30:17.90 James However, in this case, it's kind of as if you were driving those decisions smartly for it. And it's been pretty fun to use. It's just came out like. 30:25.76 Frank Mm hmm. 30:26.06 James four days ago from this recording. So it's pretty neat in general. I think Mario put out a video on it with some cool animations, but it is like. really neat, which makes me like want to work with models. 30:37.55 James and agents like very much more asynchronously than ever before I would say. 30:43.62 Frank That's cool. um It's a good, it's like upping your game. um I have not gotten into any of the orchestration stuff. And I'll tell you why. Because I have this stupid, in the in the same way FOMO prevents me from using lower models, um my knowledge of how contacts work and how contacts cost money makes me fear orchestration. because It's constantly choosing a different model for every prompt I give it. than That's costing me big bucks as it has to move my non-cashed. 31:16.30 Frank ah conversation all the way back around. I'm one of those people who has really long sessions. I don't like. start a session, say blah, blah, blah, implement the thing, and then close that session and then check on it or anything. 31:29.56 Frank I have dialogues that I've been having with AIs for six months now and that conversation has been compressed 50 different times and it fills up a one meg um token pool. instantly. 31:42.63 Frank even the compressed ones. fill that up. And so I fear orchestrators mostly because I'm like, how are they? It sounds like it's going to cost more money. But I like that you said. No, it's actually cheaper because it's... it's picking cheaper models and all that kind of stuff or and it's it's It's doing a better process. We've talked about this before. There should always be a critic. and The fact that we don't have a second model being the critic is just... 32:09.12 Frank laziness on our part or failure of our harnesses, to be honest. um they they still have that chat UI model. where critics don't come up. Enough, but. it's obviously anyone who's used these things for any little bit of time knows A critic step is awesome because it finds so many mistakes in the original implementation and all that stuff. 32:32.23 Frank Or like you said, it means you just don't have to babysit it. I'm accustomed to being the critic. Every time it does something, I write a 20. bullet points on how it did a bad job. 32:42.91 James Yeah. 32:43.40 Frank Oh, it's going to go back to the silicon and I'm going to hunt down its family. and no I didn't do any of that stuff, but I do give it 20 bullet points pretty much every time. 32:52.39 James ah do 32:53.26 Frank And it's not like they learn from it or anything. So it's really just me exhausting my wrist, typing all that kind of stuff out. Okay, that's a long-winded way of saying I need to get on the orchestration game. is Is this the best orchestrator you found? Because I know, like, what were the old ones? What was the orchestrator and co-pilot called before? there was like Not autopilot. There was fleets or whatever. Wasn't that like kind of an orchestrator? 33:17.73 James I think that this is... 33:21.34 James It... It's... The difference is this is like a multi-model. reasoning orchestrator. So I think like in the past, right, when I think of fleets and autopilot, they are trying to get to a goal and they are trying to spawn off. 33:35.57 Frank Yeah. 33:35.24 James subagents to go execute and do research. 33:36.76 Frank Right. 33:37.78 James The difference with this is I don't call an it it is an orchestrator. you know, it is a multi-model orchestrator but i think the biggest differential is that It is analyzing multiple orchestration patterns based on what it's attempting to do based on the scoring process. 33:54.10 Frank Mm hmm. 33:54.28 James um instead of just spawning off a bunch of fleets kind of we're like hey okay you have an api you have a back end you have a front end you have a another website i will go do three tasks because they're not gonna intersect with each other right this is like doing a lot more. 34:09.34 James Kind of. pre-planning and organization and and kind of rubber ducking between the two of it and drafting and critiquing and and revising and then implementing. And then I also, with the team, what they were saying is like. 34:20.50 James They don't just always... changed the model. every single time throughout. Like they're looking at the right model. The cash, the cash timing, the busting, is it worth it? 34:30.33 Frank Mm hmm. 34:31.62 James These things do they need to add tools and and not do this. And what they said they also did is like they have it. So they have like tool list contacts, like, so like there's different shared. 34:42.41 James work spaces for these isolated reviews, like help optimize cost and then like all these like different things that it's doing. So there's probably tons of stuff happening under the hood in general. And there's like a great blog post um in there. 34:55.43 James um about it. um And these checkpoints and what it's doing and these costs and how they're hill climbing it and doing all these other things with other charts and graphs that nobody knows, but they're there, but it's pretty neat in general. So at least for me, I've been. 35:08.60 James Telling people. You have to turn it on and experimental in the CLI. You got to flip on experimental and then go into slash models, do hydrofusion research preview, but. 35:20.16 James It's been my default in the CLI, pretty much exclusively doing stuff. 35:23.96 Frank Crazy. 35:24.91 James Um, 35:25.04 Frank Yeah. 35:25.82 James I don't like I think you and I always said. Right. Which was like auto mode is like pretty fantastic. And there's tons of work happening in auto mode too, that they're, they're doing, but that feels like the future. I don't want to have to pick the model. 35:38.35 Frank mm-hmm 35:38.61 James And I think that now it's evolved, which is I don't want to pick the model, but I also don't want to. babysit the orchestration of what. said model is doing is it should be able to figure out these things. And when these things are in the box, I'm sure people will be like, I built my own orchestrator. 35:52.06 James This amazing agent plugin did this thing. 35:52.88 Frank Yeah. 35:54.36 James That's cool. It's not in the box, right? When it's in the box. in And that's the default. that people can pick and it's really really good then i think that's like a game changer for vast majority of users. So I say give it a try. It's not. 36:08.40 James Fast. 36:09.92 Frank Okay. 36:10.06 James But it's not trying to be fast because it's doing a bunch of work. Right. 36:14.25 Frank Okay. 36:14.34 James So here's, what's fascinating is, and I need to actually probably test this myself. is Going through. And then saying, okay, let's say this bug fix that I had. 36:25.78 James For the multi window stuff. Okay, I was. doing individual models and reasoning. What if I just gave it a hydrofusion and I was like, go do it. Right? 36:33.54 Frank Yeah. 36:34.27 James And I bet. Probably it would be. just as good, if not better. it probably it probably would do a lot more critiquing so i'll do that and report back So that's on my much ado list. 36:38.49 Frank You have to. 36:44.42 Frank Probably. You'll have to check in tomorrow though, right? With all your infinite Microsoft crafts, I'm surprised you haven't written the James Harness, which basically whenever you type a prompt in, it just sends it to 13 different models. 36:47.39 James yeah 36:57.91 Frank And then there's like a big join at the end and you can just review 13 and pick from the best. of the 13 every time. i mean If I had infinite credits, I'd be doing ridiculous things like that. 37:10.76 Frank Hmm. 37:10.97 James Well, there, you know, one of the early things that, you know, Pierce had talked about and other people is. You can kind of do that, right? You can basically say, hey, listen. go implement this in three different models and three different flavors and three different styles and go do this stuff. 37:26.11 James And then in three different work trees, then I'll go and review it. 37:26.77 Frank Mm hmm. 37:29.13 James The problem really is like. Are you going to really review all? 30 different implementation models and things like that, right? 37:33.63 Frank Right. 37:35.52 James Probably not. So that's the. The thing. 37:37.66 Frank And there's always pros and cons. Like, I've done this with... gooey's before and you're like well there's some parts of the gooey i like from that one Parts I like from that one. 37:47.62 Frank Parts I like from that one. So that you can't really do a merge at the end. so Fair point. Good pushback, as the models say. um Yeah, it's... 37:55.63 James yeah 37:58.27 Frank It's tough. it's It's hard to A-B test. these things. um Especially when you're reviewing the code because then it's just exhausting because they will spit out code way faster than you can review it. 38:10.88 James Exactly. Yeah. Anyways, that's what I got. That's what I've been. That's my weekend. 38:15.69 Frank I like it. 38:15.58 James AI model things. 38:16.16 Frank So my takeaways are... Try the try the dumber models with a lot of thinking. I like it. 38:22.90 James ye 38:23.64 Frank That's almost the like infinite number of monkeys at a typewriter kind of solution. And then. I really do need out my game on orchestration. I feel like I'm finally doing the thing where I'm using 12 different harnesses. so Yay. um But I'm not doing any advanced orchestration stuff. I will tell you, though, I am i am so over plan files. and to generate. 200 markdown files for something so I'm not sure I'm going to try that experiment that you mentioned but definitely going to try the smaller model stuff. 38:53.02 James Yeah. Try it out. and And I think also... If you had something like super duper juicy, like I think a good example is. you know, I had been working on the performance stuff for like a long time. And I was like. 39:06.51 James Okay. This is it. Let's burn some tokens. 39:10.24 Frank Yeah. 39:09.86 James I'm like, I'm gonna give in. And it was time. right i was like let's do it instead of like you and i've talked about it's like you can just kind of go and rev and bullet point this and this and this and i was like you know what That's... Let's token it up. Let's do it. And it did it. It did the thing. um and and it's like really fantastic now i think like tiny clips um Yeah, these Windows APIs are not. 39:32.83 James Yeah, elegant. So it did a really fantastical job. um doing this stuff so really impressed by it but Yeah, I think give it a try. and don't be scared to try little baby models and um i think you'll be surprised but what i found Okay. 39:49.78 James Little models. Big reasoning. That's a goal it's a gold ticket for success because they're so fast, by the way. so Like the max reasoning on baby Luna is not the same as max reasoning on Astra. 40:02.34 Frank Sure. 40:03.26 James You know what I mean? Those are very different things, but. anyways give it a try let us know try out hydrofusion i'm serious like it's Pretty bananas. um And also, if you're a Chrome or Edge user or VS Code or soon Visual Studio, try Bananify. 40:13.11 Frank All right. 40:18.66 James Bananify .online, the unofficial sponsor of... 40:19.50 Frank Oh my god. 40:21.65 James this podcast. Heather and I vibe this out. um Now you're on Safari, so I'm not going to build you a Safari extension. 40:24.84 Frank It's cute. 40:27.65 James Maybe. But. um It's certified. 40:30.35 Frank It's just JavaScript. 40:32.54 James Hey. 40:33.24 Frank You certified it. 40:33.40 James Hey. ah It's certified on edge and it's being ah reviewed right now on Chrome. It's in the VS code marketplace and it is soon going to be in the visual studio. 40:44.42 James uh marketplace um and the visual studio and visual studio code one Putz. bananas and monkeys in the gutter and throughout your code like which is fantastic as far as like little pets also in your like explorer windows so very very cool and there's banana themes you got uh there's monkey themes 40:58.32 Frank Great. 41:02.94 James for VS Code and Visual Studio, four different themes. 41:04.33 Frank What? 41:05.65 James yeah 41:06.12 Frank I got to tell you about the themes. Like i I used to be a super typography nerd. And at some point someone did this research study on what is the correct like. contrast ratios for reading text on screens and paper and all that stuff and they found a really weird thing it was um A light yellow or faded yellow background with a strong darkish green as the text produced a the most comforting reading scenario and the best comprehension scenario. 41:35.31 Frank So I love that that's actually what you arrived at with stupid banana. 41:35.07 James Wow. 41:38.47 Frank Sorry, and I didn't mean to call it stupid. Silly banana. here That you actually went. To what I used to study as, this is the ideal typographical setting for especially long texts. 41:51.27 Frank You want that. 41:51.24 James Yeah. 41:52.47 Frank faded yellow background and green text. That's the sweet spot. 41:56.59 James It's my default ah theme now. 41:58.93 Frank but 41:59.10 James That is the one that you see on the website is called Banana Cream. 41:59.87 Frank Insane. 42:02.88 James And it's very nice. so 42:05.18 Frank I need a dark mode though. 42:07.06 James There are three dark themes in there. 42:09.58 Frank Okay. 42:09.48 James There is... ah monkey Monkey Jungle. 42:11.76 Frank Banana Sweat Sunday. Okay. 42:13.52 James Yeah, I guess I could do a... I should do a darker yellow theme. i' I'll take a look at that. so Um, All right, that's going to do it for this week's Merge Conflict, people. So until next week, I'm Jason Montemagno with an eye roll, Mr. 42:23.35 Frank And I'm Frank. 42:24.100 James frank krueger over there i see that so let's try one more time with all the eye roll and that's gonna do it for this week's merch complex until next time i'm james montemagno 42:26.09 Frank ah 42:32.26 Frank And I'm Freddy Krueger, not eye-rolling. Thanks for watching and listening. 42:37.42 James Peace.