John Nunemaker (00:00)
Hello, you are listening to the Friday Deploy, episode one, where we discuss how to ship on Friday and sleep through the night. This is brought to you by John. That's me and Garrett, who is over in Colorado. And we are the owners of Fireside, Flipper and Flare. And we decided to make this a dev focused podcast since we have kind of a marketing business one. You know, let's just go full on dev on this one.

Garrett Dimon (00:24)
Yes, sir.

John Nunemaker (00:26)
Yeah. So today we are almost exactly two years into owning Fireside. We bought an app and Garrett brought it up in Claude and he was like, you know, how far have we come in two years? And how do we actually you know the whole goal of this podcast is we want people who do the kinds of things we do to have a podcast to listen to, to learn new tips and tricks, and that's the hope. It'll either come from, you know, me or you, or we'll bring, you know, some guest on, you know, and have them go through like this is what I do.

John Nunemaker (00:52)
To make it so that I can ship on Friday and not feel nervous and not feel stressed about it and stuff like that. So that's kind of the goal and the hope. And this first episode we thought let's just go over like the last two years of Fireside, you know, a quick quick inversion, talk about like what we inherited, what we started with, and then kind of like where we are at today and just kind of go how we got there, like what were the steps in between and how did AI aid that and all that kind of stuff. So you wanna start with like how what where we actually started with?

Garrett Dimon (01:21)
Yeah, so let's see that it's 20. So we bought Fireside in 2024. and Dan started it in 2016. So it was about eight years old when we received it. And Dan had been pulled other directions and hadn't really done a whole lot beyond maintenance for about probably two and a half, three years.

Garrett Dimon (01:50)
So we got, and I mean, obviously things change so fast here in tech that you know something that hasn't been significantly updated in three years is gonna be pretty far behind. So like this was 2024, and we were on Ruby 2.7 and Rails 5. And now, right, Postgres nine.

John Nunemaker (02:08)
Ubuntu like seventeen p Postgres.

John Nunemaker (02:12)
Yeah, nine point five. I mean, it was

Garrett Dimon (02:14)
I and that's what

Garrett Dimon (02:15)
I sent you that text. We went from that nine to eighteen and I think I did it based on the nine's release date or nine five. And it was released almost ten years prior. So we were running on a database version that was ten years older. and yeah, I mean there was just and the things that exist today like caddy that make a lot of the

Garrett Dimon (02:43)
certificate stuff easier, stuff that just flat out didn't exist. So we were able to remove a lot of infrastructure and migrate it over to, you know, other things. But I mean the biggest catch was how do you improve an app for people while also making sure it doesn't fall over while as John described it, you're driving down the road and taking the wheels off and putting them back on.

John Nunemaker (03:07)
Yeah, I just remember

John Nunemaker (03:08)
that there was like a there was a meme of that like a long time do you remember that meme that was like

Garrett Dimon (03:12)
I don't.

John Nunemaker (03:13)
okay, it was like it was like it was like upgrading your SAS like live or something. It was some meme like that. And the car was like literally for people that are like watching, you can see my hand for listening, you're just gonna have to imagine like a car on two wheels. And they were

Garrett Dimon (03:26)
Like on the side.

John Nunemaker (03:27)
like yeah, they were outside the car changing the other two like two wheels. Yeah. While trying to balance

Garrett Dimon (03:30)
While trying to balance it. Yep. Yep.

John Nunemaker (03:33)
it going down the road and that's basically yeah, the last two years. So

Garrett Dimon (03:37)
Yes. And I mean, I feel like we've made a lot more progress than we would have probably been willing to bet on in that two year period. But at the same time, of course, it's never enough. And, you know, there's so much more we want to do and plan on doing and are in the middle of.

John Nunemaker (03:54)
Yeah. I feel like one of the kind of like one of the crazy things. So it's been two years and like you said, on one hand, it's like, okay, well everything has changed in those two years, not just for us, but like for the tech world as well. But also, you know, like you say, we we're like, man, we are just not getting some of the things we were hoping to have done. But you can kind of look back and you be like, my gosh, like we literally like nothing exists anymore that

Garrett Dimon (04:21)
It's

John Nunemaker (04:21)
was there at the beginning. Like

John Nunemaker (04:23)
Ruby two seven, now we're like Ruby three something, three two, three three. We got

Garrett Dimon (04:27)
Well, and we'll be f

Garrett Dimon (04:28)
we'll be fully current in a month or two, I think.

John Nunemaker (04:30)
Yeah. easily. I mean, we the only the only thing that held us back is like literally some all the infrastructure was on like Ubuntu seventeen or some

Garrett Dimon (04:37)
Right.

John Nunemaker (04:38)
super dead for many years version of of Ubuntu. And I was like, well, we have to like redo all that infrastructure, you know, which thankfully, you know, with AI this summer I was able to like basically do that while on a long camping trip, you know, just in the evenings here and there. I just like let it rip and it would, you know, backwards like extract what was in. so like that that was that's probably one

John Nunemaker (05:00)
you know, good way to go through this that's kind of interesting. And I think that's different. Is like we don't have we didn't have any like no puppet, no Ansible, no like any kind of reproducible, like this is how you recreate, you know, your infrastructure. So what I what I did on the infrastructure side is I was basically like, okay, reverse engineer everything that's on this node that it does into a markdown file.

John Nunemaker (05:28)
and then once I had that, I was like, cool, now I know like what, you know, generally where we're at on things. And then I was like, okay, now reproduce this on a brand new one. And so literally, you know, we went piece by piece over the you know over the course of several months and swapped out every single server. And then we finished that just yesterday. I I retired all of the old actually I guess still need to hit delete in our hosting provider because

John Nunemaker (05:53)
the work servers were the last ones. And those were the things that I think were that's what you went through all those Ruby and Rails upgrades, you

Garrett Dimon (05:59)
Mm-hmm.

John Nunemaker (06:00)
know, to get onto all those new versions. We were on Rails five two or five one and we got all the way up to like s you know, seven two or or whatever. And then it was like you were blocked 'cause you're like literally the your Ubuntu version is so old we can't move forward.

Garrett Dimon (06:13)
Yeah. Yeah. Well it

Garrett Dimon (06:15)
was it was some combination of the jump to Rails 'cause like we've got a branch pretty much ready on Rails eight, but the there was and it was a chain of dependencies, but basically it was a chain of dependencies where s something lower in there wasn't gonna fly on that version. And so we had to get everything upgraded.

Garrett Dimon (06:43)
So that we aren't on any EOL versions.

John Nunemaker (06:47)
And I think like the crazy thing too is like if we go back to the whole like driving on two wheels while we're changing the other two, like that was really happening. Like, you know, we had Postgres nine point five and we went all the way to eighteen. And, you know, probably a sane person would have done that in like a couple of, you know, jumps. And we did it in and not only that, but we went from self-hosted Postgres, self-managed, you know, to like a managed host and from nine point five to eighteen all in one hop. 'Cause it was like

John Nunemaker (07:15)
Everyone that I would talk to when I would tell them like 9.5 and hey, can you help us? They would just start laughing. because it was like, you're nine point five. Yikes. Okay. That's well, I'm gonna have to call this one guy with a very strong neckbeard, you know, who lives in a basement that might know how

John Nunemaker (07:31)
to go from nine point five to like twelve. and then AI just went bonkers in the last, you know, six months. And now all of a sudden it's like, yeah, we can

John Nunemaker (07:43)
do these things ourselves. We don't even need someone to help us with some of the scariest bits of these migrations and upgrades. Also, we were on Dan's Cloudflare account and we had to move off of that onto another one. And, you know, ideally with no downtime. You know, people, I I don't

Garrett Dimon (07:57)
Mm-hmm.

John Nunemaker (07:57)
know if I don't know if everyone knows this, but podcasters love their downloads, like very much.

Garrett Dimon (08:01)
Hmm.

John Nunemaker (08:02)
They love their downloads. And so if their downloads get changed in any way, it's like panic zone. And you got to be able to explain why. And if it's, you know, well, we did a thing and you lost some downloads, that's not

John Nunemaker (08:13)
The answer that they want. So you have to

Garrett Dimon (08:14)
Nope.

John Nunemaker (08:15)
be, you know, precise and careful with it, you know. And so I think those were two, the two biggest things that come to mind for me. You did like all the Ruby, the Rails, like a lot of that kind of stuff. You know, my side of things was more like the, you know, getting the Cloudflare account. And again, and most of it is like, I really like this process, especially with AI, where you extract what your current thing is. So you

Garrett Dimon (08:36)
Mm-hmm.

John Nunemaker (08:36)
create like this is what the current thing is. You document that. Exactly.

Garrett Dimon (08:39)
It's like self documents, yeah.

John Nunemaker (08:41)
Self document.

John Nunemaker (08:43)
What we have. That's a great way to put it. And then once you have that self-documentation, you say, okay, now recreate it in this new way with no downtime. Like, or maybe you say with like, you know, sub-second downtime or something that's, you know, that's

Garrett Dimon (08:55)
Right, right.

John Nunemaker (08:55)
reasonable. you know, for like some apps, you can lose a little bit, it doesn't matter, you know. but for like something like this with downloads and people's livelihoods, you know, full-time podcasters, you can't just like say, Well, sorry, we lost 10 minutes of

John Nunemaker (09:10)
of data or or an hour of data while we stopped the world so that we could migrate databases and stuff. So but yeah going going from nine five to eighteen in one shot, we had no downtime. Other than the only thing that we did, you know, you couldn't do rights. You couldn't like change your episodes and stuff like that. So we just used

Garrett Dimon (09:28)
For how long was that?

John Nunemaker (09:30)
that was that was for the rights was probably like a fifteen minute period, maybe twenty. It was b and it was only that long because ChatGPT had to like

John Nunemaker (09:39)
Codex had to compact in the middle for like five minutes. And that was the like was the slowest three minutes of my life because I was like, I was literally like just about to make the switch over. and it's like, okay, compacting. And I'm like, no, not right now. but you know, we got through it. But yeah, it was like that, it was probably like a 45 minute process and about 20 to 25 minutes where rights were off. But but all the reads were up. So like feeds are up, downloads are up, downloads were in queuing.

John Nunemaker (10:05)
So we just encued them and let them back up and then we reprocessed as soon as we turned it on. whole process was really smooth. I, you know, of course I had sweaty armpits and I didn't do it on a Friday. That would have been a great day to do it. but I

Garrett Dimon (10:18)
It was a Sunday morning.

John Nunemaker (10:20)
no, I think I did it on a Monday morning. yeah.

Garrett Dimon (10:22)
Monday morning, that's right. Yeah.

John Nunemaker (10:24)
Cause I was like looking at our, you know, again, like one of the things that you do when you're trying to be build like resilient, available you know, software that you feel confident shipping on Friday.

John Nunemaker (10:34)
is you look at traffic patterns and you say like do certain things when there's lower traffic patterns. That way the risk is lower. it's just stuff like that. but yeah that was that was those were probably the biggest ones over the last two years whereas one like self documenting every Cloudflare setting that was different than the defaults. ever

Garrett Dimon (10:53)
I mean, like the

Garrett Dimon (10:53)
amount of work that would have taken weeks to do manually

Garrett Dimon (10:57)
and had a high likelihood of missing something or overlooking one detail, you could gather it all and then double check it and

John Nunemaker (11:05)
Dude, I I I punted on that project forever. Like, I mean, we were on Dan's account for almost two years. Like I just did this like a month ago. You know, it's like we were for like a year and eight months, we were on Dan's account. Like basically our credit card's in

Garrett Dimon (11:22)
Well it

John Nunemaker (11:22)
there. So, you know, like it didn't didn't really matter, but

Garrett Dimon (11:25)
In large part though, that is like that migration was one of those things that is not straightforward because there's so many like nested things in Cloudflare that it would have been super easy to overlook a detail or configuration setting and not copy it over. So that to me is probably like the most terrifying type of move because you can't automate and test it. You've got to just like make sure I clicked on every single option and drilled down into every single thing. And that's

Garrett Dimon (11:54)
Nerve wracking. Whereas like this other s you know, it's not like you just said, Hey, Claude, migrate everything. Like the degree, the excruciating detail that I remember you talking about, at least most specifically and most recently with the database stuff and like the test runs and the exploration and the trial and error and

John Nunemaker (12:11)
Yeah, yep.

Garrett Dimon (12:12)
like cross, you know, it's not like it was just we, you know, anybody could go trust Claude to do a Postgres nine to eighteen. You just give it one prompt and then come back the next

John Nunemaker (12:21)
Yeah.

Garrett Dimon (12:21)
day and everything's fine.

Garrett Dimon (12:23)
You know, you still have to like go into everything and break it down, but it takes something that would have been ten huge chunks of work and makes them ten digestible chunks of work that can be, you know, verified in a way that's not terrifying.

John Nunemaker (12:40)
Well, and good news for us, like the models just kept getting better. So like, you know, when I first yeah,

Garrett Dimon (12:43)
right. For everybody. I mean that's

John Nunemaker (12:45)
when I first started on it, like, you know, it got a version that was like looked pretty good, but it was like you're gonna have these, all these problems and stuff. I waited like three months because I was like, This looks too hard. And then you know, Opus, you know, whatever comes out, run it against that. And it's like, yeah, okay, like you missed a couple things here, I'll fix these. And then it was like

John Nunemaker (13:03)
Fable came out and it was like, yeah, you missed these things. And then I switched to GPT Soul. And then Soul was like, yeah,

Garrett Dimon (13:09)
Yeah.

John Nunemaker (13:09)
you got these still. And by the time we got to like Astra, I was like, I mean, we're good. Like we know this is there's everything has been painstakingly went through, you know. Like it's I mean, it started all the way with like, okay, we have whatever, eighty gigabytes or something, which is not a lot, but like it's not zero. And it was like how okay, how long is it gonna take to transfer this literally like over the network or things like that?

John Nunemaker (13:33)
And then it was like, whoa, we don't even need this table to be that big. Like I can just shrink this table. And so I did and got us down to like twenty gigs or something like that. And so then it was like, it's just a 10-minute copy now. So now it's like it's not even like this hour-long read-only or or two-hour read-only, you know, thing. It's like or some automatic online migration where you, you know, put a draw a line in the sand and backfill from there and and replicate forward from there.

John Nunemaker (14:01)
It was a lot more straightforward. And the models got better. They helped with it. And it was the same thing with the the account transfer. Cause like the account transfer, you're like, I mean, that's a scary thing because it it was literally I think I joked to you. It was like Carrie Underwood, like, Claude, take the wheel or GPT take the wheel.

Garrett Dimon (14:14)
Hmm.

John Nunemaker (14:15)
Because I was like, I mean, I don't know when I actually do they have a documented process, but I don't know when I actually do that, you know, name server change, what is going to happen? Am I actually going to have no downtime or not? I don't know.

John Nunemaker (14:31)
and it did, it worked, and both of went off good. So

Garrett Dimon (14:34)
Then at the same time, and not to be self-promotional, but flipper was a non-trivial factor in much of this too, in terms of prepping it and being able to flip a switch and having t instead of having to like magically do a whole bunch of work right in the middle.

John Nunemaker (14:51)
And that's actually the cool part for the database migration, because we I basically added a database read only flag. And so if that flag's on, it it what it what it does is it basically the entire app just switches to read only mode. And then any part that like does a write, Yep.

Garrett Dimon (15:09)
And displays a friendly message that lets people know.

John Nunemaker (15:12)
Yeah, it shows it showed a message in the in the studio to say like, Hey, we're in read only mode right now for some database maintenance. It'll be over in the next, you know, ten to fifteen minutes. Just

John Nunemaker (15:21)
Check back in. we had no complaints, no support. Nope. Nothing. I

Garrett Dimon (15:24)
I don't think we got a single email about that.

John Nunemaker (15:27)
mean, it was super smooth. And so all I did that was like right, that was the go live moment was basically like I hit that button, enabled that flag. We went into read-only mode, and then I shut workers off and some stuff like that. Initially I had put the flag in there as well, but then I was like, this is a lot of complication. I'm just gonna shut the workers off. You can do that, you know. Like,

Garrett Dimon (15:44)
Yeah.

John Nunemaker (15:45)
you don't workers don't have to run. So I shut the workers off, I disabled Kron, I did all that normal stuff.

Garrett Dimon (15:50)
Mm-hmm.

John Nunemaker (15:50)
And then it was just the web app. And the web app was just in read-only mode. And people could click around. They could go to everybody's sites and feeds and all that. And then when it was all done, it was just the opposite. I hit disable, database read read only flag goes off. Write goes back through. everything turned back on. And then I turned the workers back on. It processed the backlog of downloads, which wasn't very big. I mean, we were all caught up and live and ready to go in just a couple minutes really. It was pretty fast. And now it's been on that for

John Nunemaker (16:19)
I I did it Monday. It's we're recording on a Thursday. so it's been live for several days, no and no problems, all the way from nine point five to eighteen. So it's cool.

Garrett Dimon (16:27)
Yeah. Well, and

Garrett Dimon (16:29)
one of the things that I have been leveraging more and hadn't really been thinking about that much until you were just pointing out how many different versions of models and agents ran through that database migration plan by the time we felt comfortable with it. And like

John Nunemaker (16:42)
yes. Yeah. Probably six.

Garrett Dimon (16:46)
the amount of times now that I mean it depends on what I'm working on. I'll I mean I almost always have another.

Garrett Dimon (16:55)
Like if I'm using Claude, I'll have Codex review it and vice versa. And then

John Nunemaker (16:58)
Mm, mm-hmm.

Garrett Dimon (16:59)
have Copilot review it in the PR. And then sometimes even have like Claude and Codex review it in the PR because they're they don't have all the context that was built up over the session. And so they

John Nunemaker (17:08)
Mm-hmm.

Garrett Dimon (17:09)
might not be tainted, you know, and have a clearer, clearer eyes. and so at the same time though, if you ask all of these different agents for different opinions, you're gonna get a bunch of different opinions. And then it's like, okay, how much of this really matters?

Garrett Dimon (17:24)
And doesn't matter. And so like it's kind of a double-edged sword with and and obviously they'll all happily over-engineer the hell out of anything.

John Nunemaker (17:33)
yeah.

Garrett Dimon (17:34)
like once I started, like for planning stuff, I made sure that it understood, you know, how we use Flipper and how it's available. And then every stupid little thing it started trying to use a flag. I mean, this was a while back, like, but you know, and it was like, I just need to.

Garrett Dimon (17:52)
Clean up this helper and it like creates a flag for the helper to just render something differently. I'm like, my

John Nunemaker (17:55)
Yeah.

Garrett Dimon (17:58)
gosh, no, please. so yeah, like it could definitely overcomplicate things. So you still have to, I think that's where all of the battle scars come in. And it's like, no, no, no, you don't need to do that. Like, slow your roll a little bit. Let's, let's be more realistic about what we actually need here and not, you know, over-engineer it to the point that it's actually more brittle than it would have been.

John Nunemaker (18:19)
That's the thing that you know, I've been saying a lot and I think you've been saying a lot. E everybody is like taste is what makes the difference these days. It's like that the

Garrett Dimon (18:27)
Yeah.

John Nunemaker (18:27)
ability to say no to certain things and yes to certain things, or, you know, have you thought about this thing? That's the thing that differentiates somebody just, you know, hack a lacking with Codex or with Claude and somebody who's like getting stuff done in a professional manner and, you know, not not gonna regret it someday. but

Garrett Dimon (18:43)
Right, right. Well

John Nunemaker (18:45)
it is it's easy to mess stuff up still. I mean

John Nunemaker (18:49)
My pro my good my biggest problem historically has always been I start on something and I get going on it and it's great. And then like I forget that I was working on it that it the like some script was running in the background and I like just leave, you know, and I don't remember because it was running in the background. I got doing something else. I never just sit there and do one thing. And so then I like look and all of a sudden I'm like, no, I remember three hours later I started, you know, a database migration or something.

John Nunemaker (19:16)
And I go back and I'm like, no, it's you it's in this certain state or whatever. And now with Codex or, you know, you can do it with Cloud too. I can start a thing and I can be like, you know, see this thing the whole way through. Like, like do your do your thing. Go all the way through it. Like I I was telling somebody this morning, you know, I swapped out all the workers yesterday and for fireside. And I was not involved. I mean, I all I did is I said, you know, just like we did the app servers, just like we did all these other servers.

John Nunemaker (19:45)
Make a read-only you know, this is what, like the self-documentation thing, and then apply it on these new servers and then in a zero dime time manner, like shut off the old, turn on the new as the active, you know, thing of it. And it was only like three or four servers, but it went through all of them, it did a thing. What was great is I started that in the morning. I started doing something else, completely forgot that that was running, like left.

John Nunemaker (20:13)
And I got a push notification on my phone, like waiting for your approval to like, you know, make the final switch. Cause auto

Garrett Dimon (20:18)
Mm-hmm.

John Nunemaker (20:19)
approve was like, This is this is a big enough deal, like John needs to be involved.

Garrett Dimon (20:22)
Right.

John Nunemaker (20:23)
And so it like pings me and I'm like, crap, I completely forgot that I was even working on this. Like, and what what's amazing though is like it got all the way through to that point, no problem, didn't have any issues. And so like I do the same thing now with PRs. Like I used to merge a PR, I'd forget about it, auto deploy would happen and it would be out, you know.

John Nunemaker (20:42)
Now I say, merge this, watch it. It's and it's just another level of resilience because it's not I it's it's a robot. It's gonna do what it's supposed to do. I'm a human. Sometimes I'm gonna look off the side and I'm gonna drive a little bit crazy on the road, you know? And

Garrett Dimon (20:57)
Yeah.

John Nunemaker (20:57)
the robot's not. Like if you tell the robot what to do, it'll stay within the guidelines pretty good. And so it just ripped and totally redid all of our infrastructure and I forgot it was even doing it. And then I came back and I was like, and I'm like, Well, why'd you only do one of them? Like, do them all. Come on, like what are we doing here?

John Nunemaker (21:11)
Like let's not and so it just went through the rest of So

Garrett Dimon (21:14)
Well, and

Garrett Dimon (21:15)
this is where I feel like the type of work each of us specializes on has drastically influenced our feelings about some of this stuff. Because like doing front end stuff, and I mean it's gotten better, but I don't know if that's because I've gotten better at like making it not be stupid. But you know, obviously it's been so trained on like where front-end development is right now and

Garrett Dimon (21:43)
You know, it's like, here, let's go to NPM and get all this garbage and use it. And it's like, no, no, no, no, no, no, no. None of that's necessary. Like

John Nunemaker (21:50)
Mm-hmm.

Garrett Dimon (21:51)
modern, like web standard, CSS, HTML. And so like I'd had to spend so much time getting it to like create good, semantic, useful HTML. But then, you know, it's like, wait a minute, all this stuff's changing so fast. I'm like, why didn't you use this specific, you know, CSS property there?

Garrett Dimon (22:12)
And it's like, I don't know, let me look. And it's like, because it was trained three months before it was released. And so its default documentation is what was out three months ago. And so you've

John Nunemaker (22:17)
Yeah. Mm-hmm.

Garrett Dimon (22:21)
kind of constantly got to stay on top of it and point those things out. And so now, you know, that's all kind of built into some of the tooling I've got. And so that it doesn't make those mistakes. But I feel like with a servers are just terrifying, period.

Garrett Dimon (22:38)
Because the consequences

John Nunemaker (22:38)
They're not anymore.

Garrett Dimon (22:40)
are so significant. But in a way, like that's more deterministic than like UI and design and front end stuff. And so you can have a more clear objective outcome. Whereas, like, build a good UI and it's just gonna go and vomit up whatever React stuff it, you know, has in its corpus. And so I trust it a lot less because of that.

Garrett Dimon (23:07)
And it's been how do you encode good taste for these things so they don't just, you know, do horrible or how to get them to just follow the like with fireside. I spent so much time like really building a design system that like captures every little detail of what we did with these updates. And you know, I'll be okay, we just need for like an admin tool. Like I don't care, I just want a table and I want to see the data in our admin.

John Nunemaker (23:33)
Yeah.

Garrett Dimon (23:35)
And I'm like, go build this. And it goes off. And I'm like, where did this come from? Like we literally have a whole design system. There's stuff here. You're supposed to like it's it's all pre-built. Like you just reinvented the wheel and for ten different partials for rendering a table when it's already there. And so it's yeah.

John Nunemaker (23:50)
That stuff is infuriating. That's the stuff that kills

John Nunemaker (23:52)
you on the inside. Like I I I mean, literally like, you know, screaming at the computer, you know, like I don't actually let that go into the AI. I I keep that, you know, I don't hit record when I'm doing that. But yeah, that stuff, like once in a while, when it does that, you're just like, what are you doing? Like this is the how could you do that stupid of a thing, you know?

Garrett Dimon (24:12)
So

Garrett Dimon (24:13)
the dumbest skill that I'm mad that I built and have to use is refocus. And basically I just call it and then it's like, sorry, I was getting completely sidetracked. This has nothing to do with what we originally were trying to work on and

John Nunemaker (24:31)
Yeah.

Garrett Dimon (24:32)
da da da. And I'm like, no shit. and so, like, again, I think, you know, I'm sure a year from now, two years from now, or you know, whatever, at some point.

Garrett Dimon (24:41)
Things like that aren't gonna be necessary. But right now, I feel like a lot of it is finding that towing the line of like, and I think I I put that off for so long, that refocus skill. I'm just because on principle, I was like, I should not have to do this. Like, you know, we created a plan, there's a plan to follow. It's a detailed plan, everything's worked out ahead of time and validated and verified. and then it's like, okay, refocus. And it's like, crap. And

Garrett Dimon (25:09)
I don't know. So things like that are definitely a little tedious and exhausting at times. And I feel like that happens more with front end stuff and UI just because it is more subjective. And you can definitely tell the degree to which it just reaches for whatever the most popular thing is on the internet right now and just slaps that code in there. but I don't know, you know, it's getting better and that's I think it's nowhere near as miserable as it was when we first started.

John Nunemaker (25:37)
Yeah, I think you know, the one thing that is interesting is like you know, you talked about like, okay, it it drives me insane because I spend all this time to do this and then like it doesn't make, you know, good semantic markup or things like that. And one of the things I'm noticing is like, yes, if if I look at the like like my domain would be like the model layer or the you know, database or infrastructure historically.

John Nunemaker (25:59)
pre AI. And yours is probably more front end, but you know, then and you're kind of coming from the front end towards the model layer. I'm kind of coming from the infra towards the model layer. We kind of overlap a little bit there.

Garrett Dimon (26:08)
Mm-hmm.

John Nunemaker (26:09)
But like now, it does all the things like I I'm too lazy to do like, or like I don't even know we're possible. Like when I was like when I swapped, you know, to to Caddy, like we had this Nginx, certbot.

John Nunemaker (26:26)
and all this kind of stuff, you know, to normal 2016 stuff for doing SL certificates. And you're like, man, I'd really love to switch to caddy. And I'm like, I'm like, bro, I'm on like two other things. Like I don't even want to think about caddy. And then I was like, well, I'll just spin a workspace up. And then I was like, it's done. but like it it went through all these things like to to verify that it was working the whole way through before it made the switch by like doing curl requests with like special commands to change host headers and

John Nunemaker (26:54)
all this kind of stuff that like I wouldn't have even known I could test that. I would have like, well, this is as far as I can go until I just flip the switch, you know? And then I would have flipped the switch and been like, nope, that was wrong, switch it back, you know. And what but it was able to like I'm seeing it just, you know, test the new caddy, you know, load balancers, whatever you want to call them, like for certificates being ex like everything before.

Garrett Dimon (27:15)
Mm-hmm.

John Nunemaker (27:16)
So like when I actually like, and that's one of the most common things that I think I've done infrastructure wise on all the

John Nunemaker (27:23)
the projects that I'm working on is like prove and I actually did like both of us do this thing where we like say, Okay, like go over the last, you know, three months of transcripts and tell me what skills I should have that I don't have in in my, you know, AI. And the number one one was like, prove to me that what you did works. Prove it. Prove yeah, prove it or lose it.

Garrett Dimon (27:42)
Prove it. Where's the the

Garrett Dimon (27:46)
evidence? Where's the, but so too, like, you know, it's it's it's like I was reading the other day that there are agents that literally made content and put it on the internet to claim as a source for the thing it was answering. And so it's

John Nunemaker (28:02)
Yeah.

Garrett Dimon (28:03)
like, holy shit, that that that's not, you know, and so for me, I think it's usually more data.

Garrett Dimon (28:09)
Involved, it's like, okay, we should do this, this, and this. And I'm like, whoa, like there's only 50 records in that table. Like, we do not need everything you're describing. And it's like, you're right. And so it's been more like, I want to see kind of the the proof of work. And I mean, I guess it slows you down and takes more time to like review that part, but that proof of work in that it's actually.

Garrett Dimon (28:37)
gathered the information and verified and you know, so and I I think that's one skill I haven't fully re-implemented since I've been like making some updates to my stuff, but I basically had one that was prove it. And so like anytime, especially planning, like things that come back with the plants like I'm like, okay, that's prove it. And it's like, I didn't actually investigate any of this. And so it's you know, I'm like, well,

John Nunemaker (29:03)
If I had a dollar.

Garrett Dimon (29:04)
yeah.

John Nunemaker (29:05)
Yeah. I actually along

Garrett Dimon (29:06)
So you know.

John Nunemaker (29:07)
those lines, one of the things that it like is the PSQL read only stuff. Like,

Garrett Dimon (29:12)
Mm-hmm.

John Nunemaker (29:12)
so the like we've started like all of our projects now have this like read-only script that you can pass a query into and it will execute it. And I've actually found that really helpful for what you're talking about because like if I'm working on something and I feel like it's going off script and it's overbuilding or underbuilding.

John Nunemaker (29:30)
I can be like, okay, use use the PSQL read-only script, you know, query Postgres, determine our scale size, and then make a decision. And so many again, it all comes back to context of like, and I think even going back to like the very beginning when you and I were both looking at this kind of stuff, and the thing you kept focusing on is like, how do you not just give it all the context all the time, but like how do you make context available for it? Give it tools it can use, you know, to like, and we go about it a little bit different, but like.

John Nunemaker (29:59)
How do you make tools available that it can use in situations? And once it learns that, like it'll all the time, it like it did something for Box Out the other day. And I was like, it was like, you know, any it's any picture, you know, bigger than six K images or whatever, six thousand, you know, pixels wide. And I was like, okay, I'm like, well, how many do we have? And it was like, well, let me check with the PSQL read only script. And it went in and it was like, okay, here's the percentile breakdown of, you know, five thousand, six thousand, seven thousand, eight thousand, nine thousand, bigger than ten thousand.

Garrett Dimon (30:25)
Yeah, yeah.

Garrett Dimon (30:27)
So

John Nunemaker (30:27)
And I was

John Nunemaker (30:27)
like, this is so data informed. It's amazing. Like you feel totally confident. And anytime you feel that, like, I think this is my number one tip for like anything AI related right now is anytime you feel anxious, nervous, unsettled, like you feel something inside you that doesn't you're you're you're scared, you keep prompting more prompts to like do more things. your problem is not.

John Nunemaker (30:53)
That you haven't hardened it enough is that you haven't actually determined the verification enough. Like, how do you get to that? You know, this is the

Garrett Dimon (31:00)
So

John Nunemaker (31:00)
actual verification.

Garrett Dimon (31:01)
I do that with interface stuff now. So like with Fireside, you know, there's some set of customers that have multiple podcasts. And then there's some that have like 10 or 20 podcasts, and some that have just one. And some you know, so there's all these contexts. And so basically when I'm designing something in the UI, like, all right, how many customers fall into like this?

Garrett Dimon (31:26)
percentage and you know like to what degree do I need to optimize for them versus

John Nunemaker (31:29)
yeah. Yep. Yep.

Garrett Dimon (31:31)
and so it's and it's so easy to gather the information to inform the design. I don't need to like go ask anybody or even go run the query myself. and I can know how many customers are going to actually use this feature. Or you know like when I was completely redesigning the entire interface for our customers.

Garrett Dimon (31:54)
obviously I could go down the rabbit hole on every single page every single

John Nunemaker (31:58)
Yeah.

Garrett Dimon (31:59)
time. And so instead I would just figure out, okay, how many customers are actually using this feature? And it's like, okay, this one is lower down on the priority list. And so I could focus more on, you know, the more important useful ones. And it made made it so much easier to be efficient's the wrong word, but like just prioritize things.

Garrett Dimon (32:21)
accordingly and and not just be guessing or intuition or okay we've got to build this interface so that's absolutely perfect for people with 20 podcasts. Like no it just needs to work and be usable and not broken. And you know we you know we don't need a podcast search in the podcast drop down. Nobody has, you know, that many that that matters. but

John Nunemaker (32:40)
Yeah, that's a great point. Mm-hmm.

Garrett Dimon (32:42)
like

Garrett Dimon (32:44)
so two, I feel like it was very easy for you to just fly by, like, okay, we've got this little infrastructure script that connects to a read-only database, but like it's read-only, it's a replica. It's like there's a bunch of layers to that to expose it. It's not like, hey, here's our DB produ our production DB credentials and full access. Like and I think for in a lot of cases, they're and obviously I'm not as much on the ops and the back end, so like I don't think or have all that experience.

Garrett Dimon (33:13)
But I I think for a lot of people too, it's like, well, no, there's no way you can have pr access to a production database. And it's like, well, no, there are ways if you do it right.

John Nunemaker (33:19)
Yeah. Yeah.

Garrett Dimon (33:22)
you know, and obviously it's not like that was the first thing we went and implemented. That's something we've only recently been like, okay, this is this is useful and safe and compatible. And but I feel like

Garrett Dimon (33:36)
Every time I end up talking to you and you like mention one other thing and I'm like, crap, that'd be amazing to just be able to like have that information when designing or implementing something.

John Nunemaker (33:45)
Yeah. Mm-hmm.

Garrett Dimon (33:46)
And it makes it so much easier to make more informed decisions.

John Nunemaker (33:51)
Well for like so long I just copied SQL queries back and forth. And I'm like this is

Garrett Dimon (33:53)
Right. That's what yeah. Yeah. Well, I

Garrett Dimon (33:56)
was using Active Record, but yes.

John Nunemaker (33:59)
Yeah, it's just I'm like, this is stu but then you got like if you did it if you did active record, then you'd have to like the way Fireside worked for a while,

Garrett Dimon (34:04)
I know.

John Nunemaker (34:05)
the console was just impossible to get into and and

Garrett Dimon (34:08)
Right.

John Nunemaker (34:08)
execute right and not get a shim wrong and all that kind of stuff. And it's like now it's like I I don't even use console anymore. Like if I need to do something like I just tell it what I want it to do, I confirm it and then I like let it rip on the PSQL read only or on other stuff like that.

John Nunemaker (34:26)
I just feel like those things are just like they're little things, but man, they make a huge difference because they increase the ability for context, you know, for the the agent when it's doing something. Because if it knows, you know, this well, 'cause like and flipper, it's trying to do all these locks and crazy things around everything so that it has like a hundred percent correctness. And I'm like, we have made it how many years without this hundred percent correctness? Like, do we really need this? And I I've been pushing back lately a little bit, especially if I get on soul high or something like that, it'll get real antsy about like correctness.

Garrett Dimon (34:54)
Well it

Garrett Dimon (34:55)
that's less years and more number of requests. Like it it

John Nunemaker (35:00)
Yeah, yeah.

Garrett Dimon (35:01)
is provably working reliably at very significant requests given how it works.

John Nunemaker (35:03)
Yeah. That's yeah.

John Nunemaker (35:07)
Yep. It's it's like we've went four years with with, you know, hundreds of requests a second. Like I yes, I mean technically there's a race here, but it's it's if it does happen, it can fix itself pretty quick and easily. So let's like let's chill out, you know. I've started letting some of them in because I'm like, well, it's actually they're actually not the worst thing in some spots, like to

Garrett Dimon (35:30)
Right.

John Nunemaker (35:30)
be able to achieve correctness.

John Nunemaker (35:32)
I tend to prefer, you know, op optimistic locking to pessimistic, where you actually like lock lock the world, you know. but, you know, and it tends to lean towards pessimistic, I think, because that's the easiest to be correct sometimes.

Garrett Dimon (35:46)
Mm-hmm.

John Nunemaker (35:46)
but for the most part, it's been okay. But one of one of my favorite things has been to be like, do we really need this? Like look at look at Cloudflare or look at this, look at our request stats. Like, what is

Garrett Dimon (35:56)
Right, right.

John Nunemaker (35:57)
the likelihood? Give me, look, give me help me make decision, you know, get

John Nunemaker (36:01)
Give me the likelihood this could happen. And it's like, actually looking, likelihood's very low for all the machinery we added. Like let's I think yeah. Yeah. Exactly.

Garrett Dimon (36:07)
Right. It's handled here, here, here, and here. For it to make

Garrett Dimon (36:10)
it to here, it's one in a trillion. It's like, okay, cool, let's not add more code for that edge case.

John Nunemaker (36:13)
Yeah. It

John Nunemaker (36:15)
yeah, it's gonna happen. I mean, one in a trillion's gonna happen, but it's not gonna happen for like a it's only gonna happen like once every couple of years or whatever. Like very rarely. It's like even if it was once a month, it's like, who cares? it's fine. It

Garrett Dimon (36:26)
Well, and it's not like we're

Garrett Dimon (36:27)
talking about a million dollar bank transfers or you know, we're talking about a a feature flag that is gonna be eventually consistent

John Nunemaker (36:30)
Nope. Yeah. Then I would exactly. Yeah.

Garrett Dimon (36:34)
and

John Nunemaker (36:35)
If it was astronauts in space, I would take different precautions. Yeah. Like I get

Garrett Dimon (36:38)
Yeah, it's a different story. Right, totally.

John Nunemaker (36:40)
why they're not using AI and every, you know, line has to be accounted for. That makes sense. So, but you know, if it's just you know, pressing a button, you know, letting a dolphin flip a flag, then it's a little bit easier scenario. So

Garrett Dimon (36:52)
Well, and this is something I thought about earlier and I wanted to one of the things that I have absolutely found to be completely amazing is so like long, long time ago I built Reviewer, which you're vaguely familiar with, but not necessarily

John Nunemaker (37:08)
Yep. Yeah.

Garrett Dimon (37:09)
totally on board yet. and I built it because I wanted to, you know, when you've got linters and automated tests and all of this stuff, you want to be able to

Garrett Dimon (37:20)
Not be like, okay, what are the five commands I need to go run to make sure that all of these files are in good shape and all that? And so I built it and I used it, but it still kind of wasn't quite there. And so then AI comes along and I'm like, okay, it's doing all this dumb stuff. AI doesn't need to be linting. It doesn't need to be. And so I was like, I'm gonna go back into reviewer and basically make it so that.

Garrett Dimon (37:48)
It can expose all of its capabilities. So you define your linter, your automated tests. And so now on all my projects, I have reviewer built in and all of this stuff, deterministic stuff defined. And so A, I'm not burning credits with the agents doing that work. And B, it gives me a much tighter surface area and a much more reliable outcome. And

Garrett Dimon (38:14)
It runs it all the time automatically because it can so easily run the entire batch of tools against a couple of files and then reviewer intelligently handles which ones it has to do. And so basically now I've got you know, depending on the project, you know, and then it's got like bundle audit and so all of your security things and RuboCop and all of this stuff, and it's constantly basically running it on itself every time it makes changes.

John Nunemaker (38:43)
Mm-hmm.

Garrett Dimon (38:43)
So that

Garrett Dimon (38:44)
it gives it a lot less, a lot fewer degrees of freedom to like really go off the rails. And that's been super, super handy to the point that like even more than I ever imagined it would be. And I'm constantly now coming up with new ideas, like, it could help with this, it could help with this, it could help with this. And basically, all it is is a a very simple like orchestrator of command line tools. And but it

Garrett Dimon (39:11)
presents it in a way that an agent can discover it and know when to use it and make sure to use it. And you can be confident that it actually got used. And it's been super, super handy that like I'd say that's probably kind of, at least right now, the like beating heart of my tooling that I built is that I just push more into command line tools and then expose those command line tools so that I'm not having to trust

Garrett Dimon (39:41)
The agents themselves.

John Nunemaker (39:42)
Beating Heart is a good that's a good segment. What's a what's the beating heart? I'm like my cause I right when you said that, I was like, okay, I know my beating heart. My beating heart right now is scheduled tasks. I've started going really hard into schedule tasks. So I have like every morning I get a basically a flare schedule task, which is the the APM we're working on. So I get here's, you know, all the companies, all the apps, whether they receive significantly more traffic.

John Nunemaker (40:10)
Or significantly less traffic or whether their performance went up or down. And if it went up or down, you know, what's the thing that maybe caused the up or down? and it just like every morning I get spoon fed and I'm like, cool. Like, well, I didn't realize, yeah, that I was getting scanned by Claudot and like we were talking about before, like got almost half a million requests a day 'cause cause CloudBot was like, I really like this app that you're building and it's like slurping in every page.

Garrett Dimon (40:36)
But so then

Garrett Dimon (40:38)
and you and I talked about this before briefly, under the hood, there's a lot more there than just like you created a task, like it's obviously talking to the flare MCP, like there's more going on.

John Nunemaker (40:49)
Yep. Yeah.

John Nunemaker (40:52)
So basically I just made a I made an API for flare, just a read only one. And then I made CLI, just 'cause I I think CLIs are actually generally that's the way to go because AI loves CLIs, like 'cause they can run help and they can see all the parameters and they can figure out really quickly how to they're very efficient with them.

Garrett Dimon (41:13)
Well and that's

Garrett Dimon (41:15)
why reviewer's been helpful, but reviewer doesn't reviewer has all the commands predefined based on that project. And so like otherwise I feel like it goes and does whatever with crazy CLI commands and options and parameters and gets wild. So like that's been another thing that's like nice to like lock that down just a touch.

John Nunemaker (41:33)
Yeah. Yeah. And so like it and then I built the CLI has like a s an MCP that's just a standard in, standard out one. It's not like a a self hosted one. So like you couldn't take the MCP and like install it in a hosted, you know, a chat GPT or something like that. You'd have it only runs like codex, cloud code, things that are local like that. And I threw that in there and I like, you know, list my applications. I did all the like dumb things. And I was then all of sudden I was like, Man.

John Nunemaker (42:02)
I go to like these APMs and I click around in them, you know, whenever I remember, but a lot of times I forget. And so I was like, I just want it spoon fed. And I never did a scheduled task before. So I created my first one, which is like I don't know, Codec s suggested it to me. It was like SaaS operator weekly radar or something like that. And so it's basically

Garrett Dimon (42:22)
Mm-hmm.

John Nunemaker (42:23)
like, here's the stuff that's changing in the SaaS world in last week. And so, and I got my first delivery and I was like,

John Nunemaker (42:29)
that was kind of cool. And I was like, well, what if I just did that with my APM stuff? And so I did. And then I saw like a bunch of performance things. So I was like, okay, spin up a new task to work on that, this, that, blah, blah, blah. Fixed all those. And then the next day I get a thing and it's like, yeah, they all look fixed. but now you got this other one. And then it's like spin up a task on that. And so like, it's just this it it, you know, like you said, tightened loop or reinforcing

Garrett Dimon (42:52)
Yeah.

John Nunemaker (42:52)
loop, you know. The the opposite part of it is that like.

John Nunemaker (42:56)
The AI can make it better every time every time the model gets better, it can just keep making it better. So that's why I'm not building like a, you know, a specific tool for this scenario. The scenario is just like, here's all the data and this is what I kind of care

Garrett Dimon (43:12)
Right.

John Nunemaker (43:12)
about. You figure out how to present it to me, show it to me. And then I've just tweaked it over time. So like at first it was just a flat list of all my apps. Then I was like, okay, show a percentage versus yesterday and versus a week ago.

John Nunemaker (43:25)
You know, and then I was like, okay, and then sh now show it group by companies and add a you know, now add like a little paragraph before each company that's like a highlight and all those kinds of things. And then eventually I was like, dang, I want this for like open source. I miss stuff in open source all the time. People submit a thing, I don't get a push notification or something, or it comes when I'm really busy and so I don't see it. And I just completely miss it. And so then I was like, you know, just go through all the open source public repos in these three orgs and give me

John Nunemaker (43:53)
a rundown every morning of like what's the priority. And it's like just doing that, I was like, crap, now I need a stripe one. You know, I need now I need a QuickBooks one. Or like like I just immediately I was like, I want this for everything. I just want every day I want stuff being pushed to me. And then either automatically, you know, and now I'm like, I I think I'm real close to where it's going to be automatically making pull requests for me and doing, you know, kind of random stuff like that. And like I I it's not yet because I'm like, well, I'm just kind of figuring

Garrett Dimon (44:21)
Yeah.

John Nunemaker (44:21)
it out.

John Nunemaker (44:22)
But this whole s like basically cruns plus AI and plus access and and granted, you know, kind of like whatever you wanna call it, like you said earlier, you know, context you can you can reach out and grab when you need it. Having all that there in on a recurring basis is just like extremely powerful. So

Garrett Dimon (44:42)
And this would be a whole rabbit hole to really go into it. But as a somebody who's spent a lot of time doing interface design, information architecture, like a huge part of me is wondering how much of our interfaces are going to disappear. So, like in this case, like

John Nunemaker (45:02)
I know.

Garrett Dimon (45:03)
so you're talking about clicking around APM or like Google Analytics or anything like that, right?

John Nunemaker (45:08)
Yeah. Yep.

Garrett Dimon (45:09)
And

Garrett Dimon (45:10)
You're kind of clicking around and looking for things. You're, I mean, it's not quite a needle in a haystack, but you're fishing. You're like, what do I need to like be concerned about? But once things can switch to a model where it's like, here are the things that are unusual or you need to be concerned about, or and it can automatically be surfaced.

Garrett Dimon (45:31)
Now the UI for just like poking around aimlessly and swimming around is useless. Instead, what matters is okay, give me the UI that's directly related to this specific instance of thing that you brought up so I can see that and not have to like click down into the app and go there. You click a link, you know, whether it's query string or whatever, and then

Garrett Dimon (45:51)
It opens up a totally on the fly custom view that exposes just the metrics and shows you the the relationships. So you've got it right there, right in front of you at that time. And but it's not a screen that exists or was designed. It's just a collection of pieces that can be assembled on the fly based on what you need to know at that time. you know, and so like interface design, it's still gonna be there, but I'm really, I mean, I haven't again, this is a rabbit hole in it.

Garrett Dimon (46:20)
Just my head explodes every time I start thinking about it. Like there is gonna be a lot of really significant change there, I think, to where or or you know, the other way to think about it in a simpler term is like with Siri. If this Siri AI works and you can like give Siri a URL to Honey Badger, and then

John Nunemaker (46:43)
Yeah.

Garrett Dimon (46:44)
Siri now can get you anything you need from Honey Badger.

Garrett Dimon (46:49)
and mesh it with flare. And so now there's no interface at all. You're just talking to Siri until you need a URL, need some information displayed in front of you so you can actually see it. And then you know again the interfaces I feel like with MCPs, there's going to be a lot of really interesting, crazy stuff that can happen.

John Nunemaker (47:11)
I very seriously am considering not shipping flare with an interface. Like, I'm I'm so tempted to just be like, you sign up, you get an API key, and you put it in, or you use the CLI to automatically install the API key. And that creates your org and stuff. You and you can like you can obviously you can click around in a UI to create the org and see the things, I guess if you want to. But I'm like, honestly, if I can make the org and I can do, you know.

John Nunemaker (47:41)
Like if I can make the org, see the stats, get it get all the things that I need and never go to Flare, I'm fine with that. I don't feel the need to go to Flare Like it's really just I need a place to store data in a particular manner that I can get it back out in a particular manner. And so like there's this side of me that's like I almost think

Garrett Dimon (47:59)
Yeah.

John Nunemaker (48:00)
it would be kind of crazy to just be like, let's not have an interface. The only interface is to put your credit card in. That's it. Like

John Nunemaker (48:06)
And and the rest of it is just like, and maybe for like some of the secrets and tokens and stuff, like it, you know, that you gotta be a little more careful with those kinds of things. But like

Garrett Dimon (48:15)
Mm-hmm.

John Nunemaker (48:16)
there's this side of me that's like I don't go to the flare interface anymore. I just go to codex and I say like what like what's you know, what's slow on this, what's the slowest controller? And it says, Okay, this project's using flare. I know how to get the top operations for a web controller as Rails, that means web. Boom. Okay, it's this one.

John Nunemaker (48:35)
And I'm like, cool. Like I it's way faster than me like clicking around and doing all these kinds of things.

Garrett Dimon (48:41)
Right, right.

Garrett Dimon (48:42)
No, and and like I would much rather I think what I struggle with though is once the agent has all that and it brings it back, it does a piss poor job of presenting it.

John Nunemaker (48:54)
Well, yes, I agree, but for me it doesn't because I don't I don't need to see like I don't need to see a graph. 'Cause

Garrett Dimon (49:04)
Right.

John Nunemaker (49:05)
the agent has already taken the raw time series and extrapolated the root issue.

Garrett Dimon (49:10)
Right.

Garrett Dimon (49:11)
Because really all the graphs are is to expose something that we

John Nunemaker (49:14)
Yep, it's an anomaly.

Garrett Dimon (49:15)
that we see or can't see if we're just looking at a bunch of numbers on a page.

John Nunemaker (49:17)
Visually. Yes. Yes.

Garrett Dimon (49:21)
And so then you're right, the graphs the value of that graph becomes less significant until the instant you need it to understand what it is that's going wrong or you know, whatever it is. But I mean, there's

John Nunemaker (49:34)
Yeah.

Garrett Dimon (49:34)
still like timestamps and seeing the patterns.

John Nunemaker (49:37)
Yes. I but I mean the the interesting thing though is like it will just tell me that all the time. So like I'll just I have an app performance thread every morning it gets a new, you know, brain dump in it. And it's like, okay, very good. Fireside, you know, has sustained a latency regression

Garrett Dimon (49:51)
So

John Nunemaker (49:51)
over the last twenty four hours, you know, twenty four percent from yesterday, sixteen percent from last week, blah blah blah blah blah. And I'm like, Well, that's probably something like really minor, to do with the database stuff. It's just some tuning or something.

John Nunemaker (50:04)
so I'm not worried about it. But like I I can look at Flipper and I can see, you know, like for Box Out, it was like Box Out super healthy. Paint had only f four server errors. and I was like, that's cool. Like we went an entire

Garrett Dimon (50:16)
So

John Nunemaker (50:16)
day and only four errors on our rendering system, which has been that's been a big reinforcing

Garrett Dimon (50:21)
that's huge.

John Nunemaker (50:21)
l availability, resiliency loop thing that I can definitely do a whole episode on. So

Garrett Dimon (50:27)
So this would be a conversation for the future. What if what you're saying and what I'm saying could happen that Flare doesn't have an interface? The interface is entirely query string driven from emails or links or URLs. And it so it has an interface, but you don't you don't browse the graphs. It's just

John Nunemaker (50:46)
It's just the API. Yeah. Yeah.

Garrett Dimon (50:49)
you get an alert and click on it, and it's like, here's all the evidence of what I'm talking about.

Garrett Dimon (50:54)
and what you could do and it just assembles that screen on the fly. It's not like

John Nunemaker (50:58)
Well part of was

Garrett Dimon (50:59)
drill into all of these things.

John Nunemaker (51:01)
I love that idea. I think that's super cool. Like that that more is just like this is an an API for creating a graph if you want to visually see something. or

Garrett Dimon (51:08)
Yeah.

John Nunemaker (51:09)
like if a time series, if you want to see the time series or whatever. But like

Garrett Dimon (51:11)
Right, right.

John Nunemaker (51:12)
there's this other side of me too that

John Nunemaker (51:13)
Like every APM has alerts. But

Garrett Dimon (51:17)
Yeah.

John Nunemaker (51:17)
my thing is like, do you need alerts? I I don't think you need alerts anymore. Like an alert is just a scheduled task.

John Nunemaker (51:27)
You know what mean? Like it's just you schedule a thing to run every five minutes and look at all the things and tell you if anything's wrong. That's that's basically an alert. Yeah.

Garrett Dimon (51:34)
yeah, feature wise, you're talking about not

Garrett Dimon (51:36)
implementing a dedicated alert system, given that Flare's

John Nunemaker (51:39)
In Flare Yeah.

Garrett Dimon (51:41)
entire thing will be based on like go check this information at this frequency and then bubble up anomalies.

John Nunemaker (51:48)
Flare

John Nunemaker (51:49)
kind of just becomes like a a specialized, opinionated time series database, you know, like that's all

Garrett Dimon (51:56)
Right, right. Really

John Nunemaker (51:58)
it is. Yeah. And then it's like everything else goes 'cause I'm like, well, the only reason that I can think of to have a UI or to have alerts or things like that is for people who don't have, you know, codex running locally or don't have clawed code, you know. But I'm like, will those people be looking at this and care? I don't know that they will.

John Nunemaker (52:17)
Or maybe like for in a team scenario, like you want but I'm like with if you like a hosted MCP for these, you in a team scenario, you you Grockbot could do this or you know, anything, you know, can can exist with an MCP and alert you on a regular basis. So really it's

Garrett Dimon (52:33)
Yeah.

John Nunemaker (52:34)
just like you're just kind of enforcing how many reads you're gonna allow people to do and how many writes are coming in and some kind of usage charge basis.

John Nunemaker (52:45)
And I'm like, that's kind of an interesting concept. I don't know. As I at once I got the MCP working and I started, you know, messing with it, I'm I've realized I have not gone back to the web UI at all. And like one of my tasks

Garrett Dimon (52:55)
Right, right.

John Nunemaker (52:55)
is like do the web UI. you know, make it feel like flipper cloud. And so they kind of feel like joint products and all that kind of stuff. But I'm realizing now I'm like, I don't know that I need to do that, honestly. Cause, you know, in this scenario, I'm not using the UI. I'm just telling it like

John Nunemaker (53:14)
I don't really actually want to look at anything. I just want to be like, what's wrong? Fix it. You know? And I just want my agent to be able to do that. it just needs that it needs that read only access and might need some write access to say, hey, turn on traces, any trace that fails the SLO, the service level objective, for like the next, you know, whatever, five minutes. Don't send me more than

Garrett Dimon (53:34)
Well or

John Nunemaker (53:34)
10 per second or something.

Garrett Dimon (53:36)
I mean or that's the interface. You go to Flare and just hit something's wrong.

John Nunemaker (53:42)
Yeah.

Garrett Dimon (53:43)
And then it just turns up the monitoring and starts paying closer attention and but two like so

John Nunemaker (53:50)
'Cause but you want it tied to your code

John Nunemaker (53:52)
though, and I don't want to have pe access to people's code. That's what I keep thinking. Is like I don't want their code

Garrett Dimon (53:56)
Yeah.

John Nunemaker (53:56)
because I don't want to have access, but I want it to have access to their code. So I think it's better for us to integrate into all their

Garrett Dimon (54:02)
I feel you're saying.

John Nunemaker (54:03)
systems than for us to integrate AI into like ours or something, you know? 'cause

Garrett Dimon (54:07)
Yeah, yeah, yeah, yeah.

John Nunemaker (54:08)
even I don't know like maybe it's bad if you expose a controller name to you know, to Claude or something. Maybe that's like a a thing. So

Garrett Dimon (54:15)
Well,

Garrett Dimon (54:17)
I mean it's least privilege, right? Whatever you can, you know, provide the value without needing access to information is gonna anybody's gonna like that.

John Nunemaker (54:26)
Yeah, and Flipper's got

John Nunemaker (54:27)
an API as well. And so I was like, I was starting to build this cleanup report. And then I was like, actually, this would be amazing with AI. But then I was like, actually, I don't think I can just send everyone's feature flags and all the data to AI. That seems like an opt-in thing. And so then

Garrett Dimon (54:39)
Yeah. Nah. Yeah.

John Nunemaker (54:42)
I was like, you know what? I think maybe cleanup is not like an actual UI thing. Maybe cleanup is an MCP API. Because we have an API endpoint that does all that. We just need the MCP in front of it.

Garrett Dimon (54:53)
Well and yeah,

Garrett Dimon (54:54)
yeah, and you could easily just start out with again, just read only and just here pull back the information and let me know what flags need to be cleaned up and just s you know, see if that helps. Yeah, I mean, okay. Yeah, gears gear gears are turning, as they do.

John Nunemaker (55:05)
Yeah. Yeah. They're turning. Hey,

John Nunemaker (55:09)
that's a good thing. This is a perfect time to end with gears turning. Hopefully your gears are turning also. and since Kris isn't on this one, if you enjoyed this, make sure you smash the like button and hit subscribe and all those things. And we're gonna try and do this quite regularly.