STEPH: People put microphones in front of us. That is their fault, not ours. We just show up. Hello and welcome to another episode of The Bike Shed, a weekly podcast from your friends at thoughtbot about developing great software. I'm Steph Viccari. 


CHRIS: I'm Chris Toomey. 


STEPH: And together, we're here to share a bit of what we've learned along the way. Hey Chris, happy Friday. 


CHRIS: Happy Friday. 


STEPH: How's your week been? 


CHRIS: It's been great. I did something that is wildly overdue, but I got a new chair and one day in. But it's also a very familiar chair because it's basically the same -- I think it's the same model as we had at the thoughtbot office. And it's nice to have a chair that is reasonable. And I think my old chair was maybe ten years old or something, deeply embarrassing and absurd like that for such a critical piece of infrastructure in my house.


STEPH: I mean, I guess depending on if it's a good chair. I don't know what the lifespan is of a good chair. [laughs]


CHRIS: I would not describe it as such. 


STEPH: [laughs]


CHRIS: I think it was like $100 at Staples. It was a fine chair. It served me well for many years. I'm very slow and cautious with what I consider to be large-scale purchases. I hate the idea of having a thing that I've spent a bunch of money on, but I don't actually like. And these are very solvable problems. But I just tend to drag my feet and over-research and do all those sorts of things. And so finally I was just like, nope, we're going to get a chair, got a chair. Cool. Now I have a chair, and it's good. It's got all of the adjustments, which is what makes it very nice. I'd say Steelcase Leap is the model for anyone that's interested. 


STEPH: That's funny. I tend to do the same thing. I tend to drag my feet until I get desperate enough that then I'm forced to make a decision and buy something. I do have an oddly specific question. Do you like chairs with or without the arms? 


CHRIS: Oh, with the arms.


STEPH: Really?


CHRIS: Yeah. 


STEPH: I am team, no arms. 


CHRIS: Where do your arms go if there are no arms to put on the chair?


STEPH: They're always on my lap or on my keyboard. So I just don't rest them on the armrest. 


CHRIS: Interesting. I feel like that would put -- I've definitely had small bouts of RSI strain fatigue in my forearms. And so I'm very purposeful with how I'm bracing my wrists. I have a little wrist rest that I put my hands on when I'm using my keyboard because the keyboard is slightly raised up because I have a nonsense mechanical keyboard, of course.


STEPH: Delightful, not nonsense.


CHRIS: Yeah, I love it. I would never trade that in, but I have to make it work and not actually sacrifice my body for a clackety keyboard. [chuckles] But yeah, I think I need some more support for my arms; otherwise, there's too much pressure on my wrists, and things are breaking at weird angles, and that's been my experience. I'm intrigued by the free-flying no arms on the chair approach that you're talking about. This particular model has nine degrees of freedom on the armrest. So I'm able to bring them in and forward and at the exact right height so that they perfectly meet my arm where it would naturally be, and that seems good. That seems like the thing that I want. 


STEPH: That makes a lot of sense. But yeah, I'm team no arms. Every time I have them, I can't get them at the right comfortable spot. And I like the freedom of where I can quickly get up and out of my chair and not have arms in the way, which sounds like a very small improvement in my life, but yet it's what I want.


CHRIS: I just like the idea of you sitting there and being like, I need to be able to make a quick escape at any moment; who knows what's going to happen? And I need to be able to run the other way. 


STEPH: If there's a gnarly bug, I got to be able to run. I can run away quickly as possible. [laughs] 


CHRIS: But in other news, so yeah, new chair that's great. I also recently embraced something in the Rails world that I have known about I think for forever for the entire time that I've worked in Rails, but I've never really used it, which is the tmp/restart.txt file, which my understanding of it is if you touch that file, or if that file exists, Rails will recognize that and will restart the server in development mode. And I think I've always known about this, but I've never used it. And I recognized recently that either I was trying to use a gem that I'd added to the Gemfile, but my server didn't know about it. So I was going to do the thing that I normally do, which is kill the server and then restart the server so CTRL+C and then CTRL+P in my terminal and hit enter, and then wait a bunch of minutes and get distracted, all of the bad things there. And I was like, wait; I remember that there's a thing here. And I don't know why I haven't been doing this for years. It's so much better. I actually went the one step further, and I configured a tmux binding so that tmux prefix and then R will touch tmp/restart in the local directory of the tmux session. That's been very nice, I will say. So I keep moving between branches. And I have environment variables that I need to reload or config initializers that I've made a change to, and I want to load that in. Or a gem that I've added to the Gemfile and I've now installed, but the server doesn't know about. All of these are just so quick now. And why wasn't I doing this the whole time? 


STEPH: I saw that you mentioned this on Twitter a couple of days ago, and I was so excited. But at the moment, I bookmarked it for later, but I didn't have time to actually really check it out. And I'm so glad you're bringing it up because I actually just tried it while we're chatting. So I started up my Rails server, and then I did the touch tmp/restart, and this is amazing. This is awesome. I'm very excited.


CHRIS: It just does the thing.


STEPH: It just does the thing.


CHRIS: Yeah, it's so nice. [laughter]


STEPH: Yeah, this is fabulous, almost as good as the pending migrations button. Not quite because that's a very special button, but this is also up there.


CHRIS: It's a very, very good button. [laughs] I really got very enthusiastic about that button, didn't I? But I stand by it. It's a very good button, and this is a very good file. But this file has existed for so much longer, this workflow. And so many times, I have restarted the server and have been annoyed that I had to do it. And my brain just had this answer available. I didn't read a blog post and relearn this thing. I've always known it. And it was this one particular time that my brain was like, "Hey, you know how we're always annoyed by having to restart the server? You know there's another answer, right? I know that you know it because I'm your brain, and I'm telling you this." [chuckles] This is my weird internal monologue. So I'm very happy to be on the other side of that and to share that with as many people as possible who may be, like me, know about this but haven't actually leaned into it, small things that make the Rails world very nice.


STEPH: Well, I'm glad you internalized it and then surfaced it because this is not something that I had heard of before. So I'm very appreciative of it. This is going to be great. 


CHRIS: Happy to share the wealth. But yeah, that's some of the stuff that's been up in my world. What's been going on in your world?
 
STEPH: It's been a rather busy week. Most of that week has been focused on improving the accessibility of existing pages and forms, which is an area that I don't get to spend a lot of time, but each time I do, I really would like to be a pro when it comes to accessibility. Well, that's probably a long journey to become a pro. I would like to become more knowledgeable in terms of accessibility because it is so important. And while working specifically on these accessibility tickets and improvements, I've discovered a few helpful tools that I figured I'd share here.  So one of the tools that I've started using is a color contrast tool. It's created by WebAIM or web accessibility in mind. And a number of our headers in our application have a white font that's on a background color, and we were getting warnings that this isn't very accessible and that there's not enough contrast. So with the Contrast Checker, you can provide the foreground color and the background color, and then it's going to tell you that contrast ratio. So if you're wondering, well, what's a good ratio? That's a great question. And the W3C Accessibility Guidelines recommend a contrast ratio of 4:5:1 for normal texts and 3:1 for larger texts. Larger text is anything that's typically around 18 px, 18.5 px, or larger. So the color contrast tool has been really helpful because then that's been very easy that we give the blue that we're using, and then we can just darken it a bit to improve that contrast. And then we apply that everywhere throughout the app.  The other tool that I've been using that I'm really excited about it's a browser extension called the IBM Equal Access Accessibility Checker. Is that something you've heard of or used before? 


CHRIS: I have not heard of that. 


STEPH: I would love to know what you currently use for accessibility, and I'll circle back to that in just a moment. But for this particular browser extension, I'm pretty sure they have it for multiple browsers. I'm using Chrome. So I've installed the Chrome extension. Once you have it installed, you can open up the browser console and then tell it to scan the page that you're on. And then it generates a really helpful report that has all the high-level offenses, which are called violations. It also has warnings and recommendations. And then if you click on a specific issue, then the right-hand area shows a detailed description of the offending HTML, what's wrong, why it's important, which I really appreciate that part, and then a couple of examples of how to fix it. So it's been a really nice way as we are working to improve the accessibility of form. We actually have feedback to know that we are making progress and that we are improving the accessibility of that particular page.  And then circling back, I'm curious, do you have any particular tools that you use when it comes to improving accessibility or any standards that you tend to follow? 


CHRIS: Yeah, this is a very apropos question. I'm working on a new project now, and accessibility is definitely something that I want to consider on every project, but it's all the more so important for this particular project, or it's something that we're, as a team, collectively really embracing early on and wanting it to be a core focus of how we're building out the application. That said, I will say that I'm accessibility aware but far from an expert and still very much learning. But some of the things that I have used are the axe DevTools. I forget what the acronym actually stands for there, but we can certainly include a link in the show notes. But those are DevTools that allow you to, I think, do some color contrast checking actually in the browser just right there, which is really nice. There's also AccessLint, which is a project that scans pull requests and, where possible, does static analysis of the HTML. And that's actually by some former thoughtboters. So it's always nice to have that in the reference. 


There's actually a new tool that I've been looking at. I haven't actually tried it out yet, but it's from a company called Assistiv Labs, Assistiv without an E interestingly at the end. But their tool is, as far as I can tell, it allows you to use screen readers and other tools but across various platforms so that you sort of turn on -- It's very similar to if you've ever used an emulated Internet Explorer session because you're working on not an Internet Explorer machine, but you want to make sure your site works in Internet Explorer, same sort of idea, I believe. But it allows you to do the same approach for accessibility. So using a screen reader or using what the native accessibility technologies are on various platforms and being able to test across a wide range of things. So that's definitely one that I'm going to be exploring more in the near future. 


And beyond that, there are a handful of static analysis-based tools that I've used. So Svelte actually has some built-in stuff around accessibility. Because they are a compiler, they can do some really nice things there, and I really appreciate that that is a fundamental concern that they've built into the language, and the framework, and the compiler, and all of that. And I've also used ESLint A11y, which is the acronymnified version of the word accessibility. But that again, static analysis, so it can only go so far. And unfortunately, accessibility is one of those things that's hard to get at from a static analysis point of view, but it's still better than nothing. And it allows you to have a first line of defense at the code as you're authoring it. So that's a smattering of things. I've used some of them. I'm interested in others of them. But this is definitely an area that I'm going to be exploring a bunch more in the near future.


STEPH: I like that you brought in the static analysis tools because that's the other thing that's been on my mind as we're making these accessibility improvements; that's been great. And we can run this particular browser extension to then check for warnings or issues on the page but then looking out for regressions is on my mind. Or as we're introducing new pages and new forms, how do we make sure that those are up to standard if someone forgets to run that extension? So I really like the idea of -- There's AccessLint that you mentioned, which will then scan PRs for accessibility improvements. That sounds really great. I'm also intrigued if there's a way to also -- I don't know if maybe tests are a good way to also look for any sort of regressions in terms of changes that we've made to a page. I don't know what those tests would look like. So I'll have to think on that some more, but I think some people at thoughtbot have thought about it. 


CHRIS: My understanding is the testing library suite of testing frameworks, so it's like testing library React, testing library, et cetera. It's primarily used in the JavaScript world, although there is Cypress, which is more of a browser-level automation. But it fundamentally works from not exactly an accessibility but a -- It doesn't allow you to do DOM selectors. It really tries to hide that. And it says, "No, no, no. You're not going to be digging in and finding the class name of this thing because guess what? A user of your application can't do that." What we want are – Typically, it's like find by label or find by things that are accessibility available or just generally available to users of your application. So whether it's users that are just clicking around or if they're using any sort of assistive technology, the testing library framework forces you in that direction. You can't write a test if your code is inaccessible tends to be the way it plays out, and it really nudges you in that direction. So it's one of the things that I really love about that. And I actually miss it when I'm working in a Capybara test suite because, as far as I know, there is not a Capybara testing library variant of it. And really, at the end of the day, it's just a bunch of functions to allow you to select within the context of the page. But again, it does it from that standpoint, and I'm all about that. 


STEPH: Yeah, that's really nice. That's a good point. Yeah, I don't think Capybara has that explicitly. I know that you have to use specific parameters. Like, if you want to access something on the page that is hidden, that's not something you can just do easily. You have to specify: I'm looking for an element that is hidden on the page. But otherwise, I don't think it goes out of its way to prevent you from doing that. There is an article that this conversation about accessibility made me think of. There's a really fun blog post written by Eric Bailey, who has been or who is a champion of accessibility at thoughtbot and has written a lot of great content around making the web more accessible. And in addition to publishing with the thoughtbot blog post, he has written for a number of publications. And the article that comes to mind that he published on the thoughtbot blog posts is An Introduction to macOS Head Pointer, and we'll link to it in the show notes. But he does a great job walking through what the head pointer is on macOS and then how to use it. And he uses his eyebrows to essentially move the mouse and then click on certain buttons or click on certain links on the screen. And it's incredible. So if you need a little bit of accessibility and joy in your life, I highly recommend checking out that article.


CHRIS: Yeah. Eric has absolutely just been such a fantastic champion of accessibility. And he's definitely someone that I think of constantly as being -- I think he's involved with the Accessibility Project. He writes on CSS Tricks. He's around the internet just being the hero we need because accessibility is such a critical thing. And I'm a deep believer in the idea that accessible applications are better for everyone. And I so appreciate the efforts that he's putting in out there. Thanks, Eric. 


STEPH: Thanks, Eric. And then, on a slightly separate note, I have a slight complaint that I'd like to file. And this one is with Rails specifically. And I'm filing this complaint with the understanding that I'm also very spoiled in terms of Rails does so much, and I'm very appreciative of how much Rails does for me and for us. But specifically, while working on accessibility for a date of birth form field, so it's a form field with three different selects, so you have your month, day, and year. And while creating this, there's a very helpful Rails method that's called date_select, where then you can generate all three of those select fields. And you can even specify the order in which you want them generated, but this particular function doesn't have a way to make it accessible. So you can generate a label for each option that's in the select dropdown. And there's no parameter. There's nothing you can pass through. It doesn't automatically generate it for you. 


So I was in a spot where I was updating a form that's using the Rails date_select. I can't use date_select and make an accessible dropdown selection for date of birth. So instead, what I had to do is I had to split it out. I had to move away from using date_select, and instead, I'm using select_month and then select_day and select_year because from there, I then can pass in; in my case, I'm using aria-label to provide a label because I don't actually want the label to show up on a screen, which could be another accessibility concern because we do have the birth date label for those three sections. But then we still want at least each text field to have a label, even if it's only visible to screen readers. So then that way, if someone is selecting from year, they understand they're selecting from year or for month they're selecting from month. So by using select_year and select_day and select_month, I could specify the aria-label as month, day, or year, but I couldn't do that with the date_select. And I just realized that there's probably a number of date of birth forms out there that aren't accessible because us Rails developers are leveraging this existing method. So it just seems like a really good opportunity to improve date_select to be able to pass in a label or generate one automatically.


CHRIS: Wow. I'm surprised that's the state of the art that we're currently at. I really wonder if there have been conversations or if there are fundamental limitations because I'd be surprised if such a core piece of the Rails world someone hadn't brought this up in the issues. What's the story there? Because I'm guessing there's a story there. Although flipping it around, I wonder -- I've never loved that input sequence; as an aside, like three different selects, that's not how I think of my birthday. My birthday is one thing. It's not three things that we smash together. But I wonder are we at a point now where IE 11 usage is so small that we can use a native date_select input and then have a polyfill -- And then I start to trail off because I don't know what the story is for. Like, I think Safari doesn't do a great job, and I forget where it's at right now. And what about mobile Safari? And wouldn't it be nice if everything was just easy and everybody kept up? [laughter] But that's an aside. But yeah, that's part of my question here, is like, can we just not use that thing at all? Like, the three select dropdown version of picking a date of birth because, man, that's my least favorite way to do it. 


STEPH: Yeah. I'm with you. I'm also curious if there is a story behind this and also if anyone has a different opinion, and I'd love to hear it. Because this has been my experience in digging through the docs is I would date_select, and I could not find a way to pass in a label or have one generated to make it accessible. So then that prompted me to use the three different methods, which, by the way, is fine. It made me stop and pause to think this is the method that most people recommend the usage of in terms of creating those three different select fields for a date of birth or for any particular date that you're supplying; it does not have to be a date of birth. So it also surprised me that then we couldn't make it accessible. So yeah, I was a bit miffed in the moment. [laughter] I had to walk myself back and be like, well, if I want to make the world a better place, I should help make the world a better place. And that started with changing the code in this codebase. But then also it means looking into Rails to see if there's an improvement that I could help with there. 


CHRIS: This is what we do: we take our moments of miffed, and we turn them into positive action in the world. This is what we want to see. [chuckles]


STEPH: I figure the least I can do is share a blog post or something on Twitter that shows what it was before and then using the new date_select functions because that is reasonable, although working with a form is a bit different. It got a little tricky there in terms of making sure that each value for each select field is still being passed within the expected nested parameter. And some of that was available in the public API for select_year and select_day, but it's not as well documented. So I'm like, well, this seems to be intentionally public, but it's not documented, so I feel a little nervous about using this. Yeah, that's it. I just wanted to share my annoyance with Rails [laughs] or the fact that it made me work so hard to have a date of birth field. 


CHRIS: You joke, but that's a lot of why we use Rails is because we want these common regular things that we're doing to be as easy as possible, to require as little code on our part as possible but also this sort of thing like there's a lot of subtlety and stuff. Accessibility is one of those things that I want a framework that has security, and accessibility, and ease of use, and all of these things just baked in, so I don't have to think about it every time. It turns out having a date of birth, or generically any date field, is going to come up in web applications a lot, it turns out. And so having all of that stuff covered is frankly what I expect of a framework like Rails. So I'm totally on board with your being miffed here.


STEPH: Yeah. Those are all really valid points. So I'm with you. What else has been up in your week? 


CHRIS: Well, we've been leading up to this, I think, for many weeks. I did a Rails 6.0 upgrade a while back, and a big reason for that was partly just to get on the current version of Rails but also because I wanted to open the door to database switching, and finally, this week, I tackled it. And let's tell a tale because there was a bit of an adventure, if we're being honest. Fundamentally, all the stuff there makes sense. I'm happy with the end configuration, but there was a surprising amount of back and forth. I broke the app more times than I want to actually announce on a podcast, but I broke it only for a brief period of time. It's fine. It's fine. Everybody's fine. [laughs] I feel a little bad about it, but these things happen. But yeah, it was interesting, is how I'll describe it. 


So fundamentally, Rails just has nice configuration for it. So at a high level, you're introducing your config/database.yml. Instead of it just being production is this URL, you now say primary is this replica or follower, whatever you want to name it is this. So you have now two configurations nested within your production config. And then in your ApplicationRecord, you inform Rails that it connects_to, and then you define a Hash for writing goes to the primary, reading goes to the follower. And you have to sync those up with the thing you just wrote in the config/database.yml but fundamentally, that kind of works. That makes it possible in your application to now switch your database connection. The real magic comes in the config environment production file. And in that, you specify that you want Rails to use a database resolver that says GET requests go to the replica, and anything that is not a GET request goes to the primary. So anytime you're writing data, anytime you're changing data within the system, that's going to go to the primary. 


And there's also a configuration that, as far as I can tell, gives a session affinity. So for the next two seconds after that, even if you make a GET request subsequently right following it, so you make a right, you POST, and then immediately after that, you do a GET. Like, you create an object, and then you get redirected to the show page for that object, Rails will continue to go to the primary. I think it's probably using a cookie or something to that effect, but you can configure that time span. So you can say like, "Actually, we see that our follower lags behind a little bit more, so let's give it a five-second timeout where all requests for that user will then go to the primary." But otherwise, once that timeout clears, then you're going to switch back, and you're going to go to the follower, and all GET requests will happen to the follower. And that's the story. You have to configure that, and then it works. 


STEPH: I always love when you start these out with "I have a tale to tell." I very much enjoy these adventures. And you also answered my question in regards to if you immediately just created something, but then you do a fetch that's very close to after you just created it and how that gets rendered. So that was perfect. 


CHRIS: Frankly, the core configuration is very straightforward, and it's very much in line with what we were just talking about of; this is what I want from Rails: make this thing very easy, hide the details behind the scenes. But as I said, there's a bit of a tale here. So that was the base configuration. It sort of worked but then immediately upon deploying it to production -- So we deployed it to staging first just to test it out. Staging was fine, as is often the case. Increasingly, I'm leaning into Charity Majors' idea of you got to test in production. You're testing in production even if you say you aren't. So once it got to production, we started seeing a bunch of errors raised or a handful of errors. And they were related to a handful of controller actions, which are GET requests, so they're either show or index, but in them, they were creating, or they were trying to create data. And so we were getting an error that was read-only connection error or something to that effect, ActiveRecord read-only, I think, was the error class. And that makes sense because I told it, "Hey, whenever you get a GET request, you're going to use that follower." But the follower is a read-only database connection because it's a follower, and so it was erroring. 


It was interesting because when this happened, I was like, wait, what? And then I looked into it. And it's frankly fine at all the levels. It is okay to create a record in a GET request as long as that creation is idempotent. You create if it doesn't exist, and then from there on, you use that same one. That still fits within the HTTP rules of idempotents, and everybody's fine with that, except for the database connection. Thankfully, this is relatively easy to work around. You just need to explicitly within that controller action say, "Use the right database, use the primary." And the way I implemented that, I wrote a method within ApplicationRecord that was with right DB connection, and then it takes a block, and you yield to that block. It's basically just proxying to another similar thing. And it's very similar to wrapping something in a transaction; it sort of feels like that. It's saying just for this point in time, switch over and use the primary because I know that I'm going to be having some side effect here. 


STEPH: Wow. That's so fun. I'm sure it was not fun for you. But as me hearing the story later, that's fun in regards to I hadn't thought about that idea of you're telling all the GETs you can only go to the read, and now you're also trying to create. I am feeling nervous in terms of local development. So if you're working on a new controller and if you have a fetch or GET action, but you're also creating something, you haven't seen another controller that is demonstrating that strategy that needs to be used. Is it just going to work locally? I imagine it does because it was working for the other code that you were running that didn't yet have that strategy in place. So I'm feeling nervous in terms of someone could easily miss that.


CHRIS: I think there are a couple of different questions in what you just said. So let me try and answer all of the ones that I think I heard. So for local development, your database/config.yml is still going to be the same as it was. So you're just connecting to database name_development. There's only one of them; there's no primary follower. So this is a case where you have a discrepancy between production and development, which is always interesting. And maybe that's something to poke at because ideally, I want as little gap there as possible. But this is one of those cases where I'm like, eh, I don't think I'm going to run two databases locally and have one be a follower. That feels like too much to manage. Under the hood with that right DB connection method that I talked about where you want to explicitly opt-in, in the case that we're in development, I just yield directly to the block. So instead of doing the actual database switching at that point, the method is basically saying, "If we're in production, then switch to the primary and yield and if we're not in production, then just yield." And so it'll just run that code, and it'll connect to the only database. More generally, I have the connects_to configuration; I wrapped that. So that's in ApplicationRecord where you're saying, "Hey, connect to these databases based on this logic," that is wrapped if we're in production check as well. And the same thing in the top-level configuration that says -- We're getting ahead of ourselves in the story because this is the end state that I got to. It's not where I started, and I screwed some stuff up in here, but basically, all of the different configuration points, my end result was to wrap them in a check that we are in production.


STEPH: Okay. Sorry if I rushed your story. I was already thinking ahead to how could we accidentally goof this up? That makes a lot of sense for the method that's with right DB connection, that method that then it's going to check if we're in production, then we can use a primary follower strategy; otherwise, just use the database that we know of. So that helps a lot in answering those questions. And then we can pause and then get to my question later. But my other question that I'm curious about is what helps us prevent the team from making this mistake in terms of where we're adding a new controller, we add a new GET action, and we are also creating data, but then someone doesn't know to add that strategy that says, "Hey, you are allowed to go to the primary to also get data but also to write data too." And I'll let you take it away.


CHRIS: I don't know that I have a great answer to that one if we're being honest. As I saw this, it was very easy to find -- I think there were three controller actions that had this behavior in the system that I was working on. They all threw errors. It was very easy to just wrap them in this extra method and fix that, and then we're good, and I haven't seen that error again. As for preventing it from new instances of this behavior, I don't have a good answer other than potentially you share this information within the team and then PR review. Ideally, someone's like, "Oh, this is one of those things you've got to wrap it in the fancy database switching logic." Potentially, and I don't actually think this would be possible, but there's a chance that RuboCop or other static analysis type thing could look inside any index or show action and say, "I see a create or an update or any of the methods." But again, Rails is so hard to do static analysis on that I would be surprised if were actually feasible to do that in a trustworthy way, probably worth a poke because this is the sort of thing that can easily sneak out. But potentially, my answer is, well, it'll blow up pretty loudly the first time you do it. And then you'll just fix it after that, which is not a great answer. I'm open to that being a mediocre answer at best. 


STEPH: [chuckles] Yeah. That's a fair answer. Just because I pose a question, I don't know if there necessarily is a great answer to it right away. And disseminating that information to the team to then having the team be able to point that out also sounds very reasonable but then still hashes that danger of someone overlooking it. The static analysis is an interesting idea, sort of like strong migrations. As you're introducing a new migration, strong migrations will do a wonderful job of showing you concerns that it has with the migration that you've added. And this is all just theoretical dreams and hopes because, yeah, that would help prevent some of those scenarios. 


CHRIS: It's interesting now that this is the second time we've discussed static analysis in this very episode. Clearly, it's a thing that I want more of in my world, and yet I work in languages like Ruby that are notoriously difficult to perform static analysis on.


STEPH: I had a moment today writing a method that was currently just returning a string each time but then I was about to update that method. I was looking for a way like, well, maybe I don't always want a string. Maybe I actually want a Boolean here. But in the other case, I want a string. And the person I was pairing with they're like, "You could return -- [inaudible 29:31] Boolean in one case and then a string in the other case. Like, this is Ruby." [laughs] I was like, true, but I feel bad about it, and I don't love it. And we just had a phone conversation around that. If you're in the Ruby world following the more functional programming or type strictness and where you're returning specific types or trying to return a consistent type, it's ideal. But then also in Ruby, it's like it's Ruby, so sometimes you can finagle the rules a little bit. 


CHRIS: YOLO, as they say. 


STEPH: [laughs]


CHRIS: Yeah, I'm definitely interested to see where projects like Sorbet and...I forgot what the core Ruby typed thing in Ruby 3.0 is called, but either of those. I'm really intrigued to see where they go and how the Ruby community either adopts or doesn't. I wouldn't be surprised if that were part of the outcome there. I've been impressed with the adoption of TypeScript and JavaScript, which is also a very, very free language, not quite to the degree that Ruby is. But yeah, it remains to be seen what will happen on those fronts.


But continuing back to our saga, so we've now had the read-only error, we've fixed those, just wrapped them in blocks, and said, "Explicitly connect to the primary." So the next thing that I did after that, I realized that my configuration was a little bit flimsy is probably the best word to describe it. I was explicitly creating a new environment variable with the URL, the Postgres URL of the follower. And so I was using that environment variable to define where the URL like the Postgres URL of the follower database -- But I realized if Heroku comes in and does any maintenance on that Postgres instance, it's possible that the AWS IP address or other details of it will actually change and so that Postgres URL will no longer be valid. So that's one of the things that I rely on Heroku for, is to maintain my databases for me. But they will update, say, the DATABASE_URL environment variable if they change out your database. But now, I had broken that consistency. And so I'd set us up for somewhere down the road this will break, and I realized that because Heroku reached out and said, "Hey, your follower database needs maintenance." And I was like, oh, no. So, I tried to get it from -- It turned out, in this case, it didn't actually change. They were able to swap it out in place, but I wanted to add a little bit of robustness around that. 


And so I actually reached out, and Dan Croak, former CMO of thoughtbot, actually had written a wonderful blog post about how to configure this and particularly how to configure it in the context of Heroku. And he described how to use the Heroku naming scheme for the environment variables. They happen to have colors in them. So it's like Heroku Postgres cyan URL or orange URL or purple URL. And so he defined a scheme where you set an environment variable that describes the color, and then it can infer the database URL environment variable from that. And then went the one step further to say, "If that color environment variable is set, then treat as if we are configured for database switching. But if it is not set, even if we're in production, pretend like we don't have database switching," which that was another nice feature that I hadn't built in the first place. When I first configured this, I just said, "Production gets database switching. And if we're in production, then database switching is true," but that's actually not something that I want. I want to be able to say, "Upgrade our follower," at some point or do other things like that. And so I don't want to be locked into database switching on production. So that was a handful of nice configurations that I wanted to get to. 


Unfortunately, when I tried to deploy that switch, man, did it break. It broke, and then I was like, oh, I see I did something wrong there. So then I tested again on staging. Staging was fine. And then I went to production, and it broke again. And this happened like three times in one day. I felt like a terrible programmer. I had no idea what I was doing. Turns out that staging and production had different environment config files, and so their configurations were fundamentally different. They also had a different configuration for the database level. So one of the things I did as part of this was to clean those up and unify them so that staging was production with some environment variables to config it, but identically production, which is definitely a thing that I believe in, and I want basically all the time. I don't think we should have a distinct staging environment config that is wildly different. It should only vary in very small ways, basically just variables that say, "This is where the database is for staging," but otherwise be exactly configured as production. So I eventually got on the other side of that, fixed everything, have a nicely Heroku-fied color-based environment variable scheme, which is a bit of a Rube Goldberg machine, but it works. And I was able to hide that config in one place. And then everything else just says, "If there is a database follower URL defined, then use it." But yeah, so that was the last hard, weird bit of it. 


And then the only other thing that I did was I realized that this configuration was telling the Rails server how to behave, but there are also background jobs. And this application actually happens to have a ton of background job traffic. And so I did a quick check of those, and there were a handful of background jobs that were read-only. A lot of them were actually sending data to external systems, so to analytics or other email marketing or things like that. And so constantly, as users are doing anything in the application, there are jobs that are queued that aggregate some information, maybe calculate some statistics, and then push it to another system. But those are purely read-only when those jobs execute. And so I was able to add another configuration which said, "Use the read-only connection and configure that to wrap those particular sidekick jobs." And with that, I think I have a working database switching configuration that will hopefully give us a lot of headroom in the future. That's the idea, that's the dream, but we will see. 


STEPH: That is quite the saga between having GET requests that create data and then also the environment inconsistencies, which is a nice win that then you're able to improve that to make those environments more consistent. And then the background jobs, yeah, that's something that I had not considered until you just brought it up, and then being able to opt-out of the database switching sounds really nice. In regards to moving in this direction, you're saying gave you a lot of headroom for this; when it comes to monitoring performance,  is there anything in place to let you know how it's doing?


CHRIS: I love that I knew that this was going to be your question. I love that this is your question because it's a very good question. And unfortunately, in this case, it's actually somewhat unsatisfying. So as is my typical answer for this, we're using Scout as the application performance monitoring tool on this. And I was able to go in and monitor what it looked like a week ago, what it looked like after I made the change, and it was a little better. And that's all I can say about it. But that's fine. The idea with this, and at least in the way I was thinking about it, is this should get better at the margins. On the days where we have a high spike in traffic, those are the days where the database is actually working hard. They shouldn't make the normal throughput of the application that much higher in the regular case; it's for those outlier instances. To that end, though, I did analyze it. And so the average response time got 2% to 3% better in that week-by-week comparison, which was fine. The 95th percentile response time, so starting to get out to those margins, starting to get to the long tale of where stuff gets -- a couple of requests came in at the same time, and the application had to try a little harder, those got 8% to 9% better. That shape of improvement where for most requests, nothing really changed for some of the requests that used to be a little bit slower; those got a little bit better. That's the shape of what I would hope to see here. And it remains to be seen. This application has particular traffic patterns where they'll encourage a lot of users to be using the app at the same time. And historically, those have been somewhat problematic, and we've had to really work to shore up the performance in those cases. That's where I'm really interested to see how this goes. It would be hard to replicate those traffic patterns at this point. So I don't have a good way to really stress test this, but my hope is that for those cases, things will just hum along and be happy.


STEPH: That makes a lot of sense and something that would be hard to measure, but the fact that you already see a little bit of improvements that's encouraging. 


CHRIS: But yeah, certainly, if I get a chance to see what that looks like in the near term, I will respond back and let you know how this has played out. But overall, now the configuration seems pretty stable. I think we're in a good spot. Hopefully, we won't have to do too much proactive management around this. And ideally, it just buys us a little bit of headroom. So that is certainly nice. But with that, with your wonderful question getting to the heart of the issue, I think that wraps up the saga of the database switching.


STEPH: Well, I appreciate you sharing that saga. That's really helpful. I've been very excited to hear about how this goes because I haven't gotten to work on a project that's going to use database switching just yet. And now I know all the inside baseball. I'm trying to use sports metaphors here as to how to do this for when I get to work with database switching.


CHRIS: Sports de force. 


CHRIS: Along the lines of new stuff, there is something I'm excited about. So in juxtaposition to my earlier statement or my earlier grievance where friends don't let friends use date_select in regards to trying to keep the web accessible, I do have some praise for something that's being added in Rails 6.1 that I'm excited about. And it's a really nice method. It's a query method that can be used to find orphan records. So if I'm writing a query that is then looking for some of these missing records, so if I have my table -- I didn't come with a great example today, so let's just say we have like table A and then we're going to left_joins on table B. And then we're going to look for where the ID for table B is nil, so then that way we find where we don't have that association that it's missing. And so left_joins does this for us nicely. And then I always have to think about it a little bit where I'm like, okay, I want everything from table A, and I don't want to exclude anything in table B if there's not a match on the two. And so then I can find missing records that way or orphaned records that way. The method that's being introduced or has been introduced in Rails 6.1, so anyone that's on that new-new, there is the missing method. So you could do tableA.where.missing and then provide the table name. So there's a really nice blog post that highlights exactly how this method works, so I'll use the example that they have. So for where job listings are missing a manager, so you could do JobListing.where.missing(: manager), and then it's going to perform that left_join for you. And it's going to look for where the ID is nil. And I love it. It's really nice. 




CHRIS: That sounds excellent. That's definitely one of those things that I would have to sit down and squint my eyes and think very hard, really anything involving left_joins otherwise center. Any joins always make me have to think and so having Rails embrace that a little bit more nicely sounds delightful. 


STEPH: Yeah, it sounds like a nicety that's been added on top of Rails so that way we don't have to think quite as hard for any time; we want to find these orphaned records, and we know that we can use this new missing method. 


CHRIS: On the one hand, I feel bad saying, "I don't want to think that hard." On the other hand, that's literally our job is to make it so that we encode the thinking into the code, and then the machines do it for us. So it's kind of the game, but I still feel kind of bad. [laughs]


STEPH: Well, it's more thinking about the new stuff, right? Like, if it's something that I've done repetitively, finding orphan records is something I've done several times, but I do it so infrequently that then each time I come back to it, I'm like, oh, I know how to do this, but I have to dig up the knowledge. How to do it is that part that I want to optimize. So I feel less bad in terms of saying, "I don't want to think about it," because I've thought about it before. I just don't want to think about it again. 


CHRIS: I like it. That's a good framing. I've thought about this before. Don't make me think about it again. [chuckles]


STEPH: Exactly. On that note, shall we wrap up? 


CHRIS: Let's wrap up.


STEPH: The show notes for this episode can be found at bikeshed.fm. 


CHRIS: The show is produced and edited by Mandy Moore. 


STEPH: Thanks, Mandy. If you enjoyed listening, one really easy way to support the show is to leave us a quick rating or even a review on iTunes as it really helps other people find the show.


CHRIS: If you have feedback for this or any of our other episodes, you can reach us at @_bikeshed. And you can reach me @christoomey. 


STEPH: And I'm @SViccari.


CHRIS: Or you can email us at hosts@bikeshed.fm. 


STEPH: Thanks so much for listening to The Bike Shed, and we'll see you next week. 


All: Bye.