Unknown Speaker  0:00  
Oh.

Joshua Warren  0:06  
The easiest entry point for a site that wants to explore something like this is feature flags and feature flags are a basically exactly what they sound like. They're a flag that can be turned on or off, that controls if a new feature is present on the site or not. So basically, that code you're deploying, you wrap it all around some conditional logic that says, if the feature flag is on, make this code active, if it's off, don't make this good after

Darin Newbold  0:37  
that good day, and welcome to Commerce Today. My name is Gerren. And happy to have you here and my co host, as always, Josh, to join us here for all about what is happening in the world of Commerce Today. So we've got a neat one here, Josh, that you've spooled up for us, leveling up, basically smarter deployment strategies for high volume e commerce stores. I think this is really neat, because the whole deployment thing has always been kind of a mystery from where I sit, it's that I'm three in the morning. And that's when I am awake, and I want to buy something and I can't because the store is down. So let's talk about it. I know. Talk about your insomnia. That's a Yeah, that's another episode we probably got to get to, in probably a different podcast, as well. But be that as it may, deploying is so important. It's key, obviously. But it's also scary, dangerous and fraught with all sorts of challenges.

Joshua Warren  1:40  
Oh, yeah. Nobody, nobody talks about deployments. And nobody wants to talk about deployments, you're looking at a new project, a new agency, something like that. And typically, they're going to talk about their amazing discovery process, you're going to talk about what it's like to work with them, maybe some, some user experience work they might do. But nobody says, Oh, look how awesome and cool our deployments are. And it's another one of those things. It's like so many things. In a service business, like it's kind of how the sausage gets made, so to speak. It's the part that people don't really like to think about. It's almost seen as overhead or an inconvenience, but it can have a huge impact on your project. And on your website.

Darin Newbold  2:18  
Well, it's interesting, kind of the way you describe that in a way, it can be turned into a differentiator or four, integration for integrators and those that are that are doing this, that can be a big deal. But well, all right, we kind of got to get get into it. So we start, as we always do the start at the basic level, and kind of your standard, what I run into Hey, sites in maintenance mode, backups, and those uploads.

Joshua Warren  2:45  
Talk about that, Josh? Yeah, so deployments, there's a lot of different ways to deploy sites. And we're going to start with kind of the simplest and least expensive, well, least expensive from a time you pay your agency, maybe not least expensive from a opportunity cost. Right, right. But so basically, site will be placed in a maintenance mode. And that's those lovely messages, you get saying, Hey, we're making some great updates to our site, check back later. Hopefully, your tech team is then making backups of everything of the code and the databases. That way, if something goes wrong, they can restore from those backups. And finally, it's real simple. They just upload the site, upload all the code to the site. So they transfer the new files, the new code they've written, from whatever environment it's currently in, in your production site. Everything goes well, then everybody's happy. You turned Maintenance Mode off, and you have a updated website

Darin Newbold  3:42  
and updated store. Well. So quick question, are the backups done while the site's down in maintenance mode? Or can the backups be done? Well, before it goes into maintenance mode,

Joshua Warren  3:53  
some of them can be done before maintenance mode, but on certain things like orders, if you're taking orders all the way up to the moment you're in maintenance mode, you really don't want to lose those orders. So you're going to run some backups once you go into maintenance mode,

Darin Newbold  4:05  
important safety tip. Okay, so you said this is probably the least expensive theoretically, at least from the cost of the service. That it's been that's providing it, but it could be the most expensive or a higher expense, from the opportunity cost of the sales that are lost during the time that you're in maintenance mode is that, uh, I probably didn't say that quite the way to but I think hopefully everybody made it made sense.

Joshua Warren  4:30  
Yeah, yeah. And this is where you know, for a smaller site, you're probably not talking about a lot of money, especially if you're doing it the middle of the night. But for very high volume stores, you know, literally, every minute they're down for maintenance is possibly a lot of money that they're losing. All right,

Darin Newbold  4:46  
well, that's our basic approach, or that's the basic approach out there. What we don't want to stay in just the basics, and that's why we're talking about some of the cool things that can happen and these next ones start to get pretty little bit more fun. So we have an intermediate strategy's kind of balancing the innovation, along with, we still need to be stable. So we have great new cool ideas or trying new things, but we got to keep some stability.

Joshua Warren  5:14  
So this is basically the the technical term is continuous deployments. It's the idea of being able to, at any moment, as many times a day as you want deploy changes to your website, ideally, in theory with little or no downtime. And so you know, you remove that friction, and it allows you to innovate a lot faster. And one of the really good examples of that, and probably the easiest entry point for a site that wants to explore something like this is feature flags, and feature flags are a basically exactly what they sound like they're a flag that can be turned on or off, that controls if a new feature is present on the site or not. So basically, that code you're deploying, you wrap it all around some conditional logic that says, if the feature flag is on make this code active, if it's off, don't make this code active.

Darin Newbold  6:07  
Hmm. Okay. Well, I mean, that, that's awesome, because it allows us to turn on turn off stuff. And what you're also saying is, is that this can be done without taking the site down without is, is there any kind of a pause? Is there any Is it maybe orders are, are paused for a period of time, so that,

Joshua Warren  6:28  
yeah, it depends on where the changes are, and how risky they are. So sometimes this will require maintenance mode and still have some downtime, where you can save time is, typically, if something goes wrong in the deployment, with the first method we talked about, you have to actually roll everything back. And that takes some time restoring databases, things like that, you're using feature flags, then unless it's gone very off the rails, you just turn that feature back off, and you've rolled back that deployment. So this, it doesn't eliminate the downtime, but it reduces it and it reduces the risk.

Darin Newbold  7:00  
Okay, so how are some ways maybe if they're beyond maybe these feature flags? How are some ways that we can we can have innovation and then keep that stability within our site and stuff, kind of try new things. But, again, always keeping that stability?

Joshua Warren  7:18  
Yeah, I really think feature flags are the best way to do that. But I think something we haven't talked about with them is, it doesn't have to just be on or off. You could also tie it into your customer data platform different systems and say, Okay, I want 20% of my users to see this feature on 80%, oh, AV testing, do some AV testing or even say, Okay, I want to start with this feature is going to be enabled for desktop users, but not for mobile users. So that way, you can not only kind of innovate and try some new things, and just make sure it's stable and be able to turn it off if you need to. But you can even see how different groups of users react to new features.

Darin Newbold  8:01  
Are these things that that just a merchant in themselves would be knowledgeable and have the ability to do versus needing a an integration partner, if you will, to do these things. It depends

Joshua Warren  8:12  
on the in house team. If they have a good solid in house team, you know, they could use this approach. Even if they don't, and they're working with a partner, they could at least ask these feature flags be built in a way that the merchant can turn them on and off, or can even tie them into their split testing platforms without needing additional development work? That is possible.

Darin Newbold  8:34  
Okay. All right. Well, moving on to maybe in a more advanced type of area we're talking about, you know, I always felt like this fun, the blue green deployments, I don't know what it really is, reminds me of something. But anyway, tell us about the bluegreen deployments. And what does that mean? And how does it How does it positively or negatively impact the overall rollout?

Joshua Warren  8:59  
So this is probably not the best way to introduce this topic. But I always think of the movie contact and where he says, The Big device is destroyed. And they suddenly reveal they have a second copy of it. They say, Well, yeah, first rule of government spending, why buy one for full price when you can buy two for twice the price. So the disadvantage to bluegreen deployments is unless you do a very advanced form of this, you have two copies of your production environment. So you have if you have one web server, one database server, well now you have two observers, two database servers, if you have 10 and 10. Now you have 20 and 20. It could get it gets a little exotic, exotic, little expensive, but again, this is where for these high volume sites, when you run the math of how much money they lose every minute they're down. He can actually definitely come out ahead with a setup like this but kind of got ahead of myself but the way the reason you have two identical Copy of your production environment is one is blue, and one is green. And blue is the currently live production environment. Green is an exact duplicate of it. You deploy all your changes to the green environment, and then you flip all your traffic to go from the blue environment to the green environment. Something breaks doesn't work, right? You just put all the traffic right back. So now you have done what the user never sees that maintenance mode page, the user never experiences any downtime. There's no pause in between on this switch. It's just just from one to the other. PFM gotta love it. Caching, there's

Darin Newbold  10:37  
no worry about even even in order. I mean, what happens if there's a you know, I'm, I don't know, I got things in my cart. And I'm, I hit the I hit the buy button at the same time as it's flipping. How does that

Joshua Warren  10:50  
so yeah, there's definitely some I am summarizing what would be at your hour long presentation. And actually, it's funny, I joke sometimes with our team about how years ago back in the Magento, one days Angry Birds was on Magento. And this is back when Angry Birds was the big hot brand. And they had one of these environments, it was like 30 different servers. And they were doing this setup, and it was just, they showed a slide of Oh, it's so simple. This is how it works. And I spent like 20 minutes deciphering all the arrows and notes on that slide. Because there's a lot you have to do if you want that level, as far as how you're handling the customer session where you're storing the session data, how you're handing that off from the blue environment to the green environment. So and again, this, this depends on the volume of the business for Angry Birds, it made total sense at that time, at least to invest you have zero downtime. For other brands, you might want to go to a bluegreen deployment setup and say, Hey, I'm okay with two minutes of not being able to place an order. And then that suddenly makes it a little bit simpler on the technical side of not having to worry about that session data as much. So there's

Darin Newbold  11:58  
a pause, maybe a brief maintenance mode of some variety, but or a brief pause or a message on hey, we're currently working on the site, we'll be back in 30 seconds or 90 seconds or whatever. Okay. Well, that makes sense. So I guess, putting these all together, then let's look at kind of the bigger picture of all of this. And what's the best way to reduce that risk and, and accelerate the innovation that you can have on an E commerce site? Because what I keep hearing you say, and yes, I'm not the sharpest stick in the shed. So it takes a little while for me to get it. But you got to try new things. And you got to be willing to do that. But to do that, it if it requires this enormous cost or time to take it down, you start to I can see a team saying well, I don't want to do that because it wipes out my my site for so long period of time on a deployment to try these all these cool new things.

Joshua Warren  12:54  
Yeah, and that's where you really as a brand have to decide what's most important to you kind of look at the scale that you're at this is where like goodness, ChatGPT can't replace us yet. Because there's a conversation that has to happen of the trade offs and of the investment needed as far as Do you. Are you okay? With the downtime your your brand that you're going to deploy changes once a month? And that's often enough and your site can be down for that time? Are you someone that wants to innovate and deploy more often? And you do you have to move to at least something like feature flags, I feel like if you want to deploy more often if you want to innovate, because you have to remove that risk, or you have to reduce it, because otherwise in the back of your mind, the project managers mind ecommerce managers mind everyone's going to be thinking, yeah, we can try this new feature. But my site's going to be down for a few hours to deploy it. Oh, maybe we'll wait. And so often, I've seen the businesses that are successful in E commerce are the ones that get out there and iterate the fastest and just say, Hey, we're going to try new ideas faster than our competition, we're going to split test, we're going to, you know, look at the analytics, we're gonna see how this is working. And then we're going to use that data to try again and try again and try again, those the compounding you get off of those results, you will grow so much faster than if you say, oh, once a month, once a quarter, we'll try something new.

Darin Newbold  14:18  
So real quick back on the feature flags, is that technology is that part of pretty much all the platforms is every platform, have it

Joshua Warren  14:26  
not every platform has it? Most of them have at least modules are people that have open sourced a way to do it. And in most platforms, it's fairly easy to implement. A lot of it comes down to say fairly easy and I can already hear back in my mind some of my team members saying What are you talking about, but for the on off, like it's completely on or completely off option. That's very simple to implement for the URL. I want to show this to only people on mobile devices. And I want to show this and especially if you want to be able to control that As a merchant, then you're adding a little bit more complexity. That's

Darin Newbold  15:03  
yeah, there's a lot to that I can only imagine. All right, well, wow, Josh, you've opened my eyes on the whole deployment and the complexities as well as the sophistication that can be brought to making it better, making it faster and more effective for all of our merchants out there. So there's really been able to the benefits of that advanced deployment methods for the high volume sites. That makes a lot of sense, the blue green, it really, you almost have to do that, in a way. And I think last, I guess, I guess, further summary, or in the final summary, Josh, I guess, how would you kind of put all this together or from from the merchant that's listening? What should they do next?

Joshua Warren  15:48  
If you don't have experience making these decisions, or thinking about deployments in this way, talk to someone who does and just bounce some ideas around. Because I think, again, like I've opened the show with this is that thing that no one really likes to talk about. But it has such an impact not only on project budgets, but it can even the mindset and outcomes can all change based on how everyone feels if if everyone in the company cringes when they hear the word we're going to deploy next week, you're not setting yourself up for a successful project will in

Darin Newbold  16:20  
a lot of the great ideas from this come from the mistakes that had been made in the process of trying to do these deployments. And that's where the blue green Paideia all comes from is the mistakes that were made. So yeah, definitely. Well, Josh, this is great stuff. We appreciate it. This is awesome. As always in as always, we appreciate you the listener, glad you're here. And definitely tune in next time to Commerce Today. Again, this is Darren and Josh, we appreciate you being here. Until next time,

Transcribed by https://otter.ai