Source Maps with Nicolo Ribaudo - VIDEO EDIT
===

Paul: [00:00:00] Hi there and welcome to Pod Rocket, a web development podcast brought to you by Log Rocket. Log. Rocket provides AI first to session, replay, and analytics, which helps service UX and technical issues impacting your user experiences. Start understanding where users are struggling and try it for free. Out log rocket.com.

My name is Paul Mikulski, and joined with me is Nicolo Aldo. He's the maintainer of Babel and a TC 39 Delegate. And Nicolo is joining us today to talk about source maps and how they work, and it's a fitting topic 'cause Log Rocket at its core is a application and service that helps you see what's going on for your user experiences and just in production.

Heat maps and source maps help you find out what's going on in your code. So we're gonna dig in deep, figure out where they're used, how they work. And how they're put together. So welcome to the podcast, Nicolo. Excited to dig in with you.

Nicolo: Hi. I'm happy to be here. Happy to answer any question. Uh.

Paul: So source maps feels like they go hand in hand with being the maintainer of Babel, right? Do [00:01:00] those relate a lot in your day-to-day work? Uh, did you get into Babel first and then you got really interested in source maps or what did that, uh, transpire in your development career?

Nicolo: So actually it's a very long path between the two. I started with Babel and from there I got into standards in T 39. And then from T zero nine I actually started talking with like companies involved in standardization of JavaScript and of like tools around JavaScript. And that's where then we got to source maps. So even though like I'm, I have to implement search maps things as part of my, of my mobile journey, that's not how I got like really close to them.

Paul: Are source maps one and one with JavaScript in your day-to-day. 'cause you know, battle, it's a tool that is heavily in that ecosystem when you think about it. Um, and that also kind of begs the question, are source maps used? For more than what people might initially think. Uh, I'm debugging something in my browser console.

Nicolo: [00:02:00] Yeah, so I would say source maps are like maybe 95% for JavaScript. Uh, that's how people use the, use them most. Well, JavaScript or TypeScript. Uh, but then it's not just that, like interior source maps could be used for everything. Like if you, if you're writing some C Successs code and like TransParenting it with like post CSS or maybe even some less or SaaS code and compiling that, you're still using source maps. In theory. You could even use source maps for like. A C two Wasm compiler. I actually had at some point some chrome dev tools showing some C code that was actually some web assembly but transformed through source maps to, to show me like, well what the original C code was and I was able to like step by step the back through that in the browser. Thanks. Thanks to Source Map.

Paul: So at its core, it's giving you observability, um, transparency into what's happening in your code. It could be any type of code. It could even be markup, call it like [00:03:00] CSS, 

Nicolo: they're, they're actually useful for like, whenever your code, whenever, like you have some build step that does something to your code and transforms it in some way, and then the browser executes that. Result of your build process. Um, but then you want to back that code that's running and you want to be able to see what's going on relative to the regional code that you wrote. Uh, and that's where you source maps. So basically use them to threat hand tutorial transforms your build process did not happen, and be able to debug the original code you wrote. And if that's not what's actually running in the browser.

Paul: All right, so it helps us debug, um, especially when there's build steps, transpiration going on. Um, our source, ma what, what are they nicolo, what are these magical things that help me debug?

Nicolo: They are just a simple Js o file, uh, which contains a bunch of, uh, metadata about [00:04:00] the file that you, that you might want to Deb Bug. Um, like mostly metadata about what what the original files were, so what the file names of the inputs to your bill process was and what the, what the contents of, of this original files were. And then it contains like some weirdly encoded stuff. Like if you go look at the source map file, uh, that you might find like in the output of your build tool. It's just like a bunch of letters and commas that it looks like they don't really make sense. But actually it's like some special encoding that when the coded it's mapping as almost character by character.

Um, like it's mapping characters of the code that's run in browser to locations in the, in the original code that, uh, we wrote before transpiring. And they contain just like a ton of these mappings. So that whenever you, for example, place a break point, the browser, the browser dev tools or like the editor dev tools are able [00:05:00] to look in the source map for the mapping corresponding to this break point and apply it on the other side.

Paul: So it's literally a map.

Hey, there's this weird token syntactical thing I can search for, and uh, this is where it exists in my source code mapping over to the source code. Um. Is it one js ON file that you speak of, or is it many? Because the initial takeaway here is that's gonna be a big honk and file.

Nicolo: Yeah, so there is like, it's one js on file for each file you are running in the browser. So if you have, like, if you build process generates like. 10 different files, you're gonna have 10 source maps. Um, there are ways to make them smaller. Uh, so for example, maybe instead of embedding all of the original files in the source maps, they can just link to them so that you don't get like a hundred megabytes jes on file. Uh, but in the end doesn't really matter if they're too large. Because they're not used [00:06:00] in production, right? Like if, if some user open your web, open your website, they're not gonna load the source maps. The source maps are loaded when you actually open the dev tools there. So when you went to the bag, and usually if you're the bagging, well, you're fine with fetching this huge file because that's not the, that's not the day-to-day experience of his new website anyway.

Paul: Got it. So the size doesn't realistically matter. It's not shipped to the browser. It's not even shipped to production. Um. I imagine that source maps could also be helpful in some development scenarios, uh, if it's not running minified code directly in production. Uh, do you find this useful in your workflow and how does that differ from any times that source maps might be useful in production?

Nicolo: ~So when using source maps,~ when the bugging outside of production, uh, I still use source maps even if maybe not minifying, just because yes, I'm not minifying. So the code running is, is like somewhat similar to what I wrote, but there are still so many differences. Like even, even if I [00:07:00] just strip TypeScript types, uh, or like just be able to do some basic transform of only the more modern features, trying to leave MyCode as. to the original as possible. These tools are still, are gonna move, like things all over the place. And if I'm not actually running a browser to code of right then CI maps are, are helpful when it comes to minifying. Yes. They're even more important there because there is, it's just impossible for a human to step through unified

code and see what's going on there. Um, and. Normally people minify more for production. Uh, you can, you can still use source maps for production code. Like I said, that they're not logged by default for, for by browsers. But if you have some production website, you can, and you do open the dev tools. Then the browser, it's gonna try to fetch the source maps for that.

Paul: Got it. Okay. So it's helpful in the development context as well, because I mean, everybody's using [00:08:00] TypeScript. Uh. Greater numbers these days. Um, even if it's not your go-to language, you're gonna end up using it at some point and it can be really useful to read the types so useful in production for minified code useful locally, even if you don't have minified code.

Um, depending where the source map exists though, changes how it gets ingested, viewed, utilized, loaded. Um. For anybody who hasn't ever dealt with a source map and they hear the word source map, it's probably the first thing they think of is century. Uh, they've kind of evangelized and socialized the common use of source maps when somebody might not have initially been, um, you know, interested in that sec, uh, section of web development.

So if it's not century and integrating with their CLI, how are source maps ingested? How are they produced? How are they loaded?

Nicolo: Yeah, so there, there are like two main ways source maps are used. One is when I'm mentioning we're just opening your [00:09:00] dev tools, open your browser dev tools and it like un transforms your code. And that actually does not rely on any external service. Uh, it's just the browser itself fetching the source map from the same server where you're hosting your code. And that source map has been generated not by some like third party provider, but by your own build tools. Um, and this is like when it comes more to the bagging. Like real type back game when you step through your code and try to, to like figure out what's going on there. But then there is desire way using source maps, which is, for example, what Century does, or similar companies where, so these companies add some monitoring right to your application, uh, whenever there is maybe an error or, or even just some, some trace that you need to figure out, uh, like what happened there. These. Providers will collect all this information, like all these traces or errors about your code somewhere. And these are the errors or the traces coming directly [00:10:00] from the browser. So they're gonna have information about the code that is running on browser, which is maybe the unified code. But then when you actually try to look at this, at these stack traces, they're gonna apply the source map to it. So that you see not what the code, not what code was running in your user's browsers, uh, but what, how the code maps to, to like your non build source files. So there are these two main ways, uh, I I think for most people actually, source maps is something that you probably don't even realize you're using them.

'cause like maybe you're using web pack or, or like. Roll up or any other, like label any tool that transforms your code. And most of these tools just generate the source map for you. You don't have to worry about that. Um, and if you don't know you have source maps, you're probably not gonna use one of those third party services that like do all of these analysis and apply the source [00:11:00] map to it. The only way we're gonna interact with source maps is without him realizing that, that you just open the browser dev tools and they just work. Ideally source maps are just there and you don't notice because everything is working so well, like source maps are doing their job so well that you don't have to think about that. And people start noticing that source maps are a thing when things do not work well. And so they have to figure out why they, the banking experience like feels a little bit off.

Paul: Right. Okay. So I think the realization that some folks might have, if you are, you know, not familiar with source maps, maybe you're just using them, you didn't set them up as Century and source maps, two separate things. They're symbiotic, um, source maps and production only. Not necessarily true. Also useful in development.

Um, source maps are produced by you at build time. It's not a, a special thing that's owned by any, uh, group or anything. All [00:12:00] that being true and being said, it kind of begs the question of standardization and how are source maps produced? There's this big JSON file. What if they're different? Nicolo? Uh, I'm sure this way source maps are produced.

Time over time changes. Um, and does that process differ between preparing them for production, pre preparing them for development, um, versus pair preparing them for a specific vendor like sent, or do they rely on the same standards?

Nicolo: ~Yeah, right. There are like,~ there are probably hundreds of tools that in some way interact with source maps. Like if you think about all the build tools we have. Uh, I dunno, like every two years there is a new, very popular Cript bill tool and they all implement somewhat the same support for source maps.

And then we have all of these, like all of these other tools that consumes source maps like the browsers, tools like sent, uh, even editors, like just code itself can reach resource maps. Uh, for example, it uses them when, [00:13:00] when you click on, got to definition in your editor. Um, and all these tools mean working on the same format mean that we actually do need a standard.

Like you cannot have a a chrome specific system of thing because then all the tools need to implement Chrome specific thing, and then Firefox would come with their own Firefox specific thing. And all tools, again, have to implement the Firefox specific thing, and that just does not scale. Um, so there is some coordination needed here. Um, but the fun fact is that. Source maps are like maybe 15 years old at this point. At least the conversion of source maps. And until two years ago, there was no standard and everybody was just trying to copy what each other was doing, trying to reverse engineering how source maps worked. And there was like this, this Google doc that some engineer from, from New Zealand, Google wrote down, uh, that, that had some instructions on how the encoding did the coding works, uh, and what various field mean, which was. Far from being complete and like [00:14:00] describing the full reality of source maps. So yeah, every, everybody just had to look at what other people were doing and try to kinda, like you're in school, you have a test and you just look at the test on your, on your classmates' desk and try to do the same thing. and like it was working well enough, uh, in many cases. But then we started hitting like all these edge cases where things actually working differently in different platforms. We also started thinking, okay, if we want to evolve source maps, making them more powerful, how? How do we do this? Given that there is huge number of uncoordinated tools that all try to do the same thing without really a single central place to discuss this thing, these changes. And so like two years ago we, within 39, uh, we gathered a group of multiple companies working on browsers, working on dev tools, uh, even just companies that have a lot of developers writing JavaScript code. And so they need to make sure that [00:15:00] the bagging experience for the developers is great. So we created this group and we took that Google Doc that there was. Um, completed all the parts that were missing, made sure it matched all implementations, and published the first version of a source map standard. And now we have like these, these meetings within the 39 with all of these different companies where we try to think, okay, how can we improve the experience of the bagging source maps?

So we try to a standard, but it's not just evolving the standard. They're like, the people that have to implement the standard are actually in the room talking, talking to each other.

Paul: I assume that there's some difference between, you know, you mentioned people are coming to the table browsers, um, there's Firefox and Chrome, there's other runtimes being developed. Um, are there any differences that you notice given the recent standardization push in how these different browser. Consume and then present source [00:16:00] maps to us as devs.

Nicolo: So there are difference in the ui, uh, that browsers use, different browsers support different features. Uh, there are browsers are better than others at like, I dunno, letting you place back points, right. In the place you want. Or there are browsers that only use source maps to roughly map the lines without like lines of code, without mapping the exact character indexes. But roughly they do all similar things. Something very positive, I'm not saying about this source maps like standardization group, is that it's working very well and like people from all different companies, from all different backgrounds really do have this shared goal. Like having some specific additions we want to do to source maps to improve the banking experience. While this is less frequent in larger standard groups where yes, technically everybody has the same goal of making like whatever the standard is [00:17:00] about better, but then different people have different opinions about for better means, which slows down progress. Uh, while right now here in service maps we're like very compact in a specific direction.

Paul: Do you think that is part of the nature of the domain of what the problems surrounding source maps and user stories are? Like maybe it's, um, a bit more ply and rooted in a fundamental truth of what is good and what is bad, um, to offer a good experience.

Nicolo: I think one of the main reasons is that. It's a not very large group. There are like a couple people working on dev tools at each of these companies, like, right, I don't know, Chrome has probably hundreds if not thousands of engineers. But then you can fail the engineers that work on Chrome dev tools in a single room. Uh, and the same is for other companies. Like I noticed that, for example, there is maybe one or two people working on dev tools for web [00:18:00] kit. Which is like much smaller than how many people work on web kit itself and all these people. So have a, like a very much shared experience on how source maps, uh, work and what the problem with source maps are. Also something important here is that source maps do not really affect the end user of the web, like the non-developer person that just navigates web pages. Uh, it's not like you. You add some feature to to JavaScript and that will have impacts on end users because maybe it allows the browser to do things it was not able to do before, like, I don't know, loading files from the user's laptop. Uh, or it might make the browsers lower as the user navigates through the web source Maps really only affect the developers given that they are only used either through dev tools or to these third party dev tools. But it's like things that. Defining users the web doesn't interact [00:19:00] with, and so it's much less scary to make a change to source maps because you're affecting a much smaller

group.

Yeah. You're not affecting the developers and nobody else.

Paul: Uh, what about the way that the conversations around generating source maps, ingesting source maps, uh, leveraging source maps in this group, in conference, and, uh, just from what you hear in the community, is there discourse around, um, web bundlers thinking like v web hack, uh, the developer experience of interacting with what is likely one of those tools in their day-to-day?

And how that dovetails with source maps.

Nicolo: Yeah, so we don't really talk about things like, like how you configure for tool or like how you make Webpac generates source maps. We only talk about, okay, when Webpac generates source maps, what information should it include in there? Uh, but in this group we have, we actually have one of Webpac maintainers, uh, or we don't only have, I know people from vi but [00:20:00] we talk with VI maintainers every once in a while. Uh, or like same for, I don't know, roll up. Uh, maybe it's not super active in our groove, but we make sure to think about what, what tools like those need. Um, 'cause like, yes, browsers can implement service maps, but if nobody actually generates service maps the right way, it doesn't matter what browsers implement, right?

So we need to have all these build tools on board, uh, like all those bundlers need to, need to basically do the right thing. And in some cases we. Like one of the proposals we're working on is actually to make it easier for some of those tools to generate source maps, even if it does not make, even if it does not improve the actual, the bagging experience or the implementation browsers.

Uh, but the easier we make to generate source maps, the easier it is for this tool, like the lower the bar is for all these kind of tools to, to generate them, uh, and. The more service maps we have, the better it is just because their goal is to improve [00:21:00] the debugging experience. So

Paul: Now if I was a user and I was making one of the token web app frameworks, uh, meta frameworks using the, using Webpack, um, insert other rollup, what do I need to do today to start interacting with source maps? Is it something that I would. Uh, placed directly in my next JS web hack, config in my, um, spelt configuration.

Nicolo: ~Um,~ it really depends on what framework or built tool you're using. Um. There are some tools where explicitly to ask them, please generate source maps now. Like, you know, if you're using B directly by default, it does not generate source maps and there is like some option that you need to enable. Or the same for, for webpac, maybe also for roll up. But then there are like the larger meta frameworks, like, I don't know, next js uh, they just tend to do the right thing by default, and so they just make sure that the underlying [00:22:00] tools to the user generates service maps and then they just put source maps. In the right place. And then the browser will just find the source maps without you really having to, to worry about it too much. So you should probably, depending on what tool you're using, just make sure that source maps are being generated. If, if it's not just for the docs of the tool, there is, I mean, sure, an option to enable source maps. Um, but yeah, ideally one day nobody will have to worry about them. They will just exist and be there and be hit and just do their job.

Paul: And if somebody got that hooked up and they started interacting or noticing source maps for the first time in their local development. What are some tips you would give for getting the full juice squeezed out of that experience? Um, getting the most that you can out of that source map.

Nicolo: if you have multiple tools as part of our build process. First of all, you like need to make sure that they're all [00:23:00] properly ingesting source maps from the previous step, like from the previous tool. Some common mistake have seen is that you have like, I dunno, three tools, one after the other and maybe two of them are generate source maps.

And then the third one is not, which means that the all previous source maps have been, are just run now. They're just useless because there is one tool in the pipeline that generate the wrong thing. Uh, and so just like. Broke all the connections. So if you have some like process where you're putting multiple tools together to make sure source maps work, you need to make sure that every single tool is generated source maps, and every single tool is ingesting source maps from the previous tool to, to propagate all of the information. Um, something else that people maybe don't really know, uh, but it's very helpful when dealing with source maps is that, well, sometimes they're just wrong. Where they're wrong, it can be a pain and I mean obviously report a bag to whatever tool generated the wrong source [00:24:00] map. But what you can do in these cases is just go to your browsers dev tools and you can turn off source maps.

All browsers have an option hidden somewhere to do so, 

Paul: If they're wrong.

Nicolo: if they're wrong, because obviously it's better to bug, we source must than without, but. The backing with wrong source maps is like the worst thing possible. 'cause you're thinking there is like you think you're running some code, your finger, you're in some function, but actually you're just somewhere else and that just doesn't work.

Paul: And what is one thing you think, uh. Is coming to the ecosystem surrounding, uh, developer hygiene and source maps or what is similarly something you think would be really cool that could be enabled with the additional standardization and, and socialization of source maps? Maybe you're thinking of some tool nicolo, you're like walking down the street some days and you're like, you know, it'd be really cool if we could use it for this.

Nicolo: yeah. So there is this new feature we're developing for source maps. It's called scopes. Um, I [00:25:00] mentioned earlier that source maps are basically just mapping points from one side of your build process to points to the other side of your build process. Uh, but. Code is not just so, they're basically mapping characters, but code is not just a list of characters, right?

This, those characters have some meaning, like you might have some variables in there. You might have some functions that contain those variables. Maybe even have two variables with the same name, uh, that like shadow each other. And so in some, at some point in your code, you can see one of the two and then some other point of your code with the same name, you're gonna see another variable. Um, and right now M Maps do not really allow reflecting all of this information. Which means that when you try to, to the back your code and like to like go step by step through it, you are gonna see the right code in your dev tools. But then usually dev tools have all these extra panel with extra information.

Like what variables are visible here? Uh, what is the, like are you set of [00:26:00] a function or are you inside the loop? Or like where, where are you? Um. Or you can, you can normally, when you're post the bug in your code, you can like open the cript console and try to execute some cript expression that uses the variables available in the local scope. And this doesn't really, source maps do not really make these operations transparent to build processes. So like if you have, if you have function that takes two parameters, A and B, and just returns A plus B. Then you have some build process that renames these two parameters to C and D, and then the function returns, C plus D. So you have a source map that's mapping this new code that contains C and D to the old code that contains A and B. Uh, which means that if you start, if you try to the bag in your browser, the code that is running, that is the code that contains C and D. You're actually gonna see A and B, and you're able to step through the code that contains A and B. However, if then like you check, okay, [00:27:00] what variables are available here, you're still gonna see C and D and not A and B uh, or like if you try to use a for something, the bar's gonna tell you no, actually A does not exist. It's just like, I'm just pretending that A exists, but it's not actually there. So something we're doing now is encoding more information in source maps. To allow dev tools reconstruct all of this information. So the source map now is also gonna tell the, the dev tool, look here, there isn't really a variable A, but there was original variable A and you can get its value by, I dunno, checking what the value is of this variable C. And that's gonna improve a lot the banking experience. Uh, tools are starting to work on experimental implementations of these browsers are starting also. So. Maybe as soon as your tool start supporting this, there is gonna be some experimental option to enable to get a new feature and it's gonna have problems, but you could like try whatever it is and see if it makes things [00:28:00] better.

And then if it makes things worse, just report a bag and disable it. 

Paul: ~Got it. Okay. So~ this is really pushing towards helping the live debugging experience and. Understanding like the mappings of your tokens, the variables, um, and accessing them with this rendered map in a more creative way that allows you to like, use what you see directly in the code. Uh, what was the name of the feature

Nicolo: It is called scopes. 

Paul: scopes.

Okay, got it.

Nicolo: It's probably gonna be difficult to Google for it, but just Googling for scopes. But maybe if you go for like source maps scopes or DC 39 source map scopes, something's gonna show up.

Paul: ~All right, Nico. Well, excuse me. Alright,~ Nicola. Well thank you so much for your, uh, time coming on and talking about source maps with us. Um, it's a topic that a lot of people don't have the time or space to explore, so it's great to like, have your deep knowledge of of, of being in it. To [00:29:00] surface back up into our memory, I know you do some talks online, uh, or you've be given at conferences about this.

Um, where would people find you? Do you have a channel in particular, are there any conferences that you could think about that would help socialize, source maps, or similarly, are there any social handles where you post regularly?

Nicolo: Um, so for me personally, I mostly, uh, use like Blue Sky or GitHub to talk with people. Uh, but when it comes to source maps, we actually do everything on GitHub. So if you want to interact with with us, you can like Google for the T 39 Source Maps, GitHub repository. Like, you can follow everything that's going on there. Or you can even like open issues. Like we, it feels like we're like this, this group of people that works on browsers and works on weird tools, but actually there is just a lot of like, just JavaScript developers interact with us and giving us feedback and getting involved, maybe even like trying to [00:30:00] design solutions to the problems that they have. So. Yeah, it's like very, we're very friendly. It's very easy to interact with us. People just just need to, to do it.

Paul: All right, so the, uh, TC 39 Source Maps, GitHub, uh, Nicolo, if people can find you on Blue Sky and GitHub. Um, and yeah, you can go search up, uh, Nicolo on YouTube if you wanna find some of the other conferences. And here. Him talk more. But thank you for your time, Nicola. It was a pleasure having you on and hopefully we'll have you back soon to talk about the direction of where the standardization pushes brought us.

Nicolo: Oh, thanks for having me here.