Paul:
Hi there and welcome to PodRocket. I'm your host Paul and joined with us today is Jess Sachs. So Jess is a staff engineer over at PathAI, but you might recognize her from Faker.js. She launched a fork and it's like Faker.js now. She's worked at Cypress on their component test runner and as well as being a View team member. Welcome to the podcast, Jess.

Jess Sachs:
Hey, thanks for having me, Paul.

Paul:
I'm glad you're on today because you worked at Cypress and we're talking about testing. That's the topic of the show today because everybody's favorite, right?

Jess Sachs:
I actually don't like testing, but I got my start in software as a QA engineer, so I have a lot of opinions that I've accumulated over the, I guess 12 years that I've been writing code. Makes it very easy for me to talk about and it's how I started working at Cypress.

Paul:
It's kind of funny when the first job or two that you have, it kind of formulates these strong nascent opinions that carry with you and if you're a QA engineer, you're the right person to split hairs about on this topic.

Jess Sachs:
Cypress is my third test runner that I built or worked on. The other two I built from scratch and those ideas carried through when I was hired onto Cypress to build their component test runner. It was a very formative experience. The first five years, four years of my career were very formative and I got to pour all those ideas into Cypress.

Paul:
You worked at Cypress because it was just a very... you were like, I'm getting really good at this stuff, very proficient, understanding testing and then you ended up there and then you've done a lot of other stuff as I mentioned at the beginning of the episode. You're a staff engineer now. You got the Faker.js. How did the Faker.js thing come about? We'll take a quick detour.

Jess Sachs:
Yeah, sure. I teach it in my testing workshops. So when I teach testing workshops I go over Faker and so November, 2021, I can't count years anymore, but November, 2021 I was teaching a workshop. I was using the old Faker and I really liked it. I then in January had some free time and I saw on the Boston JavaScript newsletter that the maintainer of Faker was also the maintainer of Colors.js and just had deleted both packages and I was like, oh no, I need those packages. Not had he deleted them, he also maliciously uploaded code, which was, if you can find the GitHub issues, it's pretty gnarly and historical reading through that giant GitHub issue.

Paul:
It was a turning point in the NPM ecosystem when that happened.

Jess Sachs:
Yeah, it was number one on Hacker News, so January 4th, 5th or 6th, 7th, somewhere around there. I was like life goal checked off. Our response to that incident ended up on the top post on Hacker News. It was like bucket list item for sure. But what happened was he deleted the package and in kind of the firefight moment, NPM and GitHub basically put it back up. They were, no, you can't completely bork a lot of repositories like that. And that caused a lot of outcry for people. My perspective and the, what was it, eight people, we started with, the eight other open source maintainers, our perspective was we used this package, we're going to fork it and keep maintaining it and we did a complete overhaul of the package within the first four weeks. The code was stale and we completely type scriptified it, added docs. It didn't have docs before that actually. It was a third party site that some good citizen kept mostly up to date. It was a very interesting January, so January, 2022

Paul:
And now Faker.js for anybody that's not familiar, this is just one of the handiest little things to mock everything you need to mock.

Jess Sachs:
And it's not a JavaScript thing.

Paul:
Oh, I've only used it for that, personally.

Jess Sachs:
Yeah, no, that was the funniest part is it's not a JavaScript thing, it's just a concept that has library bindings for most languages, over 12 languages or something. And so we were getting pinged by the original Pearl author was showing his support. It was a very cross language, I don't know, kumbaya moment. It felt very bonding I think to have so many people from so many different languages and stacks kind of come together and say, yeah, we just need a fake data generation library. It's very simple.

Paul:
I think that speaks to the quality of the library, too. The fact that you have such a surface area coverage of technologies coming together. If it surprised you, it must have been something profound. And Hey Jess, I learned something new today that I can use Faker for more than just my JavaScript apps.

Jess Sachs:
Yeah, we use it at my job at PathAI, we use the Python version, we actually don't use the JavaScript version.

Paul:
Well, I hope somebody else picked up that nugget of knowledge if they were lacking it like I was. So switching gears out of Faker, let's talk testing and Cypress, because you ended up in at Cypress doing testing. You're building the component test runner and let's talk about that for a sec because that's the topic of the episode today, we're talking about testing in Cypress. What is a component test runner and why was that needed at the time?

Jess Sachs:
Let's talk really quickly about testing in general. The point of testing is to make sure your code works and the simplest tests are environment agnostic. They look the same in every language. You want to make sure that an increment function does indeed increment a number, something simple like that. And then you realize that a lot of your code is not that simple and it involves other things like perhaps the UI if you're a front end developer. Then you ask yourself, well how do I test this? And a lot of people have tried to write post Node.js, which seems like eons ago, 10 years ago, people would write node browsers, emulated browsers to try to give you the ability to render DOM nodes in a mostly spec compliant way and get your browser side code to compile and run.
So we're talking 2013, 2014, this is how you do it is fake browsers that are implemented in node.js and jsdom, which is the headless browser that most people use, that was the way you did it. There's a few problems with this, one, you can't see the code that you're writing. When your test fails, you get HTML output. So that's not very easy to debug. So a component test is basically just, it actually has nothing to do with components. It's just stuff that renders in the browser and we happen to have the word component to describe that. That's the common way that you would describe functions that render stuff in today's language.
But the idea is that you have code that modifies the DOM and you need an environment to execute that code in. Today, we have the word component to describe that kind of stuff, but really what component testing is, is the way to have an environment and the kind of code you're trying to test.

Paul:
Could I call it an architectural component? Is that a fair analog in my head? Of my application, I have a piece of this application's architecture is a component, it's a piece of functionality versus I need to remove my brain from React components.

Jess Sachs:
Yeah. It's basically a function that needs a DOM to exist. The word document needs to be valid. So if you were going to run document dot create element in Node right now, if you open a node REPL and just typed a node document dot create element, that doesn't work. So a component test assumes that there is a browser that can, A, emulated browser, that can understand words that you would use in your normal browser and a component test is the act of executing that code and making sure it did it right.
That is way back in the day. The story of simple tests and then how front end code needs to be tested is you need an environment that can execute the code that we tend to write nowadays and that involves a React, that involves View, that involves Svelte, Solid, all of the things that we use to abstract away the raw DOM, that stuff needs to run in an environment that can handle it.

Paul:
And so we do it different now I guess is the follow-up statement is, we don't pretend browsers, we use something closer to the reality or the reality itself.

Jess Sachs:
That's the dream. That's the dream is that our tests are as similar to how real users will experience our apps, right? There's a trade-off, the realer the environment, the slower the test, the more stuff you have to load. The worst case is that you have an end-to-end test that spins of an entire server database, your web app, but nobody wants to do that. That's very flaky in general. There are a lot of things that could go wrong that aren't related to you changing the border radius on your button. You have a database involved. Who wants to spin up, who wants Docker involved in their normal day-to-day dev process? I know a lot of developers are cringing. They're, yeah, that's my life.

Paul:
Yep, me.

Jess Sachs:
I know, right? I was, yeah, me too. But in the dream sense, if you could have your ideal environment, the dream is that you hit the right balance of production-like environment, developer experience and speed. So today most developers still work in an emulated DOM. Cypress component testing has been out in GA, in general availability since, Cypress 11. It's been available in beta since Cypress 9. So it's been available to use for about two and a half years now as part of Cypress core. It was available since 2018 as a third party extension. And then Cypress kind of like Aqua hired that open source project as written by Dmitriy Kovalenko originally and Gleb Bahmutov, the original proof of concept for Cypress component testing.

Paul:
PodRocket has an episode with Gleb back in 2021. If you're listening and you want to hear the original coming out of this idea, go check out that episode and you can pick Gleb's brain with us.

Jess Sachs:
Yeah, Gleb and I live about five minutes walking from each other actually.

Paul:
Wow. Go figure. That's so cool.

Jess Sachs:
But yeah, I didn't create Cypress component testing, I created a lot of the ideas or I took over a lot of the ideas that I'd come up with as a QA engineer and pre-Cypress I brought those in to what eventually became general availability Cypress, but the original stuff is all Gleb and Dmitriy.

Paul:
So I feel like I am a little more knowledgeable now about what component means because I didn't know beforehand and there's a difference between my traditional view of a component and really the role of testing. It's interesting to me about your statement saying that most people still use in emulated DOM. I'm not a test expert so I don't know much about this DOMain, but to me I'm, oh, the standard must be to use the new tool. But it's interesting to me that you said that that's not the standard.

Jess Sachs:
Cypress is heavy. That is its biggest downside. It's 300 megs. Nobody wants to NPM install 300 megs at a coffee shop.

Paul:
That is not small, yeah.

Jess Sachs:
No, it has to carry around the entirety of Electron because it's trying to reduce issues that are like it works on my machine kinds of issues. So it carries around its own browser and it tries to make sure that you and your coworkers are not going to get different issues. So that is an architectural decision that I don't love about Cypress. I tried to work with the founder, Brian Mann, to re-architect Cypress just in a concept. I was, is there any way we could not download that much? But Cypress is not as fast as tools like V test. It is faster than Jest actually a lot of cases if you don't count the download time, which I was shocked by because Jest has no real browser but it is faster.

Paul:
That's an easy fix too, you cash your package, whatever.

Jess Sachs:
Well, it's to match snapshot actually. That's the biggest foot gun is everybody is always writing strengths to disk in Jest instead of looking at their components. So you do a lot of snapshot testing and for people who don't know what snapshot testing is, it's basically you take a string, which is usually the inner HTML of whatever you put on the body and you make sure it's the same between runs and people usually just say yes and they just click through it and they're, yeah, I'm sure it's fine.
So it's pretty low value test but it makes Jest slower than other solutions a lot of the time. Yeah, I didn't know that until I converted Beautifies test suite from Jest to use Cypress and the test got faster. So if you subtract out the install time and you like cash it in CI, it's actually faster between loops of when you're developing. Yeah, I was shocked.

Paul:
Could we take a quick look at using Cypress, you're talking about how it's really awesome between works on my machine sort of thing. If I wanted to rope it into making a new app today, what are some good practices for me to do to practice component driven tests? Backing up, what does that even mean?

Jess Sachs:
So good practices are basically, you should read the testing library ethos. So testing library is more of an ethos. It has bindings for every single framework you've ever heard of including vanilla bindings.

Paul:
So what's a binding?

Jess Sachs:
A binding? A binding is a layer almost I would say or a wrapper. So everything is based on testing libraries, vanilla DOM implementation, and then there are mounting layers for a View that are View specific and used View test details under the hood or Solid or Svelte. Yeah, so they're thin layers that wrap the real normal API that's agnostic to framework.

Paul:
Gotcha. Okay.

Jess Sachs:
So component driven development is like test driven development, you have an environment where you write expectations, you set up a component under test or a subject under test if it's not component specific. You set up a subject under test, you perform some actions and you make sure it behaves the same way or the expected way. That's test driven development. And, you write that before you write any source code, then you write source code that satisfies the test and you might refactor in between that, but it's a loop. You write the test, write the expectation, write the source code to make the test pass and then you continue on your way.
Component driven development is a little bit different. You can do it the same way, I am a little bit more fluid with my path. But in Cypress and in Storybook you can do things where you see the DOM as you begin to write your stepper components or tables and such like that. So component driven development is the act of isolating a component outside of the app. So you both in Storybook and in Cypress you have an iframe and the only element in the iframe is the component you're working on.

Paul:
So it's completely isolated.

Jess Sachs:
Completely isolated. I like to provide anything that the page would or the top level component would. So if you have an app component that will provide global state to any of the components below it like Material UI has these providers that are top level, right under app. I have a custom mount function that provides anything my app would generally use and then all of my components that I mount inside this top level app wrapper will just work/

Paul:
Is that sort of how Next.js has the underscored document thing?

Jess Sachs:
I have no idea.

Paul:
Okay, well, Next.js I can bring all my imports and then they'll all render out from that top level. Nevermind, I was just doing this for my own sanity check and following along.

Jess Sachs:
Well, it's a generic principle. So the idea, and it doesn't matter what stack it is in, imagine if you had an app component and you commented out literally everything inside of it except for the component you were about to test. Right, what is the minimum amount of dependencies that your app kind of expects for the component to work? So network requests, stuff like that.
I work at a very technically interesting company that's building something in between an interactive canvas app and a slide viewer that helps pathologists annotate where they think things like cancer are. So it's a technically complex app. The images are about five gigs streamed over the wire. It's impossible to test in anything but something like Cypress where you can mock out these app level network requests, you can mock out the fact that you need a canvas element on the page, that some third party deep down is going to use.
So it's a really powerful tool. A lot of things like color pickers that are canvas based, you can't really test in an emulated DOM and things like Cypress can contest them. Things like Storybook can test some of them. I won't explain why unless you want me to, but yeah, component driven development is the act of isolating a component in something like an iframe, providing the environment that it needs to render successfully and then to build out that component by itself.
Things like styles like apparent DOM node requiring a certain width. You can test what all these hidden links are that you wouldn't normally see in an app. If you were to render your whole app and build your component inside that, you wouldn't notice where all the environmental requirements are.

Paul:
That sounds huge. I hate jumping into my dev tools and changing CSS and not seeing the desired result because of some weird dependency graph.

Jess Sachs:
Yeah, you might pass in a ref and it'll be like, oh the ref isn't a div, therefore it'll break.

Paul:
Very annoying.

Jess Sachs:
And there's really gnarly bugs. You just avoid them entirely if you build your component in isolation and you're very clear about what the props are because your test will contain any wrapper information necessary. So if I do have a coupling, you can see it in the test, you can be, oh this carousel component requires you to have a fixed with parent and your component test will set that up. You'll see it in the render function.

Paul:
I like the tone of the conversation right now because it's highlighting to me one of the things of testing that causes me to overlook it, which is, you can beat the horse to death on your business logic and I don't care how good my business logic is, I'm going to mess something up in my dependency when it gets painted on the DOM. There's just so much you don't see that it sounds like component driven testing lends itself very well to clearing up.

Jess Sachs:
Yeah, it highlights a lot of unknowns that you couldn't have thought of before and it protects you against future refactors. So for example, if we at works walked out the way that we interact with our canvas currently with our Jest based, V test based unit tests, a lot of things would break because we're mocking literally the file. You've probably seen the underscore underscore mocks directory pattern. It's a file-based mocking pattern. With something like Storybook or with Cypress, you're just mounting the whole thing and it doesn't really matter how the data gets on the page so long as you fulfill the requisite contract, which is generally a network request, the rest of the business logic and stores will flow through and you don't have to worry about if you're using MobX, if you're using Redux, it doesn't matter because at the end of the day the user is going to either see your carousel or not and be able to swipe on it or not and that's what you really care about.
So when I mentioned the testing library ethos, if you go onto testing library or any of Kent C. Dodd's blog posts around testing, you'll learn that it's not how your code works, it is what your users are able to do with it.

Paul:
So Jess, these ideas we're talking about are very... if I was stepping into programming for the first time and somebody was telling me, go look up the ethos of testing and there's just a lot of complicated... we're talking about architecture components, what's the difference between the run times of how these get mounted and the work that you do for these testing frameworks is very complicated and gnarly. But I feel like it gives an interesting entry point for people trying to get into programming, maybe. You can have a very small app you could almost call it and play with it and pick it apart and hack with it in a controlled environment. Do you agree with that? Do you think Cypress is a good first thing for people to play with if they want to start learning the web on a greater detail?

Jess Sachs:
I think if they want to get used to testing, yes.

Paul:
Specifically testing,

Jess Sachs:
Yeah, if you're looking for a good entry point into how do I write scalable components, how do I write scalable UI, Cypress is a great place to start. But it's kind of like the next level of code pen or code sandbox, basically. You can have your iframe, that's all that code pen is, right? You can have your iframe, play with it, learn how your components, your styles, HTML, learn how all of that works. Sometimes I refer to it as being accidentally testable because after you mount it, all you have to do is get the right DOM element and call.click on it and then write an expectation. Make sure that the modal is no longer visible after I hit the X button.
There are so few lines of code that you write to get from the mount command that sets up the test to an actual assertion. So when I'm driving my components, when I'm making the step from a prototype, if I'm prototyping in code pen or something and I'm, yeah, I think the CSS pattern will work fine, I'll basically take that code and put it inside of a mount function in Cypress component testing and then write an assertion. So you can go straight from prototype phase where you're in an isolated environment like code pen, trying to test out if a certain CSS thing works, a certain HTML structure will work and you can grab that source code, put it in your application code, and then three lines later you have a functional test that is actually valuable.
I won't even get into all the details of what it does under the hood that's different from node emulated browsers, but the biggest one is that it actually renders your styles. So a lot of us don't delete entire modals from our DOM or maybe we do, but at the end of the day the user doesn't see it. And whether that's accomplished from the modal being removed from the DOM or whether that's accomplished through the display nun property, it doesn't really matter and no other test runner is currently rendering styles and doing such rich assertions.

Paul:
So maybe it could be valuable from the educational point of view of really understanding what you're writing and prototyping quickly and getting an almost production-ready piece of logic out of the box. Because with three lines, you take and you copy it and you have it ready.

Jess Sachs:
Yeah, it's really tiny to get a very high value test. It doesn't even feel like you're writing tests, it feels like you're developing your component in isolation and then at the end, you write three lines and suddenly you have test coverage and these same component tests that I'm at work, I'm converting from Vitetest or Jest to Cypress, these same tests are hundreds of lines and they're not comprehensive. They don't actually test the behavior of the code.

Paul:
What a difference.

Jess Sachs:
The comments I've gotten from coworkers are, this is extremely satisfying and there's no way that I could have tested this in a headless environment.

Paul:
If people wanted to do what you just described and what you're working on right now and maybe get into the testing space more, would you say Cypress is the number one thing that you would suggest people starting with because of these benefits you just mentioned?

Jess Sachs:
If you're writing UI code, you will find it most intuitive to render the UI code where you can see it and debug it easily. If you're writing headless code, you should be using something like V test, that's my recommendation to people. Yeah, if you should be able to see it, if your user cares about it being visible, then render it and test it in a browser. If it's not headed, if it's an increment function, render it in something fast.

Paul:
Awesome. Well Jess, thank you for your time and coming on and talking to us. I'm sure people might want to hear more from you, more about testing, more about Cypress. Do you have a Twitter?

Jess Sachs:
Yeah, I do. I have a Twitter. My name is a private variable. It's _JessicaSachs.

Paul:
I like how you described it, no, it's not prefixed with an underscore, it's a private variable.

Jess Sachs:
I told that to some of my friends the other day and I realized that all of my friends are nerds, when everybody understood what I said, I was like, oh, man.

Paul:
Do you write anywhere online or have any other sources you want to plug?

Jess Sachs:
Yeah, no, I haven't been writing much. I do have a blog. You can find it on my Twitter. I'm also on Mastodon which you can find on my Twitter. And I'll be at Vue Amsterdam in a few weeks. I will be at VueConf US, teaching a testing workshop on Cypress or Playwright or V test, it depends on some stuff that V test is working on actually, which has a lot of overlap with what we just talked about. They're working on an in browser runner, so hopefully that might replace Cypress component testing, maybe faster.

Paul:
All right, we'll see, maybe somebody will come find you in a few weeks and go to your testing workshop. Thank you for your time coming on.

Jess Sachs:
Thank you so much.