React 19 - Part 1: The Backstory; My journey writing a framework from scratch!

Oh boy, we're in for a ride today! I can assure you that those were 2 intense months, but I learned more about React and its inner workings than I ever thought I would (and should). All of this is condensed into 2 articles; this is Part 1, where I will tell you the backstory, the struggles, the wins, and how everything came to be. In Part 2, I will go through the code, the setup, and the implementation of the framework, and I will explain everything I learned in this journey.

So at the end of it, you will be able to write your framework from scratch, knowing all the ins and outs of it, and even create your very own Hydration Mismatch error! 🫠

NOTE:

This project was aimed to once and for all understand and desmistify React 19, how and why things work the way they do. It does not aim to be the next (no pun intended) big meta-framework, but the simplest possible implementation of a react-server!

#How did it all start?

Amidst the downpour of a Saturday, I obtained my coffee... Ok, leaving the BS aside, my latest "project" htnx got me intrigued by how RSC and RSA work, as they seemed to handle every and any miraculous thing I was trying to do. I had already seen, almost a year ago now, the Marz framework from LeahLundqvist, where she implemented a React Server from scratch with Bun, and I thought to myself: "How hard can it be? I should get this done by the weekend." (spoiler alert: it was waay harder than I expected.)

#Let's do it!

After all that, I got myself a brand new Bun project setup and running, TypeScript support out of the box, a web server, a bundler, and a transpiler, I was ready for the next step. Only then did I realize, and a thought came to my head: "What do I really need to do here? What should I handle, what does React handle, and how do I glue these two together?" That's when I realized there are no docs, no articles, no nothing on how to do this; I mean Next.js has it on stable, and Marz had it working a year ago, so why wouldn't I be able to do it? And that's when I started to dig into their source code, and I mean really dig into it.

Have you ever looked into the Next.js codebase? It's huge, and I mean huge! That's not the best sign for someone trying to implement similar functionality under 100 LOC. (Had I said it before? This was the initial idea—a questionable one, thats for sure.)

#The first steps

I will not lie, that quick glance at the Next.js codebase got me scared, so I thought maybe starting with Marz would be a better idea. And it is already way simpler, but for someone with literally no idea of what it is doing, it was still a bit too much, so I started digging into the old commits and found between the funny messages and a few curses the first implementation that seemed to work, and now that was my starting point!

#First thoughts

Ok, now we have the simplest possible implementation of a React server, but how does it work? Why are there two servers?? I can do it in one... (This decision comes back to haunt me later on; we'll get there.)

And this was when things started to click, react & react-dom are now both compatible with server implementations, but they can't handle it all, nor can they handle both at the same time. (This one took me weeks and a few @joshcstory DM's to get me on track)

#Simplicity vs. Webpack

The react team now has "react-server-dom-*" packages in their repo; those are compatible implementations of the needed methods and functions together with bundler-specific implementations, so you can use it with Webpack, Turbopack, etc. But this was way beyond the scope of my project; I didn't want any hidden magic; I wanted to know how it worked, and I decided to follow the Webpack version (as this was seemingly the most used on the 2 or 3 similar repos I had found at the time), but without Webpack, I was polyfilling everything that got in my way.

This was fine for a while; this was the same approach Marz took anyway, so it had to work, but working and working well are 2 different things, and I was not confident in my implementation, so again referring to the DM's, I got a better view of how things should work and that the Webpack package itself was not ready for server functions. Josh said: "the Webpack one doesn't implement server actions in the bundler config", which was the exact thing I was trying to implement... Time to go back to the drawing board, I guess.

#Enters ESM, and with it, mixed feelings arise.

For pure coincidence, some weeks ago I saw Dan discussing on Twitter how Tanner from the Tan Stack ecosystem could support React 19 in the new TanStack Start framework, and with it, he dropped a link, and just like that, a new world opened up to me: "Whats that, fixtures... flight... flight-esm... WAIT IS THAT IT?". And it (kind of) was what I needed; I had to jump into Dan's DMs to clarify some stuff and thank him for the unintentional help, but when everything seemed to be going well, then I learned something...

NOTE:

Up until this point, I was using a fixed 8-month-old canary build of the dependencies. Positive with the fixtures, I decided to update to the latest canary build, which now needed a --conditions="react-server" to be passed to the runtime, in this case Bun, and guess how happy I was when I discovered that Bun did not yet support this flag... 😅

Surprisingly enough, I found a PR and, out of sheer luck, Jared and the Bun contributors worked super quickly, and there it was one release later, the flag was supported, so we're back in business!

#The bad news

First of all, one thing I noted was that I had never seen the react-server-dom-esm package on NPM, and looking again, that's right, it is nowhere to be found (at least at the time of writing this article), but it is a thing I can see on the React repo, humm...

Back when I was talking to Josh, I remembered hearing about the ESM plugin, and going back to check, it was right there, and he explained why it is not available: "... The ESM one is just not production grade b/c it's not a bundler and would be wildly inefficient in prod ..." and Dan came back and confirmed: "the downside of the ESM one (why we don't publish it) is that it's going to be super inefficient. loading files one by one. bundler is better". So, back to the drawing board again? Wait...

#Taking matters into my own hands

Let me see if I got this right: the ESM one is not production-ready because it does not feature bundler integration and uses a custom loader for Node.js (this would already not work for me, as I'm using Bun and it does not support custom loaders), but what if I did the bundling myself? I could then use the ESM version, modules, functions, and helpers efficiently and with reasonable performance; that sounded like a plan to me.

IMPORTANT:

I think it is worth noting that, as the package was not available on NPM, I had to clone React on my machine and build the packages myself. And let me tell you, for the first time I saw my Macbook with a M chip getting hot, and I mean hot!

Anyways, 20 minutes of build later, I had the package ready to go. This is where the fixtures came into play; I now had the needed packages and a working example of how things should behave.

#The final steps

With all of that in my hands, I started a quick refactor.

I was using Bun.serve as my web server, but the implementation React has for ESM is based on pipeable streams, and Bun uses mainly readable streams for responses (I'm pretty sure theres a way to convert one into the other, but I was not ready to go down that rabbit hole), so I had to change that. The only server I knew that used pipeable streams was the good old Express, not the ideal, but it works.

express()
  .use(logger) // A simple logger middleware
  .use(cors) // Default CORS middleware
  .get("/*", async (req, res) => { ... })
  .listen(port)

After some other minor changes and fixes, I got to what I thought was the best version I could think of, and it worked "relatively" well; I mean, it rendered my pages and some client components, but any time I passed the content to renderToPipeableStream the function responsible for creating the RSC payloads, it would throw an error saying that the useReducer hook was not available. At this moment, the only thing I could think of was: "Of course it is not! I'm on the server, you know it!!", and on the top of my 999 IQ brain, I thought: "I should just patch that; React has to be the wrong one here!".

  .get("/*", async (req, res) => {
    // ...
    renderToPipeableStream(mod, moduleBaseURL).pipe(res)
  })

A span of console.logs later, I found the function that was throwing the error, threw a try/catch around it, ignored the error, and thought: "My work here is done!". So I guess this means I'm done, right? Client & Server components working, RSC and RSA payloads working, ESM modules working, and no errors on the console, I was ready to call it a day, but then Dan came back and said: "you're not doing SSR for now, right?"

#The last challenge (a few lessons learned)

Ok, I didn't even know I wanted this, but I for sure want an SSR setup to work here. I mean, a lot of React 19 is based around that. How could I forget? But I mean, how hard could it be? (This question is starting to haunt me)

But seriously, I just need to grab the method that _client.tsx uses and throw it in the API endpoint, then send the generated... HTML? That's when I remembered that React 19 is not just a couple of helper functions matched together; it is a whole new way of thinking about React, and that got me thinking.

After a reasonable break, I got back to it. The thing I was generating all along the way was not HTML but the RSC payload, which looks something like this:

2:"$Sreact.suspense"
3:I["components/counter.js","default"]
1:D{"name":"os_default","env":"Server"}
1:["$","b",null,{"children":["darwin"," ","arm64"]}]
4:{"id":"/build/components/actions.js#add","bound":null}
0:[["$","main",null,{"className":"m-4 border-4 border-dashed border-red-400 p-4","children":[["$","h1",null,{"className":"text-2xl font-bold","children":["Hello from ",["$","i",null,{"children":"node"}],"! "]}],["$","p",null,{"children":["You're running: ","$1"]}],["$","section",null,{"className":"mt-4 flex h-16 items-center justify-center border-4 border-dashed border-blue-400","children":["$","$2",null,{"fallback":"Loading counter...","children":["$","$L3",null,{"action":"$F4"}]}]}],["$","nav",null,{"className":"mt-4 [&_a]:text-blue-500 [&_a]:underline","children":["Follow to: ",["$","a",null,{"href":"/props?name=John&age=25","children":"/props"}]]}]]}],["$","footer",null,{"className":"mx-4","children":[["$","h2",null,{"children":"Caption:"}],["$","small",null,{"className":"text-red-400","children":"* Server components = No bundle size increase, rendered on the server"}],["$","br",null,{}],["$","small",null,{"className":"text-blue-400","children":"* Client components = Includes a JS bundle, rendered on the client"}]]}]]

React knows how to use this to create or hydrate a UI from it, but I don't. This means I now needed to make that into HTML, and I had no idea how to do it.

Back to the fixture, I saw they were using two servers, one calling the other in a way that I could not understand why, but there they used a specific method from react-dom/server that I guessed was the one I needed.

I just got that snippet onto my code, and... nothing? Errors upon errors: "'use client' is not supported," "you cannot call hooks during the first render," and the list went on. At this moment, all my hopes are lost. I had the perfect setup; everything made sense; I got the flag going; and the ESM version was working, but I could not get the HTML to render. I was not ready to give up, but close enough to message Josh once again. (I'm sorry, Josh, I know you have better things to do than help me with my silly projects; I owe you a coffee.).

Josh, came again to the rescue. After explaining to him the problem, he said, and I quote:

"So one thing to keep in mind is that you need two module graphs when you want to run RSC and SSR in the same process. When you have a React program that imports from react-dom and react, we want the version of those modules to be the RSC version. When you have a React program that imports from react-dom and react for SSR (client components), you need those modules to be the non-RSC version. It sounds like you probably have just one version of these modules in scope, and so when your client components try to access React for useReducer, they get the RSC version, which doesn't have that export."

WAIT WHAT, OH NO... IS THIS THAT FLAG DOES... AAAAAAAHH.

Man, what a sweet moment! I remember it vividly. I was just leaving the office, and I got the message, entered the car, and just started laughing. I had the solution all along; I had never stopped for once to think about the purpose of the react-server flag, and now I had the solution; I rushed back home, signed into my computer, split my server into two, one for the RSC and one for the SSR, did the server-call-server dance, and... it worked! I had now a working React 19 server, and I was happy, so so happy.

#Conclusion

Of course, this was not all; I had to do a lot of refactoring, a lot of cleaning, and a lot of testing, but at the end of the day, I had a working React 19 server with ESM modules and bundling, and I may say the first one to do so, and I was so proud of it. I learned a lot, and I mean a lot. I learned how React works internally, how to write a React framework, how to use ESM and moduleMaps; I met incredible people, and I learned that after removing a bit of the noise, the React community is always there to help you. I'm grateful for all of that.

For all of the brave of you who got here, I hope you enjoyed the journey and I hope you learned something from it. Part 2 will be out soon, and I will go through the code, the setup, and the implementation of the framework.

CAUTION:

I will explain everything I learned on the technical side in this journey, so hold on tight, the day you'll create your very own Hydration Mismatch error is coming! 🫠