jakelazaroff.com

An Interactive Intro to CRDTs

Wed, 04 Oct 2023 00:00:00 GMT

import Scripts from "./_Scripts.astro";

Have you heard about CRDTs and wondered what they are? Maybe you've looked into them a bit, but ran into a wall of academic papers and math jargon? That was me before I started my Recurse Center batch. But I've spent the past month or so doing research and writing code, and it turns out that you can build a lot with just a few simple things!

In this two-part series, we'll learn what a CRDT is. Then we'll write a primitive CRDT, compose it into more complex data structures, and finally use what we've learned to build a collaborative pixel art editor. All of this assumes no prior knowledge about CRDTs, and only a rudimentary knowledge of TypeScript.

To pique your curiosity, this is what we'll end up with:

<pixelart-demo resolution="100" style="--accent: var(--color-primary)"></pixelart-demo>

Draw by clicking and dragging with your mouse. Change the paint color by using the color input on the bottom left. You can draw on either canvas and your changes will show up on the other, as if they were collaborating on the same picture.

Clicking the network button prevents changes from reaching the other canvas (although they'll sync up again when they come back "online"). The latency slider adds a delay before changes on one canvas show up on the other.

We'll build that in the next post. First, we need to learn about CRDTs!

What is a CRDT?

Okay, let's start from the top. CRDT stands for "Conflict-free Replicated Data Type". That's a long acronym, but the concept isn't too complicated. It's a kind of data structure that can be stored on different computers (peers). Each peer can update its own state instantly, without a network request to check with other peers. Peers may have different states at different points in time, but are guaranteed to eventually converge on a single agreed-upon state. That makes CRDTs great for building rich collaborative apps, like Google Docs and Figma — without requiring a central server to sync changes.

Broadly, there are two kinds of CRDTs: state-based and operation-based.^[1] State-based CRDTs transmit their full state between peers, and a new state is obtained by merging all the states together. Operation-based CRDTs transmit only the actions that users take, which can be used to calculate a new state.

That might make operation-based CRDTs sound way better. For example, if a user updates one item in a list, an operation-based CRDT can send a description of only that update, while a state-based CRDT has to send the whole list! The drawback is that operation-based CRDTs impose constraints on the communication channel: messages must be delivered exactly once, in causal order, to each peer.^[2]

This post will exclusively focus on on state-based CRDTs. For brevity, I'll just say "CRDTs" from here on out, but know that I'm referring specifically to state-based CRDTs.

I've been talking about what CRDTs do, but what is a CRDT? Let's make it concrete: a CRDT is any data structure that implements this interface:^[3]

interface CRDT<T, S> {
  value: T;
  state: S;
  merge(state: S): void;
}

That is to say, a CRDT contains at least three things:

A value, T. This is the part the rest of our program cares about. The entire point of the CRDT is to reliably sync the value between peers.
A state, S. This is the metadata needed for peers to agree on the same value. To update other peers, the whole state is serialized and sent to them.
A merge function. This is a function that takes some state (probably received from another peer) and merges it with the local state.

The merge function must satisfy three properties to ensure that all peers arrive at the same result (I'll use the notation A ∨ B to indicate merging state A into state B):

Commutativity: states can be merged in any order; A ∨ B = B ∨ A. If Alice and Bob exchange states, they can each merge the other's state into their own and arrive at the same result.
Associativity: when merging three (or more) states, it doesn't matter which are merged first; (A ∨ B) ∨ C = A ∨ (B ∨ C). If Alice receives states from both Bob and Carol, she can merge them into her own state in any order and the result will be the same.^[4]
Idempotence: merging a state with itself doesn't change the state; A ∨ A = A. If Alice merges her own state with itself, the result will be the same state she started with.

Mathematically proving that a merge function has all these properties might sound hard. But luckily, we don't have to do that! Instead, we can just combine CRDTs that already exist, leaning on the fact that someone has proven these things for us.

Speaking of CRDTs that already exist: let's learn about one!

Last Write Wins Register

A register is a CRDT that holds a single value. There are a couple kinds of registers, but the simplest is the Last Write Wins Register (or LWW Register).

LWW Registers, as the name suggests, simply overwrite their current value with the last value written. They determine which write occurred last using timestamps, represented here by integers that increment whenever the value is updated.^[5] Here's the algorithm:

If the received timestamp is less than the local timestamp, the register doesn't change its state.
If the received timestamp is greater than the local timestamp, the register overwrites its local value with the received value. It also stores the received timestamp and some sort of identifier unique to the peer that last wrote the value (the peer ID).
Ties are broken by comparing the local peer ID to the peer ID in the received state.

Try it out with the playground below.

<lwwregister-demo style="--accent: var(--color-primary); --background: var(--color-background);"></lwwregister-demo>

Did you get a sense for how LWW Registers work? Here are a couple specific scenarios to try:

Turn the network off, make a bunch of updates to bob, and then turn it back on. When you send updates from alice, they'll be rejected until the timestamp exceeds bob's timestamp.
Perform the same setup, but once you turn the network back on, send an update from bob to alice. Notice how the timestamps are now synced up and alice can write to bob again!
Increase the latency and send an update from both peers simultaneously. alice will accept bob's update, but bob will reject alice's. Since bob's peer ID is greater, it breaks the timestamp tie.

Here's the code for the LWW Register:

class LWWRegister<T> {
  readonly id: string;
  state: [peer: string, timestamp: number, value: T];

  get value() {
    return this.state[2];
  }

  constructor(id: string, state: [string, number, T]) {
    this.id = id;
    this.state = state;
  }

  set(value: T) {
    // set the peer ID to the local ID, increment the local timestamp by 1 and set the value
    this.state = [this.id, this.state[1] + 1, value];
  }

  merge(state: [peer: string, timestamp: number, value: T]) {
    const [remotePeer, remoteTimestamp] = state;
    const [localPeer, localTimestamp] = this.state;

    // if the local timestamp is greater than the remote timestamp, discard the incoming value
    if (localTimestamp > remoteTimestamp) return;

    // if the timestamps are the same but the local peer ID is greater than the remote peer ID, discard the incoming value
    if (localTimestamp === remoteTimestamp && localPeer > remotePeer) return;

    // otherwise, overwrite the local state with the remote state
    this.state = state;
  }
}

Let's see how this stacks up to the CRDT interface:

state is a tuple of the peer ID that last wrote to the register, the timestamp of the last write and the value stored in the register.
value is simply the last element of the state tuple.
merge is a method that implements the algorithm described above.

LWW Registers have one more method named set, which is called locally to set the register's value. It also updates the local metadata, recording the local peer ID as the last writer and incrementing the local timestamp by one.

That's it! Although it appears simple, the humble LWW Register is a powerful building block with which we can create actual applications.

Last Write Wins Map

Most programs involve more than one value,^[6] which means we'll need a more complex CRDT than the LWW Register. The one we'll learn about today is called the Last Write Wins Map (or LWW Map).

Let's start by defining a couple types. First, our value type:

type Value<T> = {
  [key: string]: T;
};

If each individual map value holds type T, then the value of the entire LWW Map is a mapping of string keys to T values.

Here's our state type:

type State<T> = {
  [key: string]: LWWRegister<T | null>["state"];
};

Do you see the trick? From our application's perspective, the LWW Map just holds normal values — but it actually holds LWW Registers. When we look at the full state, each key's state is the state of the LWW Register at that key.^[7]

I want to pause on that for a moment, because it's important. Composition lets us combine primitive CRDTs into more complex ones. When it's time to merge, all the parent does is pass slices of incoming state to the appropriate child's merge function. We can nest this process as many times as we want; each complex CRDT passing ever-smaller slices of state down to the next level, until we finally hit a primitive CRDT that performs the actual merge.

From this perspective, the LWW Map merge function is simple: iterate through each key and hand off the incoming state at that key to the corresponding LWW Register to merge. Try it out in the playground below:

<lwwmap-demo add="off" delete="off" state='{ "foo": ["alice", 1, "lorem ipsum"], "bar": ["bob", 1, "dolor sit amet"] }' style="--accent: var(--color-primary); --background: var(--color-background);"></lwwmap-demo>

It's kind of difficult to trace what's happening here, so let's split up the state for each key. Note, though, that this is just a visualization aid; the full state is still being transmitted as a single unit.

Try increasing the latency and then updating different keys on each peer. You'll see that each peer accepts the updated value with a higher timestamp, while rejecting the value with a lower timestamp.

<lwwmap-demo split add="off" delete="off" state='{ "foo": ["alice", 1, "lorem ipsum"], "bar": ["bob", 1, "dolor sit amet"] }' style="--accent: var(--color-primary); --background: var(--color-background);"></lwwmap-demo>

The full LWW Map class is kinda beefy, so let's go through each property one by one. Here's the start of it:

class LWWMap<T> {
  readonly id = "";
  #data = new Map<string, LWWRegister<T | null>>();

  constructor(id: string, state: State<T>) {
    this.id = id;

    // create a new register for each key in the initial state
    for (const [key, register] of Object.entries(state)) {
      this.#data.set(key, new LWWRegister(this.id, register));
    }
  }
}

#data is a private property holding a map of the keys to LWW Register instances. To instantiate a LWW Map with preexisting state, we need to iterate through the state and instantiate each LWW Register.

Remember, CRDTs need three properties: value, state and merge. We'll look at value first:

  get value() {
    const value: Value<T> = {};

    // build up an object where each value is set to the value of the register at the corresponding key
    for (const [key, register] of this.#data.entries()) {
      if (register.value !== null) value[key] = register.value;
    }

    return value;
  }

It's a getter that iterates through the keys and gets each register's value. As far as the rest of the app is concerned, it's just normal map!

Now let's look at state:

  get state() {
    const state: State<T> = {};

    // build up an object where each value is set to the full state of the register at the corresponding key
    for (const [key, register] of this.#data.entries()) {
      if (register) state[key] = register.state;
    }

    return state;
  }

Similar to value, it's a getter that builds up a map from each register's state.

There's a clear trend here: iterating through the keys in #data and handing things off to the register stored at that key. You'd think merge would work the same way, but it's a little more involved:

  merge(state: State<T>) {
    // recursively merge each key's register with the incoming state for that key
    for (const [key, remote] of Object.entries(state)) {
      const local = this.#data.get(key);

      // if the register already exists, merge it with the incoming state
      if (local) local.merge(remote);
      // otherwise, instantiate a new `LWWRegister` with the incoming state
      else this.#data.set(key, new LWWRegister(this.id, remote));
    }
  }

First, we iterate through the incoming state parameter rather than the local #data. That's because if the incoming state is missing a key that #data has, we know that we don't need to touch that key.^[8]

For each key in the incoming state, we get the local register at that key. If we find one, the peer is updating an existing key that we already know about, so we call that register's merge method with the incoming state at that key. Otherwise, the peer has added a new key to the map, so we instantiate a new LWW Register using the incoming state at that key.

In addition to the CRDT methods, we need to implement methods more commonly found on maps: set, get, delete and has.

Let's start with set:

  set(key: string, value: T) {
    // get the register at the given key
    const register = this.#data.get(key);

    // if the register already exists, set the value
    if (register) register.set(value);
    // otherwise, instantiate a new `LWWRegister` with the value
    else this.#data.set(key, new LWWRegister(this.id, [this.id, 1, value]));
  }

Just like in the merge method, we're either calling the register's set to update an existing key, or instantiating a new LWW Register to add a new key. The initial state uses the local peer ID, a timestamp of 1 and the value passed to set.

get is even simpler:

  get(key: string) {
    return this.#data.get(key)?.value ?? undefined;
  }

Get the register from the local map, and return its value if it has one.

Why coalesce to undefined? Because each register holds T | null. And with the delete method, we're ready to explain why:

  delete(key: string) {
    // set the register to null, if it exists
    this.#data.get(key)?.set(null);
  }

Rather than fully removing the key from the map, we set the register value to null. The metadata is kept around so we can disambiguate deletions from states that simply don't have a key yet. These are called tombstones — the ghosts of CRDTs past.

Consider what would happen if we really did delete the keys from the map, rather than leaving a tombstone. Here's a playground where peers can add keys, but not delete them. Can you figure out how to get a peer to delete a key?

<lwwmap-demo delete="off" merge="naive" state='{ "foo": ["alice", 1, "lorem ipsum"] }' style="--accent: var(--color-primary); --background: var(--color-background);"></lwwmap-demo>

Turn off the network, add a key to alice's map, then turn the network back on. Finally, make a change to bob's map. Since alice sees that the incoming state from bob is missing that key, she removes it from her own state — even though bob never knew about that key in the first place. Whoops!

Here's a playground with the correct behavior. You can also see what happens when a key is deleted.

<lwwmap-demo state='{ "foo": ["alice", 1, "lorem ipsum"] }' style="--accent: var(--color-primary); --background: var(--color-background);"></lwwmap-demo>

Notice how we never remove deleted keys from the map. This is one drawback to CRDTs — we can only ever add information, not remove it. Although from the application's perspective the key has been fully deleted, the underlying state still records that the key was once there. In technical terms, we say that CRDTs are monotonically increasing data structures.^[9]

The final LWW Map method is has, which returns a boolean indicating whether the map contains a given key.

  has(key: string) {
    // if a register doesn't exist or its value is null, the map doesn't contain the key
    return this.#data.get(key)?.value !== null;
  }

There's a special case here: if the map contains a register at the given key, but the register contains null, the map is considered to not contain the key.

For posterity, here's the full LWW Map code:

class LWWMap<T> {
  readonly id: string;
  #data = new Map<string, LWWRegister<T | null>>();

  constructor(id: string, state: State<T>) {
    this.id = id;

    // create a new register for each key in the initial state
    for (const [key, register] of Object.entries(state)) {
      this.#data.set(key, new LWWRegister(this.id, register));
    }
  }

  get value() {
    const value: Value<T> = {};

    // build up an object where each value is set to the value of the register at the corresponding key
    for (const [key, register] of this.#data.entries()) {
      if (register.value !== null) value[key] = register.value;
    }

    return value;
  }

  get state() {
    const state: State<T> = {};

    // build up an object where each value is set to the full state of the register at the corresponding key
    for (const [key, register] of this.#data.entries()) {
      if (register) state[key] = register.state;
    }

    return state;
  }

  has(key: string) {
    return this.#data.get(key)?.value !== null;
  }

  get(key: string) {
    return this.#data.get(key)?.value;
  }

  set(key: string, value: T) {
    // get the register at the given key
    const register = this.#data.get(key);

    // if the register already exists, set the value
    if (register) register.set(value);
    // otherwise, instantiate a new `LWWRegister` with the value
    else this.#data.set(key, new LWWRegister(this.id, [this.id, 1, value]));
  }

  delete(key: string) {
    // set the register to null, if it exists
    this.#data.get(key)?.set(null);
  }

  merge(state: State<T>) {
    // recursively merge each key's register with the incoming state for that key
    for (const [key, remote] of Object.entries(state)) {
      const local = this.#data.get(key);

      // if the register already exists, merge it with the incoming state
      if (local) local.merge(remote);
      // otherwise, instantiate a new `LWWRegister` with the incoming state
      else this.#data.set(key, new LWWRegister(this.id, remote));
    }
  }
}

Next Steps

If you're reading this now, we're in the short period when this post has been published, but the next one has not. Subscribe to my RSS feed to be notified when it's ready!

They're also known as CvRDTs ("Cv" standing for "convergent") and CmRDTs ("Cm" standing for "commutative"), respectively, although I think "state-based" and "operation-based" are the preferred terms. ↩︎
There are also delta CRDTs, or hybrid CRDTs, which allow peers to negotiate the subset of the state they need to send each other. That's one example of blending operation-based and state-based CRDTs. But the fundamental tradeoff remains: sending less data between peers means more constraints on communication. ↩︎
Technically, a CRDT can be anything that follows the merging rules described below. This is a working definition; in practice, implementations in object-oriented languages will end up looking something like this. ↩︎
Commutativity and associativity might sound the same, and indeed most commutative operations are also associative. But there are a few math operations that are only one or the other. Matrix multiplication, for example, is associative but not commutative. And surprisingly, floating point arithmetic — i.e. any math operator in JavaScript — is commutative but not associative! ↩︎
You might ask: why not use the actual time? Unfortunately, accurately syncing clocks between two computers is an extremely hard problem. Using incrementing integers like this is one simple version of a logical clock, which captures the order of events relative to each other rather than to the "wall clock". ↩︎
[citation needed] ↩︎
If the value type is T, why is the state type a union of T | null? More on that in a bit! ↩︎
You might be wondering: if another peer deleted a key from their map, shouldn't we remove it from our local map as well? It's the same reason the state holds T | null. We're getting there! ↩︎
There are a couple ways mitigate this drawback, both of which are outside the scope of this article. One is "garbage collection": pruning tombstones from CRDTs, which prevents you from merging states with any changes made before the tombstones were removed. Another is creating an efficient format to encode the data. You can also combine these methods. Research suggests that this can result in as little as 50% overhead compared to the "plain" data. ↩︎

Building a Live Coding Audio Playground

Mon, 21 Aug 2023 00:00:00 GMT

Two weeks at Recurse Center went by fast! I started out trying to build an Audio Units extension and — in a classic yak shave — ended up building a live coding audio playground. (I have not yet finished the original project.)

If you just want to play around with the app: here it is. If you want to hear how I made it, then keep reading!

My initial goal at RC was to build an Audio Units (AU) extension. AU is Apple's audio plugin system, which works with apps like Logic and GarageBand. There are other plugin systems,^[1] but I use Logic when recording audio for myself and for my band, so I wanted to try extending it.

Apple actually has a tutorial on creating an AU extension, complete with a good starter template. Almost too good — it gives you a finished (albeit basic) AU extension, ready to compile and run. Great for getting up and running quickly, but bad for building a mental model of how it works.

I tend to learn by exploring, so I decided to try building a low-pass filter. And let me tell you: building an audio effect in Xcode is like pulling teeth.

I mean, okay. It's not that bad. But I'm a disciple of the Bret Victor school of thought, where trying to create something without immediate feedback is like working without one of your senses. So I decided to build an environment in which I could get that immediate feedback!

The rough plan: let users upload audio, provide a text field in which to write their effect, and display a visualization of both the original audio signal ("dry") and the audio signal modified by the user's code ("wet"). A perfect fit for a small single-page app!

I built the app using Svelte. I know I wrote before about defaulting to React for new projects, but what is even the point of being at RC if not to try new things? Plus, the design I had planned didn't involve any rich widgets,^[2] so I wouldn't need any component libraries — almost all the complexity was in the audio code.

Some more assorted thoughts about Svelte:

Single-file components are such a pleasant authoring experience! I never want to build an app without them. Someone please bring them to React.
I'm wary of two-way binding after years of working with Angular 1, but it's really convenient to be able to just define functions within a component without the rigamarole of useImperativeHandle.
No useEffect, my most hated hook! Reactive blocks are way less error-prone, although the lack of a "cleanup function" definitely made my life hard for a bit.

Ultimately, I really enjoyed Svelte! I'd still be hesitant to build a large app with it, but I'll definitely be using it again for smaller side projects like this.

Anyway, back to the audio code: the Web Audio API is pretty incredible. It lets you construct a full audio routing graph, with oscillators and visualizers and all sorts of processing nodes. The secret sauce here is the AudioWorklet, which lets you run audio processing code off the main thread.

An audio worklet has two parts: the AudioWorkletProcessor, which is where you write your audio processing code, and an AudioWorkletNode, which is an object that exists in the main thread and gets connected to the rest of your audio routing graph. So basically, your UI code interacts with the AudioWorkletNode, but the AudioWorkletProcessor is doing the real work "behind the scenes".

At a high level, here's how the playground works:

The user types code in the editor.^[3]
That code gets compiled into an AudioWorkletProcessor.
The app creates a corresponding AudioWorkletNode and connects it to the audio routing graph.

Making an audio worklet should be fairly simple: create a separate JS file with a class extending AudioWorkletProcessor, create an AudioContext and call its addModule method with that file's URL, then create an AudioWorkletNode with the name of that processor.^[4] That's all well and good, but the JS file doesn't actually exist — it's based on user-supplied code. I had a hunch, though, that I could fake it with URL.createObjectURL.

That hunch turned out to be correct! If this app has a beating heart, it's this compile function:

let n = 0n;

async function compile(ctx: AudioContext, code: string) {
  // each AudioWorkletProcessor needs a unique name
  const name = `${++n}`;

  // the source code of the AudioWorkletProcessor module
  const src = `
    class Filter extends AudioWorkletProcessor {
      process(inputs, outputs, params) {
        ${code}
      }
    }

    registerProcessor(${name}, Filter);
  `;

  // create a fake JS file from the source code
  const file = new File([src], "filter.js");

  // create a URL for the fake JS file
  const url = URL.createObjectURL(file.slice(0, file.size, "application/javascript"));

  // add the fake JS file as an AudioWorkletProcessor
  await ctx.audioWorklet.addModule(url);

  // revoke the URL so as to not leak memory
  URL.revokeObjectURL(url);

  // create a new AudioWorkletNode from the AudioWorkletProcessor registered with `name`
  return new AudioWorkletNode(ctx, name);
}

Yup, you're reading right: most of it is just a big ol' hardcoded string of JS code! The user's code gets interpolated into the process method body, which gets called repeatedly as audio flows through the graph. The compile function runs in a Svelte reactive block, so it creates a new AudioWorkletNode whenever the user changes the code. That node then gets connected to the larger audio routing graph. The graph is still pretty small, though: <audio> tag to MediaElementAudioSourceNode to AudioWorkletNode to the AudioContext destination.

If you've tried the app already, you might be wondering about the parameters — the table with the sliders on the bottom right. Those are actually part of the audio worklet spec! You can define a static getter on your AudioWorkletProcessor subclass called parameterDescriptors that lets you configure the processor from the main thread. Letting the user tweak them as sliders is important because they can really experiment with their effects: rather than having to type in different numbers, they can get instant feedback on a range of values.

Here's how that fits into the compile function (omitting code from the previous example):

interface Parameter {
  defaultValue: number;
  minValue: number;
  maxValue: number;
  name: string;
  automationRate: "k-rate";
}

async function compile(ctx: AudioContext, code: string, params: Parameter[]) {
  // ...

  const src = `
    class Filter extends AudioWorkletProcessor {
      process(inputs, outputs, params) {
        ${code}
      }

      static get parameterDescriptors() {
        return ${JSON.stringify(params)};
      }
    }

    registerProcessor(${name}, Filter);
  `;

  // ...
}

The Parameter data structure is exactly what the AudioWorkletProcessor expects to be returned from parameterDescriptors!^[5] The array just needs to be stringified to JSON so as not to return [Object object], which would be invalid JS.

Initially, the slider just controled the defaultValue property, and the whole audio worklet was recreated whenever it changes. The performance seemed fine, since the generated JS file is loaded from memory, but it had the annoying side effect of removing any state set in the worklet class. That made the slider much less useful for anything requiring more than a frame or so of state, since state wouldn't be retained as the slider moved. Ultimately, I just set defaultValue to the average of minValue and maxValue and made the slider call the parameter's setValueAtTime method.

In addition to hearing the effect, the app also visualizes it so the user can see how their filtered signal compares to the original one. This is where the AnalyserNode^[6] comes in: it takes a Fast Fourier Transform of the audio, so I can plot the frequencies as a graph. This is particularly important for effects like equalizers, where the point of the effect is to amplify or attenuate certain frequencies compared to the dry signal.

Did you click on that link? Then you found the share feature! It just serializes the code and parameters to the URL. When you load the page, if it detects the right query string, it pre-populates everything from the URL. That way, you can send a link to your friends, and they can play around with the effects you've made.

What next? Although there are a ton of features I've thought of — logging from user code, logarithmic frequency binning for the visualization, support for languages other than JS — I think I'm going to move onto something else at RC. The point of the program is to learn, and I've learned most of what I set out to when I built this thing. I tried out Svelte, AudioWorklets, the <dialog> element, figured out a method for incorporating user-generated code and learned how to build a simple low-pass filter.

I'll probably come back to this later (or sooner, if you try it out and let me know what you think). For now, stay tuned for more RC projects!

The main one is VST, which was developed by Steinberg and is used by basically everyone else. ↩︎
Well, I did need modals for help text. But luckily, the native <dialog> element is well supported! ↩︎
A plain <textarea> is not a great experience for exiting code. I used CodeMirror, which seems to strike a nice balance between lightweight and full-featured. ↩︎
I'm skipping some steps here for brevity. MDN has more complete documentation. ↩︎
If you read the documentation for audio worklet parameters, you may wondering why the automationRate is type "k-rate", rather than "a-rate" | "k-rate". a-rate is used when the value of a parameter varies over time, whereas k-rate is used when it stays the same. For the playground, parameters only ever have one value, so I don't need to worry about a-rate. ↩︎
I guess I can't really complain about this, because so much of the world accommodates American English where we don't deserve it, but I do wish it were spelled AnalyzerNode (with a "z") (pronounced "zee", not "zed") (I'll stop now). ↩︎

Starting at Recurse Center

Mon, 07 Aug 2023 00:00:00 GMT

Breaking character for a life update: I'm doing a batch at the Recurse Center!

I've been doing web development for a long time. I started making websites for myself in middle school. In college, I did a couple of web development internships, as of summer 2023 I've been a professional web developer for just over 11 years.

Most of it has been related to frontend. As time went on, I started working in other related areas, such as backend and infrastructure. I've been fortunate enough to work at companies that encouraged me to develop T-shaped skills — a deep specialization in one area and broad knowledge in others.

Hopefully, RC gives me a chance to pursue some other interests outside of that — at least, in a way that's a little more directed than just fumbling around for tutorials on Google. Not sure how often I plan to write about it while I'm there, but I do hope to make some of my work public, and you can look forward to at least a recap post when I'm done.

By the way, posts here about RC will have this link at the bottom encouraging you to join. If any of this sounds fun or interesting to you, check it out!

An Extremely Simple React Starter Kit

Fri, 21 Jul 2023 00:00:00 GMT

Let’s face it: the modern JavaScript ecosystem has a reputation for complexity. People feel like best practices and popular tools change out from underneath them, there are too many layers of dependencies and everything is perpetually on the brink of collapse.

For the record, I think that take lacks nuance. We expect more out of web apps than ever before: real-time updates and collaboration, fancy animations, interactive widgets like popovers and nested dropdown menus — and accessible and performant versions of all of the above.^[1] The ecosystem is complex because building UIs is complex, and if we push down complexity in one area it’ll probably just pop up in another.

Still, though — the point is well taken. When I want to quickly try out an idea, time spent on tooling is time wasted. I don’t want to have to deal with a bunch of configuration or read up on changes to a big framework or make sure different dependencies play well together. I want something boring that’s easy to spin up with as few moving parts as possible.

That’s why I feel compelled to write about one of my favorite web development tools: esbuild! It’s a batteries-included JS and CSS bundler made by Evan Wallace, co-founder and former CTO of Figma. Written in Go rather than JavaScript, it’s ridiculously fast, and it powers some other popular bundlers you may have heard of.

Goals

What features do we want when building a web app? Here’s my opinionated list:

A development server with live reload: I work significantly faster when the browser is already up-to-date with my latest changes by the time I look over from my text editor.
JavaScript and CSS bundling: This lets me structure files however I want, since the tooling will combine them to produce the final output that runs in the browser.
Syntax lowering: There are a lot of cool new JavaScript and CSS features, and it’s nice to be able to use them before they’re widely supported.
TypeScript support: This is a must for me, but if you don’t like TypeScript just forget I said anything!
CSS modularity: It’s convenient if the styles in different components are isolated from each other. Some people use Tailwind for this, but I don’t like it. My preferred way to solve this problem is CSS modules.
Speed: I suppose this is a nice-to-have, but development is so much more efficient (and fun!) when there’s a quick feedback loop between making a change and seeing the results.

Using something like Next.js will get you all that and a lot more — but it’s much heavier. According to NPM, Next.js has 25 dependencies, and Packagephobia reports a 170MB install size. esbuild has one dependency,^[2] and adds a comparatively tiny 9MB to your node_modules folder.

I should also mention that this list is for building a single-page app. There’s been a lot of debate around those lately, so let me reiterate: if you’re building a serious production app, you should probably use a framework that lets you render HTML on the server. This is about starting prototypes and fun projects quickly while keeping tooling to a bare minimum.

So why am I writing this now? As of a few days ago, esbuild added partial CSS module support. Before that, esbuild checked every one of those items except for CSS modularity; now, it’s got everything we need out of the box.^[3]

Project Structure

We’ll go over a barebones example React app just to showcase the different features mentioned above. Here’s what the directory structure looks like:^[4]

src/
├─ components/
│  ├─ App.module.css
│  ╰─ App.tsx
├─ index.html
├─ index.tsx
├─ livereload.js
├─ style.css
╰─ types.d.ts

index.html is pretty straightforward: it sets up a basic HTML document, links to the bundled CSS and JS files and has an empty <div> for the React app to render into. Note that even though the global stylesheet in our source code is named style.css, we’re linking to index.css — esbuild names the bundled CSS file based on the name of the corresponding JS entrypoint.

<!DOCTYPE html>
<html>
  <head>
    <link rel="stylesheet" href="/index.css" />
  </head>
  <body>
    <div id="app"></div>
    <script src="/index.js"></script>
  </body>
</html>

index.tsx is similarly spartan. It has two jobs: import any global styles, and render the root React component.

import "./style.css";

import { createRoot } from "react-dom/client";

import App from "./components/App";

const app = document.querySelector("#app");
if (app) createRoot(app).render(<App />);

Global styles go in style.css. The selectors in there won’t be changed, so that’s a good place to put things that apply to the whole project: fonts, CSS reset, custom properties, base styles, etc. You can also @import other stylesheets and they'll be bundled along with it:

@import "./reset.css";

body {
  font-family: sans-serif;
}

App.tsx is the root component. The project layout at this point is kinda arbitrary; this is mostly a contrived example to show off CSS modules.

import css from "./App.module.css";

export default function App() {
  return <h1 className={css.wrapper}>Hello, world!</h1>;
}

Speaking of CSS modules, did you notice types.d.ts? That’s there because TypeScript doesn’t know how to type check .module.css files (so you can skip this section if you’re not using TypeScript). You need to tell it the type of the import:

declare module "*.module.css" {
  const map: { [key: string]: string };
  export default map;
}

Finally, there’s livereload.js, which will only be included in development. It (surprise!) reloads the page whenever the esbuild server rebuilds any files. This snippet is straight from the esbuild documentation on live reloading:

new EventSource("/esbuild").addEventListener("change", () => location.reload());

Implementation

And now the big reveal: we can get the whole development environment with one dependency,^[5] two shell commands and zero config files!

Here’s the command for the dev server:

esbuild src/index.html src/index.tsx \
  --loader:.html=copy \
  --outdir=build --bundle --watch \
  --servedir=build --serve-fallback=src/index.html \
  --inject:src/livereload.js

This builds atop my TIL on using esbuild to run a dev server with live reload. Let’s go through what each line does:

esbuild src/index.html src/index.tsx runs esbuild with src/index.html and src/index.tsx as entrypoints. We don’t need to specify any CSS files because esbuild will gather them as they’re imported into JS files.
--loader:.html=copy‌ tells esbuild to copy files ending in .html unchanged to the build folder. This also gets triggered when the HTML file changes.
--outdir=build --bundle tells esbuild to bundle all the files and place them in the build folder.

Remember those flags, because we’ll use them both when developing locally and building for production. The rest of the flags are specific to development:

--watch tells esbuild to rebuild when any source files change.
--servedir=build serves all static files from the build folder
--serve-fallback=src/index.html serves src/index.html instead of a 404 page (useful for client-side routing)
--inject:src/livereload.js appends src/livereload.js to the end of the JS bundle.

Here’s how to build for production:

esbuild src/index.html src/index.tsx \
  --loader:.html=copy \
  --outdir=build --bundle --minify

It’s mostly the same! There are only two differences: the last two lines with the development server flags are gone, and --watch has been replaced with --minify.

If these commands seem kinda gnarly, remember that we don’t have to type out them every single time. They can go in the scripts section of our package.json file. I’ve consolidated this one a bit by moving the common flags into an esbuild script, and having start and build scripts add the extra flags at the end.

{
  "scripts": {
    "esbuild": "esbuild src/index.html src/index.tsx --loader:.html=copy --outdir=build --bundle",
    "start": "npm run esbuild -- --watch --servedir=build --serve-fallback=src/index.html --inject:src/livereload.js",
    "build": "npm run esbuild -- --minify"
  }
}

Extra Credit

Okay, so we’ve covered our goals. But esbuild can do a lot more. Here are some other flags you might think of enabling:

--sourcemap makes debugging easier by exporting .js.map and .css.map files that tell the browser how your bundled code relates to your source code.
--loader lets you import different file types from JS and CSS. A particularly useful option is the file loader, which copies the file to the build directory and embeds the file name as a string. For example, to be able to use url() to load .woff2 files from a stylesheet, you’d add --loader:.woff2=file.
--jsx-factory lets you change how JSX is compiled into function calls if you’d rather replace React with a different library like Preact.
This isn’t a flag, but in your tsconfig.json (or jsconfig.json if you don’t use TypeScript) you can set paths to { "~/*": ["./src/*"] } in order to use ~ to reference your files relative to the source root. For example, you’d import src/components/App.tsx with import App from "~/components/App".

That's it! Happy hacking! If you come up with any cool additions to this, let me know!

Updates

Originally, this post recommended --loader:.module.css=local-css in order to use the local-css loader for .module.css files. As of esbuild v0.19.0, that’s the default behavior, so that flag is no longer necessary!

The W3C has started chipping away at some of these. The new <dialog> element is a native way to build accessible modals, CSS anchor positioning looks to make things like popovers significantly easier and the View Transition API will let us animate page navigation with just HTML and CSS. Among others! ↩︎
Technically, NPM lists 22 dependencies of esbuild, which is misleading — they’re all first-party shims for esbuild’s own native binaries, so from a JavaScript perspective you could say esbuild has zero! But esbuild is written in Go and although NPM can’t inspect it, if you look in the go.mod file you can see that the entire project only uses a single third-party dependency. ↩︎
It was actually possible to get CSS modules before with esbuild plugins, such as this one that uses Devon Govett’s excellent Lightning CSS. But every extra dependency and config file adds friction, which is why I try to minimize them when building something simple. ↩︎
This example assumes we’re using TypeScript. If you don’t want to use TypeScript, just change the .tsx extensions to .jsx and delete that types.d.ts file. ↩︎
It probably goes without saying, but that one dependency is esbuild. ↩︎

Were React Hooks a Mistake?

Sun, 05 Mar 2023 00:00:00 GMT

The web dev community has spent the past few weeks buzzing about signals, a reactive programming pattern that enables very efficient UI updates. Devon Govett wrote a thought-provoking Twitter thread about signals and mutable state. Ryan Carniato responded with an excellent article comparing signals with React. Good arguments on all sides; the discussion has been really interesting to watch.

One thing the discourse makes clear is that there are a lot of people for whom the React programming model just does not click. Why is that?

I think the issue is that people's mental model of components doesn’t match how function components with hooks work in React. I’m going to make an audacious claim: people like signals because signal-based components are far more similar to class components than to function components with hooks.

Let’s rewind a bit. React components used to look like this:^[1]

class Game extends React.Component {
  state = { count: 0, started: false };

  increment() {
    this.setState({ count: this.state.count + 1 });
  }

  start() {
    if (!this.state.started) setTimeout(() => alert(`Your score was ${this.state.count}!`), 5000);
    this.setState({ started: true });
  }

  render() {
    return (
      <button
        onClick={() => {
          this.increment();
          this.start();
        }}
      >
        {this.state.started ? "Current score: " + this.state.count : "Start"}
      </button>
    );
  }
}

Each component was an instance of the class React.Component. State was kept in the property state, and callbacks were just methods on the instance. When React needed to render a component, it would call the render method.

You can still write components like this. The syntax hasn’t been removed. But back in 2015, React introduced something new: stateless function components.

function CounterButton({ started, count, onClick }) {
  return <button onClick={onClick}>{started ? "Current score: " + count : "Start"}</button>;
}

class Game extends React.Component {
  state = { count: 0, started: false };

  increment() {
    this.setState({ count: this.state.count + 1 });
  }

  start() {
    if (!this.state.started) setTimeout(() => alert(`Your score was ${this.state.count}!`), 5000);
    this.setState({ started: true });
  }

  render() {
    return (
      <CounterButton
        started={this.state.started}
        count={this.state.count}
        onClick={() => {
          this.increment();
          this.start();
        }}
      />
    );
  }
}

At the time, there was no way to add state to these components — it had to be kept in class components and passed in as props. The idea was that most of your components would be stateless, powered by a few stateful components near the top of the tree.

When it came to writing the class components, though, things were… awkward. Composition of stateful logic was particularly tricky. Say you needed multiple different classes to listen for window resize events. How would you do that without rewriting the same instance methods in each one? What if you needed them to interact with the component state? React tried to solve this problem with mixins, but they were quickly deprecated once the team realized the drawbacks.

Also, people really liked function components! There were even libraries for adding state to them. So perhaps it’s not surprising that React came up with a built-in solution: hooks.

function Game() {
  const [count, setCount] = useState(0);
  const [started, setStarted] = useState(false);

  function increment() {
    setCount(count + 1);
  }

  function start() {
    if (!started) setTimeout(() => alert(`Your score was ${count}!`), 5000);
    setStarted(true);
  }

  return (
    <button
      onClick={() => {
        increment();
        start();
      }}
    >
      {started ? "Current score: " + count : "Start"}
    </button>
  );
}

When I first tried them out, hooks were a revelation. They really did make it easy to encapsulate behavior and reuse stateful logic. I jumped in headfirst; the only class components I’ve written since then have been error boundaries.

That said — although at first glance this component works the same as the class component above, there’s an important difference. Maybe you've spotted it already: your score in the UI will be updated, but when the alert shows up it’ll always show you 0. Because setTimeout only happens in the first call to start, it closes over the initial count value and that’s all it’ll ever see.

You might think you could fix this with useEffect:

function Game() {
  const [count, setCount] = useState(0);
  const [started, setStarted] = useState(false);

  function increment() {
    setCount(count + 1);
  }

  function start() {
    setStarted(true);
  }

  useEffect(() => {
    if (started) {
      const timeout = setTimeout(() => alert(`Your score is ${count}!`), 5000);
      return () => clearTimeout(timeout);
    }
  }, [count, started]);

  return (
    <button
      onClick={() => {
        increment();
        start();
      }}
    >
      {started ? "Current score: " + count : "Start"}
    </button>
  );
}

This alert will show the correct count. But there’s a new issue: if you keep clicking, the game will never end! To prevent the effect function closure from getting “stale”, we add count and started to the dependency array. Whenever they change, we get a new effect function that sees the updated values. But that new effect also sets a new timeout. Every time you click the button, you get a fresh five seconds before the alert shows up.

In a class component, methods always have access to the most up-to-date state because they have a stable reference to the class instance. But in a function component, every render creates new callbacks that close over its own state. Each time the function is called, it gets its own closure. Future renders can’t change the state of past ones.

Put another way: class components have a single instance per mounted component, but function components have multiple “instances” — one per render. Hooks just further entrench that constraint. It’s the source of all your problems with them:

Each render creates its own callbacks, which means anything that checks referential equality before running side effects — useEffect and its siblings — will get triggered too often.
Callbacks close over the state and props from their render, which means callbacks that persist between renders — because of useCallback, asynchronous operations, timeouts, etc — will access stale data.

React gives you an escape hatch to deal with this: useRef, a mutable object that keeps a stable identity between renders. I think of it as a way to teleport values back and forth between different instances of the same mounted component. With that in mind, here’s what a working version of our game might look like using hooks:

function Game() {
  const [count, setCount] = useState(0);
  const [started, setStarted] = useState(false);
  const countRef = useRef(count);

  function increment() {
    setCount(count + 1);
    countRef.current = count + 1;
  }

  function start() {
    if (!started) setTimeout(() => alert(`Your score was ${countRef.current}!`), 5000);
    setStarted(true);
  }

  return (
    <button
      onClick={() => {
        increment();
        start();
      }}
    >
      {started ? "Current score: " + count : "Start"}
    </button>
  );
}

It’s pretty kludgy! We’re now tracking the count in two different places, and our increment function has to update them both. The reason that it works is that every start closure has access to the same countRef; when we mutate it in one, all the others can see the mutated value as well. But we can’t get rid of useState and only rely on useRef, because changing a ref doesn’t cause React to re-render. We’re stuck between two different worlds — the immutable state that we use to update the UI, and the mutable ref with the current state.

Class components don’t have this drawback. The fact that each mounted component is an instance of the class gives us a kind of built-in ref. Hooks gave us a much better primitive for composing stateful logic, but it came at a cost.

Although they’re not new, signals have experienced a recent explosion in popularity, and seem to be used by most major frameworks other than React.

The usual pitch is that they enable “fine-grained reactivity”. That means when state changes, they only update the specific pieces of the DOM that depend on it. For now, that usually ends up being faster than React, which recreates the full VDOM tree before discarding the parts that don’t change. But to me, these are all implementation details. People aren’t switching to these frameworks just for the performance. They’re switching because these frameworks offer a fundamentally different programming model.

If we rewrite our little counter game with Solid, for example, we get something that looks like this:

function Game() {
  const [count, setCount] = createSignal(0);
  const [started, setStarted] = createSignal(false);

  function increment() {
    setCount(count() + 1);
  }

  function start() {
    if (!started()) setTimeout(() => alert(`Your score was ${count()}!`), 5000);
    setStarted(true);
  }

  return (
    <button
      onClick={() => {
        increment();
        start();
      }}
    >
      {started() ? "Current score: " + count() : "Start"}
    </button>
  );
}

It looks almost identical to the first hooks version! The only visible differences are that we’re calling createSignal instead of useState, and that count and started are functions we call whenever we want to access the value. As with class and function components, though, the appearance of similarity belies an important difference.

The key with Solid and other signal-based frameworks is that the component is only run once, and the framework sets up a data structure that automatically updates the DOM when signals change. Only running the component once means we only have one closure. Having only one closure gives us a stable instance per mounted component again, because closures are equivalent to classes.

What?

It’s true!^[2] Fundamentally, they’re both just bundles of data and behavior. Closures are primarily behavior (the function call) with associated data (closed over variables), while classes are primarily data (the instance properties) with associated behavior (methods). If you really wanted to, you could write either one in terms of the other.

Think about it. With class components…

The constructor sets up everything the component needs to render (setting initial state, binding instance methods, etc).
When you update the state, React mutates the class instance, calls the render method and makes any necessary changes to the DOM.
All functions have access to up-to-date state stored on the class instance.

Whereas with signals components…

The function body sets up everything the component needs to render (setting up the data flow, creating DOM nodes, etc).
When you update a signal, the framework mutates the stored value, runs any dependent signals and makes any necessary changes to the DOM.
All functions have access to up-to-date state stored in the function closure.

From this point of view, it’s a little easier to see the tradeoffs. Like classes, signals are mutable. That might seem a little weird. After all, the Solid component isn’t assigning anything — it’s calling setCount, just like React! But remember that count isn’t a value itself — it’s a function that returns the current state of the signal. When setCount is called, it mutates the signal, and further calls to count() will return the new value.

Although Solid's createSignal looks like React's useState, signals are really more like refs: stable references to mutable objects. The difference is that in React, which is built around immutability, refs are an escape hatch that has no effect on rendering. But frameworks like Solid put signals front and center. Rather than ignoring them, the framework reacts when they change, updating only the specific parts of the DOM that use their values.

The big consequence of this is that the UI is no longer a pure function of state. That's why React embraces immutability: it guarantees that the state and UI are consistent. When mutations are introduced, you also need a way to keep the UI in sync. Signals promise to be a reliable way to do that, and their success will hinge on their ability to deliver on that promise.

To recap:

First we had class components, which kept state in a single instance shared between renders.
Then we had function components with hooks, in which each render had its own isolated instance and state.
Now we’re swinging toward signals, which keep state in a single instance again.

So were React hooks a mistake? They definitely made it easier to break up components and reuse stateful logic.^[3] Even as I type this, if you were to ask me whether I’ll be abandoning hooks and returning to class components, I’d tell you no.

At the same time, it’s not lost on me that the appeal of signals is regaining what we already had with class components. React made a big bet on immutability, but people have been looking for ways to have their cake and eat it too for a while now. That’s why libraries like immer and MobX exist: it turns out that the ergonomics of working with mutable data can be really convenient.

People seem to like the aesthetics of function components and hooks, though, and you can see their influence in newer frameworks. Solid’s createSignal is almost identical to React’s useState. Preact’s useSignal is similar as well. It’s hard to imagine that these APIs would look like they do without React having lead the way.

Are signals better than hooks? I don’t think that’s the right question. Everything has tradeoffs, and we’re pretty sure about the tradeoffs signals make: they give up immutability and UI as a pure function of state, in exchange for better update performance and a stable, mutable instance per mounted component.

Time will tell whether signals will bring back the problems React was created to fix. But right now, frameworks seem to be trying to strike a sweet spot between the composability of hooks and the stability of classes. At the very least, that’s an option worth exploring.

Even before this, we had React.createClass. We don’t speak of those days. ↩︎
Technically, they’re equivalent to class instances — AKA objects. Some people say “closures are a poor man’s objects”, and vice versa. ↩︎
Although I do wonder what React might have looked like if it leaned further into the class paradigm and implemented something like the component pattern in game development. ↩︎

Pathetic

Wed, 15 Feb 2023 00:00:00 GMT

There’s been a lot of drama in the web development community lately. Most of it is centered around JavaScript, and more specifically React. That’s not unusual, but what seems different right now is the amount of toxicity that’s been injected into the discourse.

I don’t really want to name names, because I’m not trying to shame anyone in particular. But all the examples I’m about to bring up are from prominent voices in the community. You’ve seen their faces. You've gotten their hot takes in your feeds.

One developer wrote a reasonable blog post saying, essentially, that developers are trying to make the best tradeoffs they can, in response to another post sharply criticizing React. The author of the original post — another prominent developer — found the response and accused its author of “white knighting” for Facebook and Vercel.

Another developer wrote a post about how most of the web actually runs on Wordpress and not the JavaScript frameworks that dominate the conversation. Fair enough! Except they also went out of their way to call the fact that only 4% of the top ten million sites use React pathetic (emphasis theirs).

When called out on using that word, they justified it by saying that React has been overhyped by unpleasant people. Earlier the same day, they'd posted on Mastodon about a “weird Mormon dude” and his take on TypeScript. Who's supposed to be unpleasant here?

A popular React influencer told someone they were trying to make themselves feel better about being left behind. It's like a crypto bro telling someone who won’t get on board to “have fun staying poor”. We don’t need to import that attitude into the web development community.

Most alarmingly, someone said in a discussion that a member of the React team contacted their boss over a critical tweet they made. I don’t even have words for how not okay that is. What the actual fuck.

Can we just… not?

Something is very wrong when your feelings about a JavaScript framework lead you to insult someone’s religion, or to contact their boss.

Again, these aren’t random trolls on Twitter. They’re influential accounts who have hundreds of thousands of followers between them. They’re the “thought leaders” of our community. And right now, they're causing a ton of toxicity.

You know what will hurt web development the most? What will discourage beginners and prevent us all from building better things? This right here. No one wants to feel like they’re doing something morally wrong by choosing a framework, or like they’re being left behind if they don’t.

I don’t really have a conclusion. Be good to each other, I suppose. We’re all just out here trying our best.

In Defense of Cargo Culting

Wed, 04 Jan 2023 00:00:00 GMT

In discussions on tech forums, you’ll sometimes hear people refer to cargo cults. The term was originally coined to describe indigenous tribes who came into contact with more technologically advanced colonizers (most famously, the US military in the South Pacific during World War II). After the colonizers left, the tribes would try to mimic their "rituals", building airstrips and marching in formation in a futile attempt to attract more of the same supplies the colonizers summoned from the sky.^[1]

With regard to software, cargo cult programming (or just "cargo culting") is a similarly pejorative term that describes someone who uses a technology or follows a pattern without understanding why. Often, it's related to over-engineering: building software to handle a scale the developers anticipate, rather than the scale they're actually at. Another common accusation is that developers mindlessly mimic large tech companies, because if something works for Google or Facebook then it must work for them too.^[2]

There's some truth to all that, but the discussion tends to myopically focus on how well a particular tool fits a problem's engineering constraints.

Popularity

Cargo cults tend to form around popular or trendy tools. Developers see people around them using something and conclude that there must be a good reason, so they should use it as well. Or so the thinking goes.

Some people choose based on the number of job opportunities a tool affords (“résumé-driven development”) and while that gets a bad rap, it’s a perfectly rational way to choose a stack for a personal project.^[3] On the flip side, for companies hiring people to write code, the ability to find developers who are already familiar with their stack might be more important than picking the absolute best tools for the job.

There are a lot of advantages to using popular tools! There’s probably documentation, tutorials and Stack Overflow answers readily available. Practices have been tried out and established. Plugins have been developed. Bugs have been found and fixed. If you're working on a team, it's much easier to arrive at a shared understanding of what's idiomatic, which patterns to use and what pitfalls to avoid. It can also help when onboarding new members, who can rely on an external community rather than the (lack of) documentation for a bespoke tool or niche framework.

One important consideration here is whether something is is popular or merely trendy. A decent metric for this is longevity: if it’s been already around for a while, then it’ll probably stick around for a while longer. By picking a tool that a lot of people have used for a long time, you can rest assured that if you run into an issue, someone else will have run into it before you, and (hopefully) will have also written words on the internet about how to fix it.

Inexperience

Another reason people rely on popularity is inexperience: it’s not clear to them what the actual best solution is, so they just pick one and get to work.

This is pretty common! Most of us have one or maybe two areas of deep expertise. But many projects we take on will involve new domains, or unexplored corners of familiar ones. Often, we specifically seek out such projects, since solving the puzzle sharpens our skills. This means we’re constantly being placed in situations where we have to make technology choices without fully understanding the constraints or the consequences.

But before we develop that domain knowledge, we still have a job to do. At that stage, cargo culting is a reasonable way to make a decision. We don’t have the experience to make an educated choice on our own, so we turn to the wisdom of the crowd. A more charitable way to describe this sort of cargo culting is “following best practices”.

Knowledge Reuse

Conversely, people sometimes use tools they know well to solve problems for which they’re ill-suited.^[4] This is the same sort of pragmatism that leads people to choose unfamiliar tools: they have a job to do, and getting it done quickly (or at all) is better than getting it done perfectly, eventually.

In Choose Boring Technology, Dan McKinley uses "innovation tokens" as a metaphor:

Let’s say every company gets about three innovation tokens. You can spend these however you want, but the supply is fixed for a long while. You might get a few more after you achieve a certain level of stability and maturity, but the general tendency is to overestimate the contents of your wallet. Clearly this model is approximate, but I think it helps.

If you choose to write your website in NodeJS, you just spent one of your innovation tokens. If you choose to use MongoDB, you just spent one of your innovation tokens. If you choose to use service discovery tech that’s existed for a year or less, you just spent one of your innovation tokens. If you choose to write your own database, oh god, you’re in trouble.

The thing is, you don't just spend your tokens on technologies that are new and shiny. You pay a price whenever you use something with a lot of unknowns to you, personally. Sometimes, that's unavoidable — if you're building a web app, you probably need a database even if you don't know the first thing about SQL — but if you can avoid it, you should.

In practice, this means reducing risk by choosing tools you know well. This is why so many people tend to use the same stacks they use at work for their side projects: sure, it might be overkill, but they already know exactly how to use it.

Ultimately, people choose technologies for a lot of reasons. From afar, it's easy to judge technology choices based on how well they fit the engineering problem. But "the problem" is usually multidimensional, and the engineering dimension not always the most important.

I wish there were a less colonial name for this. Anthropologists now argue that the term and its connotation says more about Western ideology than the people it allegedly describes. ↩︎
Some examples, like using Kubernetes for a blog or React for a marketing website, have basically become memes. ↩︎
Side projects should be entirely exempt from the cargo culting discussion. Why do you care about the stack of this app I made for fun? If you want me to build something in a specific way, hire me and pay me money. ↩︎
Technically, this isn’t cargo culting, but the resulting discourse tends to take the same shape so I’m including it here anyway. ↩︎

Integrating My Blog with Mastodon

Fri, 30 Dec 2022 00:00:00 GMT

Decentralization is cool again! Twitter is burning and people are fleeing to greener pastures. There are a bunch of alternatives, but the most promising one is Mastodon — not because of any particular feature, but because it’s built on an open standard called ActivityPub.

ActivityPub is a protocol: a bunch of conventions that apps can use to talk to each other. That means that if you build an app that understands ActivityPub, any other app that also understands it can communicate with it. Technically, this is called federation, not decentralization, and the ecosystem of apps that work together this way is lovingly referred to as the Fediverse.

I decided to dip my toes in and see how ActivityPub works. What I built is pretty simple: a tiny server that can interact with the rest of the Fediverse, plus a GitHub Actions workflow that posts whenever I publish a new blog post.

The server I made is called ActivityPub Starter Kit. It’s built with NodeJS and stores its data in SQLite. To make hacking on it easier, I also made a simpler version on Glitch. As soon as you remix it, you get an account that you can follow from any Fediverse app — no extra setup required!

In the rest of this blog post, I’ll show how easy it is to set up an integration between your blog and the Fediverse. We’ll assume that we’re using the Glitch app with an account called jake.

So: ActivityPub Starter Kit. It’s a very basic ActivityPub server with only a few endpoints. There are two that matter to us:^[1]

/jake/outbox is the Outbox for our account (if you’re following along at home, replace jake with whatever account name you choose). The response contains a list of posts in reverse chronological order. We’ll use this to check which blog posts have already been posted to the Fediverse.
/admin/create adds a new post to the Outbox and notifies all our followers that we’ve.^[2]

My blog is built on a static site generator that outputs an RSS feed. So there are four basic steps here:

Get the latest post from the RSS feed.
Get the latest post from the ActivityPub Outbox.
Compare the URLs.
If they don’t match, create a new ActivityPub post and notify followers.

There are only two libraries we need for this: node-fetch (to make HTTP requests more easily) and fast-xml-parser (to parse the RSS feed into JSON).

import { XMLParser } from "fast-xml-parser";
import fetch from "node-fetch";

There are a few variables we’ll need that we may as well define now:

const RSS_URL = "https://jakelazaroff.com/rss.xml";
const ACTIVITYPUB_HOST = "https://shine-thunder-forgery.glitch.me";
const OUTBOX_URL = ACTIVITYPUB_HOST + "/jake/outbox";
const CREATE_URL = ACTIVITYPUB_HOST + "/admin/create";
const ADMIN_USERNAME = "adminuser";
const ADMIN_PASSWORD = "secretpassword!";

RSS_URL is the URL of the RSS feed with the blog posts.
ACTIVITYPUB_HOST is the URL of the ActivityPub server (just the protocol and hostname, so we can build our other URLs).
OUTBOX_URL is the URL of the Outbox.
CREATE_URL is the URL of the endpoint we’ll use to actually create the post.
ADMIN_USERNAME and ADMIN_PASSWORD correspond to environment variables we can set in the server to prevent randos from creating posts.

Okay, onto the main event. First, we’ll fetch the RSS feed and get the latest blog post:

console.log(`Fetching RSS feed from ${RSS_URL}…`);
const rss = await fetch(RSS_URL);
const feed = new XMLParser().parse(await rss.text());
const latest = feed.rss.channel.item[0];

Next, we’ll fetch the ActivityPub feed and get the latest post there:

console.log(`Fetching ActivityPub feed from ${OUTBOX_URL}…`);
const outbox = await fetch(OUTBOX_URL);
const posts = await outbox.json();
const post = posts.orderedItems[0];

Now that we have the latest post from each, we can check to see if their URLs match:

if (latest.link === post.object?.url) console.log("No new post in feed.");

If they don’t match, we’ll make a new post in the ActivityPub feed:

else {
  console.log("Posting to ActivityPub feed…");

Before we can send the request, we need to calculate the Basic authentication header. That’s just the string Basic username:password, encoded in base 64:

const authorization = "Basic " + Buffer.from(ADMIN_USERNAME + ":" + ADMIN_PASSWORD).toString("base64");

We also need to create the request body. ActivityPub supports a bunch of different types of "Objects", but since this is a blog, the posts will all be of type Article. We’ll supply title, summary, content, url and published with data from the RSS feed:

const date = new Date(latest.pubDate);
const body = JSON.stringify({
  object: {
    type: "Article",
    url: latest.link,
    title: latest.title,
    summary: latest.description,
    content: latest["content:encoded"],
    published: date.toISOString()
  }
});

You’ll also notice that we’re nesting everything inside an object property. That’s because each item in the Outbox is actually a Create Activity that contains an Object.^[3] That Activity has its own set of properties, and we can override them by including them in the request body.

For example, if we wanted to set the ActivityPub post’s date to be the same as the blog post, we could add published to the top level alongside object, like so:

const date = new Date(latest.pubDate);
const body = JSON.stringify({
  published: date.toISOString(),
  object: {
    type: "Article",
    url: latest.link,
    title: latest.title,
    summary: latest.description,
    content: latest["content:encoded"],
    published: date.toISOString()
  }
});

After we have the body, all that’s left is to send off the request:

  const res = await fetch(CREATE_URL, {
    method: "POST",
    body,
    headers: { authorization, "content-type": "application/json" },
  });

  if (!res.ok) throw new Error(`Got ${res.status} ${res.statusText} when creating post.`);
}

And that’s it! People can now follow your blog in the Fediverse.

Here’s the whole script. It’s short — barely 40 lines. I included my full GitHub Actions deploy workflow in there, in case the context helps.

And of course, this is just one example of what to build. You could have your ActivityPub server follow people and turn their posts into a reading list or a blogroll. You could make a bot that looks up information, or uses ChatGPT to write poems for people. Check out ActivityPub Starter Kit and make something cool or quirky or fun!

Technically, there’s also a third: /jake/inbox, our account’s Inbox. We don’t ever make requests to our own Inbox, though — other servers will do that when they want to notify us of something. When we make a new post, ActivityPub Starter Kit will go through each follower and make a request to their Inbox. ↩︎
/admin/create is not, strictly speaking, an ActivityPub endpoint, but it does follow the client-to-server interaction guidance on Object creation without the Create Activity. ↩︎
In ActivityPub parlance, an Actor (the account) performs an Activity (some sort of action) on an Object (a thing). Darius Kazemi has a good blog post breaking this down a bit more. ↩︎

A React Developer’s First Take on Solid

Wed, 21 Dec 2022 00:00:00 GMT

When I start a new project, I tend to rely on tools I already know well. I have a vision in my head, and learning a new technology as I try to build something is a recipe for getting nothing done. Usually, for projects on the web, that means picking React.

That said, I do keep an eye on the JavaScript ecosystem as a whole, and one of the frameworks that has piqued my interest for a while is Solid. It’s a reactive JavaScript frameworks, which basically means it models your app as streams of data and automatically updates the UI when a dependency changes. That might sound abstract and weird, but we’ll unpack it in a bit.

I recently started a project involving server-side rendering. While looking into technologies to use,^[1] I discovered that the Solid team had recently released Solid Start — a Next.js-esque “meta-framework” that makes the experience of working with Solid easier and more seamless.

I haven’t gotten a chance to work with it in depth yet, but I have used a decent amount of what it has to offer. So far, I’m really enjoying it. It’s similar enough to React that there’s not a steep learning curve, and some of the features seem genuinely transformative.

There are two parts to this post. First, my impression of the Solid framework itself; and second, my impression of Solid Start.

So, without further ado: a React developer’s first take on Solid.

The Solid Framework

The creator of Solid, Ryan Carniato, once quoted someone saying “Svelte is to Vue as Solid is to React”. That rings true to me. It’s not “just JavaScript” in the same way React is — if you look at the compiled code, it’s definitely doing more than just de-sugaring JSX into function calls. But the code you’re writing is JavaScript, not a language that’s bespoke to the framework.

Here’s what a simple component might look like in Solid:

import { render } from "solid-js/web";
import { createSignal } from "solid-js";

function Greeter() {
  const [name, setName] = createSignal("Jake");

  const yelling = () => name().toUpperCase();

  return (
    <>
      <p class="greeting">Hello, {yelling()}!</p>
      <input
        value={name()}
        onChange={event => {
          setName(event.currentTarget.value);
        }}
      />
    </>
  );
}

render(() => <Greeter />, document.getElementById("app"));

You could be forgiven for thinking you’re looking at a React component. From a syntax point of view, the only difference at this point is the use of class instead of className.^[2]

Under the hood, however, they’re very different. In React, a component function gets called every render, and React modifies the DOM based on the difference between the return values. In Solid, a component function gets called once over the entire lifetime of the component, and Solid only updates the DOM for each property that changes.

Let’s break apart the example. There are four relevant lines:

const [name, setName] = createSignal("Jake"); looks like React’s useState, but with one key difference: name is not the value itself, but a function that returns the current value. So in this case, you could call name() and it would return "Jake".
const yelling = () => name().toUpperCase(); is where it becomes important that name is a function, rather than a primitive value. Any time yelling is called, it gets the current value, not just the value when the component was run. Under the hood, Solid remembers that the return value of yelling depends on name.
Hello, {yelling()}! is the trickiest part, because JSX in Solid works slightly different than in React. Rather than just rendering this immediately, Solid remembers that the text of this  element depends on the result of the yelling function. Then, when that function’s dependencies change, Solid figures out what actually changed and updates the DOM again.
setName(event.currentTarget.value); works roughly the same as in React. Calling it updates the value of the name signal, which causes Solid to re-evaluate anything that depends on it — in this case, the yelling function and the text inside the  tag.

Essentially, what the component does is set up a pipeline from createSignal to the DOM.^[3] When you call render, Solid remembers how that whole pipeline works. Then, when the value of the signal changes — in this case, by calling setName — Solid runs only that pipeline, not bothering with any other pipelines that it knows haven’t changed.

I know that’s kinda hard to visualize. If you’re interested, Ryan has a good talk that goes in-depth into how this works. In practical terms, though, it doesn’t have a huge impact on the developer experience. There are a couple gotchas to keep in mind, but for the most part you can write your code as if you were building a React app and it just… works.

Okay, so what are the gotchas? One, no destructuring props. Solid does some fancy Proxy stuff to track property access that gets messed up if you do.

// wrong
function Greeter({ name }) {
  return <p>Hello, {name}!</p>;
}

// right
function Greeter(props) {
  return <p>Hello, {props.name}!</p>;
}

Two, because component functions only run once, you need to wrap any derived values in their own functions. And three (or I guess two and a half) you use special components for looping and showing/hiding, rather than maps and ternaries.

// wrong
function TodoList(props) {
  const important = props.todos.map(todo => todo.toUpperCase());

  return (
    <ul>
      {important.map(todo => (
        <li>{todo}</li>
      ))}
    </ul>
  );
}

// right
function TodoList(props) {
  const important = () => props.todos.map(todo => todo.toUpperCase());

  return (
    <ul>
      <For each={important()}>{todo => <li>{todo}</li>}</For>
    </ul>
  );
}

Again, these are pretty small drawbacks. They’re kind of like React’s Rules of Hooks, in that you might initially think it’s weird to need to call functions in the same order each time until you realize that you already write most of your code like this anyway.

Let’s look at one more example — the infamous counter app:

function Counter() {
  const [count, setCount] = createSignal(0);

  console.log("component", count());
  createEffect(() => {
    console.log("effect", count());
  });

  return (
    <div>
      <Count count={count()} />
      <Button onIncrement={() => setCount(count => count + 1)} />
    </div>
  );
}

function Count(props) {
  return <p>{props.count}</p>;
}

function Button(props) {
  return (
    <button type="button" onClick={props.onIncrement}>
      Increment
    </button>
  );
}

Again, very similar to React. createSignal plays the role of useState, and the API is almost identical except that the "state" variable count is a function that returns the current value. You’ll also notice I put a console.log inside the Counter component — once in the body itself, and once in createEffect, which is Solid’s version of useEffect.

Solid knows that there’s a "pipeline" between that count variable and the  tag, it and updates the DOM automatically when the value changes. It also knows that there’s another pipeline that from count to createEffect, so it reruns the effect.

Here’s a link to a live version of this code, if it helps. If you run it and click the button a few times, you’ll see that the “component” log doesn’t happen when you click; these functions are, indeed, only called once. The "effect" log, on the other hand, happens every time the count is updated. You should see something like this in the console:

component 0
effect 0
effect 1
effect 2
effect 3

All this is nice, but what’s the advantage over React? For one, it’s small, although that makes less of a difference the larger your app gets. There are marginal developer experience improvements, such as not having to track createEffect dependencies. And finally: because it’s only updating the exact things that change, it’s also really fast.

Solid Start

If Solid is React, then Solid Start is Next.js. The Solid Start website cautions that it’s in beta, and they’re definitely not lying. But it also has some awesome features that seem genuinely transformative.

First, it has the file system routing that’s become bog standard amongst frontend frameworks these days. The default styling solution is CSS modules. It supports both server-side rendering and client-side rendering. There are provided components for managing head tags.

Let’s talk about loading data. Traditionally, single-page apps are split into two layers: a JavaScript frontend that runs in the browser, and a separate REST(ish) API that runs on the server, communicating with the client using JSON. Lately, frameworks like Next.js have begun combining the two, allowing developers to create “API routes” in the same codebase as their user-facing components.

Solid Start’s answer to this is route data — a Remix-inspired feature that makes it super easy to load data, both on the server and client. If you export a function called routeData from a page, calling the hook useRouteData within the component will give you the data you returned from routeData. The secret sauce is that Solid takes care of wiring up the HTTP request and getting the response back to your component:

export function routeData() {
  const [todos] = createResource(async () => {
    // here’s where you’d load data if it weren’t hardcoded as an example
    return ["brush teeth", "get milk", "publish blog post"];
  });

  return { todos };
}

export default function Page() {
  const { todos } = useRouteData();

  return (
    <ul>
      <For each={todos()}>{todo => <li>{todo.text}</li>}</For>
    </ul>
  );
}

That’s for reading data. For writing, there are route actions. They’re pretty similar, except you call them from within your component code and wire them up to your JSX, like a hook. They even return a <Form> component, which makes it super easy to create progressively enhanced HTML forms that work without JavaScript:

export default function Page() {
  const [name, { Form }] = createRouteAction(async data => {
    // here’s where you’d do some stuff with the data
    return data.get("name");
  });

  <>
    <Form>
      <label>
        Name
        <input name="name" />
      </label>
      <button>Submit</button>
    </Form>
    <p>Hello, {name.result}!</p>
  </>;
}

I won’t bury the lede: it’s pretty magical. Solid Start even includes functions createServerData$ and createServerAction$ for operations that should only happen on the server. You can make database queries in the same file as your components, and Solid will not only render the markup on the server when the page first loads, but also request new data and update the DOM whenever any of the parameters change. It’s way faster and less boilerplate-y than manually writing new REST endpoints or GraphQL resolvers.

Frankly, I’m a little skeptical of how well this scales. As apps grow, they tend to require additional complexity. Things like authorization and rate limiting. Although the Solid Start documentation does include an example of how to implement sessions, my gut says that bigger apps will still benefit from a separate “traditional” API. But for starting out quickly, or for apps that are likely to remain small, such as prototypes or side projects, this is perfect.

Solid Start does offer the API routes found in Next.js as well, though. In files within the routes directory, you can export functions named after HTTP verbs to respond to requests of that type. So, in case you do end up needing to scale your API beyond what route data and route actions can achieve, Solid Start gives you tools to migrate gradually.

All that said, Solid Start is definitely beta software. Some of the rough edges I’m hoping get fixed before the 1.0 release:

When rendering HTML on the server, Solid Start adds a bunch of weird data-hk attributes to all the elements.
Some naming conventions aren’t really clear, and the documentation doesn’t explain them. For example, some functions use $ as a suffix. My guess is that’s for functions that run on the server, but they all have "server" in the name anyway, so it seems redundant. Likewise, I’m not sure what the difference is between functions prefixed with use and create.
createServerData$ doesn’t accept a reactive function, and it’s not entirely clear why. Rather than automatically tracking dependencies such as query string parameters, you need to use them to construct a key which determines when to re-fetch the data. This overhead doesn’t exist on "normal" Solid functions like createEffect and createResource.
Possibly the most annoying thing is that data is inconsistently serialized when moving between the server and client. In my usage, it seems like objects in the initial page load are returned to the component as actual objects, but once you start taking actions on the page they become strings.^[4] It’s a major leak in the client/server abstraction. I’m not sure whether it’s a bug or simply an oversight, but I really hope it gets fixed.

Looking Forward

After all that, the question is: would I use Solid and/or Solid Start for a "real" project? I think the answer is "not yet".

I had a lot of fun building with it, and there are some really promising features. Despite having a lot of experience with React, building a full stack app with Solid took significantly less time and boilerplate. It’s a solid start.^[5]

But right now, React really is a behemoth^[6] when it comes to usage. Solid is definitely my favorite React alternative that I’ve tried, but that doesn’t mean the community agrees.

Usage matters because the community is one of the most important metrics to me. Maybe the most important. That’s how a technology gets tested; bugs get found and fixed; plug-ins get created. It’s a bit of a chicken-and-egg situation, because the lack of those things is what prevents the community from forming in the first place.

So: for large projects, I’m sticking to React. But for side projects, prototypes and experiments? I’m definitely adding Solid to my toolbox.

I find it difficult to choose a stack when it comes to “traditional” web apps. To me, structuring my UI code as components makes sense even if it doesn’t run on the client. And I’d much rather write each component in the programming language itself, rather than an entirely different templating language. Finally — and I don’t know how this hasn’t caught on outside of single-page app frameworks — but CSS modules are a requirement (I don’t like Tailwind). That’s pretty much the entire reason I don’t use Remix, even though it touts a “zero JavaScript by default” experience that checks all my other boxes: it doesn’t support CSS modules (yet). ↩︎
Call it Stockholm Syndrome, but I actually prefer className to class. I know class matches the HTML attribute, but className matches the JavaScript DOM property. Plus, class is a reserved word, so you can’t destructure it out of the props object — although this isn’t a problem in Solid, because you can’t destructure props at all. ↩︎
In reactive programming lingo, you can think of a signal as a source, and the DOM as a sink. ↩︎
I ran into the serialization issue while trying to return Date objects from routeData. Eventually I gave up and just returned strings that I parsed in the component on the client side. ↩︎
I’m sorry! I held out as long as I could! ↩︎
There are really weird spikes in the Svelte and Vue data that distort the graph. If you ignore those, you can see that React has over four times as many downloads as Vue and Angular, two orders of magnitude more than Svelte and three orders of magnitude more than Solid. NPM downloads are only a directionally accurate indicator of popularity, but you don’t need a statistics degree to see that Solid has some catching up to do. ↩︎

Tailwind is a Leaky Abstraction

Tue, 29 Nov 2022 00:00:00 GMT

I have to admit: as I’ve watched Tailwind enthusiastically adopted by more and more of the frontend community, I’ve remained skeptical. But, having never used it, I decided to keep quiet until I had an informed opinion.

Well, I’ve spent the past few months at work learning Tailwind with an open mind. I can now confidently say that I do, in fact, dislike Tailwind, and I wouldn’t use it for any new projects.

Tailwind is commonly described as “utility classes”, but that’s a bit of an understatement. It’s essentially a small language you write in the class attributes of your HTML that compiles to a combination of CSS rules and selectors — an abstraction over CSS. But all abstractions leak, and Tailwind is very leaky.

Let’s take transforms. I was trying to build an animation in which an element rotates in 3D space, so that it appears to come toward the viewer. To achieve that, I needed to rotate the element around the X axis. With CSS, this is easy: set a perspective on the parent element, and then set transform: rotateX on the element you want to animate:

.parent {
  perspective: 1000px;
}

.animate {
  transform: rotateX(45deg);
}

Unfortunately, Tailwind doesn’t support the perspective property. Implementing any sort of 3D design requires breaking out of Tailwind.

Not that it would have mattered, though. Tailwind only supports the unadorned rotate, which uses the Z axis. If you want to rotate along any other axis, you’re out of luck. A discussion has been ongoing for almost two years, with the official recommendation being to just write a custom JavaScript plugin to implement it.

You might protest that this is a niche issue (that’s what the Tailwind team says in the discussion). But there are issues with utilities that you’re likely to encounter in regular usage, too.

Even though it’s one of the main jobs of CSS, spacing things out has always been kind of a pain. Maybe that’s why Tailwind introduced the Space Between utilities. Just throw space-x-2 or some similar class on the parent, and the children will magically work themselves out!

Of course, it’s not magic. What it actually does is use child and sibling selectors to set margins on the child elements. The aforelinked page shows the generated CSS:

.space-x-2 > * + * {
  margin-left: 0.5rem;
}

That applies a margin-left to every child after the first of an element with the class space-x-2. The relevant consequence is that now, attempting to set a left margin using a selector with lower specificity — say, the selector generated by a Tailwind margin utility — will fail.

I ran into this issue when I was doing exactly that: trying to use a margin utility on an element whose parent already had one of these space utilities applied to it. It took me a while to realize why every margin I set seemed to get ignored. These features are mutually exclusive.

In the meantime, CSS has gotten much better at spacing! You can accomplish the exact same thing without Tailwind by using two properties:

display: flex;
column-gap: 0.5rem;

Originally, I was going to write about how Tailwind doesn’t support attribute selectors. But a few days before I started this post, they came out with a huge update that added them. It’s a big improvement, but it also includes some of the leakiest parts of the abstraction yet. I’m referring in particular to the “arbitrary variants” feature that allows you to include CSS selectors in your Tailwind classes.

Let’s look at some of the selectors here:

<div class="[&_p]:mt-4">
  <!-- ... -->
</div>

The & sigil lets you control nesting, similar to Sass and (soon) plain CSS. In CSS, the selector would be & p. Tailwind requires underscores instead of spaces, though, because a space would split the class name in two.

Here’s something a little more complex:

<ul role="list">
  {#each items as item}
  <li class="lg:[&:nth-child(3)]:hover:underline">{item}</li>
  {/each}
</ul>

To understand this class, you need to know how nth-child works. That’s an abstraction leak. But worse than that, it highlights a key shortcoming of Tailwind. In CSS, to set multiple properties using the same selector, you can write it once and group all the properties together. In Tailwind, there’s no choice but to write the full query again for every property:

<ul role="list">
  {#each items as item}
  <li
    class="lg:[&:nth-child(3)]:hover:underline lg:[&:nth-child(3)]:hover:font-bold lg:[&:nth-child(3)]:hover:text-blue-600 lg:[&:nth-child(3)]:hover:opacity-100"
  >
    {item}
  </li>
  {/each}
</ul>

Notice the long horizontal scrollbar. This isn’t just an abstraction that leaks some of the underlying details. It’s an abstraction that actively makes the experience worse.

Meanwhile, here’s the CSS that would be needed to accomplish the same thing. It’s still not simple. But it is scannable, and it doesn’t repeat the entire selector four different times.

@media (min-width: 1024px) {
  li:nth-child(3):hover {
    text-decoration: underline;
    font-weight: bold;
    color: var(--blue-600);
    opacity: 1;
  }
}

I realize this is a bit of a catch-22. Before, I was criticizing Tailwind for hiding CSS internals; now I’m criticizing it for exposing them. But that’s kind of the point. Tailwind is a layer on top of CSS, but it doesn’t actually hide any complexity in the layer below. You still need to know CSS.

This might be unfair to Tailwind. To my knowledge, the team has never promoted it as a CSS replacement. At its core, it really is just a set of class names that apply styles. But even after working with it for months, there’s still a mental translation layer between “Tailwind CSS” and “real CSS”.

These issues don’t mean Tailwind is bad. They’re just some of the tradeoffs you inevitably encounter when using a tool.

Maybe you really want to avoid coming up with names. Maybe you want to see your styles inside your markup, or use a ready-made set of design tokens. Tailwind will let you do those things. But the price you pay is that you need to know exactly how this tool interacts with CSS.

To me, that trade is a dealbreaker. It increases the number of tools I use without really giving me anything in return. It’s so leaky that I have to constantly think about the thing it’s supposed to abstract away. Yes, it’s nice to have my styles in the same file as the rest of my component code. But that convenience is heavily outweighed by the overhead of actually using Tailwind.