For those of you that are new here, welcome! I’m a partner at General Catalyst focused on AI, infra, cyber, etc., at the earliest stages. You can also find me on X here: x.com/alex__mackenzie. Thank you to the ~1.6k of you following along 😄

Oh & if you could do me a favour and engage with this X post, it goes a long way. Enjoy reading!

Ever since reading Tristan Hume’s (if anyone knows Tristan, I’d love to say hi sometime) piece whilst at Jane Street on Magic Trace, I’ve closely --watch’d JS’ engineering blog. Their bet on OCaml, Advent of FPGA, and Byrne Hobart’s “Understanding Jane Street” piece collectively made me rather JS-pilled.

So yeah, when I noticed that Jane Street was leading Antithesis’ latest funding round, my ears pricked up. Great infra companies frequently stem from mega-scale software companies: Confluent/Kafka from LinkedIn, Chronosphere, Temporal/Cadence from Uber, ClickHouse from Yandex, CockroachDB from Google, PlanetScale/Vitess from YouTube, etc. I suspect great infra companies are also invested in by mega-scale software companies; when the companies are actual users of the infra vs. corp dev shops, of course. This also played out with D. E. Shaw’s early bet on Flox.

Turns out, elite testing strategies (like the ones Antithesis incorporates) require some pretty interesting eng work. The kinda work that tends to go down well in this blog. Alright, let’s bash some bugs.

As always, we’ll begin with a definition which we’ll then break down bit-by-bit:

Antithesis uses, property-based testing, fuzzing, fault-injection and deterministic simulation, rather than traditional software testing methods. It’s an autonomous testing platform that finds bugs in your software, with perfect reproducibility to help you fix them.

Antithesis performs each test run in a simulated environment, which they provision & manage automatically. This environment contains your entire service architecture, and uses virtualization to simulate all hardware & network components in the system. The year of the sandbox, anyone? Customer production systems are never involved, so intense fault-injection can be performed with no risk of downtime.

Anyway, we’re getting a bit ahead of ourselves, let’s dig in. What’s so bad about good ole traditional software testing anyway?

The Problem with Traditional Software Testing?

Traditional software testing is largely comprised of “unit tests” and “integration tests” (using libraries like pytest, Jest, JUnit, etc). They’re known as “example-based testing”. Many of you will be all too familiar with these variants of testing, buuut for the uninitiated, let’s look at unit testing. Quickly.

Unit tests test if a specific “unit” of logic does what we expect. Let’s say we’ve created a “sort_number( )” function that, when given a list of numbers (e.g. [9, 12, 2, 6]), should return: [2, 6, 9, 12] . Oh, and before you all say it, yes I know python comes with sorted( ) out of the box. Anyway, we’d “unit test” this function like so:

# this is the actual function we will test

def sort_number(numbers):
    result = numbers.copy()
    for i in range(len(result)):
        for j in range(i + 1, len(result)):
            if result[j] < result[i]:
                result[i], result[j] = result[j], result[i]
    return result

# the tests

def test_sort_number():
    assert sort_number([9, 12, 2, 6]) == [2, 6, 9, 12]
    assert sort_number([]) == []
    assert sort_number([1]) == [1]
    assert sort_number([5, 5, 5]) == [5, 5, 5]
    assert sort_number([-3, 0, 2, -1]) == [-3, -1, 0, 2]
    assert sort_number([1, 2, 3]) == [1, 2, 3]  # already sorted
    assert sort_number([3, 2, 1]) == [1, 2, 3]  # reverse sorted

What we’re doing in our “test_sort_number( )” function is manually (this is important) specifying that when we pass a given list (e.g. [9, 12, 2, 6]) into our “sort_number( )” function, that it should equal a certain output, in this case, the output should be [2, 6, 9, 12]. If this isn’t the case, then our unit test fails and we’ll receive an error.

Anyway, I’m not really here to explain unit testing to you. But, I am here to assert its flaws. Particularly in distributed systems. Firstly, unit tests are a pain in the ass to write. You are literally hardcoding inputs and outputs. What this also means is that you need to list, via examples, all of the ways in which your function could fail. The problem is, you might not think of example data that covers all failure modes.

Wouldn’t life be easier if we could stipulate the desired end states, or “properties”, of our function’s outputs, and have tests be automatically generated to assure this end state? If this sounds confusing, it really isn’t, so bare with me.

For example, one desired end state of our “sort_number( )” function could be that our output length should equal our input length. I.e. if we receive an array with three numbers, our function should output an array containing three numbers. Cool, we can generate a bunch of tests to make sure this is the case. Similarly, we want to ensure that our “sort_number( )” function’s output contains the exact same numbers that it receives as an input, just sorted of course.

This “property”-based approach to testing is called, shock horror, “Property-Based Testing”, or “PBT”, if you wanna look cool in front of your eng homies. Like many of my favourite eng movements (i.e. Nix), PBT stems from the functional programming community via Haskell’s QuickCheck. The primary PBT library in Python is called Hypothesis. Let’s revisit the properties we mentioned we’d like our “sort_number( )” function to have and implement them via Hypothesis.

from hypothesis import given
from hypothesis.strategies import lists, integers

@given(lists(integers()))
def test_sort_properties(nums):
    result = sort_number(nums)
    
    # Same length
    assert len(result) == len(nums)
        
    # Same elements
    assert sorted(result) == sorted(nums)

Nice. Now, by using Hypothesis, we will automatically generate input & output data (i.e. example-based tests) at runtime that we previously had to hand write. It’ll also bias toward edge cases (empty, single element, negative, large numbers, duplicates, etc.). Now I know testing isn’t necessarily the “hottest” category of developer tools, but c’mon, that is pretty sweet. Antithesis takes this PBT methodology, but applies it to how your entire system should work. Don’t stress about this yet, we’ll return to it.

The Challenge With Distributed Systems?

When you have a distributed system (multiple services, databases, network calls), bugs often depend on precise timing. This is known as a race condition. For example, a race condition might only trigger when server A’s request arrives exactly 3 milliseconds before server B’s, whilst the database is mid-write, during a network blip. You could literally run your test suite a thousand times and never hit this bug. Sure, us mere mortals can conjure simple unit tests, but something this complex? Tricky. This is why production is a different beast to development / staging environments.

As Doug Patti at Jane Street puts it: “testing is still hard. It takes time to write good tests, and in any non-trivial system, your tests are an approximation at best. In the real world, programs are messy. The conditions programs runs under are always changing: user behavior is unpredictable, the network blips, a hardware failure causes a host to reboot. It’s inherently chaotic. That’s the hard thing about developing HA systems: for all the careful tests that you think to write, there are some things you can only learn by experiencing that chaos. That’s what it takes to go from merely being tested to being battle-tested.”

Nice. Now, I should point out that “regression testing” (see something bad happen in production & test for it in the future) has been a testing methodology for some time. But it’d be preferred if, yeno, the bad thing never happened in the first place? We also have some mitigations for distributed systems’ non-determinism like: locks/mutexes, atomic operations, and message passing, but they can create performance overhead & are still be difficult to reason about in advance of deploying to production.

Antithesis - the Deterministic Computer

Antithesis takes being “battle-tested up” several notches from regression testing etc. Per Jane Street: “The amazing thing that Antithesis does is run your whole system in a virtual machine controlled by a completely deterministic hypervisor, and then adds a little manufactured chaos by interfering with scheduling and networking. It uses this setup to explore many different scenarios, and to discover circumstances where your system might fail.”. Alright, now this is something for us to unpack 😄

Aware the above sounds complicated, but don’t fret. The virtual machine part is easy to grok: Antithesis clones your distributed system (e.g. your food delivery app) & runs it on their own servers (vs. your users’!). It does this as it wants to mimic your system running in production. It can then replicate this system n-many times, across n-many scenarios (we’ll get to this).

Most importantly, these virtual machines (“VMs”) are controlled by a deterministic hypervisor. This hypervisor ensures that your distributed system has a consistent set of properties. To make this more concrete, let’s revisit the race condition we described above. When not using Antithesis, our “normal” production system may work like so:

Monday 2pm - Everything works fine:
- Server A request: arrives at 14:00:00.100
- Server B request: arrives at 14:00:00.250 (150ms later)
- Database write: starts at 14:00:00.050, finishes at 14:00:00.120
- Network: stable
Result: ✓ Works perfectly

Tuesday 9am - Same exact user actions:
- Server A request: arrives at 09:00:00.100
- Server B request: arrives at 09:00:00.103 (3ms later - just unlucky timing!)
- Database write: starts at 09:00:00.098, still running at 09:00:00.103
- Network: small blip causes 5ms delay
Result: ✗ BUG! Data corruption!

Wednesday 3pm - Trying to reproduce:
- Server A request: arrives at 15:00:00.100
- Server B request: arrives at 15:00:00.180 (80ms later)
- Database write: starts at 15:00:00.095, finishes at 15:00:00.115
- Network: stable
Result: ✓ Works fine again

& herein lies the issue: you know there’s a bug, but oftentimes you don’t know what’s causing it. Was it our network blip? Was it the server request timing? Perhaps it was the presence of both? etc. Hence, the bug becomes difficult to reproduce. Our system is rather inconsistent. Gah!

Hopefully it’s now clear why we’d like for our system to have a guaranteed set of properties. If we can reproduce this 3ms latency and network blip consistently, and each time they’re present we note that our data corruption bug occurs, then we’ve likely found the problem. Nice.

By contrast, using Antithesis’ deterministic hypervisor, our system would run like the below. These are separate runs, but identical. Note - this is an oversimplification, but illustrative.

Monday 2pm - Running Scenario #1 in Antithesis:
- Server A request: arrives at T+0.100
- Server B request: arrives at T+0.250 (150ms later)
- Database write: starts at T+0.050, finishes at T+0.120
- Network: stable
Result: ✓ Works perfectly

Tuesday 3pm - Replaying Scenario #1 in Antithesis:
- Server A request: arrives at T+0.100 [IDENTICAL]
- Server B request: arrives at T+0.250 (150ms later) [IDENTICAL]
- Database write: starts at T+0.050, finishes at T+0.120 [IDENTICAL]
- Network: stable [IDENTICAL]
Result: ✓ Works perfectly [IDENTICAL]

Friday 10am - Replaying Scenario #1 in Antithesis:
- Server A request: arrives at T+0.100 [IDENTICAL]
- Server B request: arrives at T+0.250 (150ms later) [IDENTICAL]
- Database write: starts at T+0.050, finishes at T+0.120 [IDENTICAL]
- Network: stable [IDENTICAL]
Result: ✓ Works perfectly [IDENTICAL]

Nice. As Doug Patti said, based on the determinism that Antithesis gives us, we can “[use] this setup to explore many different scenarios, and to discover circumstances where your system might fail.” I.e. Antithesis will clone your distributed system across a number of VMs, and then autonomously start changing certain properties (e.g. 3ms latency could become 3.5ms) and purposefully inject issues like network blips (“fault injections”) in order to test your distributed system. Cool. Have you noticed that this autonomous testing is just property-based testing? The pieces are coming together.

If your system fails, no worries, Antithesis will help you reproduce the bug and, ultimately, bash it. This readers, is “Deterministic Simulation Testing”.

How Does This Work? The Thesis Behind Antithesis

Ok, I think it’s important to take a breather and acknowledge that you should now get the value prop of Antithesis. That’s great. But, how on earth does Antithesis actually achieve this determinism that’s evidently so hard? Guys, it’s honestly so smart.

You see, infrastructure always comes with tradeoffs. The miracles that are modern computers (i.e. the servers running production code) are optimised for speed and throughput, not predictability. Whether it’s CPU schedulers, caching, multi-core parallelism, etc., they’re doing their darnedest to make your software fast. But, this means that if your system can run 8ms faster on Tuesday vs. Monday, it will. Hence, it’s non-deterministic.

Instead of trying to recreate production’s chaotic timing (based on actual milliseconds ticking by on a clock), Antithesis creates a controlled environment where time works fundamentally differently. Don’t worry, you’re not reading Dark Matter, this is simply cool comp sci.

Time flows in Antithesis not via milliseconds, but by CPU instruction count. Whether it’s 2:45pm on Tuesday or 5:15am on Monday is irrelevant, instruction number 5,000 is instruction number 5,000. Our new unit of “time” (or “simulated clock”) removes the variability inherent in production systems.

This is important, as software applications often make timing-based decisions (timeouts, rate limiting, cache expiration). If your code sees different time values on different runs, it will behave differently. Antithesis’ “virtual time” ensures that your code always sees the same values. This took me quite a bit to grok, so let me walk through an incredibly simplified example.

start = time.now() # gets us the actual time per our computer's clock

When the above system call is made, Antithesis intercepts it, and instead, converts the instruction count into a virtual time value in milliseconds. Not sure how they actually calculate this conversion, but for simplicity’s sake, let’s pretend they divide the count by 100. In this case, instruction count 5,000 always outputs 50 milliseconds. It ultimately converts to milliseconds as our original code (the time.now( ) function) was written to process milliseconds, not instructions.

Nice. Now, remember the multi-core parallelism I mentioned above? Well, much like our friend “time”, it also creates non-determinism issues. Why? Well, once you have multiple cores each processing separate threads (i.e. CPU instructions), these threads can execute in essentially infinite different orders depending on which core happens to run faster at any given nanosecond. Of course, Antithesis has an answer here too.

Within Antithesis, each instance of the deterministic hypervisor runs on just one physical CPU core (vs., say, 48 cores). This is totally fine, as this isn’t our production environment, we don’t need the speed benefits of multi-core processing.

Alright, for those folk interested, there’s a lot more Antithesis does (e.g. deterministic I/O) to assemble their “deterministic computer”, but, I think at this point, you get my drift. The reality with all great infra is that it’s not a single “silver bullet” idea that solves the underlying problem, but a series of micro-optimisations that compound.

Incredible work team Antithesis, and congrats on your Series A! time.now( ) to get back to work 😄

Disclaimer

Views expressed in “posts” (including podcasts, videos, newsletters, substacks and social media) are those of the individual GC personnel who wrote the post or is quoted therein and are not the views of General Catalyst Group Management, LLC, “GC”) or its respective affiliates. GC is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell — or a solicitation of an offer to buy — any securities, cryptocurrencies, or any financial instrument or property, and may not be used or relied upon in evaluating the merits of any investment. Additional important information about GC, including Form ADV Part 2A Brochure, is available at the SEC’s website: http://www.adviserinfo.sec.gov

Nothing here should be construed as or relied upon in any manner as investment, legal, tax, or other advice. Any projections, estimates, forecasts, targets, prospects, or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts or figures provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in the content has been obtained from third-party sources. While taken from sources believed to be reliable, no one has independently verified such information and there are no representations about the enduring accuracy of the information or its appropriateness for a given situation.

To the extent any content authored or provided by any GC personnel in this forum or medium makes reference to any company in which any of the funds or vehicles advised by GC have an economic or financial interest, previous or current, none of the information provided or opinions expressed herein are connected in any way with GC’s business activities, and GC did not provide any information or assistance in the creation of this content.

My next primer will likely be on Antithesis if you feel like subscribing below to keep tabs. Other posts in the proverbial oven include some performance analysis of Ruff, Jane Street’s Magic Trace, and a primer on tinygrad. Suggestions are always welcome!

Ever since my Hyena Hierarchy primer I’ve been pretty into the research surrounding context optimisation, limits, scaffolding, etc. Largely because, whilst we’ve continued to make progress here, I still bump into context issues all the time. I often find myself feeling like I need to re-reference a specific file or function in Cursor after a certain period of time. A more “fun” example of this issue that I ran into recently was having a Udon recipe described as a Rust object (!). Rust’s memory safety is great, but I don’t need it in my noodles (there’s a sentence I never thought I’d write).

It’s also important to note that there’s a pretty material difference between a model’s “physical” maximum context window and its maximally effective context window. So we should always be a little skeptical with context window headlines.

So, over the break, I took note when I saw Alex Zhang’s & Omar Khattab’s work doing the rounds on X (full paper here). Perhaps even more so once I saw just how bullish the Prime Intellect folks were on its implications, describing RLMs as “The Paradigm of 2026”. Such is the way with AI these days, on the day I started writing this primer (Jan 7th), Cursor also released their take on “dynamic context discovery”, which, at a cursory glance, appears to rhyme with some of the thinking behind RLMs.

So yeah, let’s start the new year with some research + a new acronym for us to get acquainted with! & if you’d like to say hi, or have any thoughts, questions, etc., please do drop me a note on X: https://x.com/alex__mackenzie. Always happy to chat.

Context Rot

So, why are we still running into these context issues? Model performance degrades as input (“context”) length increases, a phenomenon known as “context rot”. As Alex Zhang points out, certain experiments refute this claim, but c’mon, we’ve all felt it.

To make this a little less satirical, and a tad more analytical, Chroma has done some nice work on charting performance degradation below. But, what actually causes context rot? I’ll share a few simple explanations with the usual caveat that there are a bunch of smart research directions beyond recursive language models tackling CR.

1. Attention Dilution

The softmax (wrote a post explaining softmax if you care to dig in) in attention normalizes scores across all tokens. As sequence length grows, attention weights get spread thinner. A token that’s genuinely important might get 0.3 attention weight in a 500-token context but 0.03 in a 50,000-token context. It’s worth making sure this point really clicks fyi, models aren’t “forgetting” as such, they’re distributing probability mass across more candidates.

2. Positional Encoding Degradation

Remember, “Attention” let models learn which words matter to which, regardless of distance (i.e. their respective positional encodings). The issue here is that most models use positional encodings that were trained on certain sequence lengths (i.e. we have significantly more training data that shows the relationship between tokens at positions 1–4096 than between tokens at positions 100,000–104,096). Hence, we get degradation. RoPE (rotary embeddings) and ALiBi help with extrapolation, but there’s still distributional shift.

3. Compounding Errors:

This is a pretty intuitive/obvious one. In autoregressive generation (i.e. models that predict the next token based on ~all prior tokens), small mistakes propagate. More context == more opportunities for the model to slightly mis-weight something, and for those errors accumulate. We can ofc partially attenuate this issue via chain-of-thought reasoning/scratchpadding, but, this adds additional context (see points 1 + 2 above 🥲).

++

Recursive Language Models

Ok, so, what are RLMs and how do they help deal with context rot? Oh, and, as a sidenote, using agents to generate UIs to explain technical topics is a total game changer. Would recommend opening up this interactive RLM demo (took 2 mins to spin-up?!) as we walk through the definition.

As Alex points out, the natural solution to context rot is something along the lines of, “well maybe if I split the context into two model calls, then combine them in a third model call, I’d avoid this degradation issue”. This intuition is the basis for a recursive language model.

How RLMs roughly work in practice is that we convert our long context (e.g. my primer on turbopuffer) into a large python string like so (thank you Hex for the nice notebook).

This python string is now a variable called context that can be processed. Note, so far, our “Root LM” hasn’t actually seen this context, and as a result, can mark itself as safe from context rot. Alex refers to this Root LM as having a “depth” equal to 0 (depth=0). As “recursive” suggests, we’ll spin up other LMs that have depths >0, but don’t worry about this for now. Note that Cursor is doing something similar with their dynamic context discovery whereby they’re turning long tool calls into files.

Important: all our Root LM has “seen” thus far is the query (e.g. “what indexing strategies does turbopuffer use?”) that we want answered via the context, and the knowledge that context variable exists and that it can be manipulated (the LLM’s system prompt).

Next, the Root LM (again, depth=0) “peeks” at the context. What this means is it writes some basic python that, for example, helps it 1) determine the length of the context, 2) prints the first 1,500 characters of the context.

Based on the above output, our Root LM may decide to “grep” (this is a technical way of saying that we’ll do a string search) for some additional text related to the query it originally received (remember, this query was: “what indexing strategies does turbopuffer use?”). So our string search could be the below. PS - don’t get intimidated by the regex below, it never looks pretty.

All this is doing is returning back to our Root LM every occurrence of the word “index” in the context, as well as the word’s position plus a 200-character preview around each match. Again, we’ve progressively added to our Root LM’s context. The returned object looks roughly like:

[
  (15234, “indexing. Ok, so, just like at the library, we use indexes in databases...”),

  (16102, “index (remember, the ‘asynchronous’ guy) and understand ‘ANN’ and ‘inverted BM25’...”),

  (17845, “inverted index would look like:\n\n’rubberized’: [1, 3]...”),

]

With this output, our Root LM can now infer that the indexing section of our turbopuffer primer exists between characters ~15,000-~18,000. It’s context is updated accordingly. Next is where things get a bit more interesting, as we’re about to explore depth=1 (i.e. spinning up separate or “sub” LMs). See below.

Ok, cool, to sum-up: our Root LM has written code to create a new python variable called indexing_section that contains the primer text between characters 15,000-18,000.

With this in mind, our Root LM calls a sub-LM (depth=1) and asks it to summarise the indexing strategies described in indexing_section. Through doing so, we’re minimising the amount of context our Root LM receives (e.g. perhaps our sub-LM also leverages browser-use to complement what it’s read in the primer). The sub-LM then passes its final output to the Root LM:

FINAL(”turbopuffer uses two main indexing strategies: 1) ANN (Approximate Nearest Neighbor) with SPANN optimization that stores centroids in NVMe cache and posting lists on object storage, and 2) inverted BM25 indexes for text search that map terms to document IDs with relevance scoring.”)

So, what the above (hopefully) illustrates that a RLM is a general inference strategy where language models decompose and recursively interact with their input context as a variable. It’s this input context, and the LM’s multiple interactions with it, that determines what step the LM will take next vs. some pre-ordained plan (i.e. “agentic planning”). It should also be evident that the example query-chain described above requires significantly less context to generate the desired output.

So What? What Does This All Mean?

Well, on the performance side, the RLM team demonstrated that “an RLM using GPT-5-mini outperforms GPT-5 on a split of the most difficult long-context benchmark[s] [they] got [their] hands on (OOLONG) by more than double the number of correct answers, and is cheaper per query on average!”. Nice, seems like we have a pretty material solve for context rot.

When spelt out like this, RLMs sound kinda trivial though, right? Sure, but it’s first worth noting that the depth=0 “peeking”, “grepping” and depth=1 “summarisation” steps are strategies the RLM has developed based on its constraints. The RLM wasn’t instructed to first peek, then grep, etc., nor did it make an initial plan to conduct these steps sequentially.

For more semantic queries/contexts where the likes of regex fails (regex doesn’t know what indexing is, it just knows how to search for the literal word “index”), RLMs have also identified smart chunking strategies whereby they split the context into multiple “chunks”, & then spin-up independent sub-LLMs to process each chunk. Pretty cool. No doubt we’ll see plenty of other interesting behaviours emerge.

Now to zoom out, and, as Palmer would say, think about the logical end state. Whilst we’ve yet to see if RLMs get us here (although multi-threading seems quite doable?), what does it mean to have an infinitely effective context window? Truly full codebase-reasoning (e.g. “find all the ways this API contract could be violated across these 500 files”) would be an exciting step. Perhaps we’re another leap closer to long-running personal assistants, or better yet, Udon recipes without the Rust.

RLMs also hint that we’ve found a way for AI to increasingly “self-manage” its context which could be rather profound. Will spend more time thinking about what this all means. Excited for 2026.

Disclaimer

Views expressed in “posts” (including podcasts, videos, newsletters, substacks and social media) are those of the individual GC personnel who wrote the post or is quoted therein and are not the views of General Catalyst Group Management, LLC, “GC”) or its respective affiliates. GC is an investment adviser registered with the Securities and Exchange Commission. Registration as an investment adviser does not imply any special skill or training. The posts are not directed to any investors or potential investors, and do not constitute an offer to sell — or a solicitation of an offer to buy — any securities, cryptocurrencies, or any financial instrument or property, and may not be used or relied upon in evaluating the merits of any investment. Additional important information about GC, including Form ADV Part 2A Brochure, is available at the SEC’s website: http://www.adviserinfo.sec.gov

Nothing here should be construed as or relied upon in any manner as investment, legal, tax, or other advice. Any projections, estimates, forecasts, targets, prospects, or opinions expressed in these materials are subject to change without notice and may differ or be contrary to opinions expressed by others. Any charts or figures provided here are for informational purposes only, and should not be relied upon when making any investment decision. Certain information contained in the content has been obtained from third-party sources. While taken from sources believed to be reliable, no one has independently verified such information and there are no representations about the enduring accuracy of the information or its appropriateness for a given situation.

To the extent any content authored or provided by any GC personnel in this forum or medium makes reference to any company in which any of the funds or vehicles advised by GC have an economic or financial interest, previous or current, none of the information provided or opinions expressed herein are connected in any way with GC’s business activities, and GC did not provide any information or assistance in the creation of this content.

In a few months, Maze has already been deployed across the Fortune 500 and some of the world’s most sophisticated security teams. But, as Harry Wetherald (Maze’s CEO) likes to remind me, these are but the early innings.

Read the announcement from Harry at Maze here and take Maze for a spin here.

Congratulations Harry, Santiago, Adrian and the Maze team on bringing a bit of “magic” back to the cybersecurity industry.

Why does the world need another vulnerability management product?

CVE publish rates continue to increase at an alarming pace. Concurrently, the average “time-to-exploit” (TTE) vulnerabilities has decreased from 207 days in 2021 to a jarring 8 days in 2023.

This rapid increase barely accounts for advancements in AI. As Josh Bressers points out in “Why are vulnerabilities out of control in 2024?”: hundreds of millions of pieces of open source software have been released, yet there have been ~250k CVE IDs ever created (?!). There are likely millions of vulnerabilities that can now, or very soon, be discovered at scale with AI. In fact, my friend Sean Heelan recently did just this: https://sean.heelan.io/2025/05/22/how-i-used-o3-to-find-cve-2025-37899-a-remote-zeroday-vulnerability-in-the-linux-kernels-smb-implementation/. Times they are a changin’.

Furthermore, whilst a meaningful percentage of CVEs flagged currently are benign due to poor PoC exploit code, this example code may be all an LLM needs to accurately reproduce the vulnerability. Auto-exploitation needs to be countered with auto-remediation.

Whilst (or perhaps because) there are more vulnerabilities than ever, the amount of data describing these vulnerabilities fell off of a cliff since February 12, 2024 as NIST almost completely halted enriching the world’s most widely used software vulnerability database (the “National Vulnerability Database”).

Attempts like Vulnrichment have been made to plug this gap but there is equally an opportunity for private threat intel (e.g. Filigran) and/or vulnerability management (e.g. Maze) players to help fill this void. Regardless, we suspect this event will encourage security practitioners to seek out new sources of vulnerability data, fostering new conversations.

For new folk here, hi, I’m Alex Mackenzie, a partner at General Catalyst. I’d love to hear from any of you reading along: amackenzie@generalcatalyst.com

A special thanks to Simon and the cool folk at turbopuffer for the platform access & for helping me write this. A fun primer to end 2025 on.

Last year, a well-known (how mysterious) database founder & I were debating investing in another database. Being quite a rare opportunity to ask someone of their ilk, “what do you look for when investing in databases?”, I took my shot. Cache hit.

Perhaps it’ll seem trite to some of you, but what he said has shaped several of my subsequent investments (e.g. Tracebit, Flox, Atlas). He said: “look for companies that ‘religions’ can be built around”.

I.e. look for companies that stand for something; what they stand for should probably p*ss a few people off. Why? Well, as I stated on Shomik’s podcast (hey Shomik) a year or so ago, “as degree of opinion goes up, CAC goes down.” People talk about you because they love ya, or talk about you because they hate ya. Par example, Flox is built on the “philosophy” that is Nix:

Currently in the data realm there are two “religions” that I’m tracking. Skip (portfolio) is a framework for creating reactive software, which responds in real-time to changing inputs. One way to think about this is “streaming, without the streams”. You can only imagine how Kafka zealots respond to such a claim! A primer on Skip will come soon.

The other religion forming is, as my friend Justin recently put it, “object storage is all you need”. You should watch his full presentation, but the tl;dr is that object (or, “blob”) storage (think S3) is having another “moment”, whereby open table formats (e.g. Iceberg), the proliferation of SSDs (vs. HDDs), and compatibility with modern query engines have bestowed blob storage’s 99.999999999% durability with a little more.. pace. Nice.

But how does one spot a religion? & better yet, how does one spot “companies that ‘religions’ can be built around”? In dev tools, infra, cyber, etc., my general learning is that everything washes out in developer docs. Jump into an architecture diagram & you’ll see if a company really stands for something. Seek out their explicit trade-offs across speed, cost, usability, etc. Don’t bother looking at landing pages.

So, last week, I decided to do some.. washing. I jumped into turbopuffer’s docs and it became clear that they’re a disciple of “only needing” object storage. Or, as Matt Klein put it, they’re leveraging the “blob store first architecture”. With customers like Notion, Suno, Cursor (primer here), Readwise, etc., it seems like this movement has legs. Let’s dig into “tpuf” under the hood and see what we think for ourselves? <(°O°)>

Ok, as always, let’s begin by seeing how turbopuffer describes itself. There’s quite a bit of detail in this description but, don’t worry, we’ll break it down piece-by-piece:

Cool site, huh. Let’s tackle object storage first. But we’re going to start counter-intuitively, and briefly touch upon storage hardware (e.g. SSDs).

Why? Well, there are layers-upon-layers of abstractions in data management that can make things quite complex. However, knowing that data is generally stored the same way on a given piece of hardware (irrespective of a high-level abstractions like object storage) gives us a good base to return to.

So, how is data ultimately stored? Like most things in infra: it depends. SSDs (solid state drives) encode and store data as electrical charges (watch this); HDDs (hard disk drives) use magnetic “domains” to store bits on spinning platters (watch this).

Ok, back to object storage. There are three primary (aka “the big three”) data storage abstractions or “models”: 1) file storage, 2) block storage, and 3), object storage. Think of these models as software “layers” that enable you (or your software) to reason about, and interface with, data physically contained within storage hardware.

We’re all familiar with file storage (even if you don’t know it yet). In file storage, we create folders (e.g. “why_now_primers”), and within these folders (or “directories”) files are stored that contain data (e.g. “subscribers.csv”):

File Storage System
└── why_now_primers/           # Folder (Directory)
    ├── subscribers.csv        # File
    │   ├── Name, Email, Subscription Date
    │   ├── Warren Buffett, wb@berkshirehathaway.com, 2023-01-01
    │   └── Howard Marks, hm@oaktreecapital.com, 2023-02-01
    └── readme.txt             # Another File

Thanks, WB. If you’re a Mac user you use a specific file system implementation known as APFS, or, “Apple File System” (how, err.. creative). Ok, we understand how this software models our data, but let’s walkthrough how it interfaces with it.

Let’s say we want to open that “readme.txt” file up above. When we do so in an app like TextEdit, it makes a “system call” (have a whole primer on these guys), to the macOS kernel. Then, the kernel hits up APFS and (essentially) says: get me the metadata that maps to this file path: alex/desktop/why_now_primers/readme.txt.

Buckle up. Included in this metadata is what’s known as the “logical block address” or (“LBA”). This is a virtual representation of where readme.txt’s data is physically stored (the “physical block address”). Why bother to virtualise? I asked the same thing. You see, for reasons that are beyond the scope of this primer, our data’s “PBA” changes dynamically. So, using our LBA provides us with, you guessed it, a helpful abstraction.

Nice, I guess we’ve got file storage down. The thing is… unbeknownst to you, we also just tackled block storage. Ha! Block storage is literally “just” the LBAs we discussed up top. I.e. block storage has no human-friendly abstractions like files & de minimis metadata.

Just like TextEdit, databases from SurrealDB to RocksDB access LBAs via the file system. However, certain databases like Oracle’s Real Application Clusters (“RAC”) expose underlying LBAs directly to the database engine, removing the abstraction & hence, ameliorating performance (but creating maintenance overhead, of course).

The below describes the relationship between file and object storage quite nicely:

+----------------------------+
|     Application Layer      |
+----------------------------+
            |
            v
+----------------------------+
|    File System (NTFS,      |
|    ext4, HFS+, etc.)       |
+----------------------------+
            |
            v
+----------------------------+
|      Block Storage         |
|                            |
| +--------+  +--------+     |
| | Block  |  | Block  |     |
| |   1    |  |   2    |     |
| +--------+  +--------+     |
| +--------+  +--------+     |
| | Block  |  | Block  |     |
| |   3    |  |   4    |     |
| +--------+  +--------+     |
+----------------------------+
            |
            v
+----------------------------+
|    Physical Storage        |
| (HDD, SSD, Storage Array)  |
+----------------------------+

Alright, I guess we should probably address object storage now. So, to repeat, object storage is just another model and interface for interacting with data stored on storage hardware like SSDs. So.. what’s different about it?

Well, first off, unlike file storage, where data is stored hierarchically (i.e. in a “tree” structure), object storage consists of a “flat” namespace. This means that we don’t have folders, but instead, we have a primary key (i.e. a unique identifier) that maps directly to the underlying data, and metadata. Par example:

why_now_primers/subscribers.csv: {"author": "admin", "description": "Subscriber list"}

But wait. The “why_now_primers/subscribers.csv” key sure looks like a folder? This is where I kept on getting tripped up too! To us, as mere mortals, these keys are made to look like they have hierarchical structures (for legibility reasons), but our system views them as a “pointer” to a specific piece of data. Blob storage is an “unstructured pool” of data.

I’m aware this sounds trivial. But, it has a rather material impact on object storage’s performance and scalability vs. file storage. Pourquoi? Well, when you store files in a tree structure, the storage system has to follow (or “traverse”) the tree, in order to get the data. These trees can get pretty long if you have hundreds of files. Check out the performance hit below. Yikes.

Ok, we understand why file storage’s query performance degrades. Why does blob storage hold up though? Well, the reason I find most instructive is how it handles metadata.

Let’s just take a step back for a sec. What is metadata (e.g. a timestamp) but a useful way to help describe, filter, or “query” for, some data? So, if we’re trying to optimise query performance, one way to do it is (surprise, surprise) to optimise how effective our storage system is at adding metadata.

Now remember, object storage’s metadata is included in the object itself. This stands in contrast to file storage which stores its metadata in a centralised directory table that’s shared between folders and files. Hence, as the number of files increases within object storage, there’s no centralised directory table (i.e. server) that’s “overwhelmed”. Nice.

But wait, there’s more. Object storage also enables you to add custom metadata (e.g. the camera used for a snap) to your data, whereas file storage has a fixed set of metadata that you can add. See the code for creating an S3 bucket below:

s3.put_object(
    Bucket='my-bucket',
    Key='my-image.jpg',
    Body=open('my-image.jpg', 'rb'),
    Metadata={
        'author': 'A Mack',
        'camera': 'Canon EOS 5D Mark IV',
        'location': 'London init bruv, UK',
        'tags': 'landscape,sunset'
    }
)

Have a think for a sec why custom metadata can help query performance as the no. of files (or data) grows? Remember, metadata is a filter. So, if we want to query a dataset like every Flickr image ever, and search for photos taken in Europe, having location metadata like “London” above helps the cause.

Nice one, we now have a pretty deep understanding of object storage. Let’s scrape it off our to-do list. Not to mention storage hardware like SSDs (although there’s more to come here). Let’s come back to our turbopuffer definition. Next, we’re going to tackle separating storage & compute.

Data is sometimes stored on the same machine that processes it. E.g. on your laptop you have an HDD (storage) and a CPU (processing). If you use something like a Mac or MSFT Surface (sike!), storage and compute are tightly coupled. If you want to improve your storage, you’ll likely need to replace your entire machine. This used to be the case with mainframes too.

However, what if we could decouple this relationship? Then, if we only need more storage, we can add some, vs. also having to add compute capacity (i.e. replace our machine) (i.e “scale vertically”).

This decoupling would enable us to scale storage and compute up/down to match demand (i.e. add more/less servers) (i.e. “scale horizontally”). Sounds a bit like the promise of the cloud, huh? Bingo.

However, “back in the day” having physically separate storage and compute resulted in increased latency, of course. It wasn’t until improved networking hardware (ethernet to fiber) & software (RDMA and RoCE) that made data transfer speeds viable. RDMA & RoCE are beyond the scope of this primer, but are worth digging into. Cool stuff.

“Virtualisation” (i.e. partitioning hardware resources) also made scaling compute flexible, enabling multiple “VMs” to run on the same physical server. Want more compute? Cool, “here’s another VM”. Not, “here’s another server”. Nice.

In many ways, Snowflake or Databricks’ “religion” was the separation of storage and compute. And yeno what storage model “horizontally scales” particularly well? A Simple Storage Service indeed.

Alright. We’re doing well. That’s object storage and the separation of storage and compute done here. Again, we’ll go with the slightly counter-intuitive route and tackle some of the more esoteric items like write-ahead logs & asynchronous indexing prior to talking about tpuf directly and connecting all of these concepts together <(°O°)>

This stuff sounds hard, but tbh, it’s kinda easy. I’m going to get in trouble for saying this, but you can sub out the word “log”, for “queue”. I.e. they’re data structures that preserve the order of things. We dealt with queues in my primer on Restate & Durable Execution (isn’t it cool how all of these topics connect).

So, when turbopuffer receives some new data, it first “writes” this data to a log, prior to writing it to its “primary storage” (i.e. “vanilla” object storage). Hence, it’s a write-ahead log.

For context, I say “vanilla” because tpuf’s write-ahead log (or, “WAL”) lives within object storage itself. So, think of it as an another abstraction on top of object storage:

Hmm. Why doesn’t tpuf just write data directly to its primary storage? Remember, all infra comes with trade offs. So, as rosy of a picture as I’ve painted of object storage, it has its flaws. We won’t dig into all of these flaws, but something rather important that object storage doesn’t fully (nuance here) support on its own is “atomic” transactions (logs do).

Writes are considered “atomic” when only once all of the related data in a transaction (think both debit and credit operations in a bank transfer) are complete. “Atomicity” is pretty important, as otherwise your storage system is prone to “partial writes” (think the debit operation happening, but not the credit). Partial writes = “corrupted” data. So, turbopuffer navigates this issue with a write-ahead log (or “WAL”). Nice job tpuf.

There’s a tonne more we could talk about here (alex@tapestry.vc) but let’s move onto asynchronous indexing. Ok, so, just like at the library, we use indexes in databases to speed up our searches (i.e. queries). Think of them as filters. Easy.

Next, to get your head around “asynchronous” do what I did & replace asynchronous with independent. I.e. when we update our index (post writing to the WAL), updating the index doesn’t hold up any new operations like a new write or query. Independence.

Pretty cool that we can now strike: separating storage and compute, object storage, write-ahead logs and asynchronous indexing off of our list. There are a few other gnarlier topics within tpuf’s architecture that I wasn’t going to dig into (“don’t over-load the reader Alex!”); but then I remembered… whilst I’m grateful for all of you that read along, I honestly write all of this for me first tbh. Sorry!

So, uh.. let’s touch on a few other items and then I promise we’ll actually take tpuf for a t-spin and build something cool with it. I have some fun ideas. Ok, back to some tpuf architecture topics below. Rapid fire.

Visualise the above using the image below. That “./tpuf” is just their API (written in Rust fwiw). Caching (or “cache locality”) is easy, so let’s not dig into that. NVMe SSDs were new to me, so let’s linger a bit here. Traditional SSDs use the “SATA” bus. Buses (think cables, controllers, protocols, etc) are how hardware like RAM connects to other components like the CPU.

The “SATA” bus was originally built for HDDs, so it’s been knockin’ around for a bit. & more importantly, it isn’t optimised, for SSDs (was never really built for ‘em). NVMe SSDs, however, are capable of using the more performant “PCIe” bus. These SSDs hit GA on AWS in 2018 (h/t Simon). Hence, tpuf’s cache gets a nice performance bump.

Ok, next. Let’s revisit our index (remember, the “asynchronous” guy) and understand “ANN” and “inverted BM25” indices. I bloody love all of the optimisation work this team has done (& shared). Religion building washes out in the developer docs!

You might recall “nearest neighbours” from your probability classes. Let’s do a quick refresher. Imagine we have a query vector (e.g. an image encoded as a point in a multi-dimensional space) (e.g. a picture of these Niobium C_3s), we might want our website to recommend “related products”:

We could do this ourselves manually (i.e. tag one product as related to another), but imagine this overhead at Kith-scale! So, instead, we take this query vector (remember, point in space), and find other items (i.e. other points in space) that are approximately the closest (i.e. nearest) to the our Niobiums. Think of ANN as a (alternative methods exist) series of steps that helps us make this “distance” calculation.

Next. What’s an inverted BM25 index? This is a technique that I wasn’t familiar with, but again, it sounds more intimidating than it actually is. It’s considered “inverted” as it flips the usual function of an index. Instead of storing “documents” (think ~files) & their contents (i.e. data) it stores “terms” (e.g. text like “rubberized” or “toecap”) & a list of “document IDs” that contain said term. Par example, let’s say we have three documents in our Kith database:

document 1: "Rubberized heel blue" 
document 2: "Toecap chrome finish"
document 3: "Rubberized compressed toecap with t99 cut"

Our inverted index would look like:

"rubberized": [1, 3] <--- this array is known as a "posting list"
"toecap"    : [2, 3]
"heel"      : [1]
... u get my drift

The “BM25” (or Best Matching 25) part is a “scoring function” widely used by search engines to rank documents by relevance to a query. This function takes into account variables like: 1) term frequency, 2) document length, 3) how common/rare terms are, etc etc. It should hopefully be clear which document above would be considered most relevant to a query like “Rubberized toecap”. Math buffs, if u want the BM25 function here it is:

Oh man. This optimisation work by tpuf only gets better (see below). I’m starting to believe that if an infra company can’t write architecture docs that include “optimised for X” >= 5 times they may not have “it”. Gotta lean into your thing vs. someone else’s.

Ngl, I hadn’t heard of SPANN (see here) or centroids either. To grok SPANN we need to understand centroids, so let’s start there. Centroids are essentially “representative” vectors, meaning that they themselves aren’t real datapoints in vector space. Instead, they’re at the centre (i.e. the mean) of a group of other real points in space. I’m sure that makes almost no sense… so I’ll break it down via an example:

Let’s go back to our Kith example (hope you all enjoyed this pod). When building our product recommendation engine, we might have 1 million images (i.e. vectors) in our shoe dataset. That’s a lot of data. So, we could use a clustering algorithm like k-means to “cluster” (i.e group) these 1 million vectors into 1 thousand clusters to “shrink” our “search space” a little. We then use centroids to essentially identify (i.e. give it its own point in vector space) each cluster. Smart.

Ok, so, what is SPANN? Or.. wait for it.. “Simple yet efficient Partioning-based Approximate Nearest Neighbour” (😵‍💫) search. Wow, nothing “simple” about that.

Luckily for us, because all of the work we’ve done in this primer already, it is in fact simple to grok. Basically, the core idea behind SPANN is that it stores centroids in memory (think our NVMe SSD cache) and the underlying vectors that map to these centroids (weirdly, also called “posting lists” despite not being an inverted index) on disk storage (i.e. our vanilla object storage). Simple as that. Naturally, this work leads to another performance bump.

What should we build then? Well, I’ve kept the below advice aggregated blog post up-to-date for some time. So, what if instead of visiting it to look for worldly wisdom, I could just.. query it? (p.s. this page lives on GitHub so add to it via a pull request if ya like).

Very doable with tpuf. I’ll show you how, but let’s see it in action first. What results do I get back for the query “chips on shoulders..”? So cool.

Let’s see this reflected in turbopuffer’s dashboard too, and then I promise we’ll dig into the code. 55 documents (remember, vectors) in there, nice. Looks like tpuf’ll be passing my own “toothbrush test” (see result 10; please laugh at my joke) at this rate.

Ok, now for the code. I feel compelled to add that I don’t think this is in any way a “difficult” application to build. I thought about doing something more challenging with StockX’s Sneaker Data Contest dataset. Maybe over the holidays. Anyway:

import os
from dotenv import load_dotenv
import turbopuffer as tpuf
from openai import OpenAI

# Load environment variables from .env file
load_dotenv()
turbopuffer_key = os.getenv("TURBOPUFFER_API_KEY")
openai_key = os.getenv("OPENAI_API_KEY")

tpuf.api_key = turbopuffer_key
client = OpenAI(api_key=openai_key)

# Create namespace
ns = tpuf.Namespace("advice-aggregated-search2")

# Function to get embeddings from OpenAI with error handling
def get_embedding(text):
    try:
        response = client.embeddings.create(
            model="text-embedding-ada-002",
            input=text
        )
        return response.data[0].embedding

Request to Substack — make prettier code blocks? Would some colour hurt anyone? Jeez. Anyway, as you can see I’ve bolded what matters in this section of my code. We 1) create a “namespace” (a collection of data) in turbopuffer & 2) create a function that will turn any text we supply (i.e. our advice aggregated list) into word embeddings.

Next, we create an array of “strings” (i.e. text) that contains our list of advice. This is the text that will be the input into our “get_embedding” function above. We then run (or “call”) our `get_embedding( )` function to actually turn this text into vectors. Easy.

advice_items = [
    '"People that scare me" - Graham Duncan on hiring Analysts that are capable of being better than you.',
    '"Look for infinite problem spaces" - Will Gaybrick on market selection.',
    '"Patience is arbitrage" - Mike Maples on the importance of playing long-term games.',
    '"In order to make delicious food, you must eat delicious food." - Karri Saarinen on cultivating taste (via Jiro Dreams of Sushi).',
    '"Moral authority" - Peter Fenton on a founders ability/right to create change in an industry.',
    '"Focus on both music making and conducting" - Josh Kushner on how to refine your investment picking whilst management responsibilities scale.',
    '"Unique and compelling value proposition" - Pat Grady on gross margins & operating margins.',
    '"Suck the oxygen out of the air" - Pat Grady on winning founder mind share. (about Sarah Guo!)',
    '"Do good deals" - Anon on becoming a great investor. "Obvious" - but very easy to climb the wrong hill',
    '"Look at the steepness of the slope" - Geoff Lewis on assessing talent.',

...... etc

# Create vectors for each piece of advice
for idx, advice in enumerate(advice_items):
     try:
         embedding = get_embedding(advice)
         if embedding:  # Only proceed if we got a valid embedding
             ns.upsert(
                 ids=[f"advice_{idx}"],
                 vectors=[embedding]
             )
             print(f"Processed item {idx + 1}/{len(advice_items)}")  # Progress indicator
             time.sleep(0.5)  # Add small delay to avoid rate limits

print("Vectorization complete!")

Ok, final part and then we’re all wrapped up. Next, we turn our desired search query (remember, “chips on shoulders”) into a vector too. This will enable us to query our namespace (using the `ns.query( )` function below) by searching for those vectors that are approximately “close” to the query in vector space. We then just print the results.

search_query = "chips on shoulders"
query_embedding = get_embedding(search_query)

# Query the namespace for similar vectors
results = ns.query(
    vector=query_embedding,  # The embedding of your search query
    top_k=10,  # Number of results you want to return
    distance_metric="cosine_distance",  # Assuming cosine distance is appropriate
    include_attributes=[],  # Adjust based on what attributes you need
    include_vectors=False
)

# Print the results
for idx, result in enumerate(results):
    # Debugging: Print the result object to understand its structure
    print(f"Result {idx + 1}: {result}")

    # Assuming result has attributes like 'id' and 'dist', access them directly
    item_id = result.id  # Accessing as an attribute
    score = result.dist  # Use 'dist' instead of 'score'

    # Get the original advice text using the ID (it will be something like 'advice_27')
    original_idx = int(item_id.split('_')[1])
    print(f"\nResult {idx + 1}:")
    print(f"Text: {advice_items[original_idx]}")
    print(f"Similarity Score: {score}")

& that, dear readers, is the end of our primer on turbopuffer. I hope you had as much fun reading as I did writing this one. A special thanks to the tpuf team again. Oh &, subscribe below, and, as always, feel free to say hi at alex@tapestry.vc

For new folk here, hi, I’m Alex Mackenzie, a partner at Tapestry. I’d love to hear from any of you reading along: alex@tapestry.vc

Recently I listened to Kenny Beats’ (“Whoa, Kenny!”) interview on Tetragrammaton. I’d recommend tuning in for many reasons, but it got me thinking about the ‘edges of the internet’. Shortly after, I thought about queuing for Glastonbury tickets, which got me thinking about.. browser automation.. and thus, this primer was born.

By ‘edges of the internet’, I mean places that you personally frequent that feel a little proprietary..? They’re earned. For me, that could be Factorio’s blog, PC’s fast list, or, perhaps, Polymarket’s Chess prop bets. Who ya got?

The problem with many of these ‘edges’ is that they’re often webpages, not webapps. You have to remember to visit ‘em, write ‘em down somewhere, parse them for the content you enjoy, etc. Or, even when they are webapps, you’re beholden to an API reference. What if you want access to data that an API doesn’t provide?

This is where scraping and browser automation (both of which we’ll dig into in detail) come in. Theoretically, with these two techniques, you can turn any web-page into a web-api. We’ll learn how later, by having a bit of fun and building our own HN tool.

It’s likely at this point that you’re thinking ‘isn’t scraping and browser automation old news?’. In short — yes. However, have you ever tried to build a durable web scraper? If not, just know that it’s not exactly trivial. Plus, Selenium kinda sucks.

There’s also a genuine opportunity to put LLMs to work here which we’ll dig into. Personally, I’m very excited about never having to write a line of .*(?!regex).* again. Whoa, Kenny! indeed.

Accordingly, I was happy to see Paul and the team at Browserbase taking things up a notch. Fwiw, I’ve been using Browserbase for weeks now and I’m a happy (paying!) customer. So, let’s see what all the fuss is about.

Ok, so, as discussed, we’ll learn about the evolution of scraping, browser automation, and Browserbase itself by building a Hacker News tool. It will:

1. Run every morning at 9am via a cron job (check out Northflank).

2. Pull the top 30 posts (i.e. the ‘front page’) on the HN site (Browserbase).

3. Click into the hyperlink of each post (Browserbase).

4. Summarise each post (ChatGPT API).

5. Send an email about these posts (Resend).

Note 1: I’ll include the code I use so that folk can follow along, but you don’t need to fully grok the code itself. If you get stuck, or want more context, try throwing the code in your favourite LLM and asking some Qs.

Note 2: If you’re following along, I recommend using Flox to install the dependencies we use. It’ll keep your system clean & conflict free.

1. Scraping 101

Whilst I was, correctly, far too occupied battling the Elite Four (iykyk) in the 1990s to care, I’m told that web scraping was pretty straightforward “back then”. Websites essentially looked like this: http://thegestalt.org/simon/ps2rant.html.

I.e. they were often walls of text with no real interactivity or client-side rendering. And yes, this post is the co-founder of Fastly complaining about the PS2. I did tell you, I spend far too much time on the internet.

Let’s see how we can scrape websites that look like this. Let’s first use cURL (fun fact: prev named httpget and urlget) to retrieve the webpage:

This is fine, but cURL doesn’t include any built-in text processing tools, so we have to use command line utilities like ‘grep’, ‘awk’ or ‘sed’ to process this data further. Par example, if we want to extract the URL’s included in Simon’s post we’d use ‘awk’:

‘awk’(ward) being the operative word here, this command is hard for me to interpret, and I wrote it yesterday.. things can get unwieldy, fast, if we want to do more complex text processing:

curl -s http://thegestalt.org/simon/ps2rant.html | \
sed -n 's/.*>\(.*\)<.*/\1/p' | \
tr '[:upper:]' '[:lower:]' | \
tr -c '[:alnum:]' '[\n*]' | \
awk 'NF' | \
sort | \
uniq -c | \
sort -nr | \
head -n 10 

I’d more readily read Cuneiform! Let’s keep in mind however, that Simon’s webpage is very basic by modern standards (although I do like the recent vanilla html ‘comeback’). Brace yourselves, as you see what happens when we ‘cURL’ Hacker News:

Eugh! Modern webpages have a lot more going on. Also, it’s not like Hacker News is going to win a Webby anytime soon (sorry, PG); it’s still pretty basic. ‘BeautifulSoup’ and ‘Requests’ came along in 2004 and 2011 respectively to make parsing modern webpages like this a little easier.

Let’s use Requests (for making http requests) and BeautifulSoup (for html parsing) to visit the Hacker News page and extract every title on the front page as well as their corresponding hyperlinks. I’ll bold some functionality that I think is worth noting but, don’t fret if this code is confusing. I’ll elaborate on what’s important.

file name: hacker_news_bs4.py
------------------------------------------------------------------------

import requests
from bs4 import BeautifulSoup
from typing import List, Tuple

def scrape_hacker_news() -> List[Tuple[str, str]]:
    # Base URL for Hacker News
    url = "https://news.ycombinator.com/"
    
    # Send GET request
    response = requests.get(url)
        
    # Parse HTML content
    soup = BeautifulSoup(response.text, 'html.parser')
    
    # Find all story titles (they are in 'titleline' class)
    stories = []
    
    # Find all story rows (they have class 'athing')
    for story in soup.find_all('tr', class_='athing'):
        # Find the title span
        titleline = story.find('span', class_='titleline')
        if titleline:
            # Get the first link in the titleline
            link_tag = titleline.find('a')
            if link_tag:
                title = link_tag.get_text()
                # Get href attribute, if it's a relative URL, prepend the base URL
                link = link_tag.get('href')
                if link.startswith('item?'):
                    link = url + link
                    
                stories.append((title, link))
    
    return stories

def main():
    try:
        stories = scrape_hacker_news()
        
        # Print the results
        print(f"Found {len(stories)} stories:\n")
        for i, (title, link) in enumerate(stories, 1):
            print(f"{i}. {title}")
            print(f"   Link: {link}\n")
            
    except Exception as e:
        print(f"An error occurred: {e}")

if __name__ == "__main__":
    main()

If this is your first time looking at BeautifulSoup it may look a little overwhelming; the most important item to note is that the library gives you tools (or functions) like ‘.find_all( )’ or ‘.get_text( )’ that can search for html like anchor tags or specific CSS class names like ‘athing’. These libraries ultimately make parsing webpages a hell’uva lot easier.

Let’s see what our HN code returns when we run the above script:

Nice, we’ve made progress here (Bocker looks cool). But remember, we also want to ‘click’ into the hyperlink of each post. We can technically do this with BeautifulSoup and Requests, but we’ll likely run into some difficulties which I’ll outline in the next section below.

2. Along Came Browser Automation

With the advent of JavaScript (which I wrote about in my primer on Deno), came the ‘dynamic’ webpage. JavaScript enables websites to render, and/or change, ‘client-side’ (i.e. on a user’s device) vs. on a server.

Celtic and Rangers fans are more likely to ‘get on’ than those debating the pros/cons of client-side rendering, so let’s not get into the weeds here. Instead, let’s understand why client-side rendering spoiled our (Beautiful)Soup via an example.

Recently, I tried to scrape the Public Comps (non-gated) dashboard using Requests and BeautifulSoup. When you do so, you receive a bunch of HTML (mainly just SVG coordinates), but no company data. Visit the site, and have a guess why this may be?

Go on. Visit it.

This data is rendered client-side! How this webpage likely works is that when it’s first loaded, a basic HTML skeleton is rendered. Then, the webpage itself will make an API call (or ‘fetch’) the underlying data that populates the corresponding charts. The issue with BeautifulSoup & Requests is that it:

1. Only sees a webpage’s initial HTML.

2. Can't execute JavaScript (e.g. clicking a button) to render dynamic content.

3. ~Can't make the subsequent API calls that load the actual data.

4. Can’t authenticate (i.e. log-in) via modern auth to a given website or webapp.

I’m sure many of the links returned by our HN scraper will act similarly to Public Comps, and so, we need an alternative approach. Something more.. robust. Along Came Browser Automation.

Let’s take a step back for a second. When we’re using the Google Chrome ‘app’ ourselves, we don’t run into any of the above issues, right? Cool animations run, buttons can be clicked, images are downloadable, et cetera.

We ‘control’ this Google Chrome app with our mouse and keyboard and all works swimmingly. So, what if we could control this same app, but with code? This was Jason Huggins’ core idea with Selenium back in 2004.

Let’s build our HN tool using Selenium this time. Remember, I think ‘Selenium kinda sucks’. I won’t share my Selenium code (it will overcomplicate things), but I will share snippets that’ll highlight its flaws. But first, proof that Selenium can get the job done:

Ok, so why the Selenium boeuf? Well, firstly, look at the extensive set of dependencies and complex config it requires:

# There's a bunch of 'scaffolding' that needs to be imported

from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from selenium.webdriver.chrome.options import Options
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.common.exceptions import TimeoutException, NoSuchElementException
import time
import json

# Configures the Chrome web driver

def setup_driver():
    options = Options()
    options.add_argument('--headless')
    options.add_argument('--disable-gpu')
    options.add_argument('--no-sandbox')
    options.add_argument('--disable-dev-shm-usage')
    options.add_argument('--window-size=1920,1080')
    
    service = Service(ChromeDriverManager().install())
    driver = webdriver.Chrome(service=service, options=options) 

Eugh! Again! Cursor (see my primer here if interested) has made my life so much easier now, but I used to spend hours getting this set-up right. This used to take all of the fun out of building scrapers. I guess this has changed now..

..what hasn’t changed, however, is how ‘brittle’ Selenium’s CSS selectors are. You might recall that our HN scraper looks for the CSS class called “athing”; this is fine, until the developers of HN decide to change their class naming conventions. If this class changes, and is now called “athing2”, our entire scraper breaks. No bueno.

There’s a plurality (minimal CAPTCHA support, high resource usage, memory leaks) of other small issues that add up to make the maintenance of a Selenium script a bit of a nightmare. Hopefully, you’ve built a sense for why we might want something a little different? Some names such as Playwright, Puppeteer and, now, Browserbase, came along to help us all out.

3. Browserbase - APIs For The Internet

Ok. So, what is Browserbase? Let’s go with their definition which we’ll then break down in our own way:

‘Browserbase offers a reliable, high performance serverless developer platform to run, manage, and monitor headless browsers at scale. Leverage our infrastructure to power your web automation and AI agents.’

Let’s start with what is likely Browserbase’s most experimental (yet profound) feature, Stagehand. I’ve always liked Josh Wolfe’s framing of ‘following directional arrows of progress’; in browser automation’s case, we’ve been trending towards selecting html and/or css elements via natural language for years.

Why? Well, the end goal has always been to get your code to ‘click the log-in button’, but until LLMs, we had to use weak proxies such as: ‘page.click(".login-button")’ to do so. No longer. How so?

Well, at a (very) high-level, Stagehand consists of two functions: ‘act( )’ and ‘extract( )’.

‘act( )’ allows Stagehand to interact with a webpage. If we visit End Clothing, we could provide the act function with some natural language like ‘act(“search for Palace hats”)’ or ‘act(“click on the first shoe”)’ & Stagehand will dynamically generate the underlying Playwright code to get the job done. I cannot stress enough how cool this is.

‘extract( )’ does what you’d likely expect.. it grabs structured text from the webpage. Par example, we could ‘extract(“the price of the first shoe”)’ or ‘extract(“the navbar”)’. Hopefully, you’re starting to see how leveraging natural language makes our browser automation much more robust, or, reliable (see definition above). Let’s see if these functions enable us to build our HN tool in its final form?:

There ya have it. Our Hacker News Daily Summary email. Nice one Browserbase.

Browserbase does a lot more on the reliability front though. Remember, most sites don’t like browser automation, or, ‘bots’, so they’re constantly trying to thwart our efforts. Boo! To combat bots, websites will try and ‘fingerprint’ (i.e. record and recognise) your device by tracking attributes such as your: browser version, IP address, website activity, etc. It’s really quite smart.

So, how do we fight back? We won’t dig into everything Browserbase does, but I think it’s worth understanding ‘residential proxies’ as one arrow in our quiver. As noted, one way to spot-a-bot is to track IP addresses. If a given IP address consistently interacts with a site in some predictable manner, at a predictable interval, the site may block its access or throw a CAPTCHA at it.

So, logically, one way to fight back is to not use our own IP address, but instead, route our request through another server (or, ‘proxy’) to mask our presence. The ‘top grade’ of proxy servers is a ‘residential proxy’, which is a legitimate device (think a personal computer or router) that belongs to an actual household. Browserbase enables us to tap into a configurable network of these proxies. Pretty sweet.

Alright, let’s dig into Browserbase’s developer platform and then we’ll wrap up. And before you say it.. I know my product screenshots don’t exactly look like Linear’s.. but hey, I guess my writing is ok.

So, what you’re seeing above is where the action is — Browserbase’s ‘Sessions’ tab. Each time we run a browser automation with Browserbase we get a full play-by-play of what happens.

For each ‘session’ (i.e. each time we run our script) we can see: what pages we visit, the page’s corresponding ‘DOM’ (or ‘document object model’), the network requests (e.g. API calls) a given website makes, etc. This is super helpful for understanding exactly what our scraper is doing, as well as for debugging any reliability or performance issues. Finally, browser automation has a modern developer experience!

So.. yeah, I guess we’re kind of done here. Well, publicly at least; I have so much cool sh*t I want to build behind the scenes for us at Tapestry.. but I couldn’t share that :) I would encourage you all to visit your own ‘corner of the internet’ and ask yourself, could I turn this into an API? If so, fire away and let me know what you think of Flox, Northflank, and, of course, Browserbase 😎

Oh and subscribe for my next primer on Turbopuffer! (& if someone can get me beta access to Turbopuffer, do a brother a solid). Thank you all for reading, I have a tonne of fun writing these.

Last Friday a friend and I were sipping one of the Lore of the Land’s finest stouts when he asked: “why Cursor?”.

You see, I shared with him that I’ve tried the gamut of IDEs and VS Code extensions: PyCharm to Zed, Codium to Copilot, but that Cursor has been the only one to truly.. “stick”. When expounding on my answer, I noticed myself rather haphazardly listing off product features that I enjoy vs. providing him with a crisp response.

As I’ve thought more about this moment, I’ve realised that within this feature list actually lies an explanation I’m ok with. Many of my most beloved products and brands [1][2][3] string together seemingly inconsequential product decisions that, in aggregate, make them truly different. Par example, I didn’t expect Nothing’s glyph interface to change my life, but it kinda has.

In Cursor’s case, these small decisions are derived from the LLM being a first-class citizen as opposed to an afterthought. For example, the fact that Cursor’s activity bar is horizontal vs. vertical is trivial, but it likely means that they’ve found another 100+ ways to create space and context for Cursor Chat. The collective is far less trivial.

Herein lies another learning. The word “native” (cloud-native, LLM-native, etc) gets thrown around a lot in tech, but it’s a genuine source of advantage if you know what to look for. My high-level framework is that being “native” (i.e. building from the ground-up with) to a technology should enable you to either:

1. Deliver an existing feature far better than your competitive set who’re built on alternative technologies or architectures. E.g. code linters have been a thing for a long time, but Cursor understands what you’re doing-next vs. do-ing.

2. Offer an entirely new feature that can’t be provided by your competitive set who’re built on alternative technologies or architectures. E.g. with a `@web`declaration, Cursor can visit the docs of a given api.

As is evident from the above, Cursor benefits from both types of “native” advantage. Nice. Puff pieces are a little boring though.. so I decided to try and test the limits of Cursor and write something a little more critical (and more constructive, I hope!) by attempting to build Next.js Conf’s beautiful new website (partially built by my talented friend Facundo) without writing a line of code myself.

Below is a stream of product observations and requests derived from this experiment. Enjoy! Critique! Frame on your wall! Oh & say hi if you like: alex@tapestry.vc

Observation 1 - 80% feels like 100%

Typically, when we read our favourite newspapers we find them insightful. But, have you noticed that when the publication writes about something you consider yourself an “expert” on their perspective is a little.. lacking?

This same phenomenon seems to be occurring with Cursor. For someone that’s never built a navbar before, doing so with a single prompt is an absolute game-changer (just ask Positive_Box_69). But, it took me 10+ prompts to get my navbar remotely close to Next.js’ (& I’ve built a few too many navbars in my time). For the uninitiated, getting 80% of the way there feels like they’ve completed it mate.

Now, perhaps I’m a poor prompt designer. So, see below my system prompt or, “Rules for AI” (note the use of role-playing), as well as my attempt to create said navbar via Chain-of-Thought prompting.

Note: I’m sure there are many ways that I could improve my prompt (e.g. specifying the inclusion of Next.js “Geist” font) but as you’ll see below, my prompt is reasonably extensive.

// Rules for AI

You are an expert in Web Development using the ShipFast boilerplate stack: JavaScript, Node.js, React, Next.js App Router, Tailwind
CSS, DaisyUl, NextAuth, MongoDB and Mongoose.

Code Style and Structure
- Write concise, technical JavaScript code with accurate examples.
- Use functional and declarative programming patterns; avoid classes.
- Prefer iteration and modularization over code duplication.
- Use descriptive variable names with auxiliary verbs (e.g., isLoading, hasError).

- Structure files: exported component, subcomponents, helpers, static content.

Naming Conventions
- Use kebab-case for directories.
- Use camelCase for variables and functions.
- Use PascalCase for components.
- File names for components should be in PascalCase. Rest of the files in kebab-case.
- Prefix component names with their type (e.g. ButtonAccount.jsx and ButtonSignin.jsx, CardAnalyticsMain jsx and CardAnalyticsData.jsx, etc.)

Syntax and Formatting
- Use the "function" keyword for pure functions.
- Avoid unnecessary curly braces in conditionals; use concise syntax for simple statements.
- Use declarative JSX.

Ul and Styling
- Use DaisyUl and Tailwind CSS for components and styling.
- Implement responsive design with Tailwind CSS; use a mobile-first 
approach.

Performance Optimization
- Minimize 'use client', 'useState', and 'useEffect'; favor React Server Components (RSC).
- Wrap client components in Suspense with fallback.
- Use dynamic loading for non-critical components.
- Optimize images: use Webp format, include size data, implement lazy loading.

Key Conventions
- Optimize Web Vitals (LCP, CLS, FID).
- Limit 'use client':
- Favor server components and Next.js SSR.
- Use only for Web API access in small components.
- Avoid for data fetching or state management

// Prompt
// Includes screenshot of the navbar from next.js site

I would like for you to recreate the image I have attached to this prompt.

Context:
- We currently have a "default" next.js site that's generated by running 'npx create-next-app@latest'.
- We're using an "src" file with @layout.js and @page.js .
- The image is of a "navbar" on the @https://nextjs.org/conf website.

Instructions:
1. Let's start from left to right on the navbar elements. Describe the navbar image.
2. Describe each element within the navbar. Including the elements' colors, specific shapes, likely border radii in pixels, etc.
3. First, create the "Next.js" text and the div that contains the "CONF24" text. The div should have a black border of .5px.
4. This "Next.js" text and div should be aligned left, about a "third" of the way across the page.
5. Next, create the "box" (a "short" rectangle) that contains the text "/". The border color of the box should be light grey.
6. Ensure that this "box" is at the centre of the navbar.
7. Next to this "box" should be the text "CHAT".
8. The "NEXT.JS On [insert black triangle]", "LOG IN" text should be aligned right. In the final "third" of the page.  
9. Finally, there should be a button to the right of the "LOG IN" text. It should have a dark blue background color, its border should be transparent with a 0px radius, its text should read "GET TICKETS" and the text's color should be white.
10. Please note that next to the "GET TICKETS" text is a white arrow with a circle around it with a white border color.   

See my result here. I mean, it’s pretty impressive, but it’s about “20%” off of the end result. Would I be able to discern where it’s wrong if: 1) if I didn’t have a reference image, and more importantly, 2), if I wasn’t comfortable with frontend dev? Would I have the “vocabulary” to make small padding or margin refinements? It’d be tricky.

Observation 2 - Cursor is nothing without its people models

Stripe is an incredibly important piece of infrastructure for many companies new and old. But, if you asked me to describe, say, Linear’s core value props, I likely wouldn’t throw “payment processing” into the mix.

Perhaps when describing something like Airbnb I would allude to Twilio’s involvement, but again, is receiving a text message about your upcoming stay in Azenhas do Mar what makes you use or switch over to Airbnb? Unlikely.

There is, however, a class of companies where their underlying infrastructure is a core component of the product experience: Vimeo is a video platform that uses Mux, Tide is a bank that leverages ClearBank. This isn’t necessarily a bad product trait, although it can increase a company’s platform or “steamroll” risk.

Cursor, and many LLM-native companies fall into this bucket. So much so that when folk say “Cursor helped me build X” they should be passing at least ~50% of this praise onto Claude et al. Though, Cursor is already astutely addressing this risk with cursor small.

As the company continues to add product depth by focusing on some v interesting problems [1][2] I think this 50% attribution rate will begin to shift. In fact, Cursor’s increasing ability to inject the “Optimal Context” into LLMs may end up flipping their relationship with LLM providers on its head. Will be an interesting watch.

Observation 3 - In-Context Editing

When working on my Next.js clone I often found myself hopping into Cursor Chat to make minor edits (think changing padding, easing animations, etc). As someone that’s very comfortable with CSS this was strange, why didn’t I just change 1px to 2px or #87CEEB to #000080?

The thing is, you kinda get used to “writing code” in English, so my instinct was to continue to do so: “move that div a little to the left”. This isn’t a big deal when you’re familiar with the language/library that you’re using as you have the vocab: “move the <div className=”example”> 10px to the left”, or, if all else fails, you can just jump into the code and fix things.

But what if you don’t have the words? If someone has never worked with three.js before, they’ll have no clue how “alphaMap” impacts the opacity of their 3D object. Thankfully, three.js (& more specifically, lil-gui) might suggest an answer here.

lil-gui allows you to edit certain variables graphically so that you build a sense for how they impact your 3D scene: threejs.org/examples/#webgl_animation_skinning_morph.

What if Cursor could generate similar “ephemeral” and/or “malleable” UIs, but in real-time? Having briefly thought about how you’d scale this, it seems plausible given that Cursor can crawl third party docs, etc. This would be very cool.

Observation 4 - Selective Memory

Admittedly, I thought that Bill Gurley was erring on the side of bloviation when discussing the memory challenges of SOTA models [1][2][3]. Unsurprisingly, he’s totally right. But I’ll add one slight caveat — LLMs have “selective memory”.

Let’s dig into some examples. I was recently working with Twitter’s API (which, btw, is now extortionate) and authenticated via OAuth 2.0. Sweet, all was going great. But, as I iterated through other errors within my codebase, I ran into an auth issue. Odd?

At some point, the LLM decided to replace my OAuth 2.0 boilerplate with OAuth 1.0. I.e. the model, despite having the context that Cursor provides, “forgot” that I strictly asked for it to work within the OAuth 2.0 spec. This is likely due to the bulk of twitter API code on the internet using OAuth 1.0 (Twitter’s V2 endpoint that uses OAuth 2.0 was only released in 2021).

Now, before anyone says it. Yes, I did accept the change that Cursor suggested, so the blame is technically, on me, but either way, the code suggested was incorrect. I’m only human, and perhaps LLMs are too.

What do I mean by “selective” though? Well, I’ve found that there are certain prompts/instructions that models treat as sacrosanct, even as your latter prompts contradict them (or even cancel them out entirely). In these cases, the model’s long-term memory is nearly “too good”.

Observation 5 - Local Optimum Loop (LOL)

I was trading notes on Cursor with a friend who summed up my own experience pretty well:

“I've had similar experiences with trying to do complex animations. Very easy for the output to get stuck in a local optimum and just loop. I find that I often have to "jump start" it by rewriting the existing code more significantly.”

For me, this issue was most apparent when attempting to implement the “NEXT grid”. Cursor did a truly awesome job of creating a crude “NEXT” outline by using CSS grid, but no matter what I tried (including using three.js instead), it couldn’t get anywhere near the finished product. Instead, I entered the “loop” described above.

Given my “experiment” involved writing no code myself, my attempt at “jump starting” the LLM came via narrowing the aperture of the task. So, my prompt went from detailing how the entire “NEXT” grid should look, to focusing on a single letter, and eventually, a single square within a letter.

This approach initially improved results, but its efficacy decayed and I gradually found myself back at the “local optimum loop”. This did get me thinking about how much I still have to learn about prompting though!

Ok, that’s a wrap to my “addendum” of problems that I’m equally excited to see Cursor solve. Never having to read a wall of stack traces again sounds like a pretty ideal end state to me. I’d love to say hi/spar with others who’ve been playing around with Cursor: alex@tapestry.vc. Thanks for reading, folk.

Whilst it’s not required, if you want to follow along, you can install Flox in an instant here.

Oh and thanks for the 1k subs. Pretty cool!

What’s cool about technology investing is that amongst all the hysteria, there are technologies out there, laying in plain sight, that are criminally underrated. I recently pointed to canaries as but one example, but let’s not forget that the transformer was hanging around ~5 years before ChatGPT.

Today, dear readers, I’m going to layout why I think that the world’s 20 million-something developers (not to mention agents) will, eventually, all be using Flox.

Some background: I personally use Flox for every project, pretty much everyday, and I can’t think of a reason why I would pick Docker over it. This is particularly true when running AI models locally.

Even better, I don’t use Flox for just one task. The reason why Nix (Flox’s underlying technology) is infamously difficult to grok is that it is many things. If anything, Nix is more of a philosophy, an opinion. Thankfully, Flox takes those many things and makes them a little more.. approachable.

Want a reproducible developer environment like Python’s venv or Conda? Cool, Flox can do that, but like, it actually works. Want a package manager like npm? Yeah, Flox does that better too. Want to build packages? Again, Flox is the correct way to do this.

Even better, Flox passes Sergey Brin’s “toothbrush test”. It’s something that every developer should use every hour of every day. Products with these characteristics, naturally, capture our attention at Tapestry.

So, dear readers, my gift to thee is to: 1) explain why I feel so strongly and 2) help you install Flox yourselves. Yeno, if that’s your kinda thing.

Alright, so something a few of you enjoyed during my piece on Restate & Durable Execution is when I explained Restate through sections of their website. I figured that with a website as pretty as Flox’s, it’d be rude not to flaunt it a little too: flox.dev

Ok, so firstly.. what is a “dev environment?”.

As developers, we use a tonne of third party software. For example, instead of writing the raw python code to do some specific Tapestry VC portfolio data analysis, I might use a third party library or “dependency” like Pandas.

To use Pandas on my “machine” (i.e. laptop), I need to install it. If it’s not there already of course. I typically do this with a python-specific (take note, this is important later) package manager like Pip:

Alas, this is where my problems begin. Look at what happens when I run the above command in my terminal:

It turns out that when we install the Pandas library, we’re also installing specific “versions” of other libraries that Pandas itself relies on. In this instance, we see that Pandas “requires” Numpy. Specifically, it requires any version of Numpy that is greater than 1.22.4 but less than 2.0.0.

Hmm. But what happens if I have the latest version of Numpy (which is 2.0.0) installed on my system already? I.e. before I installed Pandas. Then we’re in trouble. Why? Well, if we then install Pandas, we now have two versions of Numpy on our machines. C’est très problématique.

What’s the problem? Well, how does my machine know what version of Numpy it should use for my new project? Whilst we do have some smart ways of “resolving” these “dependency conflicts”, we still often run into issues. I assure you, there is nothing more frustrating than dependency conflicts as a developer.

This is a problem that every developer runs into, and hence, one where every developer needs some solution. This is where “developer environments” kick in. You’ve probably heard of a few of these guys in the past: venv, pipenv, conda, nvm, rvm, etc. When you learn to code in Python, venv is likely one of the first three tools you pick up. Let’s ‘av a look at how venv works:

Ok, so, here I’ve just created a developer or “virtual” environment and called it “myenv”. Let’s now “activate” this environment and then I’ll explain what this accomplishes:

Cool, I’m now “inside” my developer environment called “myenv”. Just think of developer environments as folders that store the dependencies a specific project requires. See below that “myenv” is a folder in the `whynow_code_examples` directory alongside some other Why Now folders [1][2]:

Within these environments, when we install packages like Pandas, they’re “isolated” or “contained” (sound familiar?) within the directory of our project.

Hence, this time when I run `pip3 install pandas` my terminal doesn’t tell me that the requirement of Numpy is “already satisfied”. Why? Well, my developer environment is isolated. It doesn’t have access to dependencies that are installed elsewhere. Thus, there’s no (or at least, less) room for dependency conflicts! Nice.

Ok, we’ve made a lot of progress here. One final thing to understand about developer environments is that they’re intended to be “portable”. I as SickSoftwareEng34 want to ensure that my esteemed peer BasedDev22 can successfully run the software that I’ve written, on their machine also.

In order to do this, they need to “reproduce” the exact developer environment that’s on my machine, on their own machine. I.e. they need specific versions of Pandas, Numpy, etc.

When using venv, and many other developer environments, we achieve this by using a requirements.txt file. Let me show you how:

Running `pip3 freeze` within my terminal outputs all of the dependencies installed by pip within a given directory (in this case, our developer environment). We can then save, or “pipe”, the output of this command to a text file like so:

Ok sweet, we now have a `requirements.txt` file that contains the `pip3 freeze` output:

Things are quite simple from here. I can then share this file with BasedDev22, they’ll run `pip3 install -r requirements.txt` et voilà, they’ll install the exact* same dependencies in their environment.

*Of course, I wouldn’t be writing this primer on Flox if things worked so.. swimmingly. Tools like venv are far from complete.

First up in the case of Alex Mackenzie vs. venv, 2024 is that venv isn’t a package manager (yep, I said it). I need to use a separate tool, pip, when installing packages. I’m aware that this may sound a little pedantic.. but I’d prefer a consolidated toolset here.

Next, venv is language-specific. If I’m writing a node-based (i.e. JavaScript) web app, I can’t manage my JavaScript dependencies with venv. As a developer this is a PITA, as it means that I have to use an alternate tool for JS et al. Habits aren’t formed this way.

More generally, other language-specific environment or package managers like npm (JavaScript), Cargo (Rust) and RubyGems (Ruby) are similarly not language-agnostic.

Finally, package managers like npm et al can’t install “system-level” dependencies (e.g. compilers or other utilities). For this, we need to use other package managers like Homebrew for macOS or APT for Linux. Again, I hope the fragmented toolchain is overt here.

Ok, so to sum up our current woes as developers just tryin’ to get our jobs done:

• Language-specific package managers like pip or npm aren’t versatile enough to accommodate the other programming languages I use.

• Language-specific package managers can’t install the system-level dependencies I often need to install.

• Many environment managers aren’t package managers (venv), many package managers aren’t environment managers (Homebrew).

My real “a ha!” moment was when I realised that Flox (or, Nix) solves all of the above challenges (& some other pretty gnarly ones). As mentioned, you can install Flox in an instant if you want to follow along below. It’s totally fine to follow along without doing so too.

Alright, I’m now back in my `whynow_code_examples` directory:

Let’s spin up a Flox developer environment with the `flox init` command. This should feel quite familiar as Flox’s workflow is similar to that of venv’s:

Great, I’ve now created a virtual environment. Let’s “activate” the environment, climb inside it, and then I can show you some of Flox’s magic:

Want to install Python’s Pandas library? Let’s do it. Want to install JavaScript’s Node.js? We can do that too. We can even give Homebrew a run for its money and install a system-level utility like htop. Flox is both language and platform agnostic:

At this point you should have hopefully realised that Flox is acting as a developer environment too. When we’re inside our Flox environment, we can run `htop`:

When we `exit` outside of our Flox environment, we can’t:

Hmm. It would appear to me that Flox has just solved the three problems enumerated above? Furthermore, my workflow used to require an array of tools, now it’s converging towards just one. Nice work Flox.

Next - remember our `requirements.txt` file? Well, you guessed it, with Flox we don’t need it either. If I want to use a colleague’s work, I can simply activate their “remote environment” (published to FloxHub) and with one-command, their software works flawlessly on my machine (irrespective of my OS).

Flox’s Ross Turk created the FLAIM (Flox AI Modelling) environment that let’s us run Stable Diffusion locally, so let’s ‘av a bit of fun and do that. Later, we’ll get into why Flox is particularly suited to running AI workloads locally.

Again, with one command `flox activate -r flox/flaim` we can get to work:

Ok, the final test, has Flox installed the exact dependencies that enables Ross’ software to run successfully on my machine?:

I mean, it’s not exactly the Emerald Isle’s tricolour, but hey, that’s Stable Diffusion’s problem. Ross’ software was successfully “reproduced” on my machine (macOS) and it’d run just as successfully on any Linux distribution. Now we can truly “write once, run anywhere”.

Now is when the dude in the back of the room, often with a long grey beard, usually asks: “what about Docker?”. A fair question. This is where things get particularly interesting, and where Nix really comes into a league of its own.

Firstly, we need to level set on Docker. Docker, and the technology they democratised, containers, achieves comparable levels of isolation, reproducibility and portability to that of Flox / Nix. Shall we familiarise ourselves with an example?:

First, let’s log into “Docker Hub” (hub.docker.com) and find a Docker “Image” that we want to “pull”. Docker Images are essentially developer environments, they describe the dependencies a given project requires, as well as some other important metadata like a project’s filesystem, etc.

Ok, cool, I’m logged into Docker Hub via my CLI. Next, much like when using Flox, I’ll search for a piece of software I want to reproduce and get to work via `docker search pandas`:

Ugh, I receive an error message. We’ll dissect this in a second, but note, I don’t run into this issue when using Flox:

Many of you who’re avid Docker users will immediately call me out on my bullshit (rightly so), but this is but one of many examples of working with Flox being just a little less cumbersome than Docker.

Let’s resolve this issue by running “Docker Desktop” (yes, a separate app I need to install on macOS / Windows..) which is the “Docker daemon” that wasn’t running previously.

Now all works perfectly fine when I run `docker search pandas` (a Python library), `docker search nodejs` (a JavaScript library) or, our pal,`docker search htop` (a system utility). Much like Flox, Docker is language and platform agnostic.

Next, I’ll pull the Pandas Image I want and get to work:

I’m now running a shell within my Docker “container”. Within it, I have comparable levels of “isolation” to that of a Flox developer environment. Let’s use Pandas within our container to prove it:

For those unfamiliar with Python / Pandas, the code isn’t important. Just know that I’m using Pandas (the `import pandas as pd` snippet) and it’s running successfully within my shell (i.e. I’m not running into an error message). If I run this code outside of my container, it’d fail.

So, if much like Flox / Nix, containers are language and platform agnostic developer environments, then what’s the catch? Well, like most technologies, containers come with trade-offs. We need to understand exactly how containers are “isolated”, and how they achieve this isolation, in order to appreciate their limitations.

Ok, commençons. People have a tendency to overcomplicate containers, but at a high-level, you can just think of a container as a “slice” or “partition” of your overall machine. Each container has it’s own:

• Resource Limits: a slice of CPU, memory, and I/O resources. This ensures that a container does not over-consume resources and affect other containers or the host system.

• Network Interfaces: a unique “address” within the overall system. This ensures that other containers running on the same machine can’t access any inbound/outbound traffic.

• File System: everything needed to run a specific application, such as code, runtime, system tools, and libraries. This ensures that other containers running on the same machine can’t access my code etc (which is likely proprietary).

• Etc.

Where folk often get tripped-up is on the difference between containers and “virtual machines” (or “VMs”). Put very simply, the ~one resource containers do share is the host machine’s underlying operating system. VMs are greedy, they want a copy of the operating system (a “guest OS”) and all.

Containers achieve this isolation by using two Linux kernel (note: Linux, not other OS’s) utilities: Control Groups (or “cgroups”) and Namespaces. Alas, this is where our trade-offs begin (particularly when working with macOS / Windows).

Firstly, when using macOS or Windows (80%+ of desktops), I, naturally, can’t access Linux-native utilities like cgroups or namespaces. So, when working with containers, Docker actually first spins up a Linux virtual machine, and theeen, we access cgroups and namespaces to create a container. Rather circuitous, non?

This is one of the reasons why we have to install Docker Desktop when working with macOS. Docker Desktop does a lot of heavy-lifting to make containers work across all machines. Again, there’s no such requirement for Flox / Nix.

I can get over this “virtualisation tax”. What I can’t get comfortable with is that containers’ strong isolation properties actually impede us from accelerating AI workloads via GPUs (amongst many other impediments). In order to begin to work with GPUs we need Nvidia’s Container Toolkit, which itself requires some config.

Whereas, as you may recall, when using Flox (FLAIM) earlier we had no problem using our GPU to accelerate Stable Diffusion. We used Flox in exactly the same manner as if we we running “traditional” software. This is how habits are formed.

So, to sum up, Flox is a language and platform agnostic developer environment and package manager. Whilst it rhymes with Docker, Flox doesn’t incur Docker’s virtualisation tax, it can seamlessly access system resources and it’s, frankly, a better developer experience. These properties are why Nix is often referred to as “containers without containers”. Or perhaps, containers, without Docker 😉 (I jest, they play nice).

What’s kinda incredible about Flox / Nix is that we haven’t really scratched the surface re. why Flox / Nix’s developer environments, builds, packages, etc., are objectively more reproducible (reducing performance/security issues) than Docker’s / containers also. Thankfully for you all, I’ve scratched this surface previously with a primer on Nix.

Ok, this primer is wrapping up. At this point, I want to be clear that whilst this post may have seemed “anti-Docker”, this is by no means the case. I have used Docker for most of my “professional” life, and will continue to do so for the use cases it’s suited to best.

If anything, the fact that Docker has been used for so much beyond its core use case (running cloud apps in production) is a testament to its technology and community. One of the perks of investing in infra is that it’s versatile. Just ask the Dow Chemical Company!

So, my final ask to you all is to unite these two worlds. 1) Give Flox a go if you haven’t already and 2) build a Docker image with FloxBuild (ready soon 👀).

Oh yeah, and subscribe etc if you enjoyed this and/or learned a thing or two. If you want to say hi, I’m at alex@tapestry.vc

Tracebit has now deployed close to 2,000 canaries across hundreds of production accounts including those of Riot Games, Docker and Synthesia. A few basis points closer to “100%”.

Read the announcement from Andy at Tracebit here and take a glimpse into their research here.

Congratulations Andy, Sam and the Tracebit team! 

Why canaries & what are they? There was once a time when security teams could plausibly focus solely on “prevention” (e.g. stop that employee from clicking that phishing email). However:

1. The stack of technology upon which an organisation is built has become far too complex (hello, durable execution) and ephemeral (looking at you, lambdas) to properly reason about whether it’s consistently “secure”.

2. Whilst detection engineering is important, if a false positive is a “one in a million event” but there are billions of events, then there’ll actually be a decent number of false positives.

3. The number of security breaches published publicly tells a compelling story that even the best-resourced security teams (Okta, Snowflake, MongoDB) can and will be compromised.

Canaries flip this problem on its head by “assuming breach”. Instead of preventing and/or hunting for the one in a million event that “shouldn’t happen” they instead, create a resource that shouldn’t be interacted with.

Because this resource is wholly created and owned by the person looking to do the detection, they can fully reason about what interactions with it are acceptable and significantly reduce the possibility of misleading false positives creeping in. Same same, but different.

Why now? When we first studied canaries and other deception technologies two years ago we noticed something that piques our interest at Tapestry — dogma. During this time, we were hastily told that “canaries are only for the top 1%” or, a personal favourite, “Atlassian can scale a deception program, but you can’t”.

The convenient thing about dogma is that it can break down rather quickly if you just ask “why” a few times. Answers included: alert imprecision, cumbersome maintenance, untenable costs, et cetera. All seemingly fair points.

However, during our explorations, a duo of founders (who previously helped take Tessian from seed to exit) noted that these points are somewhat moot when applied to modern cloud environments. They believed that canaries could go from the top 1% to 100%.

The team has since jotted down why the modern cloud is a fundamental unlock: 

1. Alert Precision: any interactions with these resources produce a very rich audit trail (in AWS it's CloudTrail) that doesn't just provide an IP address but also session, user, role, user agent and more, making it much more actionable.

2. Maintenance: AWS is responsible for operating systems and software patches of these resources, most are completely transparent to us (e.g. when Log4Shell happened - Amazon paged their engineering team, not their customers).

3. Cost: These resources are usage based, so either very cost effective (cents a month) or free to deploy.

Rarely in cybersecurity do you find a category with so much “space” (thank you, dogma). This admittedly made us apprehensive, but fortunately, we had no shortage of material to keep us busy: Andy has been writing about canaries since 2010, deception jobs were cropping up and “sludge for good” was making the rounds in various forums.

Our intrigue grew into conviction as we spent the year getting to know Andy, Sam and Tracebit. Security Engineers raved about their chats with Tracebit; new product pillars (like canary infra) were rolled out at a rapid clip; and Sam’s research hit the top spot on Hacker News.

This was clearly a product for security engineers, built by security engineers. We couldn’t think of a better pairing and team to bring canaries to “the rest of us”.

Thank you once again Andy & Sam for having us at Tapestry VC on this journey with you.

If you enjoy these primers, I’d appreciate it if you favourite this post’s tweet or LinkedIn post to help spread the good (?) word of Why Now!

I’m at alex@tapestry.vc should you ever wish to chat, always up for saying hello.

I recently spent some time trawling through the history of The Dow Chemical Company with some friends. Niche crew, yes. The winters are cold in London.

If you think Nix or WebGPU are complicated, try getting your noggin’ around the production of bromine in 1897! A humbling experience.

Safe to say that I left this excursion with a bruised ego, but many lessons: Dow was basically a “publisher”, it’s the “chemical company’s, chemical company”, and they spend a tonne more time dissecting price pressure in their quarterly results than.. MongoDB.

The lesson that stood out though was that: product “atomicity” leads to company durability. One of Dow’s first products was bromine (silver bromide specifically), a light-sensitive compound that was used for development in analog photography.

This development process changed from the 1970s onwards with digital photography; a calamity for film-based juggernauts like Kodak, yet Dow, rather deftly, siphoned its bromine assets towards other use cases: flame retardants, tear gas, pharmaceuticals, etc.

Turns out that when the product you produce is literally an element on the periodic table, it tends to be rather versatile. It’s no secret that “platforms” are attractive for this reason: oh you like vectors? Great, check out our Atlas Vector Search product. Think stream processing is de rigueur? Atlas Stream Processing does that.

This said, Dow gave me a deeper appreciation of what it means to be a true platform. As I wrote this primer on Restate, I saw a similar opportunity.

The reason why it’s kinda difficult to understand (& hence, “in need” of a primer) is that it’s atomic. Want Lambdas represented as code? We gotcha. Need consistent multi-writes? We can do that too. It’s difficult to think of Restate as one thing, because it is in fact, everything. Let’s dig in.

Restate is an event broker “on steroids”. It:

• Routes messages/invocations.

• Tracks message/invocation execution.

• Handles retries and recovery via persisting partial progress.

There’s a lot here, I know. We’ll break it up piece-by-piece.

Section 1 - Event Broker

Events are something that happen (e.g. a customer payment) at a point in time (e.g. 2023-01-15-14:30). Surprised? These events are represented as “logs” and collected and processed (enriched, normalised, et cetera) by titans like , (via Elastic Beats or Logstash) and .

Here’s a very simple log below. When these logs are sent somewhere they’re often called “messages”. At first glance, they may look a little intimidating, but study their contents and you’ll find they’re really quite intuitive.

{
  "event_type": "payment_intent.succeeded",
  "created_at": "2023-01-15T14:30:00Z",
  "data": {
    "object": "payment_intent",
    "amount": 2500,
    "currency": "USD",
    "status": "succeeded",
    }
  }  

It’s très important to internalise that, we collect a shit-tonne (industry measure) of logs. Why? Well, we can put ‘em to work. This is the vocation of the event broker.

Brokers do this by organising these events into “topics” (think of them as folders stored on the broker). Par example, we could create a “payment_events” topic that records Stripe payments like the one above. Each time a new payment is made, the resulting log is sent to the topic:

These events aren’t providing utility yet though, are they? They’re in need of a destination (or “sink”) that consumes them. In our payments example, the sink could be a dashboard that lists new sales in real time.

What’s cool, I guess, is that brokers can also route logs from topic-to-topic. An event broker like Confluent (Kafka), may send each new log to a processing engine (i.e. software that transforms the log) like Flink, and then route the computed result to another topic.

Allow me to illustrate the above. Notice that the “payment_events” log includes a sales price (called “amount” in our log)? Each time a new log enters “payment_events”, we could send it to Flink to recalculate the median sales price of all of the logs it’s processed. We’d then send this data to the “median_sales_price” topic. Get it now?

My walkthrough hasn’t quite done the thankless role of an event broker justice however. Distributed systems are hard. What if an event fails to deliver (known as a “dead letter”) due to network issues? Or perhaps, a “sink” is overwhelmed by the deluge of `item_added_to_cart` events it’s receiving on Cyber Monday? Told ya.

I’ve adorned Confluent’s Kafka with it’s very own `whynow_cluser` which I think best illustrates how much these event brokers do. Replication (for data resiliency), data retention (for compliance), event flushing (for performance), it’s all there:

Quite a bit there, huh? In fact, event brokers are so complex, that companies can build nice businesses (15,000 customers) providing better Kafka UIs, for Confluent’s Kafka UI. You don’t see myopia like this in PaaS too often!

Cool. You’ve hopefully learned what an event broker (& Kafka) is in ~480 words. If true, I’m kinda proud of that. So where does Restate add the, eh.. steroids?

Section 2 - RPC

So far, we’ve only discussed how event brokers like Kafka transport “messages” (remember, event logs). However, what if they could also transport “procedures”?

Well, recall that our Restate description also mentioned “invocations”. An invocation is when a function (i.e. procedure) is “called” like so:

# Define a Python function
def greet(title):
    return "Hello, {title}!"

# Invoke the function
greet("Why Now reader")

In microservice architectures, it’s common to have a number of these functions all residing in their own servers, waiting to be called-upon. For example, a user-facing application (think a checkout screen) might call, or “invoke” a microservice (like a KYC check).

Whilst there are many ways to invoke a function remotely, this “invoking” is often done through the RPC (Remote Procedure Call) protocol.

Before we delve into RPC though, let’s first look at an alternative; partially because it’s Sunday evening and re-learning RPC requires work, but mostly because doing so will serve as a helpful contrast.

Whilst I’m not prepared to get you the data (again, Sunday), I’m willing to make a bet that plain ole HTTP requests are the most common way to invoke functions remotely (i.e. from a “client” like your web browser).

Thus, many, people write web servers that.. serve.. these HTTP requests like so:

from flask import Flask

app = Flask(__name__)

@app.route('/example', methods=['GET'])
def example_route():

    return 'This is the response from the example route.'

if __name__ == '__main__':
    app.run(debug=True)

Looks complicated, eh! Most of this is “boiler plate” stuff so don’t fret. All that really matters is this:

@app.route('/example', methods=['GET'])
def example_route():

    return 'This is the response from the example route.'

Here we’re saying, when the url whynow.onrender.com/example receives a HTTP GET request, invoke the function “example_route”.

Hopefully it’s clear that we now have a way to invoke a function remotely, all we need to do is make a HTTP GET request. Great, how does one do wanna them?

Well, making GET requests is quite easy, as all you gotta do is visit (i.e. GET) the URL. Let’s do something a little more fun though. Mac users, press command + spacebar. Windows users, do better? Now, search for your “terminal” and open it.

Ok, now type: curl whynow.onrender.com/example in that fine terminal of yours. Et voilà! You’ve made a HTTP request. What’s up, h@ckɇr.

The thing is, like most general-purpose technologies, HTTP isn’t optimised for making function calls. RPC on the other hand, is.

RPC intends to make function calls appear “local”. Its goal is to make function calls look as though they’re made directly in a single file; when in reality, you’re calling functions that live in many different files, on many different servers (aka “microservices”).

Why? Well, think about it. It’s much easier to reason about functions that are “native” to the file (e.g. app.js) you’re working on. Local functions don’t need any superfluous information about handling network protocols. They’re local! There’s no need for a network to transport any data to a remote function.

Let’s make this point really hit /home. Take a look at what our app.js file (running in our browser “client”) could look like when plain ole HTTP is involved. It’s ok, your head is supposed to hurt:

const http = require('http');

// Define the server's address
const hostname = 'whynow.onrender.com';
const port = 8000;
const path = '/example';

// Create an HTTP request options object
const options = {
    hostname: hostname,
    port: port,
    path: path,
    method: 'GET',
};

// Create an HTTP request
const req = http.request(options, (res) => {
    let data = '';

    // Receive data from the server
    res.on('data', (chunk) => {
        data += chunk;
    });

    // Process the server's response
    res.on('end', () => {
        if (res.statusCode === 200) {
            console.log('Data received from server:');
            console.log(data);
        } else {
            console.error('Error:', res.statusCode);
        }
    });
});

// Send the HTTP GET request to the server
req.end();

There’s a lot of cruft here that makes it clear that you are, in fact, not making a local function call. Perhaps it’s also now clear why we need a better way?

In RPC we use “stubs” and “skeletons” to abstract network communication details (the cruft above). Stubs are used on the client-side and look like so:

// xmlrpc is just a "library" that makes using RPC easier
const xmlrpc = require('xmlrpc'); 

// Create an RPC client (your "stub")
const client = xmlrpc.createClient({ host: 'whynow.onrender.com', port: 80, path: '/example' });

// Define the remote procedure and its parameters
const methodName = 'add';
const params = [3, 4];

client.methodCall(methodName, params, (error, value) => {
    console.log('Result:', value);
});

Again, don’t get bogged down on the code above. All you need to appreciate is that the “add” function actually lives on another file (http://whynow.onrender.com/example.)

It looks kind of “local” though, right? It’s as if we’ve just imported the “add” function as if it was a library like xmlrpc. Note that there’s also no mention of what HTTP “method” (i.e. GET) we’re availing of, status codes, or anything that suggests that we’re making a request over a network. Nice.

Before I get called out by “astute” (😉) readers, much like RPC, I have abstracted things a little here. For those that wish to go deeper, look into RPC skeletons, interface definitions and protocol buffers (Buf is cool).

Section 3 - Durable Execution

Ok, we’re kinda on a roll here. You now know what an “event broker” is, what “messages” and “invocations” are, and I’ve endowed you with a new acronym to woo folk with at dinner parties. Let’s begin to tie this all together.

Let’s revisit our definition: Restate is an event broker that routes and tracks both messages and invocations. It also handles retries and recovery via persisting partial progress.

Astute readers (the good kind 😉) might have already guessed what retries are. As we learned through studying Kafka, messages and invocations may fail for a shit-tonne (again, industry metric) of reasons. Distributed systems are hard.

It’s commonplace for developers to write logic that handles when things go awry. For example, in frontend development, it’s common to use `try` and `catch` blocks to handle network errors:

A frontend developer might `try` to fetch some data from a remote database (i.e. a network request is made), and if an error occurs, `catch` it and run some alternative logic in response. Sometimes this alternative logic is simply, trying again after a period of time has elapsed (a “retry”).

The above is a rather rudimentary example though. In reality, a single RPC call may trigger a plethora of functions across a series of microservices. One big game of distributed dominos.

Let’s look at a system that’s a little more complex, shall we? Imagine a world where I sell Why Now dad caps. When an order is placed, my system needs to:

• Deduct inventory

• Charge the customer's credit card

• Send a confirmation email

This involves several remote services (perhaps called via RPC) and potential points of failure. For example, what if everything’s rosy for my inventory service, but a customer’s credit card on file has expired?

In this instance, I may have deducted my inventory count, despite an incomplete sale. C'est ne pas bon. To address this potential issue, I could send an error “message” (remember, a log) to an “event broker” like Restate, Temporal, Inngest, Resonate, etc., if a payment ever fails.

For this specific error, we might end the current workflow and “rollback” our inventory to its former state. Through our error message, we could then trigger a customer email informing them that they need to update their payment method. Notice how we’ve made our application more resilient, or, durable?

Event brokers that are predominately used to increase the durability of our distributed systems (again, Restate, Temporal, Inngest, Resonate) are known as “durable compute” or “durable execution” providers.

Their focus is slightly different to event brokers like Confluent and Redpanda whose primary function is real-time data transport. As a result, durable compute providers come fully-loaded with numerous helpful utilities like: retries, error handlers, timeouts, state maintenance, etc.

Cool, you’ve got retries down. Sometimes you want to pick-up where you left things off though, you want to “recover” a workflow.

In this case, perhaps we decide to keep our inventory count deducted for a period of time (i.e. we persist the “state” of our workflow). We’ll still send an email to the customer asking them to update their payment method, and, if they do so hastily, we can just resume our workflow from where it left off (known as “workflow continuation”).

Cool, that’s recovery and persisting partial progress down too. Want a dad cap?

Section 4 - Restate

Ok so: Restate is an event broker that routes and tracks both messages and invocations. It also handles retries and recovery via persisting partial progress.

Let’s delve into Restate itself now and how it actually achieves all of the above.

The image on the right is just a microservices architecture. Don’t worry about it, but note familiar terms like RPC, durable execution and retries? Nice. You’ll recognise most of the terms on the left, the one that might throw you is “async/await”.

Async/await is a common pair of keywords used in web development. In JavaScript, you place “async” in front of a function definition like so:

async function fetchData() {
  try {
    const response = await fetch('https://example.com/api/data');
    const data = await response.json();
    console.log(data);
  } catch (error) {
    console.error('An error occurred:', error);
  }
}

fetchData();

// subsequent code blocks:
const numbers = [1, 2, 3, 4, 5];

// Use the map function to square each number
const squaredNumbers = numbers.map((num) => num * num);

// Print the squared numbers to the console
console.log("Squared numbers:", squaredNumbers);

All async/await does here is enable a developer to request (often called “fetch” in web dev) some remote data, without impeding the execution of subsequent code blocks.

We’ve already learned why this matters, requesting data over networks can be unreliable. The last thing we want on our website is for the page not to fully render (i.e. subsequent code not execute) due to network issues. Async/await solves this.

To be honest, this section threw me for a spin when I first saw it. It’s actually fine though. It’s just a use cases section, chill.

1. Lambda Workflows as Code

Simply put, Lambdas are remote servers that execute some code (e.g., our inventory deduction function). Microservice architectures often chain many Lambdas together based on some “if this, then that” (i.e. event-driven) logic.

This “if this, then that” logic typically lives in a separate JSON (i.e. configuration file) to the Lambda itself. Note the “triggers” and “events” in the below JSON file:

.json

{
  "FunctionName": "MyLambdaFunction",
  "Runtime": "nodejs14.x",
  "Handler": "index.handler",
  "MemorySize": 256,
  "Timeout": 10,
  "Environment": {
    "Variables": {
      "API_KEY": "my-api-key",
      "DATABASE_URL": "https://my-database-url"
    }
  },
  "Role": "arn:aws:iam::123456789012:role/lambda-execution-role",
  "Triggers": [
    {
      "Type": "APIGateway",
      "Source": "MyApiGateway",
      "Method": "POST"
    },
    {
      "Type": "S3",
      "Source": "MyS3Bucket",
      "Events": ["ObjectCreated"]
    }
  ]
}

With Restate, this configuration logic is replaced by actual code.

“Who cares?” says the, em.. gentleman.. in the back. Well, code is much more flexible than simple “key:value” data structures like JSON (e.g., you can’t “loop” in JSON).

As a result, Restate can implement more sophisticated logic like: “reliable timers & schedulers” and function suspension [1][2] vs. our lowly key:value file. My friend Jack (hello, Jack) co-authored this post on the Restate blog which further extols the benefits of this approach nicely.

2. Transactional RPC Handlers

This one should be kinda easy to grok. Recall our inventory management microservice that has two “procedures” (i.e. functions): “addInventory( )” and “deductInventory( )”.

When a client calls “deductInventory( )”, this request is initially read by an “RPC Handler” that, typically, lives in the same microservice. This RPC handler then routes the request to the exact procedure requested by the client.

Restate bestows our RPC handlers with transactional guarantees. Transactional systems are those that prioritise data consistency and integrity. For example, my inventory management service better be right, as otherwise, I might charge a customer for dad hats that I no longer have.

One way that transactional systems are achieved is via the concept of “atomicity”. This means that either my entire “transaction”: deduct inventory > charge customer’s card > send confirmation email works, or none of it does and the flow rolls-back accordingly.

As we learned earlier, Restate can persist transaction state.

3. Event Processing with Kafka

We’ve already spent time understanding how event brokers build resiliency into their message transportation (e.g., “dead letter queues”). So, let’s not rehash this too much.

Basically, Restate sits as a proxy between your event broker (Kafka) and your event consumer. It brings the qualities of durable execution to event transportation.

At this point, you likely understand all of the above? If so, sweet. This stuff ain’t easy.

Software that comes as a “single binary”, is de rigueur in the event broker world of late. Not without good reason. Single binary software comes bundled with all of its dependencies; you can install Restate and get going.

Whereas, with Kafka for example, you need to also install the Java Runtime Environment, Kafka Command Line Tools, etc. It just takes more work.

Regarding the image on the right: gRPC (Google RPC), HTTP and Kafka are various protocols that we can use to transport messages/invocations to Restate. We have this down. OpenTelemetry I’ll deal with in a sec.

OpenTelemetry (or “OTel”) is an absolute game changer, I personally think it still doesn’t get enough airtime, but anyway.

OTel is an open source project that provides a set of APIs, libraries and agents to enable the collection of observability data (you guessed it, logs!). In microservices, “traces” document the sequence of messages/invocations that span across an application (e.g. each step in our inventory management flow).

It’s not lost on me that this primer on durable execution has been.. durably executing.. for a bit now. You all have Restate down. Sometimes one needs to `exit( )` whilst they’re ahead (sorry).

As always, there’s _so_ much more (coroutines! Warpstream! Flawless!) that I’d like to dig into here. However, the good news, readers, is that this primer is itself, a primer for what’s to come next.

I’ve wanted to put digital-pen to digital-paper on a “Every AWS Service Ever” post since the beginnings of Why Now. Post-studying the importance of “product atomicity”, now seems like no better time to do it.

10 primers down, and 11 more to come this year. Thank you for reading along, friends.

Welcome to new subs + thanks a million for the Hyena love! This thing has grown a whole lot faster than my inner-pessimist could have imagined. So, thank you!

If you enjoy these primers, I’d appreciate it if you favourite this post’s tweet or LinkedIn post to help spread the good (?) word of Why Now!

I’m at alex@tapestry.vc should you ever wish to chat, always up for saying hello.

I don’t mean to alarm you but, “accidents of history” are among us.

Countless decisions make sense when tethered to a particular moment in time. However, if we were to rethink them today from, dare I say it, first principles, we would conclude that we have been subject to one calcified ruse after another.

Par example, that QWERTY keyboard of yours. It was originally designed to prevent the jamming of mechanical keys in typewriters. I know it’s fundamentally flawed, but good luck coercing me into switching to DVORAK.

What else? We created legal tender to facilitate a double coincidence of wants, with internet-scale liquidity, we have this. We’ve developed modern algorithms to replace antiquated IPOs, but all the best convincing your resident Lead Left to change their ways. Don’t get me started on Dockerfiles or, better yet, the imperial system.

Whilst perhaps not quite as existential as Dockerfiles legal tender, I view SQL as comparably archaic. ORMs, SQL supersets, semantic layers and query engines all feel like calcification to me. Band-Aids vs. MMR II.

Instead of relying on Trino to rewrite our SQL for us, what if we rewrote the language itself? Thankfully, Lloyd Tabb (of Looker fame), Michael Toy, Carlin Eng, and the team at GCP asked themselves a similar question:

“If we knew all the things we know about data, and about programming with data, and about programming languages in general, and we were designing a query language today, what would it look like?”.

More broadly, this line of questioning is one that we as investors should stuff into our back pockets. Lodge it right next to Charlie’s boiling frog. In this instance, the result of Lloyd’s question is Malloy — a language for describing data relationships & transformations. Sounds.. familiar?

Sit tight, there’s more. Malloy:

• Compiles to SQL optimised for your database.

• Has both a semantic data model and query language.

• Excels at reading and writing nested data sets.

• Seamlessly handles what are complex/error-prone queries in SQL.

Interesting. It’s no secret that the proverbial: “modern data stack” has been in vogue within venture. Rightly so. However, since stumbling upon Malloy, I’ve wondered how much of this stack can be obviated by swapping out SQL itself?

Furthermore, why has SQL endured as the incumbent query language since the 1970s? New programming languages spring up often. What’s unique here? Let’s dig in.

As mentioned, Malloy is a language for describing data relationships and transformations within SQL databases. It:

• Compiles to SQL optimised for your database.

• Has both a semantic data model and query language.

• Excels at reading and writing nested data sets.

• Seamlessly handles what are complex/error-prone queries in SQL.

As always within Why Now, we’ll build towards an understanding of the definition above via breaking it into its components.

Section 1 - History of SQL & RDMS

As Pablo Picasso (“founding eng” at IBM, I think) once opined: “learn the rules like a pro, so you can break them like an artist”. Similarly, we must first understand SQL & the “relational model” writ large if we are to challenge it.

Up until 1970, database management systems (DBMSs) such as IBM’s Information Management System (IMS) organised data using “hierarchical” (think tree structures) or “network” models:

tree_structure.lua

Organization
|
|-- Department: IT
|   |-- Employee: John Doe
|   |   |-- Task: Develop Software
|   |   |-- Task: Maintain Systems
|   |-- Employee: Jane Smith
|       |-- Task: Network Management
|
|-- Department: HR
    |-- Employee: Mike Johnson
    |   |-- Task: Recruitment
    |-- Employee: Sarah Lee
        |-- Task: Employee Relations    

With what we know now (recall, “accidents of history”), it’s clear that this is an incredibly rigid, and rather abstruse data model to grok at scale. Imagine attempting to parse Zomato’s 150M logs p/m if represented this way (!).

IBs rejoice, Edgar Codd came along in 1970 with the paper: “A Relational Model of Data for Large Shared Data Banks”. Edgar proposed that data should be presented to end-users as “relations”, which are the database “tables” (yes, like Excel) that we’re all too familiar with today.

Edgar stated that each row of a table should represent an instance of data (ie a “record”) and each column, an attribute (ie a “property”). To avoid duplicates, each record should contain a unique identifier, known as a “primary key” (e.g., “student_id” below).

relational_structure.lua

| student_id | name | age | major           |
|------------|------|-----|-----------------|
| 1          | John | 21  | Computer Science|
| 2          | Jane | 22  | Mathematics     |
| 3          | Mike | 20  | Physics         |   

Edgar also equipped us with a set of operations that could be used to manipulate these “relations” (remember, tables) in order to retrieve & modify data. Operations such as: selection, projection, union, set difference, Cartesian product, and join. Hmm. Familiar?

Technical Detail: What rhymes here, is that Edgar advocated for the separation of database schema & physical storage. Many years later in 2012, Snowflake would pioneer the separation of storage and compute, and more recently, Firebolt has separated indices also.

As Josh Wolfe would instruct, pay attention to directions of progress. Accordingly, what are we separating next?

Whilst Edgar’s seminal work precipitated the modern relational database industry, he is in fact, not the creator of SQL himself. This kinda all balances out. As, as already alluded to: “SQL has no real right to be called relational at all”. It’s implausible that the father of relational algebra would make such a series of blunders.

Rather, Big Blue’s Chamberlin & Boyce developed what was originally dubbed “Structured English Query Language” in 1974. Yes folks, it is See-quill vs. es-que-el. ...Clearly.

IBM’s “System R” was one of the first RDMSs & implementations of SQL (following the University of Michigan’s “Micro DBMS”). However, by 1979, the fledgling Relational Software Inc released the first commercially available implementation of SQL. Bravo, Larry.

Ok, so what’s so wrong with Chamberlin & Boyce’s Modern Prometheus?

Section 2 - SQL & Its Issues

Good news folk. Whilst Nix expressions or Zig’s syntax might’ve required some labour to wield, SQL is, frankly, très facile. This was very much intended by Chamberlin & Boyce who designed the language for “ad-hoc” queries, often made by “non-professional” users.

Note the misplaced design decision? Herein lies a derivative of Jevon’s Paradox at play. Lowering the barrier of entry to “data programming”, increased the number of sophisticated “data programmers” over time. Unsurprisingly, these sophisticated users aren’t too content with SQL — it wasn’t designed for them, after all.

Ok, moving swiftly on from economic theory to programming reality. Welcome, class, to SQL 101.

SQL’s functionality is essentially GROUP’d into 3 components: 1) Data Definition (ie describe my data), 2) Data Manipulation (ie change my data) & 3) Data Querying (ie get my data).

Defining our data could look something like this:

CREATE TABLE table_name (
    column1 datatype,
    column2 datatype,
    ...
);

Here, we have created a table called “table_name” with two columns: “column1” & “column2”. We can specify the data types (recall “types” from our Deno primer) that should be inserted into each column (e.g., dates, integers, strings).

Next, we manipulate our data like so:

INSERT INTO table_name (column1, column2,...)
VALUES (value1, value2,...); 

Here, we’re manipulating the data structure that we originally defined by inserting “values” (ie data) that, all going well, conform to the data types we specified above.

Great. Now we have data stored in our relational database (think PostgreSQL or BigQuery). In order to actually use this data, be it to power our application or, better yet, strike fear into unsuspecting sales reps, we need to query it. A la:

SELECT column1, column2, ...
FROM table_name
WHERE condition;

All pretty straightforward, right? Things get a little more.. complicated when you realise that, despite adhering to a standards body, the SQL you write that works for PostgreSQL, may not work for MySQL.

For example, if you want to concatenate strings in PostgreSQL you can use the “concat()” function or use the following “pipe” (ie ||) operator:

postgres_example.sql

SELECT 'Hello' || ' ' || 'World';

Whereas MySQL doesn’t use the pipe operator by default (it can be enabled):

mysql_example.sql

SELECT CONCAT('Hello', ' ', 'World');

Countless examples exist to enumerate my point. Database vendors claim that these differences are due to “optimisations” for their DBMS. Sure, but do you think they’re mad about simultaneously creating switching costs? That sweet, sweet 158% NDR.

Fortunately for SQL, its soft-underbelly is often shielded by object-relational mapping (or “ORMs”) tools like SQLAlchemy or Prisma. These tools enable developers to write Python, etc., vs. raw SQL (hence, the “mapping”), whilst also acting as database abstraction layers.

See the “calcification” now? Or as Paolo Atzeni puts it: SQL is an elephant on clay feet.

Technical Detail: See below a SQL “upsert” (portmanteau of update & insert) I recently wrote using SQLAlchemy. Note I’m using the “Pythonic” function text() vs. writing raw SQL.

scraper.py

neon_conn.execute(
                text(
                "INSERT INTO nix_tweets_two (id, tweetid, tweettext, userurl) VALUES (:id, :tweetid, :tweettext, :userurl) ON CONFLICT (tweetid) DO NOTHING"
                ),
                [{"id":nix_tweets_uuid, "tweetid": int(tweet_id), "tweettext": tweet_text, "userurl": user_url}]
                )
        neon_conn.commit()

Next, we get onto the subject of “nested queries” (I’m a hit at dinner parties). As any of you who’ve waded through my Zig primer will know, I’m a sucker for eloquent programming syntaxes. SQL’s “non-professional” syntax is.. proficiently offensive.

Have a look at how SQL formats nested queries:

nested_query_example.sql

SELECT name, salary 
FROM employees 
WHERE salary > (SELECT AVG(salary) FROM employees);

Perhaps I’m being a little pedantic, but parsing this query certainly isn’t intuitive. For a start, what should I (or the “parser” itself) compute first? Also, whilst I’m on this brazen crusade, shouldn’t we start the query with the FROM clause?

However, things can get much more convoluted:

compex_nested_query_example.sql

SELECT customer_name, total_order_value
FROM (
    SELECT o.customer_name, 
           SUM(oi.price * oi.quantity) AS total_order_value
    FROM orders o
    JOIN order_items oi ON o.order_id = oi.order_id
    GROUP BY o.customer_name
) AS customer_orders
WHERE total_order_value > (
    SELECT AVG(total_order_value) 
    FROM (
        SELECT o.order_id, 
               SUM(oi.price * oi.quantity) AS total_order_value
        FROM orders o
        JOIN order_items oi ON o.order_id = oi.order_id
        GROUP BY o.order_id
    ) AS order_values
);

Anyone else feel a sense of malaise? Don’t worry, my head is pounding too. On a positive note, even dedicated SQL parsing algorithms struggle to comprehend walls of SQL like this.

Perhaps it’s increasingly clear why we should DROP this language? (sorry, couldn’t resist)

Let’s scratch off some of our newfound understanding from our original definition:

As mentioned, Malloy is a language for describing data relationships and transformations within SQL databases. It:

• Compiles to SQL optimised for your database.

• Has both a semantic data model and query language.

• Excels at reading and writing nested data sets.

• Seamlessly handles what are complex/error-prone queries in SQL.

Section 3 - Introducing the Semantic Layer

Let’s begin by unpacking the “semantic data model”.

We’ll get into the nitty-gritty, but “semantic” is the key word here. Ever squinted at thousands of table rows & struggled to understand the “meaning” of the data? Don’t fret, this is a safe space, I have too.

Let’s continue our therapy session. Perhaps you’ve also felt the anxiety of wondering if the SQL you’ve written is calculating subjective metrics like: “a magic number” or “MAUs”, correctly? Rather stressful.

Ideally, we’d make our data easy to grok. We can do this by translating abstruse data schemas and walls of SQL into intuitive natural language concepts and instructions. Hence, “semantic model” (or “semantic layer”). Ok, nice, but how does this all work?

Brace yourselves for another layer of “calcification”.

Many credit BusinessObjects (acquired for ~$6B by SAP in 2007) for creating the original semantic layer, called “Universe”. However, I think Looker’s “LookML” (created in 2012) demonstrates its ideals best.

LookML is essentially a SQL abstraction. Folk using Looker for BI purposes can write the language vs. writing sushi-grade SQL. Nice lock-in by Looker, huh? As discussed, this is common in database land: Elastic’s Query DSL, Neo4j’s Cypher, CQL, et cetera.

Anyway, LookML exposes many core concepts that bestow semantic meaning upon raw SQL. We’ll focus on LookML’s “measures” (ie data calculations) as but one example. Let’s begin:

sales.view.lkml

view: sales {
  measure: total_sales {
    type: sum
    sql: ${TABLE}.sale_price ;;
  }

Here we have a LookML file (hence “.lkml”) that stores a “view” (ie a table) called sales that’s derived from another table’s sales_price column.

Within this view, we have a “measure” called total_sales. Note the metadata within the “measure” that makes it quite intuitive to understand what’s happening here (e.g., type: sum).

This metadata also reduces the length & complexity of the raw SQL that would be written otherwise:

sales_alternative.sql

SELECT SUM(sale_price) AS total_sales
FROM sales;

Hmm. Nice… maybe? To close the loop, more recently, dbt inherited semantic layer capabilities via its acquisition of Transform. Much like Looker, dbt also encourages the extension of SQL via the Python-based templating engine, Jinja.

Super. With LookML it.. looks-ml.. like we have solved many of SQL’s shortcomings, right? Well, no. Firstly, as belaboured at this point, we’re not fixing the root cause of our woes: SQL itself.

Secondly, the issue with many of these SQL extensions & abstractions is that they’re tethered to specific vendors. If I’m the de novo database on the block, do I really want to create dependency on a future competitor? Nope. As a result, our efforts don’t yield standardisation around a better way of doing things.

It’s clear that we need a new, open, standard.

Ok, one final scratching-off exercise below. Seems like we’re now ready to tackle Malloy?

As mentioned, Malloy is a language for describing data relationships and transformations within SQL databases. It:

• Compiles to SQL optimised for your database.

• Has both a semantic data model and query language.

• Excels at reading and writing nested data sets.

• Seamlessly handles what are complex/error-prone queries in SQL.

Section 4 - Malloy

I’ll be honest, I didn’t think I’d get this excited about a query language. That said, I should know better at this point: Zig, Godot’s GDScript, & Deno (well, Fresh) have all played lead roles in Why Now.

The world runs on software, but software runs in the direction that it’s guided towards. Minimise bugs and you’ll create mission critical applications, enforce type safety and you’ll build a whole lotta enterprise SaaS.

Languages shape software, making them rather fascinating. This impact, served with comparisons of magic, made Malloy rather difficult to resist. To be fair, I think Lloyd may be onto something. Let’s delve a little deeper.

Ok, so, firstly.. Malloy starts with a run clause, which essentially serves the same purpose as (although is more “powerful” than) SQL’s FROM clause. As mentioned already, this just makes logical sense:

run: duckdb.table('../data/airports.parquet') -> {
  select:
    id
    code
    city
  limit: 10
}

Technical Detail: Don’t worry about the “duckdb” reference (thanks Hannes), this is “just” a database that’s embedded locally on your device (laptop, phone, etc).

So, all we’re doing here is accessing an airports table that’s stored as a Parquet file. Parquet files are columnar file formats which we won’t get into now, but you really should read up on them if unfamiliar.

Let’s continue examining the “scaffolding” of Malloy for a tad longer before we delve into the meatier stuff.

As I’ve alluded to previously, language ambiguity creates bugs and hinders readability. How so? Well, if multiple ways of achieving the same outcome exist, the “reader” of a particular piece of code has to do a little more grease-work to understand it.

For example, in SQL, the SELECT clause can be used for two distinct use cases. When married with the GROUP BY clause, SELECT is used for aggregation purposes (note the “SUM(sales) as total_sales” below). If the GROUP BY clause isn’t present, then SELECT is used for plain ole data retrieval.

Par example, imagine we have a table like so:

+---------+-------+
| country | sales |
+---------+-------+
| USA     | 100   |
| USA     | 50    |
| Canada  | 70    |
| UK      | 30    |
+---------+-------+

If we query this table with the following SQL (note the SELECT & GROUP BY) like so:

example.sql

SELECT country, SUM(sales) as total_sales
FROM data_table
GROUP BY country;

We end up aggregating (in this case, SUM-ming) the USA rows. Why? Because they’re present more than once (hence the 150 total sales):

+---------+-------------+
| country | total_sales |
+---------+-------------+
| USA     | 150         |
| Canada  | 70          |
| UK      | 30          |
+---------+-------------+

Accordingly, notice that the SELECT statement has a lack of “identity” to some degree here? It serves a couple of purposes. This tends to occur when a programming language is essentially developed in tandem with its own use case.

My-guy-Malloy, on the other hand, has the benefit of hindsight. Thus, Malloy is a little more.. explicit with aggregates:

run: duckdb.table('../data/airports.parquet') -> {
  group_by:
    state
    county
  aggregate:
    airport_count is count()
    average_elevation is avg(elevation)
}

Comparatively, this statement is easier to build an intuition for, despite many of you likely dabbling with a SELECT clause or two in your time? Kinda cool.

No doubt some of you think I’m being pedantic thus far, perhaps that’s fair. We’ll inch towards Malloy’s more poignant features.

Recall from my opening preamble this sub-point: “about programming languages in general”?

There’s a lot we do syntactically in general-purpose programming languages that, for whatever reason, haven’t found their way into SQL. Par example, in Python, Go, JavaScript, Rust, Kotlin, etc., we define variables like so: variable name = value.

In SQL, unfortunately, we invert this when creating column aliases (ie “max elevation” below):

waitbutwhy.sql

SELECT 
   max(base."elevation") as "max_elevation"
FROM '../data/airports.parquet' as base
ORDER BY 1 desc NULLS LAST

Why create this overhead? Again, think about it. If a software engineer was creating SQL from scratch today, this naming pattern wouldn’t fly. In Malloy, aliases are defined like variables are in programming languages:

run: duckdb.table('../data/airports.parquet') -> {
  aggregate: max_elevation is max(elevation)
}        

Nice, progress. What else can we glean from programming languages?

Well, recall that “nested datasets” (ie datasets derived from other datasets) in SQL can become a bit of a mess:

example.sql

SELECT customer_name, total_order_value
FROM (
    SELECT o.customer_name, 
           SUM(oi.price * oi.quantity) AS total_order_value
    FROM orders o
    JOIN order_items oi ON o.order_id = oi.order_id
    GROUP BY o.customer_name
) AS customer_orders

But why are they particularly unwieldy in SQL? Nested calculations aren’t unique to data queries. In programming languages, like Python for example, you’ll often “nest” a function within another function (often called a “callback function”).

To make these nested functions more legible, it’s very much common practice to assign a variable name to the “callback” like so:

example.py

def callback_function():
    print("Callback executed!")

def main_function(callback):
    # Do something...
    callback()  # Execute the callback function

main_function(callback_function)  # Outputs: "Callback executed!"

Notice how using named expressions makes the “nested function” (ie main_function()), easier to wade through? The same is true for SQL.

Whilst you certainly can assign names to expressions in SQL, they’re not required. People are some confluence of busy & lazy; if you don’t encourage a desired action, you may not see it. Malloy, on the other hand, mandates named expressions.

The below code literally won’t run if the output of its “aggregate” calculation isn’t named (ie “max_elevation”):

# This code works:

run: duckdb.table('../data/airports.parquet') -> {
  aggregate: max_elevation is max(elevation)
}

# This code doesn't work:

run: duckdb.table('../data/airports.parquet') -> {
  aggregate: max(elevation)
}
       

Perhaps these examples still seem like small wins, but they embody how historical decisions benefit from present context. In tandem, these wins inculcate a broader point: Malloy has rethought SQL to the nth degree. No suboptimal clause or pattern is safe.

Whilst I tend to feel the magnetic pull of micro-optimisations (the trade-offs are fascinating); Malloy also introduces new functionality that we should make time to appreciate. Namely, a semantic layer.

Recall, SQL doesn’t have a semantic layer itself. It outsources this work to companies that build (& often monetise) semantic layers. Nothing wrong with this per se, but it can precipitate the dreaded “vendor lock-in”.

Unsurprisingly, Malloy’s semantic layer rhymes with Looker’s. When you want to save common calculations (remember “measures” in LookML), you can do so, along with “dimensions”, “joins”, etc.

Let’s ‘av a look:

source: airports is duckdb.table('../data/airports.parquet') extend {
  dimension: county_and_state is concat(county, ', ', state)
  measure: airport_count is count()
  measure: average_elevation is avg(elevation)
}

We initialise semantic layers in Malloy with the source keyword. No ambiguity here, if you spot source, you know some sweet, sweet, semantic meaning is going down.

Note that, again, much like in LookML, our semantic layer is derived from some original table (ie duckdb.table('../data/airports.parquet')).

We then “extend” this table with one “dimension” (county_and_state), and two “measures” (airport_count, average_elevation).

Good news is, we’ve already done the work to know what measures are. They are simply instructions for how to calculate subjective metrics (remember, “MAUs” for example).

We can then use these saved “measures” in future Malloy queries like so:

run: airports -> { 
  group_by: county_and_state
  aggregate: airport_count
}

Dimensions are new to this primer, but common in semantic layers more generally. They’re used to further describe and transform your data. More good news. They’re quite simple to grok once placed under the lens of our example code:

source: airports is duckdb.table('../data/airports.parquet') extend {
  dimension: county_and_state is concat(county, ', ', state)
  measure: airport_count is count()
  measure: average_elevation is avg(elevation)
}

Here, the “dimension” (think new column) country_and_state, is the concatenation of two separate columns in our airports table (country and state). The resulting column, or “dimension”, of this column will look like so:

+---------------------+
|  county_and_state   |
+---------------------+
|  Santa A, CA        |
|  Santa B, CA        |
|  York, NY           |
+---------------------+

All quite simple, but some powerful & convenient functionality to have embedded in a query language directly. Nice work, Malloy.

No doubt some of you have picked up on the, ? subtle ?, irony present within this primer. A primer lambasting an unwieldy query language has become.. unwieldy.

There is, _so_ much more for us to dig into re. Malloy. The language’s approach towards query pipelines is particularly cool; the fact that you can structure your queries with freedom is too; and you’re all lucky I omitted several other micro-optimisations.

I’d encourage you to walkthrough Malloy’s quickstart at your leisure.

Lloyd, Michael, Carlin & the team are breaking rules in a way that would make Pablo himself proud. Art has a plethora of forms.

Welcome to new subs + thanks a million for the Zig love! This thing has grown a whole lot faster than my inner-pessimist could have imagined. So, grazie!

If you enjoy these primers, I’d appreciate it if you favourite this post’s tweet or LinkedIn post to help spread the good (?) word of Why Now!

As always, I’m at alex@tapestry.vc should you ever wish to chat. In particular, I’ve spent a lot of time thinking/writing about what new applications could exist thanks to LLMs — if you have too, I’d love to say hello.

Hyena clans are arranged in what’s known as a “linear dominance hierarchy”. This means that there’s a one-to-one chain of command: Hyena 3 reports to Hyena 2, and Hyena 2 reports to, you guessed it, Hyena 1.

This structure is also present in baboons, wolves, and perhaps most notably, chickens.. ever wondered where the phrase “pecking order” came from? (That one’s for free).

Don’t worry, Why Now isn’t expanding it’s purview into primers on the animal kingdom. Although, do expect a foray into some biotech topics soon!

Instead, I’ve been ruminating of late on the logical end state (or as Palmer Luckey puts it — the “final platform”) of LLMs. What will this technology look and feel like when it reaches maturity? What does the “iPhone” of Generative AI look like?

Well, for a start, LLM context-limits don’t feel particularly mature to me. Rather, they’re akin to the bandwidth limits that were present in early mobile network generations. Good luck using TikTok, Spotify, Discord, etc., with 2G’s 50 kbps!

So, what’s being done today to loosen this constraint?

Well, it’s fair to say that one solution — vector databases — received its fair share of.. attention (sorry). Given how models du jour price (ie per [X]K tokens), perhaps vector dbs will remain the most economically viable solution. However, another approach has recently come into play: Hyena Hierarchy.

Hyena is proposed to be a drop-in replacement (much like Zig is for C!) for the Transformer’s key mechanism — the attention operator — which, whilst a seminal innovation, is the source of LLMs’ scaling challenges to date.

Throughout this primer I’ll walk you through:

• How and why we arrived at the attention mechanism (RNNs, LSTMs, GRUs, etc)

• How the attention mechanism actually works - step by step

• Why we need an alternative to attention

• Said alternative to attention - “Hyena”

If the fastest growing application of all time was unlocked via “attention”, then it’s incumbent on us (as investors at least) to study where the puck is heading next.

Let’s see if attention, in fact, isn’t all you need?

Hyena is defined as: a sub-quadratic drop-in replacement for attention constructed by interleaving implicitly parametrised long-convolutions and data-controlled gating.

Oh boy…

Don’t worry, as always in Why Now, we’ll build towards an understanding of the (rather abstruse) definition above via breaking it into its components.

Before we dig in here, it’s worth remembering that machine learning may be full of complex math, but all of this math is ultimately attempting to recreate how we as humans, rather effortlessly, make predictions:

So, throughout this primer, I’ll push you to think about how you yourself, very naturally, make specific predictions, before I reveal how a given model does so. If you listen to Deepmind’s podcast, you’ll notice that much of the team’s time is spent ultimately running the same exercise.

Conveniently, the “attention mechanism” present in Transformers is a good opportunity to begin this exercise. For example, in the sentence: “hello, please subscribe to my…”, if you were to predict the next word in this sentence, I suspect (or hope!) some of you may predict “newsletter”.

                                        [ hello, — please — subscribe — to — my — ]                                           

Analyse this sentence, what about it would increase your likelihood of making this prediction? Do genuinely give this a go.

Well, you may pay heed (or, attention) to the word “subscribe”, and if you do, you would likely ask yourself… what’s possible to subscribe to? [SaaS products, Dollar Shave Club, …newsletters].

In tandem, you might pay attention to the word “my” given its relation to, or influence on, the word “subscribe”. If you have context (ie you’ve been “trained”) on who I am, you may pick the word “newsletter” from your word list. Why? Because you know that I’m the author of Why Now, not the founder of DSC.

Similarly, words like “please”, “hello” or “to” should have less attention placed on them. They don’t exactly contribute here.

I suspect that for most of you this invention doesn’t exactly feel, groundbreaking? Good, in many ways the idea itself isn’t. However, it’s important to understand on a technical level how we got here, so that we can then better predict what comes after Transformers.

We’ll do this first by studying Recurrent Neural Networks (RNNs), and then by delving into the actual architecture of a Transformer.

RNNs are often the architecture of choice for making predictions based on sequential data (e.g., sentences, audio, time-series, etc), because, unlike traditional “feed forward neural networks” (your run-of-the-mill neural net), they’re ~capable of preserving an intermediary state (known as a “hidden state”):

Let’s go back to our “hello, please subscribe to my..” example, to demonstrate (on an admittedly theoretically level), why this state maintenance matters.

If our RNN “remembers” that it’s already processed the word “subscribe”, then when it later processes the word “my”, this new word has a little more context. Ie, the model may (theoretically) ask itself: what does Alex own that people could subscribe to?

Without this context (or memory), our model would more generally just try to predict the most likely word to follow “my”; errors can occur here as there may be words closer to “my” in vector space than “newsletter”.

Cool, it sounds like due to RNNs’ “memory” we can make accurate predictions. Why do we need Transformers at all then? Well, it turns out that RNNs are a tad forgetful — they suffer from “memory loss”.

Key Point: “Memory loss” is what it says on the tin. An input (think a given word in a sentence) your neural network has received, is forgotten over time. This is as a result of both the “vanishing gradient” and “exploding gradient” problems (we’ll spend time on the vanishing gradient).

If you’re familiar with how neural networks learn (or Vercel’s style guide..), then the word “gradient” likely isn’t new to you.

A loss function, gradient descent, and backpropagation underpin how model parameters (weights and biases) are updated:

Quick level-setter:

1. Our loss function determines how (in)accurate our model is — the model’s “loss”.

2. Backpropagation determines the gradient of the loss function with respect to a given weight/bias (ie how “responsible” each weight/bias is for the overall loss).

3. Gradient descent is the formula above, it calculates the weights/biases to minimise this loss based on a configured leaning rate. Et voilà! We have learning.

Technical Detail: As some of you may recall from differential calculus, if we want to solve the following equation: y = f(u) and u = f(x), which can also be rewritten as y = f(u(x)), we can leverage the “chain rule”. It’s this very rule that backpropagation is built upon.

In neural networks, backpropagation calculates the gradients (remember, “loss responsibility”), for the outermost layer’s weights/biases, and then, we use these gradients to help determine the gradients of previous layers, and so on so forth.

Remember, in neural networks, each neuron is actually function: σ(w1*a1 + w2*a2, w3*a2….+ bias), whose inputs (specifically the “a” which = “activation”) is derived from another function. Sounds kind of similar to y = f(u(x)), right?

Alas, whilst backpropagation (thanks Yann), was a key innovation for making training scale, it’s not perfect. Enter vanishing gradients!

Remember, neural networks consist of many “hidden layers” (ie the layers between the input & output layers). So, when applying the chain rule to neural networks our equation may essentially look like, brace yourselves…

y = f(a(b(c(d(e(f(g(h(i))))))))).. you get me? (this isn’t the actual formula, but gets the point across)

What essentially happens here is that we use the gradients of the previous neuron (the gradient of the weight and gradient of the bias), to help calculate the gradients of the current neuron. We do so aaaaall the way to the very first hidden layer.

The “vanishing” happens if the gradients across layers are consistently small, as if you multiply say, 0.05 x 0.02, you get 0.001, now imagine taking this 0.001 gradient, and doing the same again & again. Eventually the gradient values get infinitesimally small. Some might even say that they “vanish”!

Why is this an issue? Well, per the gradient descent formula, if we calculate our weights/biases via the gradient value, if the gradient value itself is minuscule, the resulting change in the weight/bias will be too. Hence, our network’s learning adopts the pace of a Gastropoda (keeping the animal kingdom theme alive here).

Key Point: To make matters worse, this problem is exacerbated in RNNs by what’s known as backpropagation through time (or “BPTT”). We don’t have to get into the weeds on this. Instead, I’d rather you have a think about why this may be the case? (psst, focus on time). Here’s the answer courtesy of Lex. Thanks Lex.

** RNNs, Backpropagation, Vanishing Gradients - nice work! **

Hyena is defined as: a sub-quadratic drop-in replacement for attention constructed by interleaving implicitly parametrised long-convolutions and data-controlled gating.

Ok, so we’ve learned that traditional RNNs, whilst directionally correct, suffer from memory loss, due to the vanishing (or exploding) gradient problem. If these models can’t recall “long-term dependencies” (e.g., early words in a paragraph), then they can’t really be used for making predictions based on sequential data. A shame.

Enter “gating” — spot it above? At a high-level, “gates” help RNNs control what data in a sequence to keep (ie remember), and what data to forget. This gets us one step closer to paying “attention”:

Ok, ok, but how do these fabled gates actually work?

Well, it depends, choose your fighter: Long Short-Term Memory (LSTM) networks or Gated Recurrent Units (GRUs). We’ll go with the more modern of the two, GRUs, introduced by Cho et al. in 2014.

As mentioned, in “vanilla” feed-forward neural networks, a neuron is essentially an “activation function” (e.g., tanh⁡, sigmoid or ReLU) that receives a weighted sum and a bias as an input: σ(w1*a1 + w2*a2, w3*a2….+ bias) and produces an “activation” that feeds into the next layer.

Technical Detail: Sigmoid, tanh, etc., functions are known as “squishification” functions as you can pass them an array of numbers (ie a word, ie a vector), and they will “squish” (ie map) these numbers to values between 0 and 1. This is helpful for probability calculus.

For example, if I pass [2.3, 5.1, 8.9] into the sigmoid activation function: “σ(2.3, 5.1, 8.9)” like so:

It’ll return 2.3 = 0.9, 5.1 = 1, 8.9 = 1. Note, the… squishification!

RNNs introduce the concept of “time”, and hence, their activation functions are a little more complex: σ(w1*a1 + w2*a2… + w1t-1*a1t-1 + w2t-1*a2t-1… + bias).

This notation is, frankly, wrong, but what “w1t-1*a1t-1 + w2t-1*a2t-1” is attempting to get across is that our activation functions in RNNs take into account both an entirely new input (e.g., a new word in a paragraph) as well as the activation produced by previous layers (known as the “hidden state”).

Unfortunately for us, GRU neurons (often referred to as “units” or “cells”) make the complexity of an RNN activation look rather… simple:

Don’t fret. Remember, the above is the equivalent of a neuron (ie a simple activation function) in a feed forward neural network. So, each layer of a GRU will consist of multiple of these units, that all connect to the next layer of the network.

Note the presence of a “Reset Gate” and an “Update Gate” , as well as the specific insertion points of the sigmoid and tanh “squishification” functions. Let’s dig in via an analogy to root this complexity back to “real life”.

Think about the process of learning something new. You often have some base understanding of the topic (a hidden state, if you will), and then you’re gradually exposed to new information during your learning process.

Occasionally, your “hidden state” will be updated, as you realise that your prior understanding was perhaps off-kilter. Similarly, you may disregard some new information as it’s irrelevant or contradicts something you’ve learned that you consider to be true. GRU’s data-controlling gates are the machine equivalent.

Ok, back to our GRU:

Our Reset Gate is used to determine how much of the past (ie the “hidden state”) to forget. As shown in the diagram above, the current input value and the hidden state pass through the sigmoid function and produce a “reset gate value”. A value close to 0 means “forget the past”, a value close to 1 means “retain the past”.

Our Update Gate dictates how much of the remaining past information (ie the reset gate value) to keep vs. the new information to consider (e.g., a new word in a paragraph). The resulting (intermediary) value is known as the “candidate activation”.

The final hidden state (or “activation” of the unit) is then calculated by combining the past hidden state with the new candidate activation. Et voilà! Our unit is complete.

At this point I’ll (backward)pass (sorry!) this primer to you. What does all of this mean? Why do you think that “gating” helps deal with the vanishing gradient problem? Again, give it a go, it’ll help cement much of what you’ve read so far.

The truth is, gating helps in many ways. However, think about it, if gating reduces what our neural network needs to remember, then there are less available neurons (and hence, gradients) to perpetuate the vanishing gradient problem!

Hyena is defined as: a sub-quadratic drop-in replacement for attention constructed by interleaving implicitly parametrised long-convolutions and data-controlled gating.

Alright folks, it’s time for Transformers. If GRUs are capable of preserving long-term dependencies, why didn’t they unlock this current wave of AI mania? Well, there are a few reasons.

Firstly, GRUs (just like traditional RNNs) process data sequentially (duh, they’re used to process sequential data). The issue with this is that it makes “parallelisation” across time-steps difficult, leading to longer training times. What if we could process an entire body of text at once vs. doing so word-by-word?

Following closely in second, whilst GRUs are notably more performant than RNNs at recalling long-term dependencies, the Transformer’s “attention mechanism” represents relationships between inputs in a more complex and rich manner.

So, we know what “attention” means at a theoretical level, but how is this implemented technically and what is a “Transformer”, really?

Much like the European Union (news to me), the Transformer consists of three core pillars: positional encodings, self-attention, and an overarching encoder-decoder structure. (We’ll tackle the first two)

Let’s start with positional encodings as they’re reasonably easy to get our heads around. Remember this dream?: “what if we could process an entire body of text vs. doing so word-by-word?”, well, it turns out we can do exactly this.

Instead of feeding a sentence (ie sequential data) word-by-word into a neural network to preserve its order, we can encode this positional information into the vector representation of each word.

Due to their theoretical simplicity, positional encodings may seem trivial, but they’re a game changer. To contrast, within RNNs, each word is essentially given its own layer. This means that if we want to train on, say, adult fiction novels (70k - 120k avg. word count), our network would need, you guessed it, ~70-120k+ layers (!). Transformers dramatically reduce this “network depth”.

I personally don’t think positional encodings get enough publicity. Perhaps “Positional Encodings are All You Need” is missing the requisite alliteration? Anyhoo, how does the attention-grabbing, “attention mechanism”, work?

Let’s go back to my, incredibly subtle, call to action.. I mean.. example:

                                       [ hello, — please — subscribe — to — my — ]

As a reminder, the idea behind attention is that certain words in a sentence have a greater “influence” on (or, relation to) a given word than others.

In our example, we said that “subscribe” (theoretically) would have a greater influence on “my” than “hello,” or “please” would.

Ok, so how is this “influence” actually calculated? Attention begins by deriving three vectors (ie points in space) for each word in a sentence. So, our sentence above would result in 15 vectors (hello x 3, please x 3,… etc) calculated for each “attention head” (ignore for now) within the Transformer.

Let’s stay as consistent as ever and use the word “subscribe” to walkthrough this vector math:

Query Vector: This is a representation of what you’re looking for (e.g., something relevant to “subscribe”). It’s calculated by multiplying the original embedding of the word (subscribe=[.25, .17, .3]) by the query vector’s weight.

Key Point: Remember, this “weight” is often “initialised” with ~random values and learned over time through training. Just like any other neural network!

Key Vector: This is essentially an “index” or “tag” that makes a given word easier to categorise. It’s calculated by multiplying the original embedding of the word (subscribe=[.25, .17, .3]) by the given key vector’s weight.

Value Vector: This is the actual “content” that you end up retrieving (ie a vector representation of “subscribe”). It’s calculated by multiplying the original embedding of the word, you guessed it, (subscribe=[.25, .17, .3]) by the given value vector’s weight.

Don’t worry readers (or,.. Alex), this math class is over shortly. Next, we take the query vector of one word (e.g., “subscribe”) and the key vectors of every word in the given sentence, including “subscribe” itself.

Through doing so, we calculate “attention scores” — ie how relevant a list of words are to our query word (“subscribe”). Note in the example below that we’re doing matrix multiplication here to calculate these scores. The words on the left are the query vectors, and the words on the top are key vectors.

If we then repeat this process for every word in our sentence, well, our model has stored some form of “memory” or “state” associated with these words, without any need for the recurrence present in RNNs and GRUs. Without recurrence, the vanishing gradient problem literally dissipates!

We then run these scores through a “softmax” function (similar to our friend, the lowly sigmoid) to get these scores into a “normalised probability distribution”, also known as: between 0-1.

& finally, dear readers, we multiply each value from the softmax output with its corresponding value vector and sum them all up. This sum is the context-enriched output for a given set of words.

So, what’s hopefully clear is that Transformers’ adulation is arguably just. Not only are they considerably more efficient to train (thanks positional encodings), but they also store long-term dependencies much more effectively than the next best thing!

Sound too good to be true?…

Hyena is defined as: a sub-quadratic drop-in replacement for attention constructed by interleaving implicitly parametrised long-convolutions and data-controlled gating.

For those of you left, nice work! I’m aware that calling this post a “lengther” (Irish colloquialism) would be an understatement. However, there’s plenty of historical context and other antecedents whose exposure/recall are typically assumed within these ML papers. A pity really!

Alas, I am here to spoil the party slightly. Whilst the attention mechanism is clearly a positive step-change, like many technical decisions, it isn’t immune to trade-offs.

Attention is fundamentally a quadratic operation. Think about it (& revisit our matrix above), to calculate attention scores, you need to compare n number of words by n number of words (ie n x n).

Quadratic operations become inefficient as the size of the input (ie the context we provide a given LLM) grows, because the number of operations grow meaningfully:

This is (primarily) why ChatGPT et al., have context limits!

What if there was a way to resolve the quadratic nature of attention? (you may now strikethrough “sub-quadratic”). Can we remove this n x n dynamic whilst maintaining performance?

The answer has been a partial “yes” for some time (thank you H3, etc). However, Hyena, thanks to some lessons gleaned from signal processing, has made this confirmation rather resounding.

Quick level-setter: signal processing is a field of study that deals with the analysis, manipulation and, ultimately, interpretation, of “signals” (ie time or spatial-varying data like audio). Hmm. This sure sounds akin to sequential data?

Within signal processing, “convolutions” (a mathematical operation) are used to calculate how one signal (e.g. your voice) is modified by another signal or system (e.g. a room). Through leveraging convolutions, we can precisely enhance or attenuate a specific signal by adjusting its “filters” accordingly.

Hmm x 2. “Enhance or attenuate” — this also sounds a little similar to what the attention mechanism does in Transformers, or perhaps “gates” in GRUs? Both mechanisms amplify or reduce certain inputs (ie data).

So, perhaps these “filters” can similarly preserve long-range dependencies?

The truth is, they’ve already been doing so (albeit inadequately) for some time. I suspect those of you that aren’t entirely new to deep learning have been graced by “CNNs” or Convolutional Neural Networks?

Well, within these networks, it’s these “filters” that are tweaked and learned over time. Much like our weights and bias within a traditional neuron! A filter (aka “kernel” or “mask”) in a CNN is typically a matrix like so:

Specifically, “convolutional layers” (which contain many filters) are often used at the beginning of computer vision tasks to extract key “features” of an image. A very high-level example of an extractable feature may be something like rounded corners.

So, if my CNN has been sufficiently trained (and hence, had its filters tweaked) to identify rounded corners in an image, when it ingests an image whose data overlaps with the above matrix, our model may infer that this image has a rounded corner. A match! Nice.

Much like its scavenging counterpart, the Hyena Operator takes what’s good from the “bone” of the CNN. However, it then goes one step further:

Instead of relying on the “short” filters (e.g., a 3x3 matrix) present in CNNs, Hyena leverages “long” convolutions (ie filters that span the entire length of the input sequence). This means that the entire input is a feature! Meaning that each output element depends on all inputs, thus capturing the “global context” of the input image, sentence, etc.

Hyena is defined as: a sub-quadratic drop-in replacement for attention constructed by interleaving implicitly parametrised long-convolutions and data-controlled gating.

Fancy striking through “implicitly” also? Cool, implicitly just means that these filters are learned (much like we’ve already learned ourselves) vs. hard-coded. Simple!

Hyena is defined as: a sub-quadratic drop-in replacement for attention constructed by interleaving implicitly parametrised long-convolutions and data-controlled gating.

Ok, cool, definition complete, but how does Hyena’s combination of “long-convolutions” and “data-controlled gating” equate to computational complexity that is O(n log n)? Or, “sub-quadratic”.

Well, put simply, there’s no n x n calculation present in Hyena. Long-convolutions enable the learning of long-range dependencies (without a n x n attention matrix), and data-controlled gating, much like we’ve seen in GRUs, filters out unnecessary data (and hence, computation).

Post-Transformers - Hyena Hierarchy - c’est fini!

If you’ve made it this far, thank you (& fair play!). If you’ve learned something along the way, I’d love it if you share this post with a friend or two. I’m pretty close to an exciting subscriber milestone so it’d help a tonne!

As always, I’m at alex@tapestry.vc - always down to say hello! If you’re wondering where to go next, check out my posts on Nix, Zig, WebGPU or Building Pokemon Yellow in Godot (personal fave).

Taking the above into account, what would a, from the ground-up, “frontend IDE” look like?

This is my first Substack “thread” so.. um.. discuss!

Welcome to new subs + thanks a million for the Godot love! This thing has grown a whole lot faster than my inner-pessimist could have imagined. So, grazie!

If you enjoy these primers, I’d appreciate it if you favourite this post’s tweet to help spread the good (?) word of Why Now!

As always, I’m at alex@tapestry.vc should you ever wish to say hello. If you are building with Zig, I most definitely want to chat.

Not every technology is capable of direct value capture. This can be perilous as an infra investor, as it’s easy to become enamoured by something that’s innovative, yet devoid of a business model or meaningful market. Guilty.

Programming languages are a good example here. How Rust handles concurrency is a thing of beauty, Roc’s VM-less memory allocation is pretty cool.. and, well, there’s a lot to like about Zig. However, these languages aren’t licensed out, their value capture certainly isn’t tied to usage/utility.

What helps me sleep at night however is that these technologies (think languages, runtimes, protocols, etc) can serve as proxies for sophisticated & opinionated technologists. Dissenters capable of being wrong.

Now this is something that history would tell me capital is worth putting towards.

Take Zig for example. Bun is outpacing Node.js and its anagrammatic counterpart, Deno. How so? Well, Bun does a lot of things well, but it does tout Zig’s low-level control of memory and lack of hidden control flow (so good!) as a key unlock.

Continuing down this proxy, we find Stephen Gutekanst / Mach Engine working 24/7 on a game engine to help “upend the gaming industry”. Building a game engine ~from scratch is no small feat.

++ finally, we have TigerBeetle. Again, one doesn’t simply build a financial database from the ground up over a leisurely weekend. Zig has served as a beacon, attracting those who’re thinking orthogonally. I’m fond of folk like this.

As you may have guessed from, I don’t know, the title (!) we’re going to dig into Zig.

Selfishly, I want to truly get my head around why companies are opting to build with this nascent language as well as what we should expect next. I figured I’d bring you all along for the ride.

If you are building with Zig, I’d love to say hello! alex@tapestry.vc

Finally, I’d ask that, if possible, you consider supporting the Zig Software Foundation.

Zig is many [1][2][3] things [4][5] but at its core it’s:

1. A general-purpose programming language (think Python or JavaScript).

2. A “toolchain”.

The programming language part here is familiar, and hence, easy to grok. However, it’s worth lingering on. My favourite quality of Zig’s is its simplicity. What does this mean?

Well, firstly, the language is tiny.

It’s specified with a 500-line PEG grammar file. For context, a PEG (parsing expression grammar) file is typically used to define the structure and syntax of a programming language.

The tl;dr on this benefit is that a “small” language ultimately means that you have less language-specific keywords, etc., to remember. Hence, the language is “simple”.

** As someone that somehow still struggles to recall git flags, I’m --happy about this! **

Less keywords also means that there’s ideally “only one obvious way to do things.” Why is this good? Well, it becomes much easier to read your/someone else’s code when you know that a specific keyword is typically used for a handful (vs. infinite) number of things.

Fun Fact: Zig's developers took this mandate so much to heart that for a time, Zig had no for loop, which was deemed an unnecessary syntactic elaboration upon the already adequate while loop. “What you do is who you are”.

This goal stands in stark contrast with decisions made within languages like Python, where I’ve truly lost track of the number of keywords that essentially say: “if this happens then do that”. Sorry, I’m ranting.

Back to Zig. The language doesn’t stop here though, oh no. Zig also has “no hidden control flow”. This essentially means that each line-of-code written in Zig executes sequentially, as you would expect:

If you haven’t written code before this is likely confusing: doesn’t all code execute.. line-by-line (ie sequentially)? Nope. Oftentimes languages (e.g., JavaScript) will take ~helpful but “hidden” steps for the developer to ~fix/improve the execution order (ie “control flow”) of their code.

Technical Detail: Check out variable/function “hoisting” in JavaScript as an example of such hidden magic.

This action is often helpful, as intended. However, because it’s hidden from the developer, it can make their code more difficult to reason about (especially for other developers).

Make sure this point clicks as it’s important. Think about it, a developer could forget about the hidden control flow rules of a given language; all of a sudden, their code isn’t executing sequentially as expected? This happens, often in subtle ways, and is confusing for all involved.

As Zig eloquently puts it, you should “focus on debugging your application, not your programming language knowledge”.

Bun provides an equally glowing endorsement: “low-level control over memory and lack of hidden control flow makes it much simpler to write fast software.”

Alas, we have introduced a new value prop here: “Low-level control over memory”. Que?

Much like Fall Out Boy (sorry), our computers have “memory” or “mmrs” (sorry again). ie spaces (e.g., RAM) within the system where they store data or instructions that will ultimately be used/manipulated again.

For example, in my JavaScript program: `example.js` I might create a variable: `let age = 28`. JavaScript will then pull some Houdini-work once more and dynamically allocate enough space in memory at “runtime” to store the variable `age` for me. Helpful.

Zig, is less.. presumptuous. Within Zig we have to specify what type our variable is, which reveals a tad more information about how much memory our variable requires.

** If you want to visit / revisit “types” check out my primer on Deno (+ TypeScript)! **

Zig doesn’t stop here though, defining your types “statically” is the easy part.

Zig, much like C and C++, enables developers to allocate memory (remember, a space) manually. This means that Zig developers can ~precisely state how much memory they require for a given variable, function, etc., as well as when this memory should be freed, and hence, used elsewhere.

Again, to make the comparison to Brendan Eich’s darling — JavaScript handles this freeing of memory automatically. This process is known as “garbage collection”.

Technical Detail: Unlike C and C++, which use the `malloc` keyword, Zig doesn’t expose memory allocation directly in the language’s syntax.

Instead, memory access is exposed via Zig’s “standard library”.

A standard library is a set of pre-installed functions/objects that are ready to be used with a given language. For example, Python’s standard library comes with the `datetime` object.

This, again, makes memory allocation in Zig more explicit / clear.

At this point you’re likely thinking, “so what”. Fair. What’s important to know — and what relates back to Bun’s proclamation in our intro — is that this granular control of memory leads to performance gains. Why?

Well, there are a few reasons. I’ll point-er (sorry) out two:

1. Fragmentation: As memory is allocated and deallocated dynamically, free memory blocks become scattered across the “heap” (place especially for dynamic memory). This can result in fragmented memory, where there are small gaps between allocated blocks.

I’ve “drawn” a diagram that hopefully depicts this issue better than the wall of text above does. The main point being that these fragments, because they’re small, end up being a literal waste of space. (harsh, I know)

2. Garbage Collection: Yep, y’boy garbage collection (GC) packs a punch. GC introduces additional overhead. Why? Well, because it’s ultimately another “program” running in the background of your own.

Andrew Kelley, the creator of Zig, goes as far as saying that GC can result in “stop the world latency glitches”. When it comes to building critical systems (think aviation software), “latency” doesn’t cut it.

GC can also result in “non-deterministic” memory deallocation. ie, it may ultimately free memory that you would have ideally still had allocated. This is another example of a “hidden” action that other languages take.

To reiterate, whilst potentially perilous, software written in C, C++, Zig, etc., can be more performant than software written in dynamically allocated memory (DAM.. Daniel) languages a la Python or JavaScript.

Once again, Zig’s explicitness (in this case, explicit memory allocation) is what makes it simple. You, and your rag-and-tag crew of developers, don’t have to figure out how memory’s allocated/freed in your application, you literally state this in your code.

** Did I mention the word explicit? **

As hard is it may be to believe, Zig does even more to foster “simple” codebases such as omitting a “preprocessor” and “macros”. Don’t worry, we’ll get into what these terms mean.

** Ok, take a sip of water, go get some sunshine etc., we’re onto Zig’s “toolchain” **

The meaning of “toolchain” is a little more difficult to accurately scope. However, the word typically means a set of utilities: libraries, compilers, build tools, etc., that the language, or users of the language, can leverage.

1. Libraries = code that someone else has written and packaged which can now be used by others to achieve a specific task. E.g., Rust’s Pola.rs library for data manipulation.

2. Compilers = take your high-level code and convert it to “machine code” (1s and 0s) that corresponds to a specific instruction set. Do some other helpful things like optimising your code (e.g., removing “dead code”).

3. Build Tools = a build tool manages the entire build process, which includes compilation but also includes dependency management, testing, packaging, etc. I wrote about “building” software in detail on my Nix primer.

We can use Zig’s stated goals (“maintaining robust, optimal and reusable software”) to fine-tune our definition.

1. Robust = software written in Zig works consistently, even during edge-cases.

2. Optimal = software written in Zig can be.. optimised.. for a specific task.

3. Reusable = software written in Zig is simple, scalable & portable.

With these goals in mind, I consider (go easy HN!) the Zig toolchain’s most notable features to be the following two:

1. Zig’s “Comptime”.

2. Zig’s 4 build systems.

Ok, I’m done writing in lists of twos & threes, I promise. Let’s delve into Zig’s “Comptime”.

Zig touts its Comptime as “A fresh approach to metaprogramming based on compile-time code execution and lazy evaluation.” Let’s unpack each word emphasised as per. First, compile-time.

Software has a “lifecycle” that ultimately results in said software being executed (ie running on a computer):

Developers write code (think Python), “compile” this code, “link” each compiled file generated (called an “object file”) into a final “executable” and then “run” (ie execute) this.. um.. executable.

Programming languages are typically evaluated at either compile-time (e.g., TypeScript) or runtime (JavaScript). “Evaluation” essentially means checking for errors, determining the “type” of a given variable, etc., all with the aim of ultimately executing a program.

Like any technical decision, there isn’t an objectively “correct” way to evaluate a program. Rather, there are trade-offs.

For example, if you evaluate a language’s “types” at compile-time, then you’ll pick-up the incorrect usage of a “string” in a function that expects an “integer” before you compile said language and run it somewhere. Thus picking up a “bug” before your software is deployed. Phew.

The drawback of this compile-time eval is that developers have to specify the exact type of data they expect their function to receive. This can get rather tricky. Why? Well, end-users of software are unpredictable, they may end up inserting valid data types (e.g., an integer in a “first name” field on a form) that you may not expect.

Zig takes a more.. democratic, approach. The language enables developers to explicitly state which blocks of their code they’d like “evaluated” at compile-time vs. runtime. This is handled via Zig’s `comptime` keyword:

Taking all that we now know about Zig, we can assume that the primary goal of this explicit statement of compile-time vs. runtime evaluation is.. you guessed it, explicitness.

A developer reading your Zig code doesn’t have to identify/recall what’s being evaluated at compile-time, you literally tell them. Much like Zig’s control flow, nothing is “hidden” from the developer.

Ok, cool, we like explicitness. However, I want to also point out that Comptime reiterates Zig’s ability to be fine-tuned for performance.

For example, if we offload type inference to the developer who compiles their software, then the end-user (think a general “consumer”) doesn’t have to handle type inference on their own machine at runtime. Nice.

Right, so we know what evaluation is and when it happens (compile-time / runtime). What’s “lazy” evaluation?

Well, thankfully, it’s rather self-explanatory. Lazy evaluation, much like a “lazy person”, isn’t proactive, it only completes a task at the last-minute, when it must.

I’ll make this more concrete with some (simple!) Zig code which we’ll build on.

If we were to lazily evaluate this code, we would only check/determine the values of the variables: first_name (“Alex”) and second_name (“Mackenzie”), when we need them. In this case, we need these values to complete the first_name ++ second_name operation.

Why is lazy evaluation helpful you ask? Well, it means you’re not doing any heavy-lifting before you have to, which ultimately results in more-efficient resource allocation.

Why calculate the value of an expression if you’re only maybe (e.g., in the context of conditional logic) going to use it later? Smart.

This said, whilst Zig makes this lazy eval.. explicit.. I do personally feel that lazy evaluation goes against Zig’s simplicity / feels a little “hidden”. This is a primer though, so let’s leave my judgement to the side.

Well, this is a “lengther”. Sorry, but programming languages are very much the aggregation of minute technical decisions that, in aggregate, support a handful of objectives. If you want to grok a language, you’ve got to appreciate its nuances.

Next, “Metaprogramming”. Remember our brief mention of “preprocessors” and “macros”? They’re back. Kinda.

Metaprogramming is common in systems-level programming languages like C, C++, Rust. It’s what you likely expect — a program, “programming” itself. Woah, meta.

In practice, metaprogramming involves leveraging compile-time information (e.g., type declarations like: var age: i32 = 28;) to manipulate (e.g., edit/generate code) your program in some way.

For example, with this “type information” our program could automatically edit our variable age’s data type to be “i8” vs. “i32”. i8 is a smaller data type, and hence, takes up less memory. Thus, through metaprogramming, we have optimised our Zig code at compile-time. C'est très cool!

Technical Detail: In C/C++ metaprogramming is handled by a “preprocessor” program that uses “macros” (ie specific keywords like: #define).

Without getting unnecessarily into the weeds, these macros are complex/error-prone; so much so that they’re considered by some to be a “separate programming language” beyond C/C++.

Whereas Zig treats metaprogramming as a “first-class citizen”, and hence, tightly integrates the process with the rest of its “toolchain” (via Comptime).

Alright, we’re nearly wrapped up here with Zig. For those of you still here, nice job, this isn’t an easy read by any means. It certainly wasn’t a walk in the park for my ghost-writer to draft! (joke).

As mentioned, Zig is… supple. It has 4 “build modes”. Again, we discussed what “building software” is at length within my Nix primer so I shall point you there if you need a refresher.

Zig’s 4 build modes are:

• Debug = used during development (ie writing your code) and prioritises ease of debugging over performance. In this mode, code is compiled with additional debugging info.

• ReleaseSafe = used for the final build of an application when performance and optimisation are critical.

• ReleaseSmall = prioritises generating the smallest possible “executable”. Achieved through techniques such as dead code (ie unused) elimination or “function/data merging” (removing duplicates). This mode’s particularly useful for embedded systems (e.g., a Ring doorbell) that have limited resources (compute/memory).

• ReleaseFast = sits in between debug and release modes. Optimises for performance but still includes some additional debugging info.

You select one of these build modes via the command line like so:

In particular, these build modes speak to Zig’s stated goal of producing optimal and reusable software. Wanna run some Zig code on your toaster? Cool, use ReleaseSmall. Fancy building a database? Impressive, but please use ReleaseSafe.

As hard as it may be to believe, there’s so much more (build.zig, cross-compilation, etc.) that I’d like to take you through re. Zig. However, I feel like the law of diminishing returns is almost certainly kicking in already.

I suspect you “get it”, you understand the essence and purpose of Zig. I’d encourage you to jump into the following posts/videos if you’re interested in learning more:

• Mitchell Hashimoto: Zig Build Internals.

• Fastly: Build an Efficient & Portable Programming Language with Zig.

• Andrew Kelley: The Road to Zig 1.0.

As I have espoused many times, I like “Serious Software”. Think game engines, 3d modelling software, runtimes, etc., as the “real estate” of features that can (and should!) be optimised within them is tremendous. Good luck building Blender over a weekend!

Jarred puts it similarly when asked “Why is Bun Fast?”: “In one word: obsession. An enormous amount of time spent profiling, benchmarking and optimising things. The answer is different for every part of Bun”.

I suspect Zig will continue to make it easier for more folk like Jarred to lean into their obsessions and take on incumbents through fine-grained tweaks and tuning. If so, I am very excited to see what’s coming around the corner.

Finally, I’d ask that, if possible, you consider supporting the Zig Software Foundation.

Welcome to new subs + thanks a million for the Secure Enclaves love! This thing has grown a whole lot faster than my inner pessimist could have imagined. So, grazie!

If you enjoy these primers, I’d appreciate it if you favourite this post’s tweet to help spread the good (?) word of Why Now!

As always, I’m at alex@tapestry.vc should you ever wish to chat.

For awhile now I’ve been ruminating on what it would take to dethrone Unity. Why? Well, game studio revenue follows a power law similar to the one that venture obeys. So, if you back the right engine you back the right index.

This is why “infrastructure” — everything from databases to build systems — is a compelling investment category to me. Infrastructure is an index of.. innovation.. a little (lot) cringe I know, but true?

Anyway, back to Unity. We’re witnessing the “rise of indie” in gaming. WebGPU is (officially!) bringing high-performance/fidelity gaming to the browser, Xbox’s Game Pass is actually “Netflix for gaming” and LLMs mean that what used to take hours (sprite sheets in Piskel) takes minutes.

Not to mention the fact that my mother is crushing it on Candy Crush.

Godot would appear to be the indie developer’s engine of choice. So, I decided to pick it up last September to ease the imposter syndrome (I usually build “traditional” software). I recreated the very first Game Boy game I ever laid my youthful eyes on. Pokemon Yellow!

I documented the development process on my blog (mack.work) + decided it’d be fun to share it today as a *bonus* Why Now issue. I will likely write-up a similar post on my process of learning Zig soon. Lmk what you think of this format?

PS - this was pre the latest GenAI boom, so I did indeed spend ~2 hours drawing all the game assets pixel-by-pixel. I also had to learn Godot’s native scripting language (GDScript). Kinda crazy how much has changed since!

The next Why Now primer is under construction (hint, hint) so stay tuned.

Creating the Main Character

First, I created a "Player" scene that contains an AnimatedSprite & a CollisionShape2D. I created the player animations in Piskel

Pokemon_Yellow_Remastered - Godot Engine - Watch Video

I then added code to make the player's animations dynamic & to enable the player to move. The code below is written in "GDScript", Godot's DSL that resembles Python.

extends KinematicBody2D

export var speed = 400
var screen_size

func _ready():
    screen_size = get_viewport_rect().size
        
func _physics_process(delta):
    var velocity = Vector2.ZERO # The player's movement vector.
    if Input.is_action_pressed("move_right"):
        velocity.x += 1
    if Input.is_action_pressed("move_left"):
        velocity.x -= 1
    if Input.is_action_pressed("move_down"):
        velocity.y += 1
    if Input.is_action_pressed("move_up"):
        velocity.y -= 1        

    if velocity.length() > 0:
        velocity = velocity.normalized() * speed
        $AnimatedSprite.play()
    else:
        $AnimatedSprite.stop()
        
    position += velocity * delta

    if velocity.x != 0:
        $AnimatedSprite.animation = "Walking_Sideways"
        $AnimatedSprite.flip_v = false
        $AnimatedSprite.flip_h = velocity.x < 0
    if velocity.y > 0:
        $AnimatedSprite.animation = "Walking_Down"
    elif velocity.y < 0:
        $AnimatedSprite.animation = "Walking_Up"

    move_and_collide(velocity * delta)
  

Creating Pallet Town

I found a Pallet Town sprite sheet & edited it in Figma. I used CollisionPolygon2D to create custom collision shapes in order to allow the player to walk on top of doors.

In time, when a player steps on these doors they'll "enter" the house.

Godot - Player Scene - Watch Video

Creating a Menu

I'm currently creating a popup menu that's triggered by pressing the "M" key & closed via pressing the "Esc" key or pressing "Enter" on the "Exit" menu option.

Note, when the popup menu is opened, the arrow keys typically used to move the Player must now instead, be used to navigate the menu.

I used Godot's "Focus" UI properties to enable an event-based navigation arrow that corresponds to the appropriate arrow keys.

Godot - Main Menu Scene

See main menu code below. Go easy on me, I'll refactor later.

extends Node

signal menu_opened
signal menu_closed

func _ready():
    $Menu.hide()
    $Menu/VBoxContainer/Pokedex/Arrow1.hide()
    $Menu/VBoxContainer/Pokemon/Arrow2.hide()
    $Menu/VBoxContainer/Item/Arrow3.hide()
    $Menu/VBoxContainer/Ash/Arrow4.hide()
    $Menu/VBoxContainer/Save/Arrow5.hide()
    $Menu/VBoxContainer/Option/Arrow6.hide()
    $Menu/VBoxContainer/Exit/Arrow7.hide()
    
    
func _process(delta):
    if Input.is_action_pressed("Menu_Open"):
        $Menu.show()
        $Menu/VBoxContainer/Pokedex.grab_focus()
        emit_signal("menu_opened")
    
    if Input.is_action_pressed("Menu_Close"):
        $Menu.hide()
        emit_signal("menu_closed")
    
# Arrow navigation.        
    
func _on_Pokedex_focus_entered():
    $Menu/VBoxContainer/Pokedex/Arrow1.show()

func _on_Pokedex_focus_exited():
    $Menu/VBoxContainer/Pokedex/Arrow1.hide()    

func _on_Pokemon_focus_entered():
    $Menu/VBoxContainer/Pokemon/Arrow2.show()

func _on_Pokemon_focus_exited():
    $Menu/VBoxContainer/Pokemon/Arrow2.hide()

func _on_Item_focus_entered():
    $Menu/VBoxContainer/Item/Arrow3.show()

func _on_Item_focus_exited():
    $Menu/VBoxContainer/Item/Arrow3.hide()

func _on_Ash_focus_entered():
    $Menu/VBoxContainer/Ash/Arrow4.show()

func _on_Ash_focus_exited():
    $Menu/VBoxContainer/Ash/Arrow4.hide()

func _on_Save_focus_entered():
    $Menu/VBoxContainer/Save/Arrow5.show()

func _on_Save_focus_exited():
    $Menu/VBoxContainer/Save/Arrow5.hide()

func _on_Option_focus_entered():
    $Menu/VBoxContainer/Option/Arrow6.show()

func _on_Option_focus_exited():
    $Menu/VBoxContainer/Option/Arrow6.hide()

func _on_Exit_focus_entered():
    $Menu/VBoxContainer/Exit/Arrow7.show()

func _on_Exit_focus_exited():
    $Menu/VBoxContainer/Exit/Arrow7.hide()

Next, I created a sub menu for each item. This was pretty straightforward given I already built the UI focus logic. I plan on returning to the "Save" sub menu later once I grok working with databases/storage via Godot.

if $Menu.visible == true:
        if $Menu/VBoxContainer/Exit.has_focus():
            if Input.is_action_pressed("ui_select"):
                emit_signal("menu_closed")
                $Menu.hide()
        
        if $Menu/VBoxContainer/Pokedex.has_focus():
            if Input.is_action_pressed("ui_select"):
                $Menu/Haunter.show()
            if Input.is_action_pressed("ui_back"):
                $Menu/Haunter.hide()

I had a bunch of fun building Professor Oak's Lab & the "scene switching" logic. Essentially what happens here is that when my animated spite enters the collision path of Oak's Lab's door, a signal fires which causes the "Oak's Lab" scene to be loaded.

Loom Message - 9 October 2022 - Watch Video

Pallet Town script.

extends Control

func _on_Area2D_body_entered(body):
    get_tree().change_scene("res://OaksLab.tscn")

Oak's Lab script.

extends Node2D

signal exited_oaks_lab_interior

func _on_Area2D_body_entered(body):
    emit_signal("location_changed")
    get_tree().change_scene("res://Main.tscn")

Admittedly what I have not figured out yet (tips welcome!) is how to change the starting position of my animated sprite when exiting Oak's Lab so that the sprite appears as though he's just exited the door of the lab. Something to do with changing the position variable's value based on signals I suspect.

Added a camera to the game which was pretty simple but has added a real sense of presence to the world.

Loom Message - 9 October 2022 - Watch Video

Welcome to new subs + thanks a million for the WebGPU love! This thing has grown a whole lot faster than my inner pessimist could have imagined. So, grazie!

If you enjoy these primers, I’d appreciate it if you favourite this post’s tweet to help spread the good (?) word of Why Now!

As always, I’m at alex@tapestry.vc should you ever wish to chat.

Many smart friends are 50/50 on cybersecurity as a pre-seed/seed investment category. I have always been perplexed by this.

From my vantage-point, security is one of the few categories where actors are just as incentivised to “innovate” on problem vs. solution creation. Thus, upstarts are provided with wedges to take on incumbents.

The second criminally under-discussed topic in the world of technology investing is cybersecurity’s role in the “Why Now” market equation.

Without asymmetric encryption (the “s” in https) the modern internet as we know it wouldn’t have thrived. Talk about a firm grip (..sorry) on innovation! There’s more:

• Pioneering remote-first companies (a la GitLab) without VPNs (+ perhaps SSH) would likely have struggled to operate. Would Deel be a company?

• The emerging “isolate cloud” (Deno, Grafbase, etc) is partially a second-order outcome of Chrome’s need to sandbox untrusted code.

• Microservices wouldn’t have proliferated to the same degree without service meshes a la Envoy. (& more recently, eBPF).

• And, as I’ve written [1][2] about previously, broad-scale personalised ML models will not be “a thing” (technical term) without privacy-enhancing technologies (PETs).

Ok, point proven. So what’s next in cybersecurity?

Famous last words, but the “final frontier” has always been to encrypt data across its three states: at-rest, in-transit & in-use.

Why? Because if sensitive data like your FullName is perennially encrypted (e.g., “De2CYsx”) it’s rather difficult for a malicious actor to do anything untowardly.

Conveniently, how we secure data in-use has three “states” also: fully homomorphic encryption (FHE), trusted platform modules (TPM), and secure enclaves.

As you may have ..deciphered.. from the title of this primer, we’ll be delving into secure enclaves — the approach many hardware (Intel/AMD) + software/cloud (AWS/Azure) giants are hanging their hats on.

Within this primer we’ll discuss:

• A ~brief history of encryption.

• Approaches to data in-use security.

• Secure Enclaves. You’d hope!

Thanks to friends Nev, Liam & Liz at Evervault for the inspiration for this post + for reviewing!

Yes, the above caption was my attempt of humour! .. just subscribe?

Secure Enclaves are highly-constrained compute environments that allow for cryptographic verification (attestation) of the code being executed.

As always, let’s cherry-pick the complexity out of this definition & take it step-by-step. First, some history.

Encryption — A Brief History (2023), Alexander Mackenzie

Cryptography is the practice & study of techniques of secure communication in the presence of adversarial behaviour.

Encryption is the process (& in many ways, art) of scrambling data so that only authorised parties can understand the underlying information. So yes, encryption is a subset of cryptography.

In and around 100BC Julius Caesar was slinging secret messages around Caput Mundi (aka Rome) via a relatively simple substitution “cipher” (ie encrypt/decrypt function) now known as the “Caesar Cipher”.

This was a relatively simple “shift-by-three” cipher. Meaning that the letter “A” would be substituted with the letter “D”, the letter “X” would be substituted with “A”, etc.

What’s pertinent here is that Caesar Cipher is rule-based encryption. Once someone knows the rules of the game they can easily decrypt the “ciphertext” (encrypted text) into “plaintext” (decrypted text).

Astute readers will note that if letters such as “D” or “L” appear frequently in Caesar’s ciphertext, that these letters likely map to plaintext vowels (A and I respectively).

Similarly, readers may look for common bigrams (sequences of letters) in words such as: NG, ST, QU, etc. This strategy is known as “frequency analysis”. Not exactly an unbreakable code.

** Modern encryption wasn’t built in a day either! … **

Modern encryption tends to rely on what’s known as an “encryption key” (or key*s*) which stems from the work of ~Blaise de Vigenère in the 16th century.

Encryption keys are a hard-to-guess string of letters, numbers and symbols such as: b'VQfCNh**72dvh that, when combined with a secret message, produce a unique output.

Below I used the python cryptography library to:

• Generate an encryption key

• Encrypt a message (“SuBsCrIbE”.. subtle) with this key

• Print the ciphertext

• Decrypt the ciphertext with the key & print the resulting plaintext

Technical Detail: When a single encryption key is used to both encrypt & decrypt data this is known as “symmetric encryption”.

When alternate keys are used to encrypt (public key) and decrypt (private key) data this is known as “asymmetric encryption”.

Vigenère used symmetric encryption to make his cipher “polyalphabetic” — whereby each plaintext character (e.g., “C”) was combined with an encryption key character (e.g., “H”) in order to “generate” a character (“J”) from a specific alphabet.

Example below:

However, this method and its 19th/20th century successors (including the German Enigma rotor machine!) once again eventually fell short when frequency analysis was used.

** I’m oversimplifying what it took to crack Enigma to the nth degree here! **

Fast-forward to the 1970s and IBM designed a cipher (remember, an encrypt/decrypt function) named Lucifer. A bit.. disconcerting?

Yet, despite this, the cipher was later adopted by NIST (National Institute of Standards & Technology) as a US national standard and (thankfully) renamed to the Data Encryption Standard, or “DES”.

Elsewhere in the 1970s (1976) our friend asymmetric encryption (remember, public and private keys) began to bloom thanks to Whitfield Diffie & Martin Hellman. The advent of asymmetric encryption marked the dawn of “modern” cryptography.

Key Point: Thanks to asymmetric encryption, parties no longer need a shared secret (encryption key) to securely communicate.

Alice can share her public key with Bob in order to encrypt a “secret message” from Bob, but she never has to share her private (decryption) key with him.

Take a second to internalise this as it’s important!

Shortly after (1977), fuelled by Manischewitz (worth looking into), Rivest, Shamir and Adleman released the RSA asymmetric cipher. Yes… like the conference. RSA is the most popular / widely understood asymmetric encryption system to date.

However, as I alluded to in this primer’s opening paragraph: the incentive to create new problems in cybersecurity is high. Come 1997, DES was cracked. Symmetric encryption needed a new saviour.

How was it cracked? Well, thanks to Ye Olde “Moore’s Law” clock speeds (see WebGPU primer) improved.

This meant that via sheer brute-force computation (ie trying every possible solution), malicious actors could correctly guess a 56-bit DES encryption key, and hence, access an underlying message.

Thus, in 1997, another RFP was published by NIST for a new national standard. By the second millennium (yes, 2000), a new cipher was adopted known as Rijndael (pronounced rain-dahl) or the Advanced Encryption Standard.

Without getting into the mathematical weeds, one of AES’ main advantages is that it offers 128-bit, 192-bit or 256-bit encryption keys.

For context, even with a supercomputer, it would take 1 billion billion years to crack a 128-bit AES key using brute force attack.

** Cue the “but what about post-quantum cryptography!” outcry **

A fair Q — quantum is/was a very real threat.

In 2016, NIST… you guessed it, put out another RFP for cryptographers to devise and then vet encryption methods that could resist an attack from future quantum computers.

In July ‘22 the first four quantum-resistant algorithms (thank you lattices) were announced (!) which I’m, naturally, paying quite a bit of attention to, but are well-beyond the scope of this primer.

** Encryption - fin **

Secure Enclaves are highly-constrained compute environments that allow for cryptographic verification (attestation) of the code being executed.

Ok — so what does “data in-use” mean. Why is it a particularly difficult problem to solve?

As mentioned, data has three states:

• It’s “at-rest” (😢) when it’s idle (e.g., stored in a database).

• It’s “in-transit” when sent across a network (e.g., from your phone to the cloud).

• It’s “in-use” when it’s being used/processed in some way (e.g., rendering a UI).

The challenge is, when we want to use data we ~can’t keep it encrypted. Why? well, it’s tricky to do some math on a value such as: “De2CYsx”. So, at some point we need to decrypt data if we want to process it.

However, it’s when we end up decrypting ciphertext into plaintext that things get.. precarious. Hence, data in-use security = holy grail (sorry TLS).

Ok, so what have we done about this problem-space to date?

An alternative to secure enclaves is homomorphic encryption (HE). An encryption “scheme” (ie approach) proposed as early as 1978, what a time to be A[iV3 for cryptographers!

HE enables someone to process data (derive a value from it) without ever having to decrypt the data. Think about what this means for a second:

With HE, we can pass some ciphertext (yes, De2CYsx) as an argument of a function that calculates the sum of 2 + De2CYsx and returns the same answer that it would if De2CYsx was decrypted (perhaps a number like 3?). See below:

Pretty cool, huh? However, as perhaps expected in the field of cryptography, nothing is quite as it seems.

Firstly, homomorphic encryption schemes can be partial or “somewhat” vs. full.

Partial (PHE) or Somewhat (SHE) means that only certain mathematical operations (multiplication and/or addition) can be conducted on the ciphertext.

This is fine, but we’re not quite at the promised land here. Computers tend to do a little more than multiplication/addition.

However, come 30 years later and we kinda are. Craig Gentry (& co) came along in 2009 with a fully homomorphic encryption (you guessed it, FHE) scheme. FHE, in theory at least, enables anyone to perform any operation on ciphertext.

Note: Within these primers I typically try to leave no stone unturned when explaining how something works.

For homomorphic encryption I think this would be a mistake. Each homomorphic encryption scheme is nuanced & requires some pretty hardcore math.

If you’re really into your prime numbers (…), check out the Paillier cryptosystem which is an approach I quite enjoyed digging into.

At this point you’re likely thinking — “why isn’t this primer about FHE then?”. Fair.

Well, the issue with FHE is that it requires a meaningful amount of compute and/or time to calculate a new value from ciphertext inputs.

To put this into context, IBM released its improved (!) HElib C++ library for FHE in 2018. A calculation that would take a second to perform using plaintext would take an average of 11.5 days (!) to perform using the 2018 version of HElib.

As a wise individual once loosely said — no person has any time time for that.

Time for this primer’s raison d'être — secure enclaves.

Ok, so an alternative to processing ciphertext is… to process plaintext. Shocker.

However, we know that exposing plaintext is the root of all data security problems. So, what can we do?

Well, we need to first work backwards to understand how this plaintext data may be accessed and by whom. Once we know the threat, we can form a response.

Let’s use AWS EC2 as an example. In its least-flattering light, EC2 is rented “compute” (ie CPU processing).

A given AWS EC2 “instance” is broadly composed of the following components: users (employees/contractors), 3rd party libraries, applications and an operating system. These components are the whom and should not be trusted.

Moving swiftly onto the how. In computing, when we want to process some data or code, our operating system loads this data from “storage” (e.g., a hard drive) to “memory” (ie RAM). The same is hence true for EC2.

In storage our data is “at-rest”, and hence, encrypted. However, when loaded into RAM it’s decrypted so that it can be used. Et voila! Un gros problème.

This is the how. Many methods, such as “memory scraping”, exist that exploit plaintext that’s exposed in memory. Cybersecurity is hard.

So, what can be done to make the processing of this plaintext more secure?

Well, there is a handful of approaches that various secure enclave providers take. We’ll keep the example alive + analyse AWS’ Nitro Enclaves offering which is leveraged by the likes of Evervault, Crypto.com, Anjuna, etc.

Nitro Enclaves creates an isolated environment that’s a peer (just think linked in some way) to another EC2 instance (the “parent”) that you’re running. We load and process our sensitive data in this enclave.

As alluded to, this peer is a little different, more.. constrained.

For example, your Nitro Enclave has no internet access, no shell (e.g., SSH) access, no persistent storage and it runs its own kernel. Good luck to anyone trying to exfiltrate sensitive data from an enclave that isn’t connected to the internet.

Technical Detail: SSH = “secure shell”. A shell is a command-line interface (or CLI) that acts as an intermediary between a user and an operating system.

When you open up your mac “terminal” you ultimately use a shell program (e.g, “zsh” or “bash”) to interact with the OS.

SSH is a.. secure.. way to remotely access (or “tunnel” into) a given machine, remotely. We don’t want folk to do this when sensitive data’s involved.

The only way you can bi-directionally communicate with this secure enclave is via a “Unix Socket” (don’t worry about this) that connects to the enclave’s “parent” EC2 instance.

Why stop here though? “Defence in depth”, or, the more technical term: being a pain in the @$$ multiple times, is a strategy worth adhering to.

When “instantiating” (ie creating) a Nitro Enclave, a user specifies a number of parameters such as:

• What application code the enclave will run.

• The enclave’s software dependencies.

• How much memory it will require.

• How many CPU cores it will need.

These inputs are fed into a hash function that produces a ~unique “checksum” (ie hash, ie string of characters) based on its inputs. You may remember checksums from the Why Now post on Nix.

This cryptographic attestation is a bit of a game changer. Now, each time we want to use our enclave to process some sensitive data, we can first check whether or not the secure enclave’s inputs still produce the same hash.

If so, we know we’re running the same application code, with the same dependencies, in the same way. This allows us to cryptographically prove that our enclaves haven’t been tampered with in some way. Pretty cool!

As a wise Disney pop-sensation once said, with secure enclaves we have the best of both worlds: a way to provably protect data in-use, ~performantly.

Secure Enclaves are highly-constrained compute environments that allow for cryptographic verification (attestation) of the code being executed.

** Secure Enclaves — fin **

Welcome to new subs + thanks a million for the Nix love!

I am doing my best to resist the innate Irish urge to self-disparage, & instead, ask you all to please favourite (or denigrate!) this post’s tweet.

Either approach helps get Why Now out into the ether further. Thanks!

As always, I’m at alex@tapestry.vc should you ever wish to chat.

I absolutely love it (need to get out more in ‘23) when technologies have uses far-beyond their intended purpose. Niche, I know.

Lest we forget that the “Slinky” was a stabilisation spring, Berkeley Packet Filters were for ..filtering.. network packets, and GPUs were solely for rendering computer graphics. Matrix multiplication.. who dis?

This lineage, twinned with the acronym soup (a la GPGPU, WGL, WGSL, SPIR) that, per Mackenzie’s Law™, must accompany anything to do with computer graphics, has tee’d this primer up rather nicely.

Alas, things are going to get worse before they get better.

** Enter “WebGPU” **

Web browsers are increasingly converging towards “operating systems”; applications (e.g., game engines) that were once primarily used natively are now adequately running within our browsers. How?

There’s, of course, no #C0C0C0 bullet.

However — Conflict-Free Replicated Data Types, Operational Transformation, & everyone’s favourite compilation target, WebAssembly certainly aren’t impediments.

Well, we may now soon have another technological arrow in our quiver — WebGPU.

Without revealing too much, let’s just say that up until this point via “WebGL” (we will discuss) the modern GPU has been a second citizen within the browser. WebGPU is changing that.

Within this primer we’ll discuss:

• The history of GPUs.

• The “graphics pipeline”.

• The history of graphics APIs.

• WebGPU. You’d hope!

• What trends +/ applications WebGPU may unlock +/ promulgate.

WebGPU is the working name (work harder, please?) for a Web API for accelerated graphics and compute.

As always, let’s first cherry-pick the complexity out of this definition & take it step-by-step.

Act 1, Scene 1 — the Graphics Processing Unit.

** If you’re intimately familiar (weird!) with CPU/GPUs you can skip this section **

To appreciate the GPU, we must first understand the CPU.

Central Processing Units are “chips” (ie billions of transistors et al packaged) used for “general purpose” computation; meaning that these chips are “architected” to handle a broad range of, ultimately mathematical (thank you George Boole), operations.

You know when your Salesforce app totals the (de minimis 😢) number of leads in your pipeline? Cool, that mathematical operation is handled by your CPU’s “Arithmetic-Logic Unit”. As is the bold toggle when you’re writing a Substack post.

Key Point: CPUs handle (or, “process”) the general, often very different, data and actions that your computer takes to run itself and third-party software.

It’s important to note that CPUs are designed to process these disparate operations as quickly as possible — also known as being optimised for “low latency”.

The “why” re. low latency should be clear here — we are modern, entitled computer users that expect almost prescient (thanks React) execution (!). However, this demand for low latency is also inextricably linked with how CPUs process instructions.

Each CPU’s (or CPU cores’!) operations are executed serially (ie one at a time). At an almost offensively high-level (ignoring interrupts, etc), revisiting our example, this means that a CPU would execute our Salesforce pipeline arithmetic first, and then once complete, it would execute our bold operation in Substack.

Technical Detail x2:

“Cores” are essentially mini-CPUs, CPUs within a CPU. They enable(d) some degree of parallel (vs. serial) execution. You may remember how big of a deal the first Intel “Dual-Core” processor was when launched in 2006?

CPU performance is often measured by “clock speed”. Clock speed == the number of compute cycles a CPU handles per second. Let’s not get into “compute cycles” in this post — just know that, all else being equal, the more compute cycles a CPU can handle per second, the more “performant” a computer is.

GPUs on the other hand are a little more picky, less.. general purpose.

How so? Broadly speaking, GPUs have a much more narrowly-scoped “instruction set architecture” (or “ISA”).

Without getting into the weeds of ISAs, they basically determine the range of operations a chip can take on the data it receives. I elaborated further on them within the first (🥹) Why Now post on eBPF!

Once upon a time, GPUs were considered “ASICs”, or application specific integrated circuits. Ie chips optimised for a specific task — in GPU’s case, rendering graphics.

In order to grok (spoiler alert!) why GPUs have since become used for matrix multiplication, proof-of-work cryptomining, &, making a pretty penny on StockX, we need to go back to their ASIC roots.

** Buckle in, as we’re about to tackle comp graphics, a rather gnarly (but fun!) topic **

Your computer screen’s display is made up of minute points of illumination known as “pixels” — aka “picture elements”. Colour, coat & shade these little guys & you “render” graphics. Do so with great detail and dynamism, and you generate something as breathtaking as BOTW.

A copious amount of work goes into designing what you see above. However, at a very high-level: Computer graphics is the process of converting data (e.g., cartesian coordinates) into images on a computer screen.

This conversion gets a whole lot more laborious when converting a 3D “scene” (more on this in a second). Why? Because your computer screen is a 2D environment.

We’ll focus on 3D graphics rendering as I think it better illustrates the “graphics pipeline”. (Also something we’ll be revisiting A$AP).

A “scene” is a foundational concept in 3D whereby each scene consists of:

• 3D geometry (ie “vectors”).

• Transformations.

• Lights.

• Material Properties.

• ~A Camera / View.

For graphics newcomers I suspect this all seems a little abstruse. To ground what I just said into reality, pick up an object around you.. perhaps a computer mouse?

Yes, really.. pick it up.

Hold the mouse close to your face (the “camera” in this case), then move it away from you. Note how the object inhabits more of your field of view when close? This is the result of an object (ie the mouse) being “transformed”.

Notice how light sources bounce off of the mouse when you hold it at various angles?

Ok, now pick up something else, perhaps your keyboar..efouvwvrrnibtv (I jest!).

Notice how light sources bounce differently off of your keyboard vs. mouse? This is due to their respective materials.

If we want to render images on computer screens that look 3D, these images need comparable properties (light, materials, etc) & to exhibit similar behaviours (shadow casting, light refraction, etc).

For those of you that write code, below is some JavaScript I wrote (TQ three.js) to render this cube. Note the creation of a “scene” with some geometric points, a transformation, some lighting, etc.

** To be clear — you do not need to understand this code to understand computer graphics! **

Ok, so we know what a 3D scene is comprised of & why. How does this scene get converted into the aforementioned 2D image?

** Yes, that’s right Jensen, via the “graphics pipeline”! **

Graphics pipelines vary. However, at a very high-level, what happens here is:

• We first create this 3D scene via a 3D creation application such as Godot (my favourite piece of software on this here earth!). We then export this 3D scene in an appropriate file format, e.g., .glTF.

• We then pass this file (ie data) from our CPU to our GPU. This data is an input into the graphics pipeline. This pipeline is a chain of functions (aka “shaders”) that are executed by the GPU.

• These functions ultimately “draw” (ie place, colour and shade) pixels on a screen that represent the supplied 3D scene.
  

Technical Detail: functions in programming are exactly the same concept that you learned in math. They (typically) receive data (arguments), do something with this data (e.g., add two integers together) and “return” a result.

There are a lot of pixels to draw on modern high-resolution screens.

To make things worse, modern content (think a YouTube video) is often rendered at 60 frames per second — meaning all of those pixels have to be re-drawn 60 times a second.

Suddenly, CPUs in all their general-purpose, serialisable goodness seem a little.. inefficient?

Graphics Processing Units were designed to do something about this.

Whilst modern CPUs may have something to the tune of 10 general-purpose cores (remember, mini-CPUs), modern GPUs like Nvidia’s GeForce GTX 1080 have 2560 specialised cores (!).

Technical Detail: These cores have less general-purpose instruction sets to those present in CPUs. They’re “specialised” for specific, graphics-based, operations.

With this many cores, GPUs can pass many “pieces” of 3D geometry (called “vectors”) and later in the pipeline, pixels, into multiple instances of our beloved graphics pipeline at once.

** Re-read that previous sentence to make sure it clicks. **

Meaning that GPUs process data (ie vectors & pixels) in a parallel manner vs. the serial approach taken by CPUs.

Key Point: CPUs are designed for “low-latency” — doing a single computation as quickly as possible. GPUs are designed for “high-throughput” — doing as many computations as possible.

If you’re interested in learning about the history of GPUs I’d highly recommend listening to Acquired’s episode(s) on Nvidia. However, I have taken the liberty of bringing you right to the punchline — high-throughput capabilities are useful for a whole lot more than just graphics rendering 😧:

Proof-of-worked based cryptocurrency mining, matrix multiplication (ie ML), scientific computing, video editing, etc, et al.

This offloading of specific tasks from CPUs to GPUs (or any specialised hardware for that matter!) is known as hardware acceleration. GPUs accelerate graphics rendering as well as certain types of computation.

…ok

If I’ve done my “job” here you should have a high-level understanding of CPUs, GPUs & computer graphics. Not bad! All necessary conduits for truly grokking WebGPU.

** Next up, graphics APIs **

WebGPU is the working name for a Web API for accelerated graphics and compute.

Cool, we’ve picked off a few items here in our definition. Progress.

We’re at a point now where we know why our applications would want to communicate with GPUs. Hardware acceleration!

But, how does this communication happen?

Graphics APIs, my dear Watson.

** Proceeds to whip out a history textbook **

The history of the modern graphics pipeline starts in a galaxy far, far away (sorry..) at LucasFilm in 1980. Back then, computer graphics was a particularly onerous process; every GPU model had its own software (!) (ie “drivers”).

Technical Detail: “drivers” expose the GPU’s capabilities to the operating system in a way the OS understands and expects. The operating system in turn can expose it to applications, using various graphics APIs.

This meant that if a developer wanted to get a GPU to execute their graphics pipeline (remember, chain of functions), they would need to know the exact “instruction set” for that particular make of GPU. Talk about “doing things that don’t scale”!

Thankfully, the OpenGL (Khronos Group) & DirectX (Microsoft) “Graphics APIs” were released in 1992 + 1995 respectively.

These Graphics APIs provided a common interface for developers, whereby they could learn the API’s specification vs. the specification of a specific GPU driver. The API would then handle an individual GPU’s idiosyncrasies. C’est simple!

Alas, the new APIs on the block and their GPU counterparts weren’t perfect. They only ran the “fixed-function pipeline”, or “FFP”. I promised many acronyms!

For once in graphics, the parlance here is semi-informative. “Fixed Function” meant that there was a specific.. fixed.. set of functions (e.g., make these pixels look “foggy”) exposed to the developer (by these APIs) that they could avail of when rendering graphics.

Put another way, during the fixed-function pipeline epoch, we didn’t have truly “programmable GPUs”.

Meaning that developers couldn’t use GPUs to accelerate arbitrary +/ custom graphics functions that they wrote themselves (aka “shaders”). E.g., make these pixels look “foggy” based on my specification (ie my code) of foggy.

Technical Detail: ~~Pretty much~~ any function that’s executed on a GPU is also considered a “shader”. Common shaders include vertex shaders, geometry shaders, tesselation control/evaluation shaders and fragment shaders — this is a topic for another day!

Then 2002 came along, and with it, Nvidia unveiling the GeForce FX GPU, a “fully programmable” GPU.

Of course, this innovation primarily initially equated to “faster horses”. Ie better graphics.

However, some enterprising folk realised that if they encoded any data they wanted processed as if it was graphics data (remember our 3D scenes?) that they could use these GPUs to accelerate a much broader set of calculations (ie beyond graphics).

C’est cool, non?

** This is a seminal moment in the history of computing, make sure it clicks **

Skip forward a wee bit to 2006 and Nvidia released “CUDA”. AKA, the succinct, Compute Unified Device Architecture. This was real hat-tip to those using Nvidia GPUs for general-purpose computing.

Or — GPGPU, if you will.

At an embarrassingly ( ͡° ͜ʖ ͡°) high level, CUDA was released as a new API, for a new set of Nvidia GPUs.

These GPUs had a more general purpose instruction set. Meaning that they could be used for general purpose computing (shocker!), without the complex encoding work I described a few paragraphs back.

Even better, developers could interface with CUDA in higher-level programming languages like CUDA-C / C++ or.. Fortran.. 😷.

Ok — cool, thanks to a spate of innovation in graphics, GPUs were unshackled from their singular use case. Why stop here though?

** We are nearly at WebGPU itself FYI **

Fast-forward to 2011 and Khronos released “WebGL” (ie “Web Graphics Library”) a browser-native, JavaScript API. Unsurprisingly, WebGL is really a thin “wrapper” around OpenGL.

This API ultimately runs “on top of” (ie gets converted into) the OpenGL spec when used on macOS / Linus or DirectX on windows.

Technical Detail: “browser-native” just means that the API is part of the standard
set of APIs that web browsers (e.g., Google Chrome) expose to web developers.

Meaning that developers don’t have to “install” WebGL like they would install a library like React.

This API enables web developers to perform 2D and 3D rendering (remember, drawing pixels on screens!) within an HTML <canvas> element.

If you’re unfamiliar with HTML do not fret, just think of <canvas> as a space on a webpage where graphics can be rendered by a GPU. Pretty simple.

Here’s an illustration by Figma Co-Founder, Evan Wallace, using WebGL in the early 2000s to showcase realtime water rendering. Press the G key on your keyboard when in the demo 😎

Ok — so as of 2011, GPU APIs are also present in web browsers. Rejoice!

However, note that there’s not much chat about WebGL for the “GPGPU” use case. Perhaps there’s room for a new.. GPU API for the Web..?

** If you’ve continued reading this far, fair play &.. sorry? **

WebGPU is the working name for a Web API for accelerated graphics and compute.

As one might expect in an industry that lionises being “high-throughput”, further progress wasn’t too far away.

Outside of the web in 2013, AMD began developing “Mantle”. This graphics API espoused even “lower-level” access to modern GPUs (hence, greater performance).

Unfortunately, to cut a long-story-short, Mantle’s public development was discontinued 2 years after its first breath in 2015. However, they graciously (hmm..) donated the API to the Khronos Group in 2016.

This donation led to the release of “Vulkan” by Khronos. As one would expect, Vulkan exposes Mantle’s low-level GPU access, but the API also offers:

• A single API for both desktop and mobile graphics devices; whereas previously these platforms were split between the OpenGL and OpenGL ES (“embedded systems”) APIs respectively.

• Cross-platform compatibility. Vulkan is available on multiple operating systems and devices; where as Direct3D 12 is, as you might expect, only available on Windows/Xbox (boo).

• Reduced CPU load by offloading (to the GPU) computations traditionally handled by the CPU.

• A pretty badass, if not a little disconcerting.. new brand identity.
  

In.. parallel.. (ha!) with Vulkan, Microsoft didn’t sit back idly — they released Direct3D 12 in 2015. Apple, unwilling to be left out, also joined in the modern GPU API fray in 2014 avec “Metal”.

** What does this have to do with WebGPU you may be asking yourself? Fair.. **

Well, as I said, WebGL didn’t exactly orient itself towards the GPGPU use case. Why, retrophilia..? WebGPU on the other hand, has first-class support for performing general purpose computations on the GPU.

WebGL also doesn’t offer many of the modern features that native solutions, a la Vulkan, provide. How so?

Well, WebGL is broadly a thin “wrapper” (ie a similar spec) based on OpenGL 1.0 (released in 1992), which itself, traces back to Silicon Graphics’ “IRIS GL” — released in the 1980s (!). It old.

Meaning that WebGL isn’t really built to match the design of modern GPUs, causing GPU & CPU performance issues. This ageing spec also means it’s difficult to layer WebGL on top of our modern native friends: Vulkan, Direct3D 12, Metal, etc.

Thus, work on a new Web GPU API began! & with it, a modern, bold, provocative, name.. WebGPU..

Whilst WebGPU’s working name is lacking in some imagination, the API’s spec and goals most certainly aren’t.

Firstly — Surma puts it best, WebGPU feels webby ( / wɛbi / ). Whilst I can’t exactly provide you with a technical definition here, I agree. WebGPU works in a familiar manner to its Web API counterparts.

Generally it’s viewed that WebGPU is a mixture of Direct3D 12, Metal & Vulkan. So, not only is WebGPU ~~ webby ~~ but it also maps well to native APIs, reducing the aforementioned GPU & CPU performance issues.

As mentioned, WebGPU also defies WebGL’s parochialism (bit harsh I know) by exposing two “pipelines” (remember, a chain of functions) to web developers — a “render pipeline” and a “compute pipeline”.

Finally, the web as rolled out the red carpet for GPGPU!

At this point, we could go deeper into WebGPU’s spec, tackling its abstractions (logical devices, adapters, etc) but I suspect that this may be overkill for the 3 readers still left… sorry, I know this was a lengther!

** For those interested in going deeper, Alain Galvan’s work is the-best-in-the-biz **

So instead, I figured we’d move onto some final predictions about what trends and applications WebGPU may unlock and/or promulgate:

• BOTW-Esque Graphics: Ironically, the real magic here is when render and compute pipelines are blended. Leveraging the compute pipeline facilitates modern graphics rendering techniques a la ray tracing within the web browser. Expect cool new occlusion / refraction-based UIs.

• ML in-Browser: Tensorflow.js (or ONNX.js) provides a bunch of great demos on why you may want to train a ML model in a web browser. Again, this isn’t exactly “novel” but WebGPU will make the process more performant +/ specific. There’s also a world where offloading training to your users’ local machine makes sense from a latency, cost + privacy-preserving (long-live federated learning) perspective.

• “Serious Software” in-Browser: Broadly, serious software (think Blender, Unreal, AE, etc) is OS-native vs. web-native due to its performance requirements. Collaboration flows (as Figma has proven) change entirely when you can afford to be “multiplayer-first” (thanks to web browsers). I’m curious if this creates sufficient breathing room to dethrone the likes of Unity or Autodesk.

• “Serious Games” in-Browser: there have been numerous false starts in this space and it’s still very much TBD on the local vs. cloud gaming debate — perhaps not mutually exclusive! Either way, with WebGPU it’s far more plausible that there’s a future where I’ll be playing The Elder Scrolls with some mates within a web browser. How cool.

• Browser-OS: As teased, the browser is increasingly being bestowed with the functionality of native operating systems. Future applications will all be expected to look/work like Linear — offline mode enabled, multiplayer-first, etc. This trend is being driven by many innovations, WebGPU is a constituent.

** WebGPU — fin **

Firstly, some thoughts.

This whole thing has been growing much faster than I expected, I’ve had some time to reflect on the “why”. I promise (hope?) this paragraph has not wandered itself into this post as a thinly veiled “humble brag”, but rather, as a brief, ~unconventional, take.

For really the first time, I’m writing in a purely self-serving way. What primer will I write next? Whatever complex topic I find satiating. When will I release it? Whenever I feel the pull to do so.

I’ve found that this mindset has removed all of the (self-inflicted) pressure that comes with running this type of thing. Which has, ironically, made me more consistent. Go forth, and write selfishly!

** Updates. What I’ve been up to, etc. **

Ok, I will cede to recency bias and give a brief shoutout to the wonderful team at Charm. A friend mentioned them last Friday + I spent more time than I care to admit last weekend tinkering around with their various CLI libraries.

Here’s a shell script I wrote with Charm (well, gum) that enables any of you that are new here to subscribe to Why Now. ! How convenient !

I also stumbled upon some of Basement’s work (that GameCube throwback 🥲) who have worked with the likes of Vercel, Scale + MrBeast. This led me down the rabbit hole of finally tinkering around with three.js: mack.work/cube.html.

Here’s the code if, you too, want a rotating cube on your site.

Modeling in 3D again brought back a bunch of nostalgia — I used to design YouTube intros using Cinema 4D in my early teens. Long ..really cool.. story. So I decided to pick up Blender a couple of weeks ago.

** Thankfully, I also spent every other waking moment of my youth playing rugby **

I am generally a fan of “serious software” like Blender, Godot, and After Effects, where you can granularly edit every property, pannel, view, etc. Having a lot of fun (pain) atm modeling a 3D Gengar which will replace my rotating cube! </rip>

Blender is very much the deep end though. Bezel +/ Spline does a good job of abstracting some of the complexity of 3D away if any of you are interested in dabbling too.

I, almost contemporaneously, came across Kyle at Banana.dev’s “let’s build the internet from scratch!” series of live streams. Highly recommend.

There’s something to watching someone iterate on their understanding of a technology live vs. reading a polished primer (guilty) async.

Hardware folk, I have something for you — as a result of the above thought, I picked up the Razor Kiyo Pro and Razor Seiren X condenser capsule. The condenser mic has a “super cardioid” pickup pattern, meaning that it picks up audio from the front, but suppresses audio from the side/back.

..I also definitely didn’t pick up a 12” ring light which is kind enough to accommodate 192 LEDs..

The thought here is that I may emulate Kyle’s approach & occasionally release a “raw” mack.work primer! Figured it may be interesting, if not voyeuristic, for some to see the trials and tribulations (Wikipedia) I go through when learning something new.

** Why Now **

Given all of the work/thinking I’ve been doing in the realm of computer graphics of late, I think it’d be remiss of me to not tackle WebGPU.

Here are some of the other technologies I’ve been looking into recently:

• Mongo’s queryable encryption. A bit of a fallacy? Maybe. But, all roads lead to FHE, IMO.

• Nextflow: a bioinformatics workflow scheduler. *Broadly* similar to Lyft’s Flyte (which I believe LatchBio uses).

• Unison: a whole new approach to distributed computing. Worth digging into this over an afternoon. I may revisit this in a future primer.

• Kindelia: an arguably more fundamental model of computation — Interaction Combinators. This admittedly took me a while to grok but was a lot of fun.

• Wireguard: something hoping to eat OpenVPN’s lunch. A simple yet fast and modern VPN that utilizes state-of-the-art cryptography.

I’d love your thoughts on the above + I’m always open to suggestions. Feel free to say hello at alex[at]tapestry[dot]vc or on the thread associated with this newsletter.

Leave a comment

Thanks for reading as always + lmk what you think of these occasional updates/polls.

If you have a friend that you think may enjoy reading future primers with a sprinkle of some Irish self-disparagement, I’d appreciate you sharing this newsletter with them.

Wow. “Something Different”, truly was.. different. Thank you for the kind words & a hello to many new subscribers! To continue to haphazardly ride on the coattails of experimentation, I’m trying something.. new: I’d appreciate love +/ lambasting on this post’s tweet to help get Why Now out into the ether further.

As always, I’m at alex@tapestry.vc should you ever wish to chat.

Stepping on a lego brick, not saving your old Pokémon game.. software running incorrectly on your machine.. all sources of excruciating pain.

Cue “hasn’t {build tool / package manager} already solved this issue?”

In short, somewhat. Docker, Buildpacks, etc., all do a decent job of building + packaging software that can be “run” elsewhere. For example, I use Dockerfiles in Northflank to great effect (plug, et al) if I do say so myself.

However, a proclamation in software development — “works on my machine” is still necessary; implying that it’s possible, despite using these tools, that software may indeed, not work on your machine.

This is where Nix comes in. Well, came in 19 years ago. 10 years Docker’s senior!

Nix is notoriously difficult to grok, but in its most reductive form, it’s a build tool + package manager. More flattering descriptions have extolled the technology with credentials such as “the future of computing” or as “an outrageously underrated technology”.

Whatever your Nix headline of choice is, build tools & package managers bridge single-player development to multitenant production — history would have me believe that any paradigm shift in this realm is worth paying attention to. + If Tobi Lütke compels you to learn about a specific technology, jump on that track.

** Niche tech reference, iykyk **

Nix enables developers to build: reproducible, declarative, and reliable software.

To the uninitiated, the above definition will likely be rather abstruse. That’s ok, I spent many-a-year building software without fully appreciating what was going on under the proverbial hood.

Alright, let’s drop our egos and go back to basics. If you’re intimately (weird!) familiar with how software is built +/ packaged and want to skip to the section on Nix, that’s cool.

** Turns upright framed picture of Chris Lattner **

** I’m going all in on niche tech references today, sorry **

Let’s begin with “building” software.

Applications are often comprised of many files. For example, my blog (thanks Deno) is comprised of a TypeScript file, a JSON file for some config, a “posts” folder that contains more “WIP” markdown files than I’d care to admit, etc.

These files, dear readers, are what’s known as “source code”.

Within this source code (the main.ts file specifically), I’m also “dependent” on a bunch of other folks’ code (often called “packages”) that I’ve installed. This collection of third-party code is known as a piece of software’s dependencies.

You: “Why stand on the shoulders of another developer?”

Alex: “Why re-invent the wheel?”

Ok ok. Pourquoi est-ce relevant?

~~Firstly, build tools take that there source code and convert + bundle it into a format that can be executed by a “target environment” (ie ultimately by a CPU). Remember, “execute” = make software do what it’s intended to do.

This conversion process is known as “compilation” — source code is compiled into machine code (also called “binary code”). You may also hear people refer to this compiled code as an “executable”.

Technical Detail: Compiling source code actually creates multiple machine code files called “object files” which are then “linked” by a.. “linker”.. which ultimately creates a single executable file.

So — source code + compilation = object files, object files + linking = single executable. More here if you wish to delve into the weeds of it all.

** Build tools fin **

Cool, we now know what’s required to build software on my computer. Astute readers, if I sent you this executable, would it “do what it’s supposed to do” on your computer?

Unlikely. Why? Well, I suspect you don’t have the “blog”, “ga”, and “redirects” packages installed. Remember, our software is dependent on them.

Even if you do, did you install the same version of those packages as I did? I doubt we’re even running the same macOS version currently, never mind some comparatively esoteric package.

Thankfully, a second utility resides within our arsenal — software “packaging” tools. The most famous (infamous?) of which is Docker.

Packaging tools.. package.. (ie bundle together) your source code alongside the specific (e.g., ga v1.2.3) dependencies it’s reliant on. Convenient.

These tools often go a step further and supply additional information that a 3rd party (e.g., a cloud provider) may need in order to correctly build your software.

** The line can get a little blurry between build / packaging tools **

I think this packaging is best illustrated via Docker’s “Dockerfile”. Below is one I use within Northflank. (to deploy a pretty neat twitter scraper 😎)

The above Dockerfile is a list of consecutive instructions (each called a “build step”) that when followed, compose an “image” of your application.

Think of images as snapshots in time of the “environment” (ie source code, file system, dependencies, etc) required to successfully build your software. These images can then be shared with others, and thus, your environment can be reproduced by third-parties.

Or as Peter Fenton rather poetically puts it — images give us a way to “program the global computer”.

** Packaging fin **

Look, Docker undoubtedly deserves the adulation it’s received. This said, it has some acute flaws that can only be solved by a fundamentally different approach to building and packaging software. Enter the old timer, Nix.

Let’s study one such flaw before we delve into Nix itself. As shown above, Dockerfiles contain a set of instructions that build an image. One of these build steps (e.g., RUN pip install..) involves downloading dependencies from an external source called “pip”.

“Fun” Fact: Pip is a recursive acronym. It stands for “Pip installs python”. Because acronyms in tech weren’t obtuse enough already!

Herein lies a problem. What if the dependency being downloaded from this external source has changed (e.g., maybe it’s been patched!) vs. the same dependency downloaded by another server/computer a week prior?

In this instance, two computers are ultimately relying on different environments to build software, despite using the same Dockerfile (!). Thus, using Dockerfiles != true reproducibility.

This is as close to a mic-drop moment as you get in building software. Take a second to internalize it.

** Disclaimer: Yes, I know you can technically use strict dependency pinning in Docker **

** This still doesn’t entirely solve the problem **

Ok, 6,917 characters later, and Nix’s “Why” is partially clear. Let’s peek under the hood.

— A big thanks to Burke Libbey who has done a bunch of the heavy lifting here! —

Nix’s core belief is that the dependencies that a piece of software relies upon should be explicitly stored within a graph database. A simplified version of my blog’s “dependency graph” would look like this:

Why? Because without Nix’s explicit graph (dubbed the “Nix Store”!), my blog may be ~unintentionally reliant on unknown dependencies that exist within my system. Given these dependencies are unknown, I wouldn’t include them in a Dockerfile, and hence, my environment wouldn’t be accurately reproduced elsewhere.

** We are running out of mics to drop **

The Nix Store is created with three building blocks: 1) Derivations, 2) Sandboxing, and 3), the Nix language.

1. Derivations: similar in principle to a Dockerfile. Derivations are files that contain build steps and are used to add a node to the Nix Store. For our “main.ts” node above, the build steps (conceptually!) would be:

• Point “ga” and “redirects” to “blog”.

• Point “blog” to a new node called “main.ts”.

• Build the “main.ts” node.

2. Sandbox: how derivations are “enforced”. Nix creates an isolated environment within your system (limiting network/file access, etc) in order to ensure that when you’re building a node, you only have access to the dependencies declared in the derivations. No more unintentional dependencies!

Technical Detail: Nix goes as far as using patched versions of compilers and linkers that don't look in default locations (/usr/lib, and so on) for dependencies. Thus creating a truly isolated environment. Pretty cool!

Nix language: is the language used to write these derivation files. Comparable (although considerably more powerful!) to the commands (“COPY” / “RUN”) used to structure a Dockerfile.

Let’s tie this all together:

If I’ve done my job correctly (?), the core tenets + components of Nix should be clear. Now, in order for someone to accurately reproduce your software, all they have to do is install the package from the Nix Packages registry. (the world's largest repo!)

I’m a selfish writer, however, so there’s a little more %magic (niche reference v3) I want to linger on.

Firstly, each node is added as a unique file path within the Nix Store. Examples:

• /nix/store/bv6znzsv2qkbcwwa251dx7n5dshz3nr3-zlib-1.2.11/lib/libz.so.1

• /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/libc.so.6

• /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-glibc-2.27/lib/ld-linux-x86-64.so.2

The hash (e.g., bv6znzsv2qkbcwwa251dx7n5dshz3nr3) is derived from the derivation file. Remember, the derivation file contains the specific build steps required to build a node.

Here’s an example derivation file. Warning — they’re rather gnarly. You don’t need to understand the contents but you do need to understand that any changes to the derivation file (perhaps an updated dependency?) will result in a different hash.

This derived hash allows developers to cross-reference with one another whether or not they’re building a given piece of software (ie a node) within the exact same environment.

Par example — two developers may both have Python 3.2.1 installed. However, if one developer’s Python node’s path in the Nix Store is:

» /nix/store/bv6znzsv2qkbcwwa251dx7n5dshz3nr3-python—3.2.1

and the other’s is:

» /nix/store/fg4yq8i8wd08xg3fy58l6q73cjy8hjr2-python-3.2.1

Then it’s pretty clear that they’re building Python differently from one another, despite having the same version installed. Smart!

** Nix — fin **

To the handful of you likely left, nice work.. and.. sorry? Hopefully, it’s clear that the explicitness + strictness of Nix allows you to create truly reproducible build environments.

I should also state that whilst comparing Docker vs. Nix is helpful for explanatory purposes, Docker is a full containerization suite. If anything, these technologies largely complement one another. Have a look into building Docker images with Nix 🫳🏻🎤

Fancy programming the global computer anyone?

As always, I’m at alex@tapestry.vc should you ever wish to chat about Nix — or perhaps grab a well-pulled pint of Guinness if ever in London!

Bonus content:

• Nix is hard to grok + harder to use. Flox is changing that.

• 2022 Nix Survey. Thank you, Ron!

• # Nix based projects (as of Oct ‘22) on GitHub:

  • 2964 repos with a default.nix

  • 2053 repos with a flake.nix (up from 1718 on 2022-08-01)

  • 3147 repos with a shell.nix

Firstly, some updates.

I was somehow allowed onto eBPF Summit’s (virtual) stage. This was a cool experience, as the very first instance of “Why Now” was a primer on eBPF. Thanks to those of you who subbed from day one! Talk’s here.

** breaks golden rule of not directing readership elsewhere **

Most of my “dev” time of late has been within the sublimely crafted walled garden that is the Godot game engine. I’m currently remaking Pokemon Yellow which has been a bunch of fun; the code+commentary can be found on my blog: mack.work/pokemon_yellow

Any residual dev time has been spent on learning Zig + on tinkering around with Manim. I’m currently animating some graphs associated with the world of venture investing which I’ll share on my Twitter.

Manim is a math animation library created by Grant Sanderson of 3Blue1Brown fame. If you haven’t frequented his YT channel yet then check out this primer on neural nets — you truly won’t be disappointed. Grant is a big source of inspiration for my technical writing.

** Something different **

Anyway, much like what you’ve come to expect from these newsletters, I am bloviating. Below I’m going to share the topics I’m thinking about tackling next; the thought here is that I won’t get to all of these areas/technologies but you may wish to explore them on your own accord.

I’d love your thoughts on them + I’m always open to suggestions. Feel free to say hello at alex[at]tapestry[dot]vc or on the thread associated with this newsletter.

• Nix: a build tool + package manager that explicitly defines an application’s dependencies via a graph database. Tobi Lutke is a fan.

• WebGPU: Remember WebGL? A next-gen web GPU API based on Vulkan, Metal, and Direct3D 12. Superb primer by Surma.dev here.

• AlphaFold: Honestly I don’t think AlphaFold’s receiving enough hype en masse given how profound of an unlock it is. + I know very little about computational biology so this could be fun/humbling.

• Nitro Enclaves: an isolated compute environment avec some cryptographic attestation. Leveraged by companies like Evervault.

• Godot: my weapon of choice for game development, not the play. If you’re keen to learn about the engine / game dev more broadly I’m happy to chat also.

• Zig: general-purpose programming language/toolchain. Learning new languages/their data structures is just fun. Bun is written in Zig, fyi.
  

Leave a comment

Thanks for reading as always + lmk what you think of these occasional updates/polls.

If you have a friend that you think may enjoy reading future primers with a sprinkle of some Irish self-disparagement, I’d appreciate you sharing this newsletter with them (few subs away from 100!) » whynowtech.substack.com

The theoretical physicist, Wolfgang Pauli, is widely attributed for the counter-intuitive critique: “it’s not even wrong”.

By this, Pauli means that ideas devoid of strong opinions are categorically worse than those ideas that are deemed to be incorrect, or perhaps even offensive, by some.

I’ve found this same sentiment to be true for developer tools — which I’m sure is exactly what Pauli was indirectly referring to.

The “Developer Community” is known for being particularly opinionated. Make an explicit product decision and you will be simultaneously worshipped and chastised for it. This was perhaps most evident in Python’s addition of the Walrus Operator, whose reception led to the benevolent dictator for life himself stepping down. Team Guido over here!

Arguably one of the most opinionated — particularly due to its lineage — developer tools of late is a runtime called Deno. This emerging runtime is important not only because I built my personal blog on top of it, but also because it (along with Cloudflare Workers) represents a new paradigm in cloud computing. Deno [Deploy] is an “Isolate Cloud”.

If the Isolate Cloud really is akin to its predecessors, virtual machines & containers, ~VMware & ~Docker, then it’s likely worth paying some attention to. Let’s see what the cloud v3.0.0 is all about.

Deno (Dee-no) is defined as a [batteries included], simple, modern, and secure runtime [based on web standards] for JavaScript, TypeScript, and WebAssembly that uses V8 and is built in Rust.

There’s a medley of terms and foundational technologies here that we’re going to need to unpack. But first, one explanation and then some history.

A “runtime” is a piece of software that helps developers create, compile and execute programs. To grossly oversimplify, execute = make a program do what it’s supposed to do.

So, let’s say I’ve written a JavaScript program called “grossly.js” that calculates 1+1 and returns the answer.

To execute this program I would open up my terminal, type “deno run grossly.js”, et voilà, my program will compile, execute and then return 2 (quick mafs).

** A brief history of JavaScript runtimes **

Initially, JavaScript was really only executed by web browsers. It was a language invented by Brendan Eich — who also founded Mozilla & Brave! — to make webpages served by Netscape Navigator “dynamic” (ie the webpage changes based on some logic such as a mouse click).

This was kind of a painintheass.js though as it meant developers had to write their frontend code in JavaScript and their backend (aka server-side) code in another language (PHP, etc). Put simply, it’s lessofapainintheass.js for developers to reason about their code if it’s written in one consistent language.

So, sure enough, server-side runtimes for JavaScript such as the OG Netscape LiveWire Pro emerged to address (and monetize!) this issue. However, these early server-side runtimes were lacking in what many developers considered to be essential features/characteristics such as:

• Standard libraries

• Non-blocking execution

• Utilizing V8 vs. JVM

Enter our protagonist, Ryan Dahl.

During JSConf 2009, Dahl shared a new server-side runtime called Node.js. It touted itself as a “purely evented, non-blocking infrastructure to script highly concurrent programs”.

Technical Detail: “Non-blocking” means that whilst some code - perhaps a database query - is waiting for a response (ie still being executed), subsequent code is executed in the meantime. Thus, your program is “concurrent” because it’s executing multiple lines of code at once!

Fast-forward to 2022 — I promised this would be brief! — and Node.js is ubiquitous. As a consumer, you likely used an application built on top of Node.js today (a la Uber). With 11m+ developers using Node.js & its concurrent comrades, it’s hard to miss.

However, on May 13, 2018 Node.js saw some developer churn in the shape of our protagonist, Ryan Dahl, who had a new PowerPoint and new idea.

** A new runtime has entered the groupchat **

Yes, the creator of the Node.js runtime is now building its ~adversary, Deno. This is about as Hollywood as developer tools get & I’m here for it. Let’s pick back up our definition, now armed with some history and new domain knowledge.

Deno (Dee-no) is defined as a [batteries included], simple, modern, and secure runtime [based on web standards] for JavaScript, TypeScript, and WebAssembly that uses V8 and is built in Rust.

Unlike my publishing cadence, I’m going to stay consistent here and keep tackling the foundational technologies before we delve into the simple, modern, and secure characteristics of Deno. Next candidate, TypeScript.

TypeScript is what’s known as a “superset” of JavaScript. This means that the language is derived from, but extends (ie does more) JavaScript.

Without getting too into the weeds here, JavaScript is a “dynamically typed” language, whereas TypeScript — thank you $MSFT! — is “statically typed”.

In programming languages, “types” are a set of categories that describe what a value in a program is (ie is it a number, a string, a boolean, etc.), and as a result, what operations can be taken on the value.

For example, it doesn’t make sense to multiply (an operation!) two values that are strings (e.g., “subscribe” * “to-my-newsletter”), but multiplying numbers does.

Let’s see what happens when we try to perform a multiplication operation on strings.

That’s a lotta “red” + “technical jargon”. What’s happening here is we’re running into what’s known as a TypeError. Our code isn’t being successfully executed, as string types can’t be multiplied by one another.

Remember, JavaScript code is “dynamically typed”. This means that developers aren’t required to explicitly declare the type of a variable (e.g., that “hello” is a string) in their code. Why? Because the types of variables in JavaScript are inferred upon execution (hence, dynamic!) by a runtime.

Whereas in TypeScript, variables lack imagination. A variable needs to be told that it should be a string. Par example:

Why go through the additional effort? Usability. Being explicit about what your code should consist of, and hence, do, ensures it’s used by you, and others, as intended.

Back to Deno.

Deno supports TypeScript “out of the box” (ie it just works). Whereas if you want to execute TypeScript with Node.js you have to install a specific software library & do a little more grease work.

** TypeScript **

Deno (Dee-no) is defined as a [batteries included], simple, modern, and secure runtime [based on web standards] for JavaScript, TypeScript, and WebAssembly that uses V8 and is built in Rust.

Ah, WebAssembly, Wasm, the ordained cool kid on the proverbial block. As I very briefly helped lay out in Luke Byrne’s canonical post, Wasm is a compilation target. What this means is, that a higher-level language (e.g., Go) gets compiled (ie turned into) a format that more closely resembles “machine code”.

** Wasm savants lay down your weapons **

Naturally, for a technology to receive this much fanfare, there’s much more to it. However, this is a rabbit hole for another day. What’s important to know is that Deno exposes utilities (e.g., a WebAssembly class) that developers can use to execute WebAssembly code.

To Node.js’ credit, their support for Wasm seems pretty good too.

** WebAssembly (kinda)(you’ll thank me) **

Deno (Dee-no) is defined as a [batteries included], simple, modern, and secure runtime [based on web standards] for JavaScript, TypeScript, and WebAssembly that uses V8 and is built in Rust.

V8, as you motor aficionados might expect, is an “engine”. You likely benefit from its labor every day when you use web browsers such as Chrome, Brave, Edge (really?), etc. It was released in 2008 with Chrome in order to compile and execute the JavaScript code present in Chrome-served webpages.

Astute readers — yes, this engine handles the “compilation” and “execution” part of our initial runtime definition.

The compilation step is easy to understand in principle; turn high-level (JavaScript, TypeScript, WebAssembly) code into optimized 1s and 0s. Whilst a crude synopsis, this is all the detail required regarding Deno.

Execution is due a little more time. Sure, you now have 1s and 0s that your computer understands, but your computer requires some additional help to ensure these 1s and 0s are understood and dealt with (ie executed) in the best way possible.

Let’s unpack execution a little further.

** Notably simplistic analogy inbound **

Imagine opening whatever book you’re currently reading on its current page.

You’ll likely read the text on the page in some order (remember, “call stack”), know who the characters mentioned are based on prior pages (remember, “memory heap”) and perhaps change your view on how the plot will pan out (remember, “garbage collection”).

How V8 “executes” machine code is ..um.. no different. Ok, again, a stretch, but stay with me here and replace your book’s “page” with a compiled JavaScript program (ie some lines of machine code).

V8 uses a “call stack” to prioritize what lines of code are executed when, a “memory heap” to store information that can be used subsequently, and a “garbage collector” to delete information that’s no longer relevant/useful.

Again, the details as they pertain to Deno aren’t super important here at a first glance, so don’t stress. All you need to know is that a program (V8) is used to compile JavaScript, TypeScript, and WebAssembly to machine code, and that this same program is then used to ensure the appropriate execution of said machine code.

We’ll return to V8 when we delve into the “Isolate Cloud”.

** V8 ..fait accompli.. **

Deno (Dee-no) is defined as a [batteries included], simple, modern, and secure runtime [based on web standards] for JavaScript, TypeScript, and WebAssembly that uses V8 and is built in Rust.

If you’ve frequented the vestibules of the WebAssembly FC (football reference not a technical acronym) then there’s no doubt you’ve stumbled into the compilation target’s comrade, Rust.

Rust, per Stack Overflow, has been the most popular programming language for 6 years straight. So, we’ll linger a little here.

Rust is a statically typed (like TypeScript!) “systems” programming language. It’s comparable to lower-level languages such as C++ or Go vs. higher-level abstractions like JavaScript or Python.

Systems languages (C, C++, Go, Rust, etc.) are generally used to write programs / utilities that will be used by developers to interact with some underlying piece of hardware (think a CPU).

For example, V8 itself is written in C++, the Mac operating system is primarily composed of C code, the Python language is written in C (ooh, meta), and the Deno runtime is.. “built in Rust”.

The big difference vs. languages such C++ (aside from its preferential ergonomics!) is that Rust is “memory safe”.

Technical Detail: “memory safe” means that information your programs store (e.g., a name in an array) can’t be improperly accessed or changed. Your stored information (ie memory) is.. safe. With C++ this isn’t the case.

To put the importance of memory safety into perspective: in 2019 Microsoft estimated that 70% (!) of all security vulnerabilities in their products over the last decade have been due to memory safety issues.

As it relates to Deno, Rust is a modern language used to build system-level utilities (such as a runtime!) due to its memory safety et al. It’s the best tool for the job — you don’t need to know anything else.

** Rust has been electrolysis’d!! ha **

Deno (Dee-no) is defined as a [batteries included], simple, modern, and secure runtime [based on web standards] for JavaScript, TypeScript, and WebAssembly that uses V8 and is built in Rust.

I salute you if you’ve made it this far. Now that we’ve laid the technical foundations, let’s dig into our rather opinionated runtime, Deno.

What makes Deno “simple” is a subjective matter. I would sum up its simplicity as a collection of characteristics:

1. Web Compatability: Deno comes packaged with existing “web standards” — ie the same APIs that the browser exposes to frontend developers. Whereas Node.js doesn’t.

2. Single Executable: You can build complex websites and contain them within a single file of JavaScript.

3. “Batteries Included”: Deno comes bundled with a number of useful utilities. Want to build a web app? Use their web framework, Fresh. Want to format your Fresh code? Use deno fmt. Want to deploy said web app? Use Deno Deploy. Want me to stop? … fair.

4. Dependency Management: Deno imports dependencies (ie other people’s code you’re using) via a URL import vs. a “package registry” like npm. No more node modules / package.json / the “require()” syntax 🥲 (those who know, know).

** simplicity + web standards + batteries included in one fell swoop **

“Modern” is easy. As mentioned already, Deno is unshackled by the technical inertia of Node.js — the runtime uses/supports new kid on the block technologies: Rust, TypeScript, and WebAssembly.

** modern **

“Secure” is deserving of a little more attention. I suspect Pauli would speculate that this is where Deno could.. be wrong.

Deno is secure by default. The runtime requires you to explicitly state what a program’s allowed to do/access.

For example, if you have a JavaScript program that wants to use Deno’s fetch API to make a request over a network, you have to explicitly state that this is allowed.

For an example of the above example: deno run --allow-net fetch.ts tells Deno to compile and execute the file “fetch.ts” and allows (--allow) the file to access the computer’s network (-net).

** secure --is scratched off the list.ts! **

Deno (Dee-no) is defined as a [batteries included], simple, modern, and secure runtime [based on web standards] for JavaScript, TypeScript, and WebAssembly that uses V8 and is built in Rust.

Scratching those definitions felt good. You did it, you know what Deno is + a host of some of the most important technologies of the modern web.

However, those of you with no garbage collection (..good memory..)(..I’m sorry..) may remember at the beginning of this primer I mentioned Deno Deploy being an “Isolate Cloud”.

** Before we move forward to cloud v3.0.0, consider sharing this primer with a friend? **

Share

In Cloud Computing Without Containers, Zack Bloom of Cloudflare fame explains that Cloudflare “Workers” (their Serverless product) don’t rely on virtual machines or containers. At the time this was unique vs. serverless competitors such as AWS Lambda. This was kinda a cloud infra equivalent of the proverbial mic drop.

Technical Detail: “Serverless” doesn’t actually mean no-servers. It’s really a business model. With serverless, developers don’t have to rent a specified amount of backend services (ie servers) from their cloud provider. Instead, they dynamically (ie automatically) rent based on their usage.

Instead, Cloudflare Workers rely on a little something called V8 (!!) Isolates — a particular ~feature of Chrome’s V8 engine. It’s this same technology that Deno Deploy uses to make application deployment an absolute breeze. It nearly feels.. wrong.

..How so?

Isolates allow disparate cloud computing users (ie “tenants”) to not only share an operating system (this is what containers enable), but to inherit the same runtime environment (yes, e.g., Deno!) too.

Why does this matter?..

Well, firstly, developers get to think about the cloud infrastructure their code runs on even less than when they use containers. With isolates, ~all they have to think about is writing their application code.

For context, managing infrastructure is analogous to “admin”, y’don’t want to do it, you just want to play rugby outside (write code).

Secondly, isolates are incredibly “lightweight”, meaning they take up very little "space” on a given server. Why? Again, because isolates solely contain application code vs. a bunch of other contexts needed when deploying to containers.

More space unlocks some rather interesting features for the users of isolate clouds. In Deno Deploy’s case, every time you deploy some code, Deno Deploy provides an entirely new server to host that code. Then, when you commit some changes to your code and deploy those changes, Deno will persist the old code on the original server and “spin up” a new server to host your updated code.

This makes “rolling back” to previous versions of your application (as well as testing) a walk in the park.

In Cloudflare’s case, more space™ means that it’s economical (among other things) for them to make copies of your application code and deploy these copies across their global mesh of servers. As a result, you have a truly global, low latency, application.

! How cool !

So yeah, when someone’s inevitably bloviating about the “future of cloud” think (but never say out loud) “more space” and you’ll likely be able to wade through the technical jargon and land on the same endpoint. Nice work!

** isolate cloud fin **

If you’ve come this far, thanks for reading! If you have a friend that you’d like to torture with some “light” runtime literature, sharing this post is sincerely appreciated.

Share

If you’re building the cloud v4.0.0 or just interested in sparring on new technologies more broadly I’d love to talk with you / grab coffee if you’re in London >> alex@tapestry.vc