Chris' Blog.

My occasional thoughts on software development, careers, and updates on life in general.

Angry Cloud from Commander Keen 4

A few weeks ago, there was a huge Cloudflare outage that knocked out half the internet for a while. As someone who has written a fair bit of Rust in my spare time (23KLOC according to cloc over the last few years), I couldn’t resist the urge to add some constructive thoughts to the discussion around the Rust code that was identified for the outage.

And I’m not going full Rust-Evangelism-Strike-Force here, as my pro-Swift conclusion will attest. Basically I’d just like to take this outage as an opportunity to recommend a couple tricks for writing safer Rust code.

The culprit

So, here’s the culprit according to Cloudflare’s postmortem:

pub fn fetch_features(
        &mut self,
        input: &dyn BotsInput,
        features: &mut Features,
) -> Result<(), (ErrorFlags, i32)> {
    features.checksum &= 0xffff_ffff_0000_0000;
    features.checksum |= u64::from(self.config.checksum);
    let (feature_values, _) = features
        .append_with_names(&self.config.feature_names)
        .unwrap();
    ...
}

Apparently it processes new configuration, and crashed at the unwrap because configuration with too many features was passed in.

Code Review

Keep in mind that I’m not seeing the greater context of this function, so the following may be affected by that, but here are my thoughts re the above code:

It returns a Result, with nothing for success case, and a combo of ErrorFlags and i32 for the failure case.
The presence of the &dyn for input indicates this uses dynamic dispatch, which means this isn’t intended as high-performance code. Which makes sense if this is just for loading configuration. Given that, they could have simply used anyhow’s all-purpose Result to make their lives simpler instead of this complex tuple for the error generic.
unwrap() is called. This is the big red flag, and something that should only generally be done in code that you are happy to have panic eg command line utilities, but less so for services. Swift’s equivalent is the force-unwrap operator !. When Swift was new, it was explained that the ! was chosen because it signifies danger, and stands out like a sore thumb in code reviews to encourage thorough examination. Rust’s unwrap isn’t as obvious at review time, and thus can sneak through unnoticed.
Since we’re already in a function that returns Result, it would be more idiomatic to use ? after the call to append_with_names, so that this function would hot-potato the error to the caller, instead of panicing.
If append_with_names returns an Option not a Result, ok_or(..)? would be a tidy option.

Alternative

Here I’ve changed the fetch_features function to be safer, with a couple options for how to gracefully handle this if append_with_names returns either a Result or an Option (it isn’t clear which it is from Cloudflare’s snippet, so I’ve done both). Note that I’ve also added some boilerplate around all this to keep the fetch_features code as similar as possible, but also commented out some stuff that’s less relevant.

fn main() {
    let mut fetcher = Fetcher::new();
    let mut features = Features::new();
    if let Err(e) = fetcher.fetch_features(&mut features) {
        // ... Gracefully handle the error here without panicing ...
        eprintln!("Error gracefully handled: {:#?}", e);
        return
    }
}

enum FeatureName {
    Foo,
    Bar,
}

struct Fetcher {
    feature_names: Vec<FeatureName>,
}

impl Fetcher {
    fn new() -> Self {
        Fetcher { feature_names: vec![] }
    }
    
    // This is the function Cloudflare said caused the outage:
    fn fetch_features(
        &mut self,
        // input: &dyn BotsInput,
        features: &mut Features,
    ) -> Result<(), (ErrorFlags, i32)> {
        // features.checksum &= 0xffff_ffff_0000_0000;
        // features.checksum |= u64::from(self.config.checksum);
        
        // If append_with_names returns a Result,
        // the question mark operator is safer than unwrap:
        let (feature_values, _) = features
            .append_with_names_result(&self.feature_names)?;
        
        // If append_with_names returns Option,
        // ok_or converts to a result, which forces you to be
        // explicit about what error is relevant,
        // which is then safely unwrapped using the question mark operator.
        let (feature_values, _) = features
            .append_with_names_option(&self.feature_names)
            .ok_or((ErrorFlags::AppendWithNamesFailed, -1))?;
        
        Ok(())
    }
}

#[derive(Debug)]
enum ErrorFlags {
    AppendWithNamesFailed,
    TooManyFeatures,
}

struct Features {
}

impl Features {
    fn new() -> Self {
        Features {}
    }
    
    // This is for if it returns a Result:
    fn append_with_names_result(
        &mut self,
        names: &[FeatureName],
    ) -> Result<(i32, i32), (ErrorFlags, i32)> {
        if names.len() > 200 { // Config is too big!
            Err((ErrorFlags::TooManyFeatures, -1))
        } else {
            Ok((42, 42))
        }
    }

    // This is for if it returns an Option:
    fn append_with_names_option(
        &mut self,
        names: &[FeatureName],
    ) -> Option<(i32, i32)> {
        if names.len() > 200 { // Config is too big!
            None
        } else {
            Some((42, 42))
        }
    }
}

Feel free to paste this into the Rust Playground and see if you have better suggestions :)

Suggestions

Instead of unwrap, the ? operator is a great option, particularly if you are already in a function that returns a Result, so please take advantage of such a situation.
ok_or is a great way to safely unwrap Options inside a Result function. If forces you to think about ‘what error should I return if there’s no value here?’.
Consider Swift! The exclamation point operator is a great way of drawing attention to danger in a code review, which is a fantastic piece of language ergonomics.

Summary

If anyone from Cloudflare is reading this, I hope this critique does not come across as unkind, much of my code is not amazingly bulletproof either! And kudos to Cloudflare for allowing us to see some of their code in the postmortem :)

Thanks for reading, I pinky promise this was written by a human, not AI, hope you found this useful, at least a tiny bit, God bless!

Sequoia vs Tahoe

Are you curious to know if upgrading from macOS Sequoia to Tahoe will affect compilation speeds? Everyone seems to be piling onto the anti-Tahoe bandwagon, so I thought I’d add some anecdata to the anecdotes going around.

Note that I have two identical laptops, the only difference is that one has Tahoe:

Mac                   macOS         Speed (lower is better)
---                   -----         -----
2025 M2 Air 16GB RAM  Sequoia 15.6  361.54s
2025 M2 Air 16GB RAM  Tahoe 26.1    360.88s

My core point is: Tahoe isn’t slower in my (admittedly simplistic) Rust compilation benchmark. It’s technically 0.2% faster, but that’s statistically insignificant.

To the mix, I’ve added a few other Macs I had lying around, to add some colour to the conversation:

M1 Studio Ultra   Sequoia 15.6.1  512.63s
M4 Air, 16GB RAM  Sequoia 15.6    378.13s
M2 Air, 8GB RAM   Sequoia         343.97s

Note that all macs are ‘base models’ of their generation.

Benchmark details

So, this benchmark is, as mentioned above, admittedly simple. I recently wrote a Rust tool to extract the sprites and maps from the Commander Keen episodes, and this benchmark times how long it takes to compile its 16 source files from scratch 400 times. Despite its simplicity, the two identical-hardware Mac’s scored within 0.2% of each other, so it is at least consistent.

If you’d like to repeat it:

Fresh install of macOS if possible
Install default Rust via rustup.rs
My macs were running Rustc 1.91.1
Install homebrew via brew.sh
git clone https://github.com/chrishulbert/dopefish-decoder.git
Do your best to ensure other things aren’t running in the background
make bench

Conspiracy theory!

It’s surprising that the M4 doesn’t trounce the M2’s! I wonder if Apple is actually putting M4 chips into the 2025 batch of “M2” laptops that have been updated to have 16GB RAM. Given the RAM is integrated with the CPU, maybe it was just simpler for them to put M4 chips in, rather than dust off the M2 designs, add more RAM, and restart the production line? And maybe they just didn’t bother to throttle them in some way. Maybe?

Alternatively… perhaps this was just a poor benchmark? After all, my older M2 somehow came out fastest. But the performance consistency between the two identical laptops is remarkably tight, indicating at least some level of accuracy. My M4 also has a corporate security rootkit installed too, which may slow things. Lots to think about.

Ultra

It’s unfortunate to see the M1 Ultra taking a lot longer than the others. I guess the M1 is showing its age! I can see why Apple’s rumoured to have given up on the Mac Pro: by the time the Ultra team has managed to release an Mn Ultra, the Mn+1 Max is out and faster. If I were to make any recommendations here, I’d say forget previous-gen Ultras, instead buy latest-gen Studio Max. Perhaps Ultra will become more relevant once the yearly pace of improvement in M processors slows down.

Summary

So there you have it: Benchmarking is hard. Kudos to those who arguably do it well. If nothing else though, I wouldn’t be too worried about Tahoe slowing things down, it’s a perfectly cromulent ~~word~~ operating system. Thanks for reading, I pinky promise this was written by a human, not AI, hope you found this fascinating, at least a tiny bit, God bless!

Devex

Having worked with React Native projects on and off for years now, I’ve come to appreciate that there are significant productivity and developer experience (devex) gains on the table, that tend to be derailed the moment a native library is added to the mix. But what if you could keep that productivity flowing?

Most people (somewhat rightly) think of Expo Go as the training wheels that nobody uses for serious React Native development. But you’re probably like me: the vast majority of daily work is simple Create-Read-Update-Delete (CRUD!) data manipulation. And what if, for that daily work, we didn’t need to fight with getting Xcode or Android Studio to compile, code sign, deal with cocoapods, ruby, gradle, etc etc? What if most of your team didn’t even need to install Xcode/Studio at all? I believe this strategy can be beneficial for keeping you and your team productive, and isolate all the pain of the native integration to the CI builds.

So, how to get to this point? Some thoughts:

When considering libraries, ask yourself ‘is this pure-js or native?’. For instance, when evaluating options for a feature flag library, you could use FooFlags (not a real product) or LaunchDarkly. FooFlags has a react native library that wraps native code, however LaunchDarkly is a pure-js library. You should use the one that has a pure JS library, because that gets you one step closer to being able to do your daily work in Expo Go.
Sometimes, companies release newer versions of their libraries that are pure JS. LaunchDarkly did this in the last year or two: their older library was native + JS shim, but their newer one is pure JS. In cases like these, you can upgrade to the latest pure JS one to make your life easier.
If you have an unavoidably native component, you can wrap it in a pure-JS component that shows a placeholder. If this is a part of the app that you don’t need to work on very often, this can be a great way of having your cake and eating it too: Have native components for part of the app, yet still be able to spend most of your productive workday zipping along with Expo Go.
If you have native modules, you can shim them to perform no-ops (or whatever is reasonable) when in Expo Go. I’ll demonstrate some strategies for achieving these last 2 points next:

Your native expo modules

If you’ve made your own native module, you can ‘shim’ it out in such a way that it does nothing when run in the Expo Go environment. To do so, as an example, I modify the generated modules/my-foo-module/src/MyFooModule.ts file as follows:

import { NativeModule, requireNativeModule } from 'expo';
import Constants, { ExecutionEnvironment } from "expo-constants";
import { EventSubscription } from 'expo-modules-core';
import { MyFooModuleEvents } from './MyFooModule.types';
declare class MyFooModule extends NativeModule<MyFooModuleEvents> {
    PI: number;
    getValueSync(): string;
    setValueAsync(value: string): Promise<void>;
    doSomething(): void;
}
function requireOrMock(): MyFooModule {
    if (Constants.executionEnvironment ===
        ExecutionEnvironment.StoreClient) { // Expo Go:
        return {
            // My stuff:
            PI: 3.141,
            getValueSync: function (): string { return '' },
            setValueAsync: async function (value: string): Promise<void> {},
            doSomething: function (): void {},
            // Generic expo module stuff:
            addListener: function <EventName extends keyof MyFooModuleEvents>(eventName: EventName, listener: MyFooModuleEvents[EventName]): EventSubscription {
                return { remove: function(): void {} }
            },
            removeListener: function <EventName extends keyof MyFooModuleEvents>(eventName: EventName, listener: MyFooModuleEvents[EventName]): void {},
            removeAllListeners: function (eventName: keyof MyFooModuleEvents): void {},
            emit: function <EventName extends keyof MyFooModuleEvents>(eventName: EventName, ...args: Parameters<MyFooModuleEvents[EventName]>): void {},
            listenerCount: function <EventName extends keyof MyFooModuleEvents>(eventName: EventName): number { return 0 }
        } 
    } else {
        return requireNativeModule<MyFooModule>('MyFooModule');
    }
}
export default requireOrMock();

Imported library components

In our case, we use a native library for VOIP calling. We only have one component that uses this library, so I’ve added a ‘wrapper’ component that replaces our component with a placeholder when we’re using Expo Go. The wrapper works as follows:

import Constants, { ExecutionEnvironment } from "expo-constants";
import { Text, View } from "react-native";
import { MyComponentProps } from "./MyComponent";
// This wraps a MyComponent in such a way it is not instantiated for Expo Go.
export default function MyComponentWrapper(props: MyComponentProps) {
    if (Constants.executionEnvironment === ExecutionEnvironment.StoreClient) { // Expo Go:
        return (
            <View style=>
                <Text>This is disabled while using Expo Go</Text>
            </View>
        );
    } else { // Production:
        const { default: MyComponent } = require('./MyComponent'); // Lazy import.
        return <MyComponent {...props} ></MyComponent>
    }
}

This wrapper has the same props as the actual component, thus everywhere our component is used, this wrapper component is to be simply used instead.

For this to work, you have to edit MyComponent.tsx and export its props:

export interface MyComponentProps { ...

Summary

Hope you find this helpful! I strongly recommend using Expo Go for the sake of your team’s productivity if possible, and with the above tips, I think it is reasonably achievable. Thanks for reading, I pinky promise this was written by a human, not AI, hope you found this fascinating, at least a tiny bit, God bless!

You can see older posts in the right panel, under 'archive'.

Chris' Blog.

Cloudflare Rust Analysis 5 Dec, 2025

The culprit

Code Review

Alternative

Suggestions

Summary

Rust Compilation: Sequoia vs Tahoe 4 Dec, 2025

Benchmark details

Conspiracy theory!

Ultra

Summary

Better React Native devex through Expo Go 5 Sep, 2025

Your native expo modules

Imported library components

Summary

Archive