Next.js and caching - is anything really fast?

Some devs at $JOB have latched on to Next.js as a useful tool for bridging that client-side code to server gap gracefully. And it does seem to do that fairly well! You get a nice build step for static assets and a reasonable API server in one package? Sounds great. Let’s do it.

Ah. But maybe there’s a problem? (I mean…of course there is. You don’t write blg posts about things that worked right the first time…)

So Next.js works fine…for p50. And even mostly for p90. But when we’re talking p99, we’ve seen outliers at 10+ seconds for a single request. This is…not tenable as a site grows. Chewing through p99 means surprise alerts, draining SLOs, and surprise latency for users. Time to dig in. See what we can do.

Initially, we were at a bit of an impasse. Devs were asking for a CDN to address the performance issues, and my co-workers and I were hoping that the problem was purely with mobile connections (the majority of traces we found that were “bad” p99 were from mobile browsers). Our first attempts at improving this performance were focused on improving basic server performance (increase mem/cpu used). Adding vertical scaling wasn’t helping, and horizontal scaling really only masks the problem without solving it, so what can we actually do?

Obviously a CDN is on the table, but that’s expensive, (relatively) complicated to add to our deployment pipeline, and makes reasoning about our application harder. Hang on, though. This reminds me of PHP. Is there a way to put a proxy in front of the Next.js app to handle static assets?

So okay…static assets proxy. We’re basically gonna do a CDN in front of each individual instance of the Next app instead of a single shared CDN. That…might work pretty well! And putting nginx/apache in front of services is easy. We’ve all done that at some point, right?

After some basic experimentation, we landed on Caddy for our proxy as it provided better metrics out of the box. There were two major draws for Caddy over nginx:

  1. we could get latency stats for free, where nginx required parsing the logs to get request latency.
  2. no manual management of mime types
    • This is a bit tricky. Obviously the mime-type files exist, but they don’t exist on consistent paths across Linux/OS X, so dev work got quickly painful if we tried to leverage the existing mime type files.

And as a nice bonus, both nginx and Caddy leverage sendfile(), so we really can’t get much faster, can we?

The real challenge here is we can’t easily compare apples to oranges. The p99 measurements we were getting before no longer exist because Caddy is handling them now instead of Next.js. But anecdotally, we’re now seeing significantly better performance for users on first page load.

As an additional improvement, we switched a number of images from png to webp (just using the webp cli). This generally shrank the images (so less network traversal) and retained the same image quality, so it was worth the extra effort to get a bit more speed.

The moral? Next.js is…really not very fast as a server. And while it provides a lot of useful functions, it struggles to perform them well in more resource-constrained environments.

And lastly, a bit of actual code. Here’s a Caddyfile that forwards to Next.js:

# Global config options
# https://caddyserver.com/docs/caddyfile/options
{
    # disable auto-cert generation
    auto_https off
    # turn off admin API endpoint
    admin off
    # enabled metics
    # https://caddyserver.com/docs/metrics
    servers {
        metrics
    }
    log {
        level warn
        format json
    }
}

# Internal endpoint for serving healthchecks etc.
http://:8081 {
    # https://caddyserver.com/docs/caddyfile/directives/respond
    respond /healthz 200
    # no log output for internal healthchecks
    log {
        output discard
    }
    # expose metrics endpoint
    # https://caddyserver.com/docs/caddyfile/directives/metrics
    metrics /metrics
}

# actual site
# because of how Next.js builds its static files, we actually
# end up with (at least) three unique paths we need to look for
# static files, plus a websocket endpoint we need to properly handle
http://:8080 {
    # first, let's handle static assets (js/css/etc.)
    # all these are compiled under `/app/.next/static`
    # so we capture the path `/_next/static/(.*)` and attempt to
    # load a file at `/app/.next/static/$1` (where $1 is our capture group above).
    # If that file doesn't exist, we need to pass through to the actual node app
    handle_path /_next/static/* {
        root * /app/.next/static
        @notStatic not file {path}
        reverse_proxy @notStatic http://localhost:3000
        header X-Real-IP {remote_host}
        header X-Forwarded-For {remote_host}
        file_server
    }

    # Next, the favicon.
    # Because this file is actually _generated_ on first call
    # (at least, that appears to be what Next does)
    # so we need to (similar to our static assets)
    # look for favicon.ico (which lives at /app/app/favicon.ico)
    # and if it doesn't exist, pass the request through to Next
    handle /favicon.ico {
        root * /app/app
        @notStatic not file {path}
        reverse_proxy @notStatic http://localhost:3000
        header X-Real-IP {remote_host}
        file_server
    }

    # Our websocket
    # Really, we just need to add the additional needed headers
    # that web sockets require for this path
    handle /_next/webpack-hmr {
        reverse_proxy http://localhost:3000
        header X-Real-IP {remote_host}
        header Connection *Upgrade*
        header Upgrade websocket
    }

    # default to fallthrough to node server
    # There are also static assets (images/etc.) that live
    # under /app/public, so we'll look for them
    # in our default fallthrough
    handle {
        root * /app/public
        @notStatic not file {path}
        reverse_proxy @notStatic http://localhost:3000
        header X-Real-IP {remote_host}
        file_server
    }

    # default compression setup is solid, so don't worry about configuring mime types
    # and using zstd is faster than gzip, so if it's supported...great!
    encode zstd gzip

    header {
        # disable FLoC tracking
        Permissions-Policy interest-cohort=()

        # enable HSTS
        Strict-Transport-Security max-age=31536000;

        # disable clients from sniffing the media type
        X-Content-Type-Options nosniff

        # clickjacking protection
        X-Frame-Options SAMEORIGIN
    }
}