Skip to main content
  1. Posts/

GOMAXPROCS Demystified

·1953 words·10 mins·
Go Kubernetes Containers Scheduler Performance
Dave Amit
Author
Dave Amit
Principal Architect with nearly two decades designing distributed systems and cloud-native platforms. I bring deep systems expertise, hands-on AI-assisted engineering workflows, and a track record of shipping things that actually scale. Currently writing Go and Rust, running on Kubernetes, and exploring what happens when strong architectural judgment meets modern AI tooling.
Table of Contents
Go in Containers - This article is part of a series.
Part 1: This Article

GOMAXPROCS sets the number of logical processors the Go scheduler uses. Before Go 1.25, it was computed from the host’s logical CPU count and the process’s CPU affinity mask; cgroup CPU quotas were ignored. A Kubernetes Pod limited to 500mCPU on a 64-core node started with runtime.GOMAXPROCS(-1) == 64, spawned 64 Ps, and spent its CPU allowance quickly enough to be throttled every scheduling period. Latency rose, GC pressure worsened, and this was the default behavior.

Go 1.25 makes the runtime cgroup-aware: it reads CPU quota from cgroup v1 or v2, rounds fractional limits up, refuses to auto-set GOMAXPROCS below 2 unless the host or CPU affinity has only one CPU, and can update the value when limits or affinity change. The new default is similar to Uber’s go.uber.org/automaxprocs, but the two differ on rounding, the minimum, and what counts as an explicit override. A bigger concern is that a lot of pre-1.25 code now silently opts out of the new behavior.

This post covers what GOMAXPROCS actually controls, how the Go 1.25 default works, and why runtime.GOMAXPROCS(runtime.NumCPU()) is no longer the harmless snippet it used to be.

Prefer video? Watch the full breakdown on YouTube.

What GOMAXPROCS Actually Controls
#

GOMAXPROCS sets the number of logical processors (Ps) the Go scheduler maintains. Each P can run a goroutine on an OS thread (M). In plain terms, GOMAXPROCS is the ceiling on how many goroutines execute simultaneously.

That matters for:

  • CPU-bound parallelism.
  • Scheduler overhead, per-P caches, and lock contention.
  • GC worker count: the GC targets roughly 25% CPU utilization during concurrent phases using a combination of dedicated workers (approximately GOMAXPROCS/4) and idle workers on free Ps.

It does not limit:

  • Goroutines blocked in syscalls or I/O.
  • Threads created by cgo.

So GOMAXPROCS=2 with a thousand goroutines waiting in epoll is fine; the OS threads that actually run code are capped at two.

Why the Pre-1.25 Default Breaks in Containers
#

Before Go 1.25, if the GOMAXPROCS environment variable was not set, the default was computed as:

min(host logical CPUs, CPU affinity logical CPUs)

Cgroup CPU quota was invisible to the runtime. A 2-CPU container on a 64-core node got GOMAXPROCS=64. The damage came in four flavors.

1. Cgroup throttling
#

With 64 Ps and a 2-CPU quota, goroutines burn their 200ms of CPU time per 100ms period quickly, then the kernel throttles every thread in the cgroup until the next period. Tail latency explodes.

2. GC amplification
#

The GC targets roughly 25% CPU utilization using dedicated workers plus idle workers on free Ps. At 64 that is 16 dedicated threads, plus idle workers on any free P, so the GC can briefly consume far more than the quota, triggering throttling even when user code is under budget.

3. Scheduler overhead
#

64 Ps means 64 per-P caches, more coordination, more context switches, all for parallelism that the cgroup will not allow.

4. Mutex holder descheduling
#

Many runnable threads, few real CPUs: the kernel may deschedule a goroutine that holds a lock. Every waiter stalls until it gets rescheduled.

Uber’s automaxprocs README published a classic table from a real service in a 2-CPU quota container. Your numbers will vary by workload, but the shape is consistent — matching the quota gave the best throughput without catastrophic tail latency. Treat it as a directional example, not a universal benchmark:

GOMAXPROCS RPS P50 (ms) P99.9 (ms)
1 28,893 1.46 19.70
2 44,715 0.84 26.38
3 44,213 0.66 30.07
8 33,112 0.43 64.32
24 (host default) 22,191 0.45 76.19

That table is why automaxprocs became standard equipment in Kubernetes deployments.

Go 1.25’s Cgroup-Aware Default
#

Go 1.25 changes the default to:

default = min(
  host logical CPUs,
  CPU affinity logical CPUs,
  cgroup CPU quota / period (if limited)
)

default = max(default, 2)   # unless host or affinity has only 1 CPU

The runtime now respects the actual CPU ceiling available to the container. Several details are worth understanding.

Fractional limits round up
#

A 500mCPU limit gives ceil(0.5) = 1, then max(1, 2) = 2. A 1500mCPU limit gives 2. A 2500mCPU limit gives 3.

CPU Limit Quota / Period Rounded Final GOMAXPROCS
250mCPU 25000 / 100000 ceil(0.25) = 1 2
500mCPU 50000 / 100000 ceil(0.5) = 1 2
1500mCPU 150000 / 100000 ceil(1.5) = 2 2
2500mCPU 250000 / 100000 ceil(2.5) = 3 3
8 CPU 800000 / 100000 ceil(8) = 8 8
unlimited max / 100000 no limit host / affinity

This differs from Uber’s automaxprocs, which defaults to rounding down (floor) unless you override the rounding with its RoundQuotaFunc option. Go 1.25 prefers utilization and avoids under-provisioning. Neither choice is universally right; what matters is that they are different.

Minimum of 2
#

Go 1.25 refuses to auto-set GOMAXPROCS=1 unless the host or affinity really has one CPU. A single-core machine or container can still yield GOMAXPROCS=1; the floor of 2 applies only to the cgroup-quota path. GOMAXPROCS=1 serializes the scheduler: GC workers and user goroutines fight over a single P, producing visible pauses. Sub-1-CPU quotas usually belong to bursty workloads, and two Ps let the runtime itself burst. If you genuinely need GOMAXPROCS=1 for a tiny latency-sensitive service, set it explicitly and measure.

Cgroup v1 vs. v2
#

The runtime walks /proc/self/cgroup and /proc/self/mountinfo to find the CPU controller and reads the leaf cgroup:

cgroup v1 cgroup v2
Limit files cpu.cfs_quota_us, cpu.cfs_period_us cpu.max
Format <value_us> (-1 = unlimited) <quota> <period> (max = unlimited)
Mount type cgroup with cpu super-option cgroup2

Mixed v1/v2 setups are supported; if a v1 CPU controller exists, it takes precedence. cpuset.cpus is not read as a cgroup file directly; it is observed through sched_getaffinity(2), which the runtime already uses.

CPU requests are ignored
#

In Kubernetes terms, resources.limits.cpu becomes the cgroup quota and affects GOMAXPROCS. resources.requests.cpu becomes cpu.shares (cgroup v1) or cpu.weight (cgroup v2); those are relative priorities, not a hard ceiling, so the runtime explicitly ignores them. A Pod with no CPU limit will still default to the host’s CPU count.

Periodic updates
#

GODEBUG=updatemaxprocs=1, the default for language version ≥1.25, enables sysmon to rescan logical CPU count, affinity, and cgroup quota about once a second, or less frequently when idle, and adjust GOMAXPROCS accordingly. This is useful for:

  • Kubernetes in-place vertical scaling.
  • Hot CPU pinning/unpinning.
  • Live container runtime limit changes.

Updates stop if you set the GOMAXPROCS environment variable or call runtime.GOMAXPROCS(n). Calling runtime.GOMAXPROCS(n) with n equal to the current value still counts as an explicit override and disables auto-updates. They can be re-enabled by calling runtime.SetDefaultGOMAXPROCS().

Precedence Rules: What Wins?
#

The hierarchy, from strongest to weakest:

  1. $GOMAXPROCS environment variable — if set to a positive integer, always wins. Also disables auto-updates.
  2. runtime.GOMAXPROCS(n) with n > 0 — explicit code call wins. Also disables auto-updates.
  3. runtime.SetDefaultGOMAXPROCS() — re-enables default computation from current host/affinity/cgroup state. It causes the runtime to ignore the $GOMAXPROCS environment variable and any prior explicit runtime.GOMAXPROCS(n) call, so auto-updates resume. If code later calls runtime.GOMAXPROCS(n) again, that explicit value wins and disables updates once more.
  4. Default computation — used when nothing above has happened.

The migration trap is real: any code that calls runtime.GOMAXPROCS(runtime.NumCPU()) at startup disables the new behavior, even on Go 1.25. That pattern used to be a harmless “make sure we use all cores” ritual. Now it is an opt-out. Worse, some web frameworks, test runners, and libraries do it for you. Audit your dependency tree.

GODEBUG can also flip behavior regardless of code. These flags still apply when the runtime computes the default, including inside runtime.SetDefaultGOMAXPROCS():

  • GODEBUG=containermaxprocs=0 — ignore cgroup limits, behave like Go ≤1.24.
  • GODEBUG=containermaxprocs=1 — consider cgroup limits. Default for language version ≥1.25.
  • GODEBUG=updatemaxprocs=0 — do not periodically update.
  • GODEBUG=updatemaxprocs=1 — periodically update. Default for language version ≥1.25.

The key phrase is language version, not toolchain version. A binary built with Go 1.25 but go 1.24 in go.mod gets the old defaults unless GODEBUG is set explicitly.

Fractional CPUs and Scheduler Trade-offs
#

Cgroup quota is an average throughput limit over a period, not a cap on instantaneous parallelism. A 2-CPU quota / 100ms period means 200ms of CPU time available in 100ms of wall time. You could run two threads at full throttle for the full period, or four threads at full throttle for half the period and idle the rest.

GOMAXPROCS caps instantaneous parallelism. Matching it to the quota average:

  • reduces kernel throttling;
  • may improve tail latency;
  • sacrifices the ability to burst within a period.

Go 1.25 rounds up, favoring utilization. Automaxprocs rounds down by default, favoring quota headroom. For a bursty latency-sensitive service, either default might be wrong; the only honest answer is to benchmark.

When to Override, and When to Leave It Alone
#

Leave it alone if:
#

  • You run on bare metal or a VM where the process owns the CPUs.
  • You run in a container with CPU limits, on Go 1.25+, with go 1.25 or later in go.mod.
  • Your workload is CPU-bound and you want the runtime to match available parallelism.

Set it explicitly if:
#

  • You have a bursty workload that benefits from idle headroom within a quota period and you can tolerate occasional throttling.
  • You run a latency-sensitive RPC in a tiny quota (e.g., 500mCPU) and measurements show GOMAXPROCS=1 actually wins.
  • You are stuck on Go <1.25: use go.uber.org/automaxprocs or compute from /sys/fs/cgroup/... yourself.
  • You want to reserve CPU headroom for a sidecar.
  • You need deterministic parallelism on a shared CI runner.

Avoid:
#

  • runtime.GOMAXPROCS(runtime.NumCPU()) in main() on Go 1.25+. It is a regression now.
  • Libraries that set GOMAXPROCS without your knowledge.
  • Base container images that set a one-size-fits-all GOMAXPROCS value.

Debugging What the Runtime Picked
#

Read the current value:

fmt.Println(runtime.GOMAXPROCS(-1))

Read cgroup limits directly:

# cgroup v2
cat /sys/fs/cgroup/cpu.max

# cgroup v1
# Exact paths depend on the container runtime; these are typical paths.
cat /sys/fs/cgroup/cpu/cpu.cfs_quota_us
cat /sys/fs/cgroup/cpu/cpu.cfs_period_us

Force the new behavior explicitly:

GODEBUG=containermaxprocs=1,updatemaxprocs=1 ./myapp

If code has already disabled auto-updates and you want them back:

runtime.SetDefaultGOMAXPROCS()

This ignores the environment variable and recomputes from the current host/affinity/cgroup state, still honoring GODEBUG flags.

Kubernetes-Specific Notes
#

  • resources.limits.cpu → cgroup quota → GOMAXPROCS.
  • resources.requests.cpu → shares/weights → ignored by the runtime.
  • CPU Manager with the static policy sets cpuset.cpus, which the runtime sees through affinity.
  • Workloads without CPU limits behave the same as before Go 1.25.
  • In-place vertical scaling pairs naturally with updatemaxprocs.

Opinionated Takeaway
#

For Go 1.25+, running in a container, with a CPU limit: stop touching GOMAXPROCS. Remove automaxprocs unless you specifically need its round-down behavior, its Min option, or its logging hook (its real options are Logger, Min, and RoundQuotaFunc). More importantly, hunt down every runtime.GOMAXPROCS(runtime.NumCPU()) call and any framework that sets it behind your back. That single line is the biggest remaining regression source.

My recommendation:

  • Go 1.25+, container, CPU limit set: do nothing. Make sure go.mod says go 1.25 or later.
  • Go 1.24 or earlier: use automaxprocs, but audit for anything that overrides it.
  • Bursty latency-sensitive services: measure before overriding. Theory loses to a wrk2 run every time.
  • No CPU limit: this whole discussion does not apply; GOMAXPROCS still follows the host.

Go 1.25 replaces “mostly works by accident” with “correct by default.” There will be surprises during migration, especially for fractional-CPU services that assumed automaxprocs-style round-down behavior. But a 500mCPU Pod owning 64 Ps was never healthy. This change finally makes the runtime honest about the world it lives in.


Next in the Go in Containers series: Garbage Collection Under Pressure — memory limits, GOMEMLIMIT, and the OOMKill cycle. Follow along if you are into this stuff.


Go in Containers series: Intro — A Silent Crisis · 1. GOMAXPROCS Demystified · 2. GC Under Pressure · 3. GC Pauses and Latency · 4. The Container Debugging Workflow

Go in Containers - This article is part of a series.
Part 1: This Article

Related

Your Primary Key Is a Bottleneck (And Scaling Won't Fix It)
·3160 words·15 mins
Postgresql Performance B-Tree Uuid Database-Design
GitHub Issues as an AI Agent Task Manager
·2087 words·10 mins
Ai Go Automation Github Devops
Stop the Plumbing Hell: Building `bs`
·486 words·3 mins
Go Devops Build-Systems Bs Engineering
Go's Container Problem: A Silent Crisis
·706 words·4 mins
Go Kubernetes Runtime SRE
Designing a Modular Monolith in Go: Structure, Boundaries, and Practical Patterns
·2209 words·11 mins
Go Architecture Golang Modular-Monolith Architecture