Go Concurrency Limits: Prevent Goroutine Explosions with Backpressure

In this post we will review how to limit concurrent work in GO, why a semaphore can still leave us with too many goroutines, and what should happen when the application cannot accept more work.

Goroutines make it easy to run operations concurrently. However starting a goroutine is not the same as having enough capacity to complete its work. Under sustained load, this difference can cause high memory usage, timeouts, and restarting pods.

Let's start with a simple example.

The Goroutine Explosion Problem

Assume our application receives 100 requests per second. Each request starts 10 independent checks, and each check takes two seconds.

At a steady rate, we can have approximately 2,000 checks in flight. If the dependency slows down and each check takes ten seconds, the number grows to approximately 10,000, assuming we keep accepting work at the same rate.

Each operation can retain more than its goroutine stack: request data, parsed objects, buffers, and references to other application state. The dependency might also be unable to handle so many concurrent calls.

This creates a feedback loop. A slower dependency leaves more operations in flight. The additional work increases pressure, which can make the dependency even slower.

A concurrency limit puts a boundary on this growth. However the location of the boundary matters.

The Semaphore Mistake

A buffered channel can act as a semaphore. Sending a value acquires a slot; receiving a value releases it.

A tempting implementation looks like this:

slots := make(chan struct{}, 10)

for _, item := range items {
    go func(item int) {
        slots <- struct{}{}
        defer func() { <-slots }()

        process(item)
    }(item)
}

This limits how many process calls can run concurrently. However it does not limit how many goroutines we create.

If items contains one million entries, the loop can still create one million goroutines. Most of them will wait for a slot while retaining their arguments and other reachable data.

We have limited the active work, but moved the backlog into waiting goroutines.

Acquire Before Starting the Goroutine

For a batch with one producer, the simple solution is to acquire the slot before starting the goroutine. The producer waits when all slots are occupied.

The following complete example uses only the standard library. Save it as main.go:

package main

import (
    "flag"
    "fmt"
    "sync"
    "time"
)

func main() {
    concurrency := flag.Int("concurrency", 10, "maximum concurrent tasks")
    count := flag.Int("count", 100, "number of tasks")
    flag.Parse()

    if *concurrency <= 0 || *count < 0 {
        fmt.Println("concurrency must be positive and count must not be negative")
        return
    }

    slots := make(chan struct{}, *concurrency)
    var workers sync.WaitGroup

    for item := 0; item < *count; item++ {
        slots <- struct{}{}
        workers.Add(1)

        go func(item int) {
            defer workers.Done()
            defer func() { <-slots }()

            process(item)
        }(item)
    }

    workers.Wait()
}

func process(item int) {
    time.Sleep(100 * time.Millisecond)
    fmt.Printf("processed item %d\n", item)
}

Run it using:

go run main.go -concurrency 10 -count 100

The sleep simulates a slow operation. In a real application, process could call an API or handle a database record.

The producer cannot keep creating task goroutines while all slots are occupied. At most ten process calls run concurrently with the default configuration, and the main goroutine waits before scheduling more work.

The deferred receive releases the slot when the worker returns. The WaitGroup makes the program wait for the scheduled workers before exiting. It does not limit concurrency by itself.

This example illustrates admission and cleanup. A production implementation also needs task error handling and cancellation. Completion order is not guaranteed.

What Backpressure Actually Means

Backpressure means that a downstream capacity limit slows or stops upstream production.

In the batch example, the producer waits for a slot. If it reads records incrementally from a database, it can stop reading ahead while the workers are busy.

However acquiring a slot before starting a goroutine does not automatically bound an entire HTTP service. The HTTP server already runs handlers concurrently. Thousands of handlers can wait for the same semaphore while retaining their request data.

For a service, we need an admission policy as well as an execution limit.

Bound Active Work and Waiting Work

There are two separate limits:

• Active work: how many operations may execute concurrently.
• Waiting work: how many accepted operations may wait to execute.

A worker pool with 20 workers and a queue of 100 jobs bounds those two parts of the system. However if producers block indefinitely while sending to the full queue, the backlog can still accumulate outside it.

When capacity is exhausted, choose an explicit behavior:

• Wait for a bounded duration, provided the waiting callers are also bounded.
• Reject the operation and let the caller decide whether to retry.
• Save it to a durable queue for later processing.

For HTTP APIs, temporary service overload often maps to HTTP 503. A caller exceeding its rate allowance often maps to HTTP 429. The exact response should follow the API contract.

Do not silently drop work or report success for an operation that was never accepted.

Scope the Limit Correctly

A semaphore created separately for every request only limits each request. One hundred requests with ten slots each can still run 1,000 operations.

If the goal is protecting a shared dependency, the limiter should usually be shared across the code paths calling that dependency within the process.

Also notice that a process limit is not a cluster limit. Ten pods with a limit of 20 can issue approximately 200 concurrent calls. Scaling out can increase pressure on the dependency unless we account for the total.

Different resources may need different limits. Database writes and external model calls do not necessarily consume the same capacity.

Concurrency Limits Are Not Rate Limits

A concurrency limit controls how many operations are in flight. A rate limit controls how many operations may start during a period.

Twenty slots can permit thousands of fast operations per second. The same twenty slots might complete only a few operations per second when the dependency is slow.

If an external API allows 100 requests per second, a semaphore alone does not enforce that allowance. We may need both a rate limiter and a concurrency limiter.

Cancellation and Nested Work

A production task should not wait for a slot after its request has been canceled. Waiting for admission should observe the request context, and the operation itself should use cancellation and appropriate timeouts.

Canceling a context does not forcibly stop a goroutine. The code must observe it, and blocking dependencies must support cancellation or have their own timeouts.

Another subtle problem is nested parallelism. Suppose every parent task holds a slot, starts child tasks using the same limiter, and waits for them. If all slots belong to parents, the children cannot start and the parents cannot finish.

Avoid this circular wait. Depending on the design, run child work inline when another slot is unavailable, or schedule the work so a parent does not hold a scarce slot while waiting for children that need it.

Choosing the Limit

There is no concurrency value that works for every application.

Start with the resource we want to protect: CPU, memory, database connections, or the capacity of an external service. For CPU-heavy work, available CPU capacity matters. For operations that mostly wait on a network, a higher concurrency may help until another resource becomes the bottleneck.

A useful starting estimate is:

required concurrency ≈ target operations per second × average operation duration in seconds

For 100 operations per second taking 0.2 seconds each, the estimate is 20 operations in flight. This is a steady-state average, not a safe production setting by itself. Latency variation and bursts still matter.

Measure active tasks, queue depth, admission wait time, rejected tasks, task duration, and memory usage. Increasing a limit is useful only while throughput improves without unacceptable latency, errors, or resource consumption.

Verify the Overload Behavior

A useful check submits more work than the configured capacity and holds the active tasks behind a synchronization barrier. Count the active operations and verify that the maximum never exceeds the limit.

Then fill the waiting capacity and verify the defined behavior for the next operation. Check cancellation while waiting, slot release after a task error, and shutdown with work still outstanding.

Also test with a slow dependency. That is when an apparently harmless backlog often becomes visible.

Final Note

Goroutines are inexpensive, but the work they retain is not free.

Limit active execution, bound waiting work, and decide what happens when both capacities are full. With these boundaries, a traffic spike becomes controlled waiting or an explicit rejection instead of uncontrolled growth inside the application.