Module 3 · Lesson 1 · ~25 min read

Goroutines and the Scheduler

A goroutine is a function running concurrently with the rest of the program, scheduled by the Go runtime onto a small pool of OS threads. Cheap to start (kilobytes of stack), cheap to run (cooperative scheduling). The hard part isn't starting them — it's coordinating them.

The keyword

go handleRequest(req)
go func() {
    log.Println("async work")
}()

Prefix any function call with go and it runs concurrently. The call returns immediately; the function runs on its own goroutine.

That's the entire syntax. Everything else is consequences.

Goroutines are not OS threads

An OS thread costs ~1MB of stack and a kernel context switch (~10μs). You can have a few thousand on a modern machine.

A goroutine starts with a 2KB stack (grows dynamically) and is scheduled by the Go runtime in user space. You can have hundreds of thousands. A Canton-adjacent indexer streaming transactions might routinely run tens of thousands of goroutines without strain.

The runtime maps goroutines onto OS threads via the GMP scheduler:

The scheduler distributes runnable G's across P's, and P's are bound to M's. When a goroutine blocks on I/O or syscall, the runtime parks it and runs another G on the same M. The mechanics are interesting; they're rarely something you tune. Defaults are good.

"Go boldly" — but with a plan to wait

This is the most common newcomer bug:

func main() {
    go doWork()
}  // program exits BEFORE doWork starts

main is itself a goroutine. When it returns, the program exits — the runtime does not wait for other goroutines to finish. They're killed mid-flight.

You need to coordinate. The simplest tool is sync.WaitGroup:

var wg sync.WaitGroup
for _, item := range items {
    wg.Add(1)
    go func(item Item) {
        defer wg.Done()
        process(item)
    }(item)
}
wg.Wait()  // blocks until every Done is called
Loop variable capture

Notice func(item Item) takes item as an argument. In Go versions before 1.22, the loop variable was shared across iterations; capturing it directly meant every goroutine saw the same (final) value. Go 1.22 fixed this — each iteration gets its own variable. Pass it as an argument anyway, for compatibility and clarity.

The model — communicate, don't share

Go's slogan is "do not communicate by sharing memory; share memory by communicating." Translation: instead of using locks around shared state, pass values between goroutines through channels (next lesson). The result is code that reads as a dataflow rather than a tangled lock graph.

This is an ideal, not an absolute. Real Go code uses both: channels for orchestration ("here's the next batch to process") and mutexes for protecting bounded shared state (a counter, a cache). Don't reach for a channel when a mutex is right; don't reach for a mutex when a channel is right. Lesson 3 covers when to use which.

What can go wrong

Goroutine leak

func leakyServer() {
    ch := make(chan int)
    go func() {
        heavyWork()
        ch <- 42  // blocks forever if no one reads
    }()
    // ... maybe we read from ch, maybe we don't, depending on a request path
}

If nothing ever reads from ch, the goroutine sits there forever, holding its stack, references, and any resources it acquired. Leaked goroutines kill long-running services.

Defenses: explicit cancellation via context.Context (Lesson 4), buffered channels for fire-and-forget signals, careful ownership ("who shuts this down?").

Race conditions

var counter int
for i := 0; i < 1000; i++ {
    go func() { counter++ }()  // DATA RACE
}

counter++ is read-modify-write on a shared variable across goroutines. The result is undefined — could be 1000, could be 700, could segfault under enough optimization. go test -race will catch this. Production code that touches shared mutable state without synchronization is broken, even if it appears to work.

Deadlock

ch := make(chan int)
ch <- 42            // blocks: unbuffered channel, no receiver

Unbuffered channel sends block until a receive is ready. If you never spawn the receiver, the program is stuck. The runtime detects all-goroutines-deadlocked at the program level and panics with a useful message — but partial deadlocks (3 of 50 goroutines stuck) won't trigger it.

Patterns to internalize

Bounded worker pool

func processBatch(items []Item, workers int) {
    jobs := make(chan Item)

    var wg sync.WaitGroup
    for w := 0; w < workers; w++ {
        wg.Add(1)
        go func() {
            defer wg.Done()
            for item := range jobs {
                process(item)
            }
        }()
    }

    for _, item := range items {
        jobs <- item
    }
    close(jobs)
    wg.Wait()
}

Fixed concurrency, no goroutine explosion. Workers exit cleanly when the channel is closed.

errgroup — for "wait for many goroutines, propagate the first error"

import "golang.org/x/sync/errgroup"

g, ctx := errgroup.WithContext(ctx)
for _, item := range items {
    item := item
    g.Go(func() error {
        return processWithContext(ctx, item)
    })
}
if err := g.Wait(); err != nil {
    return err
}

errgroup wraps WaitGroup with error propagation and ctx cancellation — first goroutine that errors cancels the rest. Used everywhere in production Go.

Don't go too far

Goroutines are cheap, not free. A goroutine for every line of code is bad design. Reach for them when you have:

If your code is purely CPU-bound and sequential, goroutines won't help — the GIL doesn't exist, but you still only have so many cores.

Takeaways