How I Practice Worker Pools in Go
The worker pool is one of those patterns I keep rebuilding from scratch. Not because it’s hard — it isn’t. But because every time I do it, something clicks a little more.
This is my practice routine. I write the struct, spin up the workers, feed them jobs, and shut it down cleanly. Then I do it again with a different task. By the third rep, it’s in my hands.
First, I refresh my mental model of goroutines
Before I write a single line of pool code, I remind myself what a goroutine actually is. It’s easy to skip this and end up confused later.
A goroutine is Go’s way of doing two things at the same time. When you write go doSomething(), Go starts it in the background and immediately moves on — your program doesn’t wait.
go checkURL("https://example.com") // starts in the background
go checkURL("https://google.com") // also starts, right away
// both are running at the same time nowThis is concurrency — multiple things in flight at once. That’s the foundation everything else builds on.
Then I remind myself why unbounded goroutines break things
The first version I ever wrote looked like this:
for _, url := range urls {
go checkURL(url)
}1,000 URLs → 1,000 goroutines at the exact same moment. All of them hammering the network. All of them fighting for CPU time. It felt clever until things started timing out and crashing.
Goroutines are light, but they’re not free. Spinning up too many at once gives you:
- Slow responses (everyone is competing)
- Crashed connections (network gets overwhelmed)
- A program that’s hard to reason about
You: "Everyone go do your task RIGHT NOW"
1000 workers: 🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃
🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃
🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃🏃
... 950 more ...
Door: 😰 (only fits 5 at a time)That’s the problem the worker pool solves. You want concurrency with a ceiling.
The picture I draw before I write any code
When I’m about to build a pool, I sketch this out first. It keeps me honest about what I’m actually building.
Instead of one goroutine per task, you start a fixed number of workers. They sit and wait. A task comes in, a free worker picks it up, does the work, and goes back to waiting.
┌─────────────┐
│ Job Queue │
tasks ──────────► │ [url1] │
│ [url2] │
│ [url3] │
│ [url4] │
└──────┬──────┘
│
┌────────────┼────────────┐
▼ ▼ ▼
Worker 1 Worker 2 Worker 3
(busy) (waiting) (busy)3 workers, 4 jobs in queue — only 3 run at once. The fourth waits. When a worker finishes, it grabs it.
Speed (multiple things at once) and control (never more than N). That’s the whole idea.
Channels are the queue
The job queue isn’t a slice or a list — it’s a channel. A tube: drop items in one end, pull them out the other.
┌──────────────────────────────┐
send ──► ── │ item │ item │ item │── ──► receive
└──────────────────────────────┘
channelWhat I like about channels for this: they’re safe for multiple goroutines to use at the same time. No locks, no races — Go handles the coordination.
In the pool, the main program sends jobs into the channel. Workers pull jobs out. That’s it.
Building it, one piece at a time
I always use URL checking as my practice task — it’s concrete and the I/O makes the concurrency visible. Here’s how I build it up.
Step 1: define the struct
type Pool struct {
numOfWorkers int
jobs chan string
wg sync.WaitGroup
}Three fields. I make myself name what each one does before moving on:
numOfWorkers— the ceiling. I pick this number.jobs— the channel. The shared queue between the main program and the workers.wg— the scoreboard. Goes up when a worker starts, down when it finishes. Zero means everyone’s done.
Step 2: write the constructor
func NewPool(numberOfWorkers int) *Pool {
return &Pool{
numOfWorkers: numberOfWorkers,
jobs: make(chan string, 10),
}
}make(chan string, 10) gives the channel a buffer of 10. That means you can drop 10 jobs in before anyone needs to be reading. Like a waiting room with 10 seats — once they’re full, the next arrival waits at the door.
Step 3: write the worker
This is the piece I spend the most time on, because it’s where the pattern actually lives.
func (p *Pool) worker() {
for job := range p.jobs {
result := checkURL(job)
fmt.Println(result)
}
}for job := range p.jobs blocks and waits until a job appears. When one does, it grabs it, does the work, and loops back to wait. When the channel is closed and empty, the loop ends and the worker exits.
start
│
▼
wait for job ◄──────────────────────────┐
│ │
│ job arrives │
▼ │
do the work │
│ │
▼ │
handle result ───────────────────────────┘
│
│ channel closed AND empty
▼
done, exitStep 4: start the workers
func (p *Pool) Start() {
for wId := range p.numOfWorkers {
p.wg.Go(func() {
p.worker()
})
}
}p.wg.Go(...) launches a goroutine and increments the WaitGroup in one call. I do this numOfWorkers times to get N workers all sitting and waiting.
wg.Gowas added in Go 1.25. Before that:wg.Add(1)thengo func() { defer wg.Done(); ... }(). Same idea, more ceremony.
Step 5: submit jobs
func (p *Pool) Submit(job string) {
p.jobs <- job
}The <- sends a job into the channel. If the buffer is full, Submit blocks until a worker frees up a slot — which is exactly what I want. It naturally throttles the producer.
Step 6: stop cleanly
This is the step I used to skip, and it’s the one that matters most in production.
func (p *Pool) Stop(ctx context.Context) error {
close(p.jobs)
done := make(chan struct{})
go func() {
p.wg.Wait()
close(done)
}()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("pool stop: %w", ctx.Err())
}
}Three things happen here:
close(p.jobs)— signals that no more jobs are coming. Workers drain whatever’s left in the queue, theirrangeloop ends, and they exit.p.wg.Wait()— waits for every worker to actually finish.- The
select— if workers finish before the context deadline: clean exit. If something’s hanging (a URL that never responds): we return an error instead of waiting forever.
close(jobs)
│
▼
workers finish current job
drain remaining jobs in queue
workers exit
│
▼
wg.Wait() returns (scoreboard = 0)
│
▼
done ✓The full thing — I run this to confirm it works
Once I’ve written all the pieces, I assemble them and run it. Seeing the output — 3 URLs resolving in parallel while 2 wait — is what makes it stick.
package main
import (
"context"
"fmt"
"net/http"
"sync"
"time"
)
// --- the pool ---
type Pool struct {
numOfWorkers int
jobs chan string
wg sync.WaitGroup
}
func NewPool(numberOfWorkers int) *Pool {
return &Pool{
numOfWorkers: numberOfWorkers,
jobs: make(chan string, 10),
}
}
func (p *Pool) Start() {
for range p.numOfWorkers {
p.wg.Go(func() {
p.worker()
})
}
}
func (p *Pool) Submit(job string) {
p.jobs <- job
}
func (p *Pool) Stop(ctx context.Context) error {
close(p.jobs)
done := make(chan struct{})
go func() {
p.wg.Wait()
close(done)
}()
select {
case <-done:
return nil
case <-ctx.Done():
return fmt.Errorf("pool stop: %w", ctx.Err())
}
}
func (p *Pool) worker() {
for url := range p.jobs {
status := checkURL(url)
fmt.Printf("%s → %s\n", url, status)
}
}
// --- the actual work ---
func checkURL(url string) string {
resp, err := http.Get(url)
if err != nil {
return "DOWN"
}
defer resp.Body.Close()
if resp.StatusCode == 200 {
return "UP"
}
return fmt.Sprintf("UNKNOWN (%d)", resp.StatusCode)
}
// --- main ---
func main() {
urls := []string{
"https://example.com",
"https://google.com",
"https://github.com",
"https://definitely-not-a-real-site-xyz.com",
"https://go.dev",
}
pool := NewPool(3) // 3 workers, checking 5 URLs
pool.Start()
for _, url := range urls {
pool.Submit(url)
}
ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
if err := pool.Stop(ctx); err != nil {
fmt.Printf("shutdown error: %v\n", err)
}
}3 workers, 5 URLs. You’ll see 3 results arrive in roughly the same breath, then the last 2 come after. That staggered output is the pool working.
The three mistakes I made (and still catch myself making)
1. Forgetting to close the channel
I did this in my first attempt. Never called close(p.jobs), so workers sat in their range loop forever, waiting for a job that was never coming. wg.Wait() also waited forever. The program just hung.
Rule I’ve internalised: close the channel as soon as you’re done submitting work.
2. Closing the channel while still sending to it
This one panics loudly, which is actually helpful:
panic: send on closed channelIt happens when you call close(p.jobs) while something else is still calling pool.Submit(...). I hit this when I added a ticker that kept feeding jobs while Stop was already running.
WRONG order: RIGHT order:
close(jobs) stop the ticker / feeder
submit(job) → PANIC close(jobs)
wg.Wait()Stop the thing feeding the pool first. Then close. Then wait.
3. Buffer too small
A buffer of 0 means every Submit blocks until a worker is free. You’ve just made the pool single-threaded. A buffer of 10 lets you queue up 10 jobs ahead so the submitting loop can run at full speed.
My rule of thumb: buffer ≈ one full batch of work.
When I actually reach for this
Before I use a pool, I ask myself four questions:
- Do I have a list of tasks to work through — URLs, files, records, messages?
- Do I want a fixed ceiling on how many run at once?
- Are the tasks independent — task 3 doesn’t need task 2’s result?
- Do I need a clean shutdown — finish what’s in flight, don’t abandon it?
If yes to all four, pool. If I’ve only got 5 tasks, I just launch 5 goroutines directly. The pool earns its complexity at tens, hundreds, or thousands of tasks where you actually need the control.
Further reading
- Go sync package docs
- Go channel tour — interactive, takes 10 minutes
- Concurrency in Go by Katherine Cox-Buday — chapters 3 and 4 if you want to go deep