Fan Out Fan In Pattern in Go: Explanation and Example

The fan out fan in pattern in Go is a way to run multiple goroutines concurrently to process tasks (fan out) and then collect their results into a single channel (fan in). It helps efficiently handle parallel work and combine outputs using Go's channels and goroutines.

How It Works

Imagine you have a big job that can be split into smaller tasks. Instead of doing them one by one, you ask several friends to work on different parts at the same time. This is the fan out part, where you start many goroutines to do tasks concurrently.

Once all your friends finish their parts, they send their results back to you. You collect all these results into one place. This is the fan in part, where multiple goroutines send their outputs into a single channel to be processed or combined.

In Go, this pattern uses channels to send tasks out and gather results back, making your program faster and more efficient by using concurrency.

Example

This example shows how to fan out work to multiple goroutines and fan in their results into one channel.

go

package main

import (
	"fmt"
	"time"
)

func worker(id int, jobs <-chan int, results chan<- int) {
	for j := range jobs {
		fmt.Printf("Worker %d started job %d\n", id, j)
		time.Sleep(time.Second) // simulate work
		fmt.Printf("Worker %d finished job %d\n", id, j)
		results <- j * 2
	}
}

func main() {
	jobs := make(chan int, 5)
	results := make(chan int, 5)

	// Fan out: start 3 workers
	for w := 1; w <= 3; w++ {
		go worker(w, jobs, results)
	}

	// Send 5 jobs
	for j := 1; j <= 5; j++ {
		jobs <- j
	}
	close(jobs)

	// Fan in: collect results
	for a := 1; a <= 5; a++ {
		result := <-results
		fmt.Println("Result received:", result)
	}
}

Output

Worker 1 started job 1
Worker 2 started job 2
Worker 3 started job 3
Worker 1 finished job 1
Worker 1 started job 4
Worker 2 finished job 2
Worker 2 started job 5
Worker 3 finished job 3
Worker 1 finished job 4
Worker 2 finished job 5
Result received: 2
Result received: 4
Result received: 6
Result received: 8
Result received: 10

When to Use

Use the fan out fan in pattern when you have many independent tasks that can run at the same time to speed up processing. For example:

• Processing multiple files or data streams concurrently.
• Making many network requests in parallel.
• Running CPU-bound tasks on multiple cores.

This pattern helps improve performance by using Go's lightweight goroutines and channels to manage concurrency cleanly.

Key Points

• Fan out means starting multiple goroutines to do work concurrently.
• Fan in means collecting results from many goroutines into one channel.
• This pattern uses channels to safely communicate between goroutines.
• It improves performance by parallelizing tasks.
• It is useful for IO-bound and CPU-bound concurrent workloads.

Key Takeaways

Fan out fan in uses multiple goroutines to run tasks concurrently and gather results via channels.

It helps speed up programs by parallelizing independent work.

Channels coordinate communication safely between goroutines.

Use it for tasks like parallel processing, network calls, or CPU-heavy jobs.

This pattern is a core concurrency technique in Go.