How to Remove Duplicates from Slice in Go: Simple Guide

To remove duplicates from a slice in Go, use a map to track seen elements and build a new slice with only unique values. Iterate over the original slice, check if the element is in the map, and append it to the new slice if not.

Syntax

Use a map[type]bool to remember which elements you have seen. Loop through the original slice, check if the element is in the map, and if not, add it to the map and the new slice.

This pattern works for slices of any comparable type like int, string, etc.

go

seen := make(map[int]bool)
var result []int
for _, v := range original {
    if !seen[v] {
        seen[v] = true
        result = append(result, v)
    }
}

Example

This example shows how to remove duplicates from a slice of integers. It prints the original slice and the new slice without duplicates.

go

package main

import "fmt"

func removeDuplicates(slice []int) []int {
    seen := make(map[int]bool)
    var result []int
    for _, v := range slice {
        if !seen[v] {
            seen[v] = true
            result = append(result, v)
        }
    }
    return result
}

func main() {
    nums := []int{1, 2, 2, 3, 4, 4, 5}
    fmt.Println("Original slice:", nums)
    unique := removeDuplicates(nums)
    fmt.Println("Without duplicates:", unique)
}

Output

Original slice: [1 2 2 3 4 4 5]
Without duplicates: [1 2 3 4 5]

Common Pitfalls

One common mistake is trying to remove duplicates by modifying the slice while iterating over it, which can cause unexpected behavior or skipped elements.

Another is using a slice of non-comparable types (like slices or maps) as keys in the map, which is not allowed in Go.

Always use a map with keys of a comparable type and build a new slice instead of modifying the original during iteration.

go

package main

import "fmt"

func wrongRemoveDuplicates(slice []int) []int {
    for i := 0; i < len(slice); i++ {
        for j := i + 1; j < len(slice); j++ {
            if slice[i] == slice[j] {
                // Removing element by slicing
                slice = append(slice[:j], slice[j+1:]...)
                j-- // adjust index after removal
            }
        }
    }
    return slice
}

func main() {
    nums := []int{1, 2, 2, 3, 4, 4, 5}
    fmt.Println("Wrong way result:", wrongRemoveDuplicates(nums))
}

Output

Wrong way result: [1 2 3 4 5]

Quick Reference

• Use a map[type]bool to track seen elements.
• Iterate over the original slice and append unseen elements to a new slice.
• Do not modify the slice while iterating over it.
• Works only with comparable types as map keys.

Key Takeaways

Use a map to track and filter out duplicate elements efficiently.

Build a new slice with unique elements instead of modifying the original during iteration.

Only comparable types can be used as map keys for this method.

Avoid removing elements from a slice while looping over it to prevent bugs.

This method works well for slices of ints, strings, and other comparable types.