File globbing (wildcards *, ?, []) in Linux CLI - Deep Dive

Overview - File globbing (wildcards *, ?, [])

What is it?

File globbing is a way to find and list files using special symbols called wildcards. These wildcards let you match many file names with simple patterns instead of typing each name. The most common wildcards are * (matches any characters), ? (matches one character), and [] (matches any character inside the brackets). This helps you work faster with groups of files in the command line.

Why it matters

Without file globbing, you would have to type every file name exactly, which is slow and error-prone. Globbing saves time and effort by letting you select many files with one pattern. It is essential for automating tasks like copying, moving, or deleting files in bulk. Without it, managing files on Linux or Unix systems would be much harder and less efficient.

Where it fits

Before learning file globbing, you should know basic Linux commands like ls, cp, and rm. After mastering globbing, you can learn about regular expressions for more powerful pattern matching and scripting techniques to automate complex file operations.

Mental Model

Core Idea

File globbing uses simple symbols to match groups of file names so you can work with many files at once without typing each name.

Think of it like...

Imagine you have a big box of keys and you want to find all keys that look similar. Instead of checking each key one by one, you use a shape template that fits many keys at once. Wildcards are like that template for file names.

Pattern Matching Flow:

  Input Pattern
      │
      ▼
  ┌─────────────────────┐
  │ Contains wildcards?  │
  └─────────┬───────────┘
            │Yes
            ▼
  ┌─────────────────────┐
  │ Match files in folder│
  └─────────┬───────────┘
            │
            ▼
  ┌─────────────────────┐
  │ List matched files   │
  └─────────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding the Asterisk Wildcard

Concept: The asterisk (*) wildcard matches zero or more characters in file names.

In the command line, * can replace any number of characters. For example, *.txt matches all files ending with .txt, like notes.txt or report.txt. It can also match files with no characters before or after, like just .txt if it exists.

Result

Using ls *.txt lists all files with the .txt extension in the current folder.

Knowing that * matches any number of characters lets you quickly select many files sharing a pattern without typing each name.

2

Foundation

Using the Question Mark Wildcard

Concept: The question mark (?) wildcard matches exactly one character in a file name.

The ? replaces a single character. For example, file?.txt matches file1.txt or fileA.txt but not file10.txt because 10 is two characters. This helps when you want to match files with names of a certain length or pattern.

Result

ls file?.txt lists files like file1.txt and fileA.txt but skips file10.txt.

Understanding ? helps you match files with precise character counts, useful for files with numbered or patterned names.

3

Intermediate

Matching Specific Characters with Brackets

Concept: Square brackets [] match any one character inside them, allowing selection from a set or range.

For example, file[123].txt matches file1.txt, file2.txt, or file3.txt but not file4.txt. You can also use ranges like file[a-c].txt to match filea.txt, fileb.txt, or filec.txt. This lets you target specific characters in file names.

Result

ls file[abc].txt lists files named filea.txt, fileb.txt, and filec.txt only.

Using [] lets you precisely control which characters to match, making patterns more flexible and targeted.

4

Intermediate

Combining Wildcards for Complex Patterns

🤔Before reading on: do you think you can combine * and ? wildcards in one pattern? Commit to your answer.

Concept: You can mix *, ?, and [] wildcards in one pattern to match complex file name sets.

For example, data_??_[a-c]*.csv matches files starting with data_, followed by any two characters, then an underscore, then a letter a to c, and ending with .csv. This powerful combination helps select very specific groups of files.

Result

ls data_??_[a-c]*.csv lists files like data_01_a.csv and data_zz_b_backup.csv.

Knowing how to combine wildcards unlocks precise and powerful file selection for automation and scripting.

5

Intermediate

Globbing vs Regular Expressions

🤔Before reading on: do you think globbing patterns and regular expressions are the same? Commit to your answer.

Concept: Globbing uses simple wildcards for file matching, while regular expressions are more complex patterns used in many tools for text matching.

Globbing is built into the shell for file names and is simpler. Regular expressions can match more complex patterns but require different syntax and tools like grep or sed. Understanding the difference helps choose the right tool for the task.

Result

Using ls *.txt uses globbing, while grep '^file.*txt$' uses regex to find matching lines in files.

Knowing the difference prevents confusion and helps pick the best pattern tool for your needs.

6

Advanced

How Shell Expands Globbing Patterns

🤔Before reading on: do you think the shell or the command handles wildcards? Commit to your answer.

Concept: The shell expands globbing patterns before running the command, replacing wildcards with matching file names.

When you type ls *.txt, the shell looks in the current folder, finds all files ending with .txt, and replaces *.txt with their names before running ls. If no files match, behavior depends on shell settings (sometimes the pattern is passed as-is).

Result

The command receives a list of file names, not the wildcard pattern.

Understanding shell expansion clarifies why some commands behave differently with wildcards and helps debug pattern matching issues.

7

Expert

Globbing Pitfalls and Edge Cases

🤔Before reading on: do you think hidden files (starting with .) are matched by *? Commit to your answer.

Concept: By default, globbing does not match hidden files (those starting with a dot). Also, some shells have options that change globbing behavior, and patterns can behave unexpectedly with special characters.

For example, * does not match .hiddenfile unless you use specific options or patterns like .*. Also, characters like spaces or newlines in file names can cause issues. Advanced users use shell options like dotglob or quote patterns carefully to handle these cases.

Result

Without special handling, hidden files are skipped, and some file names may cause errors or unexpected matches.

Knowing these edge cases prevents bugs in scripts and helps write robust file matching patterns.

Under the Hood

When you enter a command with wildcards, the shell scans the current directory for files matching the pattern. It replaces the wildcard pattern with the list of matching file names before running the command. This process is called glob expansion. The shell uses simple matching rules: * matches any string, ? matches one character, and [] matches any character in the set. If no files match, behavior depends on shell settings, sometimes passing the pattern unchanged.

Why designed this way?

Globbing was designed to simplify file selection without complex syntax or external tools. Early Unix shells needed a fast, easy way to handle multiple files. Using simple wildcards keeps commands short and intuitive. More complex matching was left to separate tools like grep. This separation keeps the shell lightweight and flexible.

User types command with pattern
        │
        ▼
  ┌─────────────────────┐
  │ Shell receives input │
  └─────────┬───────────┘
            │
            ▼
  ┌─────────────────────┐
  │ Shell scans folder   │
  │ for matching files   │
  └─────────┬───────────┘
            │
            ▼
  ┌─────────────────────┐
  │ Shell replaces      │
  │ pattern with files  │
  └─────────┬───────────┘
            │
            ▼
  ┌─────────────────────┐
  │ Command runs with   │
  │ expanded file list  │
  └─────────────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does * match hidden files starting with a dot? Commit to yes or no.

Common Belief: The * wildcard matches all files, including hidden ones.

Reality: By default, * does NOT match hidden files (those starting with a dot). You must use special patterns or shell options to include them.

Why it matters: Assuming * matches hidden files can cause scripts to miss important files or behave unexpectedly when hidden files are involved.

Quick: Does the command receive the wildcard pattern or the expanded file list? Commit to one.

Common Belief: The command itself processes the wildcard pattern to find files.

Reality: The shell expands the wildcard pattern into matching file names before the command runs. The command only sees the list of files.

Why it matters: Misunderstanding this leads to confusion about how commands handle patterns and why some commands behave differently with wildcards.

Quick: Can ? match zero characters? Commit to yes or no.

Common Belief: The ? wildcard can match zero or one character.

Reality: The ? wildcard matches exactly one character, never zero.

Why it matters: Incorrectly thinking ? can match zero characters causes wrong pattern expectations and missed matches.

Quick: Are globbing patterns the same as regular expressions? Commit to yes or no.

Common Belief: Globbing patterns and regular expressions are interchangeable and work the same way.

Reality: Globbing uses simple wildcards for file names, while regular expressions are more complex and used for text matching with different syntax.

Why it matters: Confusing these leads to errors when writing patterns and choosing tools for matching.

Expert Zone

1

Globbing behavior can be altered by shell options like 'nullglob' and 'dotglob', which change how unmatched patterns and hidden files are handled.

2

Some shells support extended globbing with additional wildcards like @(pattern) or !(pattern), providing more powerful matching beyond basic *, ?, and [].

3

File names with special characters (spaces, newlines, quotes) can break globbing unless properly quoted or escaped, requiring careful scripting.

When NOT to use

Globbing is limited to simple filename patterns and cannot match complex text patterns inside files. For advanced text matching or filtering, use regular expressions with tools like grep or sed. Also, for recursive directory matching, tools like 'find' or 'fd' are better suited than basic globbing.

Production Patterns

In real-world scripts, globbing is used to batch process files, like renaming all .jpg files or deleting temporary files. Professionals combine globbing with loops and conditionals to automate backups, log rotations, and deployments. Extended globbing and shell options are often enabled for more precise control in production environments.

Connections

Regular Expressions

Related but more powerful pattern matching system

Understanding globbing helps grasp the simpler pattern matching that underpins regular expressions, which extend these ideas to text processing.

Unix Shell Scripting

Globbing is a fundamental feature used extensively in shell scripts

Mastering globbing patterns is essential for writing effective shell scripts that automate file management tasks.

Database Query Wildcards

Similar wildcard concepts used in SQL LIKE queries

Recognizing that * and ? in globbing correspond to % and _ in SQL LIKE helps transfer pattern matching skills across domains.

Common Pitfalls

#1 Assuming * matches hidden files starting with a dot.

Wrong approach:

ls *

Correct approach:

ls .* *

Root cause:

Misunderstanding that shell globbing excludes hidden files by default unless explicitly included.

#2 Using ? to match zero or multiple characters.

Wrong approach:

ls file?.txt to match file.txt

Correct approach:

ls file.txt (no wildcard) or ls file*.txt to match file.txt and others

Root cause:

Confusing ? as zero-or-one character wildcard instead of exactly one character.

#3 Expecting the command to handle wildcards instead of the shell.

Wrong approach:

grep *.txt pattern

Correct approach:

grep pattern *.txt

Root cause:

Not knowing that the shell expands wildcards before the command runs.

Key Takeaways

File globbing uses simple wildcards (*, ?, []) to match multiple file names quickly and efficiently.

The shell expands globbing patterns before running commands, replacing patterns with matching file names.

By default, globbing does not match hidden files starting with a dot unless explicitly specified.

Globbing is simpler than regular expressions and is designed specifically for file name matching in shells.

Understanding globbing is essential for effective file management and automation in Linux command line environments.