Recovering lost commits with reflog in Git - Deep Dive

Overview - Recovering lost commits with reflog

What is it?

Git reflog is a tool that records updates to the tip of branches and other references. It helps you find commits that seem lost, like after a reset or branch deletion. Using reflog, you can recover these commits by identifying their references and restoring them. This is essential for undoing mistakes in your Git history.

Why it matters

Without reflog, losing commits due to mistakes like resets or branch deletions could mean permanent data loss. Developers would waste time rewriting work or lose important changes. Reflog acts like a safety net, allowing recovery of lost work and reducing stress and errors in version control.

Where it fits

Before learning reflog, you should understand basic Git concepts like commits, branches, and resets. After mastering reflog, you can explore advanced Git recovery techniques and workflows for safe collaboration and history rewriting.

Mental Model

Core Idea

Reflog is a hidden diary that records every change to your Git references, letting you find and restore lost commits.

Think of it like...

Imagine your Git history as a path through a forest. Reflog is like footprints left behind even when you take a wrong turn or erase part of the path, helping you find your way back.

┌─────────────┐       ┌─────────────┐       ┌─────────────┐
│ Commit A   │──────▶│ Commit B   │──────▶│ Commit C   │
└─────────────┘       └─────────────┘       └─────────────┘
       ▲                     ▲                     ▲
       │                     │                     │
   reflog records every move and update of HEAD and branches

Commands like 'git reset' move HEAD pointer, but reflog keeps old positions.

Build-Up - 7 Steps

1

Foundation

Understanding Git commits and HEAD

Concept: Learn what commits and HEAD mean in Git to grasp how history is tracked.

A commit is a snapshot of your project at a point in time. HEAD is a pointer to the current commit you are working on. When you make new commits, HEAD moves forward. Commands like 'git log' show commit history.

Result

You understand that commits form a chain and HEAD points to the latest commit.

Knowing that HEAD moves with commits helps you see why losing HEAD's position can hide commits.

2

Foundation

What happens when commits seem lost

Concept: Explore how commands like 'git reset' or branch deletion can hide commits.

When you run 'git reset --hard' to an earlier commit, HEAD moves back and newer commits are no longer referenced by branches. Similarly, deleting a branch removes its pointer. These commits still exist but are unreachable by normal commands.

Result

You see that commits can become 'lost' but are not deleted immediately.

Understanding that commits are objects stored in Git helps you realize they can be recovered if you know their references.

3

Intermediate

Introducing reflog to track reference changes

Concept: Reflog records every change to HEAD and branch pointers, even those not visible in normal logs.

Run 'git reflog' to see a list of recent HEAD positions with commit hashes and actions. Each entry shows where HEAD pointed after commands like commit, reset, checkout, or merge.

Result

You get a timeline of all recent moves of HEAD and branches, including lost commits.

Knowing reflog tracks all reference moves means you can find commits that seem lost by looking at past HEAD positions.

4

Intermediate

Recovering lost commits using reflog

🤔Before reading on: do you think you can restore a lost commit by its hash or by resetting to a reflog entry? Commit to your answer.

Concept: Learn how to use reflog entries to restore commits that are no longer on any branch.

Find the commit hash from 'git reflog' output. Then use 'git checkout ' to inspect it or 'git branch ' to create a branch pointing to it. Alternatively, 'git reset --hard ' moves HEAD back to that commit.

Result

You successfully restore access to commits that were lost after resets or deletions.

Understanding reflog entries as pointers to past HEAD states lets you recover work that normal logs hide.

5

Intermediate

Cleaning up reflog and commit expiration

Concept: Reflog entries and unreachable commits are kept only for a limited time before Git cleans them up.

Git keeps reflog entries for 90 days by default. After that, unreachable commits may be garbage collected and permanently deleted. You can view reflog expiration with 'git reflog expire' and force cleanup with 'git gc'.

Result

You know reflog is a temporary safety net, not permanent storage.

Knowing reflog's time limits helps you act quickly to recover lost commits before they vanish.

6

Advanced

Using reflog in complex recovery scenarios

🤔Before reading on: do you think reflog can help recover commits lost after branch renames or merges? Commit your guess.

Concept: Explore how reflog tracks all reference changes, including merges, rebases, and branch renames, enabling recovery in complex cases.

Reflog records every HEAD and branch update, so even after merges or rebases, you can find previous commit states. For example, after a bad rebase, use 'git reflog' to find the pre-rebase commit and reset to it. Similarly, after renaming branches, reflog entries still show old HEAD positions.

Result

You can recover from complicated history rewrites using reflog.

Understanding reflog as a complete history of reference movements empowers recovery from almost any Git mistake.

7

Expert

Internal storage and limits of reflog

🤔Before reading on: do you think reflog entries are stored inside commits or separately? Commit your answer.

Concept: Learn how reflog data is stored internally and how Git manages its size and expiration.

Reflog entries are stored as files inside the .git/logs directory, separate from commits. Each branch and HEAD has its own reflog file. Git appends new entries on changes and prunes old entries based on expiration policies. This design keeps reflog lightweight and efficient.

Result

You understand reflog is a separate mechanism from commit history, optimized for tracking reference changes.

Knowing reflog's internal storage explains why reflog can track lost commits even after branch deletion and why reflog entries expire.

Under the Hood

Git stores commits as objects identified by hashes. Branches and HEAD are pointers to these commits. Reflog records every update to these pointers by appending entries to log files in .git/logs/. Each entry includes the old and new commit hashes, the action performed, and a timestamp. This allows Git to reconstruct past pointer states and find commits no longer referenced by branches.

Why designed this way?

Reflog was designed to provide a safety net for users to recover from mistakes without cluttering the main commit history. Storing reflog separately avoids polluting commit logs and keeps recovery data lightweight. The expiration policy balances disk usage with recovery needs, preventing indefinite storage of unreachable commits.

┌───────────────┐
│  .git/refs   │
│  (branches)  │
└──────┬────────┘
       │ points to
       ▼
┌───────────────┐
│   Commits     │
│ (objects)     │
└──────┬────────┘
       │ referenced by
       ▼
┌───────────────┐
│    HEAD       │
│ (pointer)     │
└──────┬────────┘
       │ updates logged in
       ▼
┌───────────────┐
│  .git/logs/   │
│  (reflog)     │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does deleting a branch immediately delete its commits? Commit yes or no.

Common Belief: Deleting a branch deletes all commits that were on it permanently.

Reality: Deleting a branch only removes the pointer; commits remain in the repository until garbage collected and can be recovered via reflog.

Why it matters: Believing commits are gone leads to panic and unnecessary rewrites instead of using reflog to recover work.

Quick: Does reflog keep entries forever? Commit yes or no.

Common Belief: Reflog stores all history permanently, so you can recover any commit anytime.

Reality: Reflog entries expire after a default period (usually 90 days), and unreachable commits can be garbage collected after that.

Why it matters: Assuming reflog is permanent can cause loss of commits if recovery is delayed too long.

Quick: Can reflog show commits from other clones or remote repositories? Commit yes or no.

Common Belief: Reflog tracks all commits including those from remote repositories automatically.

Reality: Reflog only tracks local reference changes; commits from remotes must be fetched and managed separately.

Why it matters: Misunderstanding this can cause confusion when commits appear missing after fetch or pull operations.

Quick: Does reflog record changes only for branches? Commit yes or no.

Common Belief: Reflog only tracks branch pointer changes, not HEAD or other references.

Reality: Reflog tracks all reference updates including HEAD, branches, and tags if configured.

Why it matters: Knowing this helps recover commits lost after detached HEAD states or tag moves.

Expert Zone

1

Reflog entries are local and not shared with remotes, so recovery is only possible on the local clone where the changes happened.

2

Reflog can be manipulated or cleared manually, which can cause permanent loss if done unknowingly.

3

Using reflog in scripts requires care because entries can expire or be pruned, so relying on reflog for long-term recovery is risky.

When NOT to use

Reflog is not suitable for recovering commits lost on remote repositories or after garbage collection. For distributed recovery, use backups, remote branches, or reflog on the remote server. Also, reflog cannot recover commits deleted by force garbage collection or repository corruption.

Production Patterns

In professional workflows, reflog is used to undo accidental resets, recover commits after failed rebases, and restore deleted branches. Teams often combine reflog with branch naming conventions and backups to minimize data loss. Automated scripts may parse reflog to detect and alert on dangerous history rewrites.

Connections

Undo and Redo in Text Editors

Similar pattern of tracking changes to allow reversal of actions.

Understanding reflog as a version of undo history helps grasp its role in recovering lost states.

Database Transaction Logs

Both record sequences of changes to enable rollback and recovery.

Seeing reflog like a transaction log clarifies why it stores pointer changes and timestamps for recovery.

Forensic Data Recovery

Both involve finding hidden or deleted data by tracking metadata and history.

Knowing reflog is like forensic recovery shows how Git preserves data beyond normal visibility.

Common Pitfalls

#1 Trying to recover lost commits by only using 'git log' without reflog.

Wrong approach:

git log
# No lost commits appear here after reset or branch deletion

Correct approach:

git reflog
# Shows all recent HEAD movements including lost commits

Root cause:

Misunderstanding that 'git log' only shows reachable commits, missing reflog's tracking of all reference changes.

#2 Waiting too long to recover commits, causing reflog entries to expire.

Wrong approach:

# After 3 months
# Try to recover commit from reflog but entries are gone
git reflog
# Empty or missing old entries

Correct approach:

# Recover commits promptly within reflog expiration period
git reflog
git checkout

Root cause:

Not knowing reflog entries expire and unreachable commits get garbage collected.

#3 Deleting reflog files manually to save space without understanding consequences.

Wrong approach:

rm -rf .git/logs/refs/heads/master
# reflog deleted, no recovery possible

Correct approach:

# Use git reflog expire and git gc to safely prune reflog
git reflog expire --expire=now --all
git gc --prune=now

Root cause:

Lack of knowledge about safe reflog maintenance commands leads to accidental data loss.

Key Takeaways

Git reflog records every change to branch and HEAD pointers, enabling recovery of commits that seem lost.

Lost commits are not deleted immediately; reflog entries let you find and restore them before expiration.

Reflog entries expire after a default period, so timely recovery is essential to avoid permanent loss.

Reflog is stored separately from commits, tracking pointer movements rather than commit content.

Understanding reflog empowers you to undo mistakes like resets, branch deletions, and bad rebases safely.