Document ID strategies (auto vs manual) in Elasticsearch - Visual Side-by-Side Comparison

Concept Flow - Document ID strategies (auto vs manual)

Start Document Indexing

↓

Check if ID Provided?

No→Auto-generate ID

↓

Assign Auto ID

↓

Use Manual ID

↓

Index Document

↓

Index Document

↓

End

This flow shows how Elasticsearch decides to use a manual ID if provided, or auto-generate one if not, before indexing the document.

Execution Sample

Elasticsearch

POST /my_index/_doc/123
{
  "name": "Alice"
}

POST /my_index/_doc
{
  "name": "Bob"
}

This example indexes two documents: one with a manual ID '123', and one without an ID, so Elasticsearch auto-generates it.

Execution Table

Step	Action	ID Provided?	ID Used	Result
1	Index document with ID '123'	Yes	123	Document indexed with ID '123'
2	Index document without ID	No	Auto-generated (e.g., 'abc123')	Document indexed with auto-generated ID
3	End	-	-	Indexing complete

💡 All documents indexed; manual IDs used when provided, otherwise auto-generated IDs assigned.

Variable Tracker

Variable	Start	After Step 1	After Step 2	Final
document_id	None	123	Auto-generated (e.g., 'abc123')	Last used ID

Key Moments - 2 Insights

Why does Elasticsearch generate an ID automatically?

When no ID is provided (see execution_table step 2), Elasticsearch creates a unique ID to identify the document.

What happens if I provide the same manual ID twice?

The second document will overwrite the first one with that ID, because IDs must be unique in an index.

Visual Quiz - 3 Questions

Look at the execution_table, what ID is used in step 2?

A Manual ID '123'

B Auto-generated ID like 'abc123'

C No ID used

D ID '456'

💡 Hint

Check the 'ID Used' column in execution_table row for step 2.

At which step does Elasticsearch use a manual ID?

A Step 1

B Step 2

C Step 3

D None

💡 Hint

Look at the 'ID Provided?' column in execution_table.

If you provide the same manual ID twice, what happens?

A Elasticsearch throws an error

B Both documents are stored separately

C The second document overwrites the first

D A new auto-generated ID is assigned

💡 Hint

Refer to key_moments about manual ID uniqueness.

Concept Snapshot

Document ID strategies in Elasticsearch:
- Manual ID: You provide a unique ID when indexing.
- Auto ID: Elasticsearch generates a unique ID if none is provided.
- Manual IDs overwrite documents with the same ID.
- Auto IDs ensure uniqueness automatically.
- Choose manual for control, auto for simplicity.

Full Transcript

This visual execution shows how Elasticsearch handles document IDs during indexing. When you send a document with a manual ID, Elasticsearch uses that ID to store the document. If you don't provide an ID, Elasticsearch creates one automatically. The execution table traces two documents: one with ID '123' and one without an ID, which gets an auto-generated ID like 'abc123'. Variables track the document ID used at each step. Key moments clarify why auto IDs exist and what happens if manual IDs repeat. The quiz tests understanding of these steps. Remember, manual IDs give control but must be unique, while auto IDs simplify indexing by ensuring uniqueness automatically.

Practice

1. What is the main difference between automatic and manual document IDs in Elasticsearch?

easy

A. Automatic IDs are generated by Elasticsearch; manual IDs are set by the user.

B. Automatic IDs require user input; manual IDs are generated by Elasticsearch.

C. Automatic IDs are always numeric; manual IDs are always strings.

D. Automatic IDs cannot be used for updates; manual IDs cannot.

Solution

1. Step 1: Understand automatic ID generation

   Elasticsearch creates unique IDs automatically when you don't specify one.

2. Step 2: Understand manual ID assignment

   Users can assign their own IDs to documents for control and easier updates.

3. Final Answer:

   Automatic IDs are generated by Elasticsearch; manual IDs are set by the user. -> Option A

4. Quick Check:

   Auto = Elasticsearch generated, Manual = user set [OK]

Hint: Auto = system creates ID, manual = you create ID [OK]

Common Mistakes:

• Confusing which side creates the ID
• Thinking automatic IDs are numeric only
• Believing manual IDs cannot be updated

2. Which of the following is the correct syntax to index a document with a manual ID in Elasticsearch using JSON?

easy

A. { "index": "my_index", "_id": "123", "body": { "name": "Alice" } }

B. { "index": "my_index", "id": "123", "body": { "name": "Alice" } }

C. { "index": "my_index", "id": 123, "doc": { "name": "Alice" } }

D. { "index": "my_index", "_id": 123, "doc": { "name": "Alice" } }

Solution

1. Step 1: Identify the correct field for manual ID

   Elasticsearch uses "_id" (with underscore) to specify manual document IDs.

2. Step 2: Check the correct structure for indexing

   The "body" key holds the document content; "doc" is used for updates, not indexing.

3. Final Answer:

   { "index": "my_index", "_id": "123", "body": { "name": "Alice" } } -> Option A

4. Quick Check:

   Manual ID uses "_id" and "body" for document [OK]

Hint: Use "_id" for manual IDs and "body" for document content [OK]

Common Mistakes:

• Using "id" instead of "_id"
• Confusing "doc" with "body" for indexing
• Passing numeric ID without quotes

3. Given this indexing request with manual ID:

{ "index": "products", "_id": "p100", "body": { "name": "Pen", "price": 1.5 } }

What happens if you send the same request again with updated price 1.8?

medium

A. A new document is created with a different ID.

B. The existing document with ID "p100" is updated with the new price.

C. An error occurs because duplicate IDs are not allowed.

D. The document is ignored and no changes happen.

Solution

1. Step 1: Understand manual ID behavior on indexing

   Using the same manual ID overwrites the existing document with that ID.

2. Step 2: Check the effect of sending updated data

   The document with ID "p100" will be updated to have price 1.8.

3. Final Answer:

   The existing document with ID "p100" is updated with the new price. -> Option B

4. Quick Check:

   Same manual ID means update existing document [OK]

Hint: Same manual ID updates document, no new doc created [OK]

Common Mistakes:

• Thinking a new document is created with same ID
• Expecting an error on duplicate ID
• Assuming no change happens on reindex

4. You tried to index a document with this JSON:

{ "index": "users", "id": "u123", "body": { "name": "Bob" } }

But the document was not saved with ID "u123". What is the likely problem?

medium

A. The "body" field is missing required fields.

B. The index name "users" is invalid.

C. The field "id" should be "_id" to set manual ID.

D. Manual IDs are not allowed in Elasticsearch.

Solution

1. Step 1: Check the ID field name

   Elasticsearch requires "_id" (with underscore) to specify manual IDs, not "id".

2. Step 2: Confirm impact of wrong field

   Using "id" means Elasticsearch ignores it and generates an automatic ID instead.

3. Final Answer:

   The field "id" should be "_id" to set manual ID. -> Option C

4. Quick Check:

   Manual ID needs "_id", not "id" [OK]

Hint: Use "_id" to set manual ID, not "id" [OK]

Common Mistakes:

• Using "id" instead of "_id"
• Assuming missing fields cause ID issues
• Thinking manual IDs are disallowed

5. You want to avoid duplicate documents in Elasticsearch. Which strategy can help you achieve this?

hard

A. Use a random string as manual ID without relation to document content.

B. Always use automatic IDs and ignore duplicates.

C. Index documents without IDs and rely on timestamps to find duplicates.

D. Use a manual ID based on a unique field like email to prevent duplicates.

Solution

1. Step 1: Understand duplicate prevention

   Duplicates can be avoided by assigning manual IDs based on unique fields like email.

2. Step 2: Compare with automatic ID approach

   Automatic IDs are random and do not prevent duplicates because each document gets a new ID.

3. Final Answer:

   Use a manual ID based on a unique field like email to prevent duplicates. -> Option D

4. Quick Check:

   Manual unique IDs prevent duplicates, auto IDs do not [OK]

Hint: Manual unique IDs prevent duplicates better than auto IDs [OK]

Common Mistakes:

• Thinking auto IDs prevent duplicates
• Using random manual IDs unrelated to content
• Relying on timestamps instead of IDs