Document ID strategies (auto vs manual) in Elasticsearch - Quick Revision & Key Differences

Recall & Review

beginner

What is an auto-generated document ID in Elasticsearch?

An auto-generated document ID is a unique identifier automatically created by Elasticsearch when you index a document without specifying an ID. It ensures each document has a unique key without manual input.

beginner

What does manual document ID mean in Elasticsearch?

Manual document ID means you provide your own unique identifier for each document when indexing. This allows you to control the ID and update or overwrite documents using the same ID.

intermediate

What is a key advantage of using manual document IDs?

Manual IDs let you update or replace documents easily by reusing the same ID. This is useful when you want to keep track of specific documents or avoid duplicates.

intermediate

What is a downside of using auto-generated document IDs?

Auto-generated IDs make it harder to update or delete specific documents because you don’t know the ID in advance. You must search for the document first to find its ID.

intermediate

When should you prefer auto-generated IDs over manual IDs?

Use auto-generated IDs when you don’t need to update documents later and want Elasticsearch to handle unique ID creation automatically, such as for logging or event data.

What happens if you index a document without specifying an ID in Elasticsearch?

A The document overwrites the last indexed document.

B The document is rejected due to missing ID.

C You must manually add an ID before indexing.

D Elasticsearch creates a unique auto-generated ID for the document.

If no ID is provided, Elasticsearch automatically generates a unique ID for the document.

Which is a benefit of using manual document IDs?

A No need to worry about duplicate IDs.

B Elasticsearch handles ID uniqueness automatically.

C Easier to update or overwrite specific documents.

D Faster indexing speed.

Manual IDs let you update or overwrite documents by reusing the same ID.

What is a challenge when using auto-generated IDs?

A You must search to find the document ID before updating.

B You can easily overwrite documents by ID.

C You control the ID format.

D You can prevent duplicates manually.

Auto-generated IDs are unknown beforehand, so you must search to find them before updating or deleting.

When is it best to use auto-generated IDs?

A When documents are write-once and not updated later.

B When you want to control document IDs.

C When you need to update documents frequently.

D When you want to prevent duplicates manually.

Auto-generated IDs are ideal for write-once data like logs where updates are not needed.

What happens if you index a document with a manual ID that already exists?

A Both documents are stored with the same ID.

B The existing document is overwritten with the new one.

C A new document with a different ID is created.

D Elasticsearch rejects the new document.

Indexing with an existing manual ID overwrites the old document with the new content.

Explain the difference between auto-generated and manual document IDs in Elasticsearch.

Think about who creates the ID and how it affects updating documents.

Describe scenarios when you would choose manual document IDs over auto-generated ones.

Consider when controlling the ID helps manage your data better.

Practice

1. What is the main difference between automatic and manual document IDs in Elasticsearch?

easy

A. Automatic IDs are generated by Elasticsearch; manual IDs are set by the user.

B. Automatic IDs require user input; manual IDs are generated by Elasticsearch.

C. Automatic IDs are always numeric; manual IDs are always strings.

D. Automatic IDs cannot be used for updates; manual IDs cannot.

Solution

1. Step 1: Understand automatic ID generation

   Elasticsearch creates unique IDs automatically when you don't specify one.

2. Step 2: Understand manual ID assignment

   Users can assign their own IDs to documents for control and easier updates.

3. Final Answer:

   Automatic IDs are generated by Elasticsearch; manual IDs are set by the user. -> Option A

4. Quick Check:

   Auto = Elasticsearch generated, Manual = user set [OK]

Hint: Auto = system creates ID, manual = you create ID [OK]

Common Mistakes:

• Confusing which side creates the ID
• Thinking automatic IDs are numeric only
• Believing manual IDs cannot be updated

2. Which of the following is the correct syntax to index a document with a manual ID in Elasticsearch using JSON?

easy

A. { "index": "my_index", "_id": "123", "body": { "name": "Alice" } }

B. { "index": "my_index", "id": "123", "body": { "name": "Alice" } }

C. { "index": "my_index", "id": 123, "doc": { "name": "Alice" } }

D. { "index": "my_index", "_id": 123, "doc": { "name": "Alice" } }

Solution

1. Step 1: Identify the correct field for manual ID

   Elasticsearch uses "_id" (with underscore) to specify manual document IDs.

2. Step 2: Check the correct structure for indexing

   The "body" key holds the document content; "doc" is used for updates, not indexing.

3. Final Answer:

   { "index": "my_index", "_id": "123", "body": { "name": "Alice" } } -> Option A

4. Quick Check:

   Manual ID uses "_id" and "body" for document [OK]

Hint: Use "_id" for manual IDs and "body" for document content [OK]

Common Mistakes:

• Using "id" instead of "_id"
• Confusing "doc" with "body" for indexing
• Passing numeric ID without quotes

3. Given this indexing request with manual ID:

{ "index": "products", "_id": "p100", "body": { "name": "Pen", "price": 1.5 } }

What happens if you send the same request again with updated price 1.8?

medium

A. A new document is created with a different ID.

B. The existing document with ID "p100" is updated with the new price.

C. An error occurs because duplicate IDs are not allowed.

D. The document is ignored and no changes happen.

Solution

1. Step 1: Understand manual ID behavior on indexing

   Using the same manual ID overwrites the existing document with that ID.

2. Step 2: Check the effect of sending updated data

   The document with ID "p100" will be updated to have price 1.8.

3. Final Answer:

   The existing document with ID "p100" is updated with the new price. -> Option B

4. Quick Check:

   Same manual ID means update existing document [OK]

Hint: Same manual ID updates document, no new doc created [OK]

Common Mistakes:

• Thinking a new document is created with same ID
• Expecting an error on duplicate ID
• Assuming no change happens on reindex

4. You tried to index a document with this JSON:

{ "index": "users", "id": "u123", "body": { "name": "Bob" } }

But the document was not saved with ID "u123". What is the likely problem?

medium

A. The "body" field is missing required fields.

B. The index name "users" is invalid.

C. The field "id" should be "_id" to set manual ID.

D. Manual IDs are not allowed in Elasticsearch.

Solution

1. Step 1: Check the ID field name

   Elasticsearch requires "_id" (with underscore) to specify manual IDs, not "id".

2. Step 2: Confirm impact of wrong field

   Using "id" means Elasticsearch ignores it and generates an automatic ID instead.

3. Final Answer:

   The field "id" should be "_id" to set manual ID. -> Option C

4. Quick Check:

   Manual ID needs "_id", not "id" [OK]

Hint: Use "_id" to set manual ID, not "id" [OK]

Common Mistakes:

• Using "id" instead of "_id"
• Assuming missing fields cause ID issues
• Thinking manual IDs are disallowed

5. You want to avoid duplicate documents in Elasticsearch. Which strategy can help you achieve this?

hard

A. Use a random string as manual ID without relation to document content.

B. Always use automatic IDs and ignore duplicates.

C. Index documents without IDs and rely on timestamps to find duplicates.

D. Use a manual ID based on a unique field like email to prevent duplicates.

Solution

1. Step 1: Understand duplicate prevention

   Duplicates can be avoided by assigning manual IDs based on unique fields like email.

2. Step 2: Compare with automatic ID approach

   Automatic IDs are random and do not prevent duplicates because each document gets a new ID.

3. Final Answer:

   Use a manual ID based on a unique field like email to prevent duplicates. -> Option D

4. Quick Check:

   Manual unique IDs prevent duplicates, auto IDs do not [OK]

Hint: Manual unique IDs prevent duplicates better than auto IDs [OK]

Common Mistakes:

• Thinking auto IDs prevent duplicates
• Using random manual IDs unrelated to content
• Relying on timestamps instead of IDs