Message retention and acknowledgment in GCP - Step-by-Step Execution

Process Flow - Message retention and acknowledgment

Message Published

↓

Message Stored in Topic

↓

Subscriber Receives Message

↓

Subscriber Processes Message

↓

Subscriber Sends Acknowledgment

↓

Message Removed from Subscription

↓

If No Ack Before Deadline

↓

Message Redelivered

Messages are published and stored. Subscribers receive and process them, then send acknowledgment. If ack is missing before deadline, message is redelivered.

Execution Sample

GCP

Publish message -> Subscriber receives -> Process message -> Send ack -> Message removed

Shows the flow of message delivery, processing, acknowledgment, and removal from subscription.

Process Table

Step	Action	Message State	Ack Deadline	Result
1	Message published to topic	Stored in topic	N/A	Waiting for subscriber
2	Subscriber receives message	Delivered to subscriber	10 seconds	Subscriber starts processing
3	Subscriber processes message	In processing	7 seconds left	Processing ongoing
4	Subscriber sends acknowledgment	Acknowledged	N/A	Message removed from subscription
5	No new message	No message pending	N/A	Waiting for next message
6	If ack not sent before deadline	Not acknowledged	0 seconds	Message redelivered to subscriber

💡 Message removed after acknowledgment or redelivered if ack deadline missed

Status Tracker

Variable	Start	After Step 2	After Step 3	After Step 4	After Step 6
Message State	Stored in topic	Delivered to subscriber	In processing	Acknowledged	Redelivered
Ack Deadline (seconds)	N/A	10	7	N/A	0

Key Moments - 3 Insights

Why does the message get redelivered even if it was already sent once?

If the subscriber does not send acknowledgment before the ack deadline (see step 6 in execution_table), the message is considered unprocessed and is redelivered to ensure it is handled.

What happens to the message after the subscriber sends acknowledgment?

According to step 4 in execution_table, once acknowledgment is sent, the message is removed from the subscription and will not be delivered again.

Does the message stay in the topic after delivery to subscriber?

Yes, the message stays in the topic. It is delivered to the subscription and tracked there until acknowledged or redelivered (see steps 1 and 2).

Visual Quiz - 3 Questions

Look at the execution_table, what is the message state after the subscriber sends acknowledgment?

A In processing

B Delivered to subscriber

C Acknowledged

D Stored in topic

💡 Hint

Check step 4 in the execution_table under 'Message State'

At which step does the message get redelivered if no acknowledgment is received?

A Step 3

B Step 6

C Step 4

D Step 2

💡 Hint

Look at the 'Result' column in step 6 of the execution_table

If the ack deadline is 10 seconds, what happens if the subscriber processes the message for 12 seconds without sending ack?

A Message is redelivered after 10 seconds

B Message stays in processing indefinitely

C Message is removed immediately

D Message is deleted from topic

💡 Hint

Refer to the ack deadline and redelivery logic in execution_table steps 2, 3, and 6

Concept Snapshot

Message retention means messages stay until acknowledged.
Subscribers must send acknowledgment before deadline.
If no ack, message is redelivered.
Ack removes message from subscription.
This ensures reliable message processing.

Full Transcript

In Google Cloud Pub/Sub, messages are published to a topic and stored until subscribers receive them. When a subscriber gets a message, it has a limited time called the acknowledgment deadline to process and acknowledge it. If the subscriber sends acknowledgment in time, the message is removed from the subscription and won't be sent again. If the subscriber fails to acknowledge before the deadline, the message is redelivered to ensure it is processed. This flow guarantees messages are not lost and are processed reliably.

Practice

1. What happens to a message in Google Cloud Pub/Sub if it is not acknowledged within the retention period?

easy

A. The message is automatically acknowledged.

B. The message is deleted and no longer available for delivery.

C. The message is moved to a dead-letter queue.

D. The message is duplicated and sent again.

Solution

1. Step 1: Understand message retention

   Message retention defines how long unacknowledged messages are kept before deletion.

2. Step 2: Effect of no acknowledgment within retention

   If a message is not acknowledged before retention expires, it is deleted and lost.

3. Final Answer:

   The message is deleted and no longer available for delivery. -> Option B

4. Quick Check:

   Unacknowledged message + retention expired = deleted [OK]

Hint: Unacknowledged messages expire and get deleted after retention [OK]

Common Mistakes:

• Thinking messages auto-acknowledge after retention
• Assuming messages move to dead-letter queue automatically
• Believing messages duplicate on retention expiry

2. Which of the following is the correct way to acknowledge a message in Google Cloud Pub/Sub client libraries?

easy

A. Call message.ack() after processing the message.

B. Call message.delete() to remove the message.

C. Call message.complete() to finish processing.

D. Call message.finish() to acknowledge.

Solution

1. Step 1: Identify acknowledgment method

   In Pub/Sub client libraries, acknowledging a message is done by calling ack() on the message object.

2. Step 2: Verify other options

   Methods like delete(), complete(), or finish() are not valid for acknowledgment.

3. Final Answer:

   Call message.ack() after processing the message. -> Option A

4. Quick Check:

   Use message.ack() to acknowledge [OK]

Hint: Use message.ack() to confirm message processed [OK]

Common Mistakes:

• Using incorrect method names like delete or finish
• Not acknowledging messages at all
• Confusing acknowledgment with message deletion

3. Consider this code snippet using Google Cloud Pub/Sub subscriber client:

def callback(message):
    print(f"Received: {message.data.decode('utf-8')}")
    # No acknowledgment called here

subscriber.subscribe(subscription_path, callback=callback)

What will happen to the messages if the callback never calls message.ack()?

medium

A. Messages will be acknowledged automatically after printing.

B. Messages will be lost immediately after printing.

C. Messages will be redelivered until acknowledged or retention expires.

D. Messages will be sent to a dead-letter topic instantly.

Solution

1. Step 1: Understand callback behavior without ack

   If message.ack() is not called, Pub/Sub considers the message unacknowledged.

2. Step 2: Effect on message delivery

   Unacknowledged messages are redelivered repeatedly until acknowledged or retention period expires.

3. Final Answer:

   Messages will be redelivered until acknowledged or retention expires. -> Option C

4. Quick Check:

   No ack means redelivery until retention ends [OK]

Hint: No ack means message redelivery until retention ends [OK]

Common Mistakes:

• Assuming messages auto-acknowledge after callback
• Thinking messages are lost immediately without ack
• Believing messages go to dead-letter topic without config

4. You have set a message retention duration of 7 days on your Pub/Sub subscription. However, messages are disappearing after 1 day. What is the most likely cause?

medium

A. Messages are being acknowledged by the subscriber before 1 day.

B. The retention duration setting is ignored by Pub/Sub.

C. The subscription is a push type and does not support retention.

D. Messages are automatically deleted after 24 hours regardless of settings.

Solution

1. Step 1: Check retention vs acknowledgment

   Retention controls how long unacknowledged messages stay. If messages are acknowledged, they are removed immediately.

2. Step 2: Analyze disappearance cause

   If messages disappear early, likely they were acknowledged by the subscriber before 1 day.

3. Final Answer:

   Messages are being acknowledged by the subscriber before 1 day. -> Option A

4. Quick Check:

   Acknowledged messages removed before retention ends [OK]

Hint: Acknowledged messages delete immediately, ignoring retention [OK]

Common Mistakes:

• Assuming retention is ignored by Pub/Sub
• Believing push subscriptions don't support retention
• Thinking messages auto-delete after 24 hours always

5. You want to ensure that messages in your Pub/Sub subscription are retained for 3 days even if they are not acknowledged, but also want to avoid message duplication after acknowledgment. Which configuration should you apply?

hard

A. Set message retention duration to 0 and rely on subscriber retries.

B. Set message retention duration to 3 days and disable acknowledgments.

C. Set message retention duration to 7 days and never acknowledge messages.

D. Set message retention duration to 3 days and ensure subscriber acknowledges messages promptly.

Solution

1. Step 1: Understand retention and acknowledgment roles

   Retention keeps unacknowledged messages for specified time. Acknowledgment removes messages to avoid duplicates.

2. Step 2: Choose configuration to meet goals

   Setting retention to 3 days keeps messages if unacknowledged. Prompt acknowledgment removes processed messages, preventing duplicates.

3. Final Answer:

   Set message retention duration to 3 days and ensure subscriber acknowledges messages promptly. -> Option D

4. Quick Check:

   Retention + prompt ack = no duplicates, messages kept [OK]

Hint: Retention keeps unacked; ack removes to prevent duplicates [OK]

Common Mistakes:

• Disabling acknowledgments causing duplicates
• Setting retention to 0 losing unacked messages quickly
• Never acknowledging causing message buildup and duplicates