How to use ordering in pub sub

GcpHow-ToBeginner · 4 min read

How to Use Ordering in Google Cloud Pub/Sub

To use ordering in Google Cloud Pub/Sub, enable messageOrdering on your topic and publish messages with the same orderingKey. Subscribers will then receive messages with the same key in the order they were published.

📐

Syntax

Enable ordering by setting messageOrdering to true on the topic. When publishing, include an orderingKey string with each message to group messages that must be ordered.

messageOrdering: Boolean flag on the topic to enable ordering.
orderingKey: String key on each message to define ordering group.
Subscribers receive messages ordered per orderingKey.

python

gcloud pubsub topics create my-topic --message-ordering

# Publishing a message with ordering key
from google.cloud import pubsub_v1

publisher = pubsub_v1.PublisherClient()
topic_path = publisher.topic_path('my-project', 'my-topic')

# Data must be bytes
data = b'Message with ordering key'
ordering_key = 'key1'

future = publisher.publish(topic_path, data, ordering_key=ordering_key)
print(f'Published message ID: {future.result()}')

💻

Example

This example shows how to create a topic with ordering enabled, publish messages with ordering keys, and receive them in order.

python

from google.cloud import pubsub_v1

project_id = 'my-project'
topic_id = 'ordered-topic'
subscription_id = 'ordered-subscription'

publisher = pubsub_v1.PublisherClient()
subscriber = pubsub_v1.SubscriberClient()

topic_path = publisher.topic_path(project_id, topic_id)
subscription_path = subscriber.subscription_path(project_id, subscription_id)

# Create topic with message ordering enabled
try:
    topic = publisher.create_topic(request={"name": topic_path, "message_ordering": True})
    print(f'Topic created: {topic.name}')
except Exception as e:
    print(f'Topic exists or error: {e}')

# Create subscription
try:
    subscription = subscriber.create_subscription(request={"name": subscription_path, "topic": topic_path})
    print(f'Subscription created: {subscription.name}')
except Exception as e:
    print(f'Subscription exists or error: {e}')

# Publish messages with ordering keys
ordering_keys = ['key1', 'key2']
for key in ordering_keys:
    for i in range(3):
        data = f'Message {i} for {key}'.encode('utf-8')
        future = publisher.publish(topic_path, data, ordering_key=key)
        print(f'Published: {data.decode()} with ordering key: {key}')

# Function to process messages
def callback(message):
    print(f'Received message: {message.data.decode()} with ordering key: {message.attributes.get("ordering_key")}')
    message.ack()

# Listen for messages
streaming_pull_future = subscriber.subscribe(subscription_path, callback=callback)
print('Listening for messages...')

import time
try:
    time.sleep(10)  # Listen for 10 seconds
finally:
    streaming_pull_future.cancel()

Output

Topic created: projects/my-project/topics/ordered-topic Subscription created: projects/my-project/subscriptions/ordered-subscription Published: Message 0 for key1 with ordering key: key1 Published: Message 1 for key1 with ordering key: key1 Published: Message 2 for key1 with ordering key: key1 Published: Message 0 for key2 with ordering key: key2 Published: Message 1 for key2 with ordering key: key2 Published: Message 2 for key2 with ordering key: key2 Listening for messages... Received message: Message 0 for key1 with ordering key: key1 Received message: Message 1 for key1 with ordering key: key1 Received message: Message 2 for key1 with ordering key: key1 Received message: Message 0 for key2 with ordering key: key2 Received message: Message 1 for key2 with ordering key: key2 Received message: Message 2 for key2 with ordering key: key2

⚠️

Common Pitfalls

Not enabling messageOrdering on the topic disables ordering.
Publishing messages without an orderingKey will not guarantee order.
If a subscriber fails to acknowledge a message, ordering for that key pauses until the message is acknowledged.
Using too many unique ordering keys can reduce throughput.

python

from google.cloud import pubsub_v1

publisher = pubsub_v1.PublisherClient()
topic_path = publisher.topic_path('my-project', 'my-topic')

# Wrong: Publishing without ordering key when ordering is enabled
future = publisher.publish(topic_path, b'No ordering key')
print(f'Published message ID: {future.result()}')

# Right: Publishing with ordering key
future = publisher.publish(topic_path, b'With ordering key', ordering_key='key1')
print(f'Published message ID: {future.result()}')

📊

Quick Reference

Enable ordering: gcloud pubsub topics create my-topic --message-ordering
Publish with ordering key: publisher.publish(topic_path, data, ordering_key='key')
Subscriber receives messages ordered per key.
Acknowledge messages promptly to avoid blocking ordering.

✅

Key Takeaways

Enable message ordering on the topic by setting messageOrdering to true.

Publish messages with the same orderingKey to guarantee their order.

Subscribers receive messages ordered per orderingKey but must acknowledge promptly.

Messages without orderingKey are not ordered even if ordering is enabled.

Too many unique ordering keys can reduce throughput and increase complexity.