Topics and subscriptions in GCP - Commands & Configuration

Introduction

When you want to send messages between different parts of your app without them needing to be connected directly, you use topics and subscriptions. Topics are like message boards where you post messages, and subscriptions are like mailboxes that receive those messages.

When you want to send notifications from one service to many other services without waiting for them.

When you need to decouple parts of your app so they can work independently.

When you want to process messages asynchronously, like handling orders or user actions later.

When you want to broadcast updates to multiple systems at once.

When you want to ensure messages are delivered even if the receiver is temporarily offline.

Config File - topic-subscription.yaml

topic-subscription.yaml

apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubTopic
metadata:
  name: example-topic
spec:
  projectRef:
    name: example-project
---
apiVersion: pubsub.cnrm.cloud.google.com/v1beta1
kind: PubSubSubscription
metadata:
  name: example-subscription
spec:
  topicRef:
    name: example-topic
  ackDeadlineSeconds: 20
  retainAckedMessages: false
  messageRetentionDuration: 604800s

This file creates a Pub/Sub topic named example-topic where messages can be published.

It also creates a subscription named example-subscription that listens to that topic.

The subscription settings control how long to wait for acknowledgments and how long messages are kept.

Commands

This command creates a new topic named example-topic where messages can be sent.

Terminal

gcloud pubsub topics create example-topic

Expected Output

Created topic [projects/example-project/topics/example-topic].

This command creates a subscription named example-subscription that listens to messages from example-topic. The ack-deadline sets how long to wait for message acknowledgment.

Terminal

gcloud pubsub subscriptions create example-subscription --topic=example-topic --ack-deadline=20

Expected Output

Created subscription [projects/example-project/subscriptions/example-subscription].

→

--topic - Specifies the topic to subscribe to.

→

--ack-deadline - Sets the acknowledgment deadline in seconds.

This command sends a message 'Hello, world!' to the example-topic. Subscribers will receive this message.

Terminal

gcloud pubsub topics publish example-topic --message="Hello, world!"

Expected Output

messageIds: ["1234567890123456"]

→

--message - Specifies the message content to publish.

This command pulls one message from example-subscription and automatically acknowledges it, meaning the message is marked as received and won't be sent again.

Terminal

gcloud pubsub subscriptions pull example-subscription --auto-ack --limit=1

Expected Output

ACKNOWLEDGED 1 message(s).

Received message:

  data: Hello, world!

  messageId: 1234567890123456

→

--auto-ack - Automatically acknowledges the pulled messages.

→

--limit - Limits the number of messages to pull.

Key Concept

If you remember nothing else from this pattern, remember: topics send messages and subscriptions receive them independently so your app parts don’t have to wait for each other.

Common Mistakes

Trying to pull messages from a subscription before creating it.

The subscription does not exist yet, so no messages can be received.

Always create the subscription first using the correct topic reference.

Publishing messages to a topic that does not exist.

Messages cannot be sent to a non-existent topic, causing errors.

Create the topic before publishing messages to it.

Not acknowledging messages after pulling them.

Messages remain unacknowledged and will be redelivered, causing duplicates.

Use --auto-ack flag or explicitly acknowledge messages after processing.

Summary

Create a topic to send messages to with 'gcloud pubsub topics create'.

Create a subscription to receive messages from the topic with 'gcloud pubsub subscriptions create'.

Publish messages to the topic with 'gcloud pubsub topics publish'.

Pull and acknowledge messages from the subscription with 'gcloud pubsub subscriptions pull'.

Practice

1. What is the main role of a topic in Google Cloud Pub/Sub?

easy

A. To authenticate users

B. To receive messages from subscribers

C. To store messages permanently

D. To send messages to subscribers

Solution

1. Step 1: Understand the role of a topic

   A topic is where messages are sent or published by a publisher.

2. Step 2: Differentiate topic from subscription

   Subscriptions receive messages from topics, not the other way around.

3. Final Answer:

   To send messages to subscribers -> Option D

4. Quick Check:

   Topic = message sender [OK]

Hint: Topics send; subscriptions receive messages [OK]

Common Mistakes:

• Confusing topic as receiver instead of sender
• Thinking topics store messages permanently
• Mixing roles of topic and subscription

2. Which of the following is the correct way to create a subscription to a topic using gcloud CLI?

easy

A. gcloud pubsub subscriptions create my-sub --topic=my-topic

B. gcloud pubsub topics create my-sub --subscription=my-topic

C. gcloud pubsub subscriptions create my-topic --topic=my-sub

D. gcloud pubsub topics create my-topic --subscription=my-sub

Solution

1. Step 1: Identify correct command syntax

   The command to create a subscription uses subscriptions create with the subscription name and --topic flag for the topic.

2. Step 2: Verify gcloud pubsub subscriptions create my-sub --topic=my-topic matches syntax

   gcloud pubsub subscriptions create my-sub --topic=my-topic correctly uses gcloud pubsub subscriptions create my-sub --topic=my-topic.

3. Final Answer:

   gcloud pubsub subscriptions create my-sub --topic=my-topic -> Option A

4. Quick Check:

   Subscriptions create with --topic flag [OK]

Hint: Subscriptions created with 'subscriptions create' and --topic [OK]

Common Mistakes:

• Using topics create instead of subscriptions create
• Swapping topic and subscription names
• Using --subscription flag instead of --topic

3. Given the following code snippet, what will be printed if the subscription receives a message?

from google.cloud import pubsub_v1

subscriber = pubsub_v1.SubscriberClient()
subscription_path = subscriber.subscription_path('my-project', 'my-sub')

def callback(message):
    print(f'Received: {message.data.decode()}')
    message.ack()

streaming_pull_future = subscriber.subscribe(subscription_path, callback=callback)
print('Listening for messages...')

medium

A. Only Listening for messages...

B. Only Received:

C. Listening for messages... followed by Received:

D. Error because callback is not called

Solution

1. Step 1: Understand code flow

   The code subscribes to a topic and sets a callback to print received messages and acknowledge them.

2. Step 2: Identify output behavior

   It prints 'Listening for messages...' immediately, then prints 'Received: ...' when a message arrives.

3. Final Answer:

   Listening for messages... followed by Received: <message content> -> Option C

4. Quick Check:

   Subscribe prints listening then message received [OK]

Hint: Callback prints message after initial listening print [OK]

Common Mistakes:

• Assuming callback runs before print
• Thinking no output until message arrives
• Ignoring message acknowledgment

4. You created a subscription but messages are not being received. Which of the following is the most likely cause?

medium

A. The subscription is not attached to the correct topic

B. The topic does not exist

C. The subscription name is too long

D. The messages are too large

Solution

1. Step 1: Check subscription-topic link

   If subscription is not linked to the right topic, it won't receive messages.

2. Step 2: Evaluate other options

   Topic existence is important but usually creation fails if topic missing; name length or message size usually cause errors, not silent no messages.

3. Final Answer:

   The subscription is not attached to the correct topic -> Option A

4. Quick Check:

   Subscription-topic mismatch blocks messages [OK]

Hint: Check subscription's topic link first if no messages [OK]

Common Mistakes:

• Ignoring subscription-topic association
• Assuming message size silently blocks delivery
• Thinking subscription name length causes no messages

5. You want to ensure that messages published to a topic are delivered to multiple independent services. Which setup should you use?

hard

A. Create multiple topics, one per service, and one subscription per topic

B. Create one topic and multiple subscriptions, one per service

C. Create one subscription and multiple topics linked to it

D. Create one topic and one subscription shared by all services

Solution

1. Step 1: Understand message fan-out

   One topic can send messages to many subscriptions, allowing multiple services to receive independently.

2. Step 2: Evaluate options

   Create one topic and multiple subscriptions, one per service uses one topic with multiple subscriptions, which is the recommended pattern for multiple independent consumers.

3. Final Answer:

   Create one topic and multiple subscriptions, one per service -> Option B

4. Quick Check:

   One topic, many subscriptions for multiple services [OK]

Hint: Use multiple subscriptions on one topic for multiple receivers [OK]

Common Mistakes:

• Creating multiple topics unnecessarily
• Using one subscription shared by all services
• Linking subscriptions to multiple topics