How to Sync Data Between Microservices: Best Practices and Examples

To sync data between microservices, use event-driven architecture with message brokers or API calls for direct communication. Event-driven syncing via asynchronous messaging ensures loose coupling and scalability, while synchronous REST/gRPC calls provide immediate consistency when needed.

Syntax

There are two main ways to sync data between microservices:

• Event-driven: Services publish events to a message broker like Kafka or RabbitMQ. Other services subscribe and update their data.
• API calls: One service calls another's REST or gRPC endpoint to fetch or update data directly.

Example event message format:

{
  "eventType": "UserCreated",
  "data": {
    "userId": "123",
    "name": "Alice"
  },
  "timestamp": "2024-06-01T12:00:00Z"
}

javascript

const event = {
  eventType: "UserCreated",
  data: {
    userId: "123",
    name: "Alice"
  },
  timestamp: new Date().toISOString()
};

// Publish event to message broker
messageBroker.publish('user-events', event);

Example

This example shows two microservices syncing user data via an event-driven approach using a simple in-memory event bus.

javascript

class EventBus {
  constructor() {
    this.listeners = {};
  }

  subscribe(eventType, callback) {
    if (!this.listeners[eventType]) {
      this.listeners[eventType] = [];
    }
    this.listeners[eventType].push(callback);
  }

  publish(event) {
    const listeners = this.listeners[event.eventType] || [];
    listeners.forEach(cb => cb(event.data));
  }
}

// Service A: publishes user created event
const eventBus = new EventBus();

// Service B: subscribes and updates its data store
const userStore = {};
eventBus.subscribe('UserCreated', (user) => {
  userStore[user.userId] = user;
  console.log('User stored in Service B:', user);
});

// Simulate user creation in Service A
const newUser = { userId: '123', name: 'Alice' };
eventBus.publish({ eventType: 'UserCreated', data: newUser });

Output

User stored in Service B: { userId: '123', name: 'Alice' }

Common Pitfalls

• Strong coupling: Direct synchronous calls create tight dependencies and reduce fault tolerance.
• Data inconsistency: Without eventual consistency patterns, data can get out of sync.
• Missing retries: Not handling message failures or retries leads to lost updates.
• Ordering issues: Events arriving out of order can cause incorrect state.

Use idempotent event handlers and dead-letter queues to handle failures gracefully.

javascript

/* Wrong: Direct synchronous call causing tight coupling */
async function updateUserInServiceB(user) {
  await fetch('http://service-b/api/users/' + user.userId, {
    method: 'PUT',
    body: JSON.stringify(user),
    headers: { 'Content-Type': 'application/json' }
  });
}

/* Right: Publish event asynchronously */
function publishUserCreatedEvent(user) {
  messageBroker.publish('UserCreated', user);
}

Quick Reference

• Event-driven sync: Use message brokers like Kafka, RabbitMQ, or AWS SNS/SQS.
• API sync: Use REST or gRPC for synchronous calls when immediate consistency is needed.
• Idempotency: Ensure event handlers can safely process repeated events.
• Retries and dead-letter queues: Handle failures and avoid data loss.
• Event ordering: Use partitioning or sequence numbers to maintain order.

Key Takeaways

Use event-driven architecture with message brokers for scalable, loosely coupled data syncing.

Avoid tight coupling by minimizing direct synchronous API calls between microservices.

Implement idempotent event handlers to handle repeated or out-of-order events safely.

Use retries and dead-letter queues to handle message delivery failures.

Choose synchronous API calls only when immediate data consistency is critical.