Environment-based configuration in Microservices - System Design Exercise

Design: Environment-based Configuration Management for Microservices

Design focuses on the configuration management system and its integration with microservices. It excludes microservice business logic and deployment pipelines.

Functional Requirements

FR1: Support multiple environments such as development, testing, staging, and production

FR2: Allow dynamic configuration changes without redeploying microservices

FR3: Secure sensitive configuration data like API keys and database passwords

FR4: Provide a centralized configuration management system accessible by all microservices

FR5: Ensure configuration consistency and version control across environments

FR6: Support rollback to previous configuration versions if needed

FR7: Allow microservices to fetch configuration at startup and refresh periodically or on demand

Non-Functional Requirements

NFR1: Handle up to 100 microservices with independent configurations

NFR2: Configuration fetch latency should be under 100ms

NFR3: System availability target of 99.9% uptime

NFR4: Configuration data size per microservice should not exceed 1MB

NFR5: Secure communication between microservices and configuration system using TLS

NFR6: Support at least 10 configuration updates per minute without performance degradation

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Centralized configuration server or service

Configuration storage (database or key-value store)

Secure secret management system

Configuration client libraries for microservices

API gateway or proxy for secure access

Version control and audit logging system

Design Patterns

12-Factor App configuration pattern

Client-side caching with refresh tokens

Feature flag management

Secrets management integration

Blue-green or canary configuration rollout

Reference Architecture

                    +-------------------------+
                    |  Configuration Management|
                    |         Service          |
                    +-----------+-------------+
                                |
               +----------------+----------------+
               |                                 |
       +-------v-------+                 +-------v-------+
       | Configuration |                 | Secret Vault  |
       |   Database    |                 |  (e.g., Vault) |
       +---------------+                 +---------------+
               |                                 |
               +----------------+----------------+
                                |
                    +-----------v-------------+
                    |  Microservices Cluster   |
                    |  (100+ independent apps) |
                    +-------------------------+

Components

Configuration Management Service

Spring Cloud Config Server / Consul / etcd

Central service to store, serve, and manage configuration data for all microservices.

Configuration Storage

Relational DB (PostgreSQL) or Key-Value Store (etcd, Consul KV)

Persist configuration data with versioning and environment separation.

Secret Vault

HashiCorp Vault or AWS Secrets Manager

Secure storage and access control for sensitive configuration like passwords and API keys.

Configuration Client Library

Custom SDK or Spring Cloud Config Client

Used by microservices to fetch, cache, and refresh configuration securely.

API Gateway / Proxy

NGINX / Envoy

Secure and control access to configuration service endpoints.

Audit and Version Control

Git-backed storage or database audit logs

Track configuration changes, enable rollback, and provide history.

Request Flow

1. 1. Configuration admin updates configuration via UI or API to Configuration Management Service.

2. 2. Configuration Management Service stores data in Configuration Storage with version and environment tags.

3. 3. Sensitive data is stored or referenced securely in Secret Vault.

4. 4. Microservices start and use Configuration Client Library to fetch environment-specific configuration from Configuration Management Service.

5. 5. Configuration Client caches data locally and periodically refreshes or listens for change notifications.

6. 6. Configuration Client retrieves secrets securely from Secret Vault when needed.

7. 7. On configuration update, Configuration Management Service notifies microservices or they pull updates on schedule.

8. 8. Audit logs record all configuration changes for traceability and rollback.

Database Schema

Entities: - ConfigurationEntry: id, key, value, environment, version, created_at - Environment: id, name (dev, test, staging, prod) - SecretReference: id, key, vault_path, environment - AuditLog: id, config_entry_id, changed_by, change_type, timestamp Relationships: - ConfigurationEntry belongs to one Environment - SecretReference belongs to one Environment - AuditLog references ConfigurationEntry

Scaling Discussion

Bottlenecks

Configuration Management Service becomes a single point of failure under high load

Latency increases when many microservices fetch configuration simultaneously

Secret Vault throughput limits when many microservices request secrets

Configuration Storage performance degradation with large number of versions and entries

Network overhead for frequent configuration refreshes

Solutions

Deploy Configuration Management Service in a highly available cluster with load balancing

Implement client-side caching and exponential backoff for configuration fetches

Use secret caching and token renewal strategies to reduce Vault load

Archive old configuration versions and optimize database indexing

Use push-based configuration updates (e.g., Webhooks or message queues) instead of polling

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain importance of environment separation and secure secret management

Describe how centralized configuration improves consistency and reduces errors

Discuss caching strategies to reduce latency and load

Highlight security measures for sensitive data

Show awareness of scaling challenges and mitigation techniques

Mention audit and version control for safe configuration changes