Environment-based configuration in Microservices - System Design Exercise

Design: Environment-based Configuration Management for Microservices

Design focuses on the configuration management system and its integration with microservices. It excludes microservice business logic and deployment pipelines.

Functional Requirements

FR1: Support multiple environments such as development, testing, staging, and production

FR2: Allow dynamic configuration changes without redeploying microservices

FR3: Secure sensitive configuration data like API keys and database passwords

FR4: Provide a centralized configuration management system accessible by all microservices

FR5: Ensure configuration consistency and version control across environments

FR6: Support rollback to previous configuration versions if needed

FR7: Allow microservices to fetch configuration at startup and refresh periodically or on demand

Non-Functional Requirements

NFR1: Handle up to 100 microservices with independent configurations

NFR2: Configuration fetch latency should be under 100ms

NFR3: System availability target of 99.9% uptime

NFR4: Configuration data size per microservice should not exceed 1MB

NFR5: Secure communication between microservices and configuration system using TLS

NFR6: Support at least 10 configuration updates per minute without performance degradation

Think Before You Design

Questions to Ask

❓ Question 1

❓ Question 2

❓ Question 3

❓ Question 4

❓ Question 5

❓ Question 6

Key Components

Centralized configuration server or service

Configuration storage (database or key-value store)

Secure secret management system

Configuration client libraries for microservices

API gateway or proxy for secure access

Version control and audit logging system

Design Patterns

12-Factor App configuration pattern

Client-side caching with refresh tokens

Feature flag management

Secrets management integration

Blue-green or canary configuration rollout

Reference Architecture

                    +-------------------------+
                    |  Configuration Management|
                    |         Service          |
                    +-----------+-------------+
                                |
               +----------------+----------------+
               |                                 |
       +-------v-------+                 +-------v-------+
       | Configuration |                 | Secret Vault  |
       |   Database    |                 |  (e.g., Vault) |
       +---------------+                 +---------------+
               |                                 |
               +----------------+----------------+
                                |
                    +-----------v-------------+
                    |  Microservices Cluster   |
                    |  (100+ independent apps) |
                    +-------------------------+

Components

Configuration Management Service

Spring Cloud Config Server / Consul / etcd

Central service to store, serve, and manage configuration data for all microservices.

Configuration Storage

Relational DB (PostgreSQL) or Key-Value Store (etcd, Consul KV)

Persist configuration data with versioning and environment separation.

Secret Vault

HashiCorp Vault or AWS Secrets Manager

Secure storage and access control for sensitive configuration like passwords and API keys.

Configuration Client Library

Custom SDK or Spring Cloud Config Client

Used by microservices to fetch, cache, and refresh configuration securely.

API Gateway / Proxy

NGINX / Envoy

Secure and control access to configuration service endpoints.

Audit and Version Control

Git-backed storage or database audit logs

Track configuration changes, enable rollback, and provide history.

Request Flow

1. 1. Configuration admin updates configuration via UI or API to Configuration Management Service.

2. 2. Configuration Management Service stores data in Configuration Storage with version and environment tags.

3. 3. Sensitive data is stored or referenced securely in Secret Vault.

4. 4. Microservices start and use Configuration Client Library to fetch environment-specific configuration from Configuration Management Service.

5. 5. Configuration Client caches data locally and periodically refreshes or listens for change notifications.

6. 6. Configuration Client retrieves secrets securely from Secret Vault when needed.

7. 7. On configuration update, Configuration Management Service notifies microservices or they pull updates on schedule.

8. 8. Audit logs record all configuration changes for traceability and rollback.

Database Schema

Entities: - ConfigurationEntry: id, key, value, environment, version, created_at - Environment: id, name (dev, test, staging, prod) - SecretReference: id, key, vault_path, environment - AuditLog: id, config_entry_id, changed_by, change_type, timestamp Relationships: - ConfigurationEntry belongs to one Environment - SecretReference belongs to one Environment - AuditLog references ConfigurationEntry

Scaling Discussion

Bottlenecks

Configuration Management Service becomes a single point of failure under high load

Latency increases when many microservices fetch configuration simultaneously

Secret Vault throughput limits when many microservices request secrets

Configuration Storage performance degradation with large number of versions and entries

Network overhead for frequent configuration refreshes

Solutions

Deploy Configuration Management Service in a highly available cluster with load balancing

Implement client-side caching and exponential backoff for configuration fetches

Use secret caching and token renewal strategies to reduce Vault load

Archive old configuration versions and optimize database indexing

Use push-based configuration updates (e.g., Webhooks or message queues) instead of polling

Interview Tips

Time: Spend 10 minutes understanding requirements and clarifying scope, 20 minutes designing architecture and data flow, 10 minutes discussing scaling and trade-offs, 5 minutes summarizing.

Explain importance of environment separation and secure secret management

Describe how centralized configuration improves consistency and reduces errors

Discuss caching strategies to reduce latency and load

Highlight security measures for sensitive data

Show awareness of scaling challenges and mitigation techniques

Mention audit and version control for safe configuration changes

Practice

1. What is the main purpose of environment-based configuration in microservices?

easy

A. To store configuration only in the database

B. To hardcode all settings inside the service code

C. To make services dependent on a single environment

D. To separate configuration settings from code for flexibility

Solution

1. Step 1: Understand configuration separation

   Environment-based configuration means keeping settings like URLs, keys, and flags outside the code so they can change without rewriting code.

2. Step 2: Identify the benefit in microservices

   This separation allows microservices to adapt easily to different environments (development, testing, production) without code changes.

3. Final Answer:

   To separate configuration settings from code for flexibility -> Option D

4. Quick Check:

   Configuration separation = Flexibility [OK]

Hint: Configuration outside code means flexibility across environments [OK]

Common Mistakes:

• Thinking configuration must be hardcoded
• Assuming one config fits all environments
• Storing config only in databases

2. Which of the following is the correct way to access an environment variable named DB_HOST in a microservice using Node.js?

easy

A. process.env.DB_HOST

B. env.DB_HOST()

C. getEnv('DB_HOST')

D. System.getenv('DB_HOST')

Solution

1. Step 1: Recall Node.js environment variable syntax

   In Node.js, environment variables are accessed via the global object process.env.

2. Step 2: Match the correct syntax

   The correct way to get DB_HOST is process.env.DB_HOST. Other options are invalid or from other languages.

3. Final Answer:

   process.env.DB_HOST -> Option A

4. Quick Check:

   Node.js env var = process.env.VAR [OK]

Hint: Node.js env vars use process.env.VAR_NAME [OK]

Common Mistakes:

• Using function calls like getEnv() which don't exist
• Confusing syntax with other languages like Java
• Trying to access env vars without process.env

3. Given this Python snippet in a microservice startup:

import os

env = os.getenv('ENVIRONMENT', 'development')
if env == 'production':
    db_url = os.getenv('PROD_DB_URL')
else:
    db_url = os.getenv('DEV_DB_URL')
print(db_url)

What will be printed if ENVIRONMENT is not set and DEV_DB_URL is set to "localhost:5432/dev"?

medium

A. "localhost:5432/prod"

B. None

C. "localhost:5432/dev"

D. Error: ENVIRONMENT not set

Solution

1. Step 1: Check default environment value

   The code uses os.getenv('ENVIRONMENT', 'development'), so if ENVIRONMENT is missing, it defaults to 'development'.

2. Step 2: Determine which DB URL is selected

   Since env is 'development', the else branch runs, setting db_url to os.getenv('DEV_DB_URL'), which is "localhost:5432/dev".

3. Final Answer:

   "localhost:5432/dev" -> Option C

4. Quick Check:

   Default env 'development' selects DEV_DB_URL [OK]

Hint: Default env triggers DEV_DB_URL print [OK]

Common Mistakes:

• Assuming ENVIRONMENT must be set or error occurs
• Confusing production and development branches
• Expecting None if variable missing

4. A microservice fails to load its configuration from environment variables and crashes. Which is the most likely cause?

medium

A. The service code has a syntax error unrelated to config

B. Environment variables were not set before service startup

C. The service uses hardcoded values instead of env vars

D. The database is down

Solution

1. Step 1: Understand environment variable loading

   Microservices read environment variables at startup. If variables are missing, config loading fails.

2. Step 2: Identify cause of crash

   If env vars are not set before starting, the service cannot find needed config and may crash or error out.

3. Final Answer:

   Environment variables were not set before service startup -> Option B

4. Quick Check:

   Missing env vars cause config load failure [OK]

Hint: Set env vars before starting service [OK]

Common Mistakes:

• Blaming unrelated syntax errors
• Assuming hardcoded values cause crashes
• Confusing database issues with config loading

5. You have a microservice deployed in three environments: development, staging, and production. You want to use environment-based configuration to manage database URLs securely and avoid code changes. Which approach is best?

hard

A. Use environment variables for DB URLs and load them at startup

B. Store all DB URLs in code and comment/uncomment per environment

C. Use a single DB URL for all environments to simplify config

D. Hardcode production DB URL and pass dev/staging URLs as query params

Solution

1. Step 1: Evaluate configuration management options

   Hardcoding or commenting code per environment is error-prone and unsafe. Using a single DB URL ignores environment differences.

2. Step 2: Choose secure, flexible best practice

   Using environment variables allows each environment to have its own DB URL securely without code changes, loaded at service startup.

3. Final Answer:

   Use environment variables for DB URLs and load them at startup -> Option A

4. Quick Check:

   Env vars for config = secure + flexible [OK]

Hint: Env vars separate config from code for all environments [OK]

Common Mistakes:

• Hardcoding config in code for each environment
• Using same DB URL everywhere ignoring environment needs
• Passing sensitive info in query parameters