REST API design for systems in HLD - Deep Dive

Overview - REST API design for systems

What is it?

REST API design is about creating clear rules and structures for how different software systems talk to each other over the internet. It uses simple web methods like GET, POST, PUT, and DELETE to let systems read, create, update, or remove data. REST APIs organize data into resources, each with a unique address called a URL. This makes it easy for different programs to work together smoothly.

Why it matters

Without well-designed REST APIs, software systems would struggle to communicate clearly, leading to confusion, errors, and slow development. Good REST API design makes it easy to build, maintain, and scale systems that work together, saving time and money. It also helps users and developers understand how to use the system quickly, improving overall experience and reliability.

Where it fits

Before learning REST API design, you should understand basic web concepts like HTTP methods and URLs. After mastering REST API design, you can explore advanced topics like API security, versioning, and microservices architecture.

Mental Model

Core Idea

A REST API is like a well-organized library where each book (resource) has a clear address (URL) and you use simple actions (HTTP methods) to read, add, update, or remove books.

Think of it like...

Imagine a restaurant menu where each dish is a resource you can order, change, or remove from your order using simple commands like 'get me this dish', 'add this dish', or 'remove this dish'. The menu is organized so everyone understands how to order without confusion.

┌───────────────┐       ┌───────────────┐
│   Client      │──────▶│   REST API    │
└───────────────┘       └───────────────┘
        │                      │
        │ HTTP Request         │
        │ (GET, POST, PUT, DELETE)
        │                      │
        ▼                      ▼
┌───────────────┐       ┌───────────────┐
│   Resource    │◀─────│  Server Logic │
│ (Data Entity) │       └───────────────┘
└───────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding HTTP Basics

Concept: Learn the basic HTTP methods and how they map to actions on data.

HTTP methods are simple commands used by clients to communicate with servers. The main ones are GET (to read data), POST (to create data), PUT (to update data), and DELETE (to remove data). Each method has a clear purpose and rules about what it should do.

Result

You can identify which HTTP method to use for common actions like fetching a list, adding a new item, or deleting an entry.

Understanding HTTP methods is essential because REST APIs rely on these simple commands to perform all operations on resources.

2

Foundation

Resources and URLs

Concept: Learn how data is organized as resources, each with a unique URL.

In REST, everything you want to work with is a resource, like users, products, or orders. Each resource has a unique URL, for example, /users/123 refers to user number 123. This URL acts like an address to find or change that resource.

Result

You can design URLs that clearly identify resources and their relationships.

Organizing data as resources with clear URLs makes APIs intuitive and easy to navigate.

3

Intermediate

Stateless Communication

🤔Before reading on: Do you think REST APIs remember previous requests or treat each request independently? Commit to your answer.

Concept: REST APIs treat each request independently without storing client state on the server.

Stateless means the server does not keep any information about past requests from the client. Each request must contain all information needed to understand and process it. This simplifies server design and improves scalability.

Result

You design APIs where clients send all necessary data each time, avoiding hidden dependencies.

Knowing REST is stateless helps prevent bugs caused by unexpected server memory and makes scaling easier.

4

Intermediate

Using HTTP Status Codes

🤔Before reading on: Do you think HTTP status codes are optional or essential for REST APIs? Commit to your answer.

Concept: HTTP status codes communicate the result of a request clearly to the client.

Status codes like 200 (OK), 201 (Created), 400 (Bad Request), and 404 (Not Found) tell the client what happened. Using them correctly helps clients handle responses properly and improves debugging.

Result

You can design APIs that clearly signal success, errors, or missing data.

Proper use of status codes creates clear communication between client and server, reducing confusion.

5

Intermediate

Designing Resource Relationships

Concept: Learn how to represent connections between resources using URLs and nested paths.

Resources often relate to each other, like orders belonging to users. You can show this in URLs, for example, /users/123/orders lists orders for user 123. This helps clients understand data structure and access related information easily.

Result

You create intuitive URLs that reflect real-world relationships between data.

Representing relationships in URLs improves API clarity and usability.

6

Advanced

Handling API Versioning

🤔Before reading on: Should API changes break old clients or support multiple versions? Commit to your answer.

Concept: Versioning allows APIs to evolve without breaking existing clients.

You can include version numbers in URLs (e.g., /v1/users) or headers. This lets you add features or fix bugs while keeping old versions working. Planning versioning early avoids costly rewrites later.

Result

You can update APIs safely and support many clients over time.

Understanding versioning protects your system from breaking changes and supports growth.

7

Expert

Balancing REST Purity and Practical Needs

🤔Before reading on: Do you think strict REST rules always make the best APIs? Commit to your answer.

Concept: Sometimes practical needs require bending REST rules for performance or simplicity.

Strict REST means using only standard HTTP methods and statelessness, but real systems may need batch requests, caching strategies, or custom actions. Experts balance REST principles with real-world constraints to build efficient APIs.

Result

You design APIs that are both clean and practical, avoiding unnecessary complexity.

Knowing when to adapt REST principles leads to better, more maintainable systems in production.

Under the Hood

REST APIs work by clients sending HTTP requests to server endpoints identified by URLs. The server processes these requests using logic that maps HTTP methods to actions on data stored in databases or other storage. Each request is independent, so the server does not keep client session data. Responses include status codes and data in formats like JSON. This stateless, resource-oriented design allows servers to handle many clients efficiently and scale horizontally.

Why designed this way?

REST was designed to use the existing web infrastructure simply and effectively. By leveraging HTTP methods and URLs, it avoids inventing new protocols. Statelessness was chosen to simplify server design and improve scalability. Alternatives like SOAP were more complex and harder to scale. REST’s simplicity and alignment with web standards made it popular for building scalable, maintainable APIs.

┌───────────────┐       ┌───────────────┐       ┌───────────────┐
│   Client      │──────▶│   REST API    │──────▶│   Database    │
│ (Browser/App) │ HTTP  │ (Server Logic)│ Query │ (Data Store)  │
└───────────────┘       └───────────────┘       └───────────────┘
        ▲                      │                      │
        │                      │                      │
        └──────────────────────┴──────────────────────┘
                 HTTP Response (Status + Data)

Myth Busters - 4 Common Misconceptions

Quick: Does REST require using only JSON for data exchange? Commit to yes or no.

Common Belief: REST APIs must always use JSON format for data exchange.

Reality: REST is a style, not a strict protocol, so it can use any data format like XML, plain text, or even images, as long as the client and server agree.

Why it matters: Assuming JSON is mandatory limits flexibility and can cause problems when integrating with systems that use other formats.

Quick: Do you think REST APIs must be completely stateless with no exceptions? Commit to yes or no.

Common Belief: REST APIs cannot store any client state on the server between requests.

Reality: While REST encourages statelessness, some practical systems use tokens or sessions for authentication, which is a controlled form of state.

Why it matters: Believing in absolute statelessness can lead to ignoring necessary security or performance features.

Quick: Is it true that REST APIs must always use nouns in URLs and never verbs? Commit to yes or no.

Common Belief: REST URLs should only contain nouns representing resources, never verbs or actions.

Reality: While REST prefers nouns, sometimes verbs are used for actions that don't fit CRUD operations, like /users/123/reset-password.

Why it matters: Rigidly avoiding verbs can make APIs awkward or force unnatural designs.

Quick: Do you think REST APIs automatically guarantee security? Commit to yes or no.

Common Belief: Using REST automatically makes an API secure.

Reality: REST is about design style, not security. Security must be added separately using authentication, authorization, and encryption.

Why it matters: Assuming REST equals security can lead to vulnerable systems.

Expert Zone

1

Understanding that hypermedia controls (HATEOAS) can make REST APIs self-descriptive but are rarely fully implemented in practice.

2

Knowing that caching strategies using HTTP headers can drastically improve performance but require careful design to avoid stale data.

3

Appreciating that designing APIs for evolvability means anticipating future changes and minimizing breaking changes through careful resource modeling.

When NOT to use

REST APIs are not ideal when you need real-time communication or complex transactions; alternatives like WebSockets or gRPC may be better. Also, for highly interconnected data queries, GraphQL can be more efficient than REST.

Production Patterns

In production, REST APIs often use layered architecture with gateways, load balancers, and microservices. They implement authentication with tokens (OAuth), use pagination for large data sets, and apply rate limiting to protect resources.

Connections

Microservices Architecture

REST APIs are commonly used as communication interfaces between microservices.

Understanding REST API design helps build clear, scalable microservices that interact smoothly.

HTTP Protocol

REST builds directly on HTTP methods and status codes.

Knowing HTTP deeply improves REST API design and troubleshooting.

Library Cataloging Systems

Both organize items (books or data) with unique identifiers and structured access methods.

Seeing REST as a cataloging system clarifies resource organization and retrieval.

Common Pitfalls

#1 Using verbs in URLs for standard CRUD operations.

Wrong approach:

POST /createUser
GET /getUser/123
PUT /updateUser/123
DELETE /deleteUser/123

Correct approach:

POST /users
GET /users/123
PUT /users/123
DELETE /users/123

Root cause:

Misunderstanding that URLs should represent resources (nouns), not actions (verbs), leading to inconsistent API design.

#2 Ignoring HTTP status codes and always returning 200 OK.

Wrong approach:

Always respond with HTTP 200 even when errors occur, putting error details only in the response body.

Correct approach:

Use appropriate status codes like 400 for bad requests, 404 for not found, and 500 for server errors.

Root cause:

Not leveraging HTTP protocol features for clear communication, causing clients to misinterpret responses.

#3 Storing client session state on the server between requests.

Wrong approach:

Server keeps user login state in memory between API calls, requiring sticky sessions.

Correct approach:

Use stateless tokens like JWT passed with each request to authenticate without server memory.

Root cause:

Misunderstanding REST statelessness principle, leading to scalability and reliability issues.

Key Takeaways

REST API design organizes data as resources accessed via URLs using standard HTTP methods.

Stateless communication and proper use of HTTP status codes make APIs scalable and clear.

Designing intuitive URLs and handling resource relationships improves usability.

Versioning and practical adaptations ensure APIs evolve without breaking clients.

Understanding REST deeply helps build robust, maintainable, and scalable systems.