Problem Details (RFC 7807) format in Rest API - Deep Dive

Overview - Problem Details (RFC 7807) format

What is it?

Problem Details (RFC 7807) is a standard way to format error responses in web APIs. It defines a simple JSON structure to describe errors clearly and consistently. This helps clients understand what went wrong and how to fix it. The format includes fields like type, title, status, detail, and instance.

Why it matters

Without a standard error format, APIs return errors in many different ways, making it hard for clients to handle them properly. This inconsistency causes confusion, bugs, and poor user experience. RFC 7807 solves this by providing a clear, uniform way to communicate problems, improving interoperability and debugging.

Where it fits

Learners should know basic REST API concepts and HTTP status codes before this. After understanding RFC 7807, they can learn advanced API error handling, custom error types, and client-side error processing.

Mental Model

Core Idea

Problem Details format is a universal error message template that makes API errors easy to understand and handle by using a fixed set of descriptive fields.

Think of it like...

It's like a standardized accident report form used by police everywhere, so everyone knows exactly what information to expect and how to read it, no matter where the accident happened.

┌───────────────────────────────┐
│         Problem Details        │
├─────────────┬─────────────────┤
│ Field       │ Description     │
├─────────────┼─────────────────┤
│ type        │ URI identifying the error type
│ title       │ Short, human-readable summary
│ status      │ HTTP status code
│ detail      │ Detailed error explanation
│ instance    │ URI identifying this specific occurrence
└─────────────┴─────────────────┘

Build-Up - 7 Steps

1

Foundation

Understanding API Error Responses

Concept: Learn what API error responses are and why they matter.

When an API request fails, the server sends back an error response. This response tells the client what went wrong. Usually, it includes an HTTP status code like 404 or 500 and sometimes a message. But without a standard format, these messages vary widely.

Result

You understand that error responses are essential for clients to know why a request failed.

Knowing that error responses communicate failure reasons sets the stage for why a consistent format is needed.

2

Foundation

Basics of HTTP Status Codes

Concept: Learn common HTTP status codes used in error responses.

HTTP status codes are numbers that indicate the result of a request. Codes like 200 mean success, 404 means not found, and 500 means server error. These codes help clients quickly understand the general error type.

Result

You can recognize common HTTP status codes and their meanings.

Understanding status codes helps you see why additional error details are often necessary.

3

Intermediate

Introducing RFC 7807 Problem Details Format

Concept: Learn the structure and purpose of the Problem Details JSON format.

RFC 7807 defines a JSON object with specific fields: 'type' (a URI identifying the error type), 'title' (a short summary), 'status' (HTTP status code), 'detail' (a detailed message), and 'instance' (a URI for this error occurrence). This standardizes error messages.

Result

You can identify and explain the key fields in a Problem Details response.

Knowing the fixed fields helps you create and parse error responses consistently.

4

Intermediate

Creating a Problem Details Response Example

🤔Before reading on: do you think the 'type' field should be a simple string or a URI? Commit to your answer.

Concept: Practice building a real Problem Details JSON response.

Example: { "type": "https://example.com/probs/out-of-credit", "title": "You do not have enough credit.", "status": 403, "detail": "Your current balance is 30, but that costs 50.", "instance": "/account/12345/transactions/abc" } This shows a clear, structured error message.

Result

You can write a valid Problem Details JSON error response.

Seeing a concrete example makes the abstract format tangible and easier to apply.

5

Intermediate

Extending Problem Details with Custom Fields

🤔Before reading on: do you think adding extra fields breaks the RFC 7807 format? Commit to your answer.

Concept: Learn how to add extra information without breaking the standard.

RFC 7807 allows extra fields beyond the five standard ones. For example, you can add 'balance' or 'accounts' fields to provide more context. Clients should ignore unknown fields if they don't understand them.

Result

You know how to customize error responses while staying compatible.

Understanding extensibility helps you provide richer error info without losing interoperability.

6

Advanced

Using Problem Details in Real APIs

🤔Before reading on: do you think all APIs should always use Problem Details for errors? Commit to your answer.

Concept: Explore how Problem Details fits into real-world API design and error handling.

Many APIs use Problem Details for client and server errors to improve clarity. It works well with HTTP status codes and helps automated clients handle errors gracefully. However, some APIs use other formats or none at all, depending on legacy or simplicity needs.

Result

You understand when and how to apply Problem Details in production APIs.

Knowing real-world usage helps you decide when to adopt this standard or adapt it.

7

Expert

Internals and Pitfalls of Problem Details Usage

🤔Before reading on: do you think the 'type' URI must always resolve to a webpage? Commit to your answer.

Concept: Deep dive into design choices, common mistakes, and subtle behaviors of Problem Details.

The 'type' field is a URI but does not have to be dereferenceable; it serves as a unique identifier. Misusing status codes or omitting 'instance' can confuse clients. Also, mixing Problem Details with other error formats in the same API can cause parsing issues.

Result

You can avoid common traps and understand the rationale behind each field.

Understanding these subtleties prevents integration bugs and improves API robustness.

Under the Hood

When an API detects an error, it creates a JSON object with fixed fields defined by RFC 7807. The server sets the HTTP status code and sends this JSON in the response body with the 'application/problem+json' media type. Clients parse this JSON to extract error details systematically.

Why designed this way?

Before RFC 7807, APIs used many different error formats, causing confusion. The standard was designed to be simple, extensible, and compatible with HTTP semantics. Using URIs for 'type' allows unique error identification without forcing web hosting, balancing flexibility and clarity.

┌───────────────┐
│ Client sends  │
│ API request   │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Server detects│
│ error         │
└──────┬────────┘
       │
       ▼
┌───────────────────────────────┐
│ Create Problem Details JSON    │
│ {                             │
│  "type": URI,                │
│  "title": string,            │
│  "status": number,           │
│  "detail": string,           │
│  "instance": URI             │
│ }                             │
└──────┬────────────────────────┘
       │
       ▼
┌───────────────┐
│ Server sends  │
│ HTTP response │
│ with status   │
│ and JSON body │
└──────┬────────┘
       │
       ▼
┌───────────────┐
│ Client parses │
│ Problem      │
│ Details JSON  │
└───────────────┘

Myth Busters - 4 Common Misconceptions

Quick: Does the 'type' field have to be a webpage URL you can visit? Commit yes or no.

Common Belief: The 'type' field must be a URL that points to a webpage explaining the error.

Reality: The 'type' field is a URI used as a unique identifier and does not have to resolve to an actual webpage.

Why it matters: Expecting a webpage can cause confusion or errors if the URI is not hosted, leading to wasted effort or broken links.

Quick: Can you omit the 'status' field if the HTTP status code is already set? Commit yes or no.

Common Belief: Since the HTTP status code is in the response header, the 'status' field in Problem Details is optional.

Reality: The 'status' field is recommended to be included in the JSON to make the error self-contained and easier to process.

Why it matters: Omitting 'status' can make client error handling inconsistent, especially when logs or intermediaries only see the JSON body.

Quick: Is it okay to mix Problem Details format with other error formats in the same API? Commit yes or no.

Common Belief: You can mix different error formats freely in an API as long as clients can handle them.

Reality: Mixing formats causes client confusion and parsing errors; consistency is key for reliable error handling.

Why it matters: Inconsistent error formats increase bugs and maintenance costs, harming user experience.

Quick: Does adding extra fields beyond the standard ones break the Problem Details format? Commit yes or no.

Common Belief: Adding any extra fields is not allowed and breaks the standard.

Reality: RFC 7807 explicitly allows extra fields to provide additional context without breaking compatibility.

Why it matters: Knowing this encourages richer error information while maintaining interoperability.

Expert Zone

1

The 'type' URI acts as a unique error code namespace, enabling clients to programmatically distinguish error kinds without parsing text.

2

Including the 'instance' URI helps trace specific error occurrences, which is invaluable for debugging distributed systems.

3

Clients should gracefully ignore unknown extra fields, allowing APIs to evolve error details without breaking clients.

When NOT to use

Problem Details is not ideal for very simple APIs where plain status codes suffice or for binary protocols without JSON support. Alternatives include custom error formats or HTTP headers for minimal error info.

Production Patterns

In production, APIs often define a catalog of 'type' URIs for common errors, document them for clients, and use middleware to generate Problem Details automatically. They also log 'instance' URIs to correlate errors with logs.

Connections

HTTP Status Codes

Builds-on

Understanding HTTP status codes is essential to grasp why Problem Details includes a 'status' field and how it complements the HTTP response.

JSON Schema

Related standard

JSON Schema can be used to validate Problem Details objects, ensuring error responses conform to the expected structure.

Legal Contracts

Similar pattern

Just like legal contracts use standardized clauses to avoid ambiguity, Problem Details uses fixed fields to prevent confusion in error communication.

Common Pitfalls

#1 Using plain text error messages without structure.

Wrong approach:

HTTP/1.1 404 Not Found
Content-Type: text/plain

User not found

Correct approach:

HTTP/1.1 404 Not Found
Content-Type: application/problem+json

{
"type": "https://example.com/probs/user-not-found",
"title": "User not found",
"status": 404,
"detail": "No user with ID 123 exists.",
"instance": "/users/123"
}

Root cause:

Not using a structured format makes it hard for clients to parse and handle errors programmatically.

#2 Omitting the 'status' field in the Problem Details JSON.

Wrong approach:

{
"type": "https://example.com/probs/out-of-credit",
"title": "Out of credit",
"detail": "Your balance is 0.",
"instance": "/account/123"
}

Correct approach:

{
"type": "https://example.com/probs/out-of-credit",
"title": "Out of credit",
"status": 403,
"detail": "Your balance is 0.",
"instance": "/account/123"
}

Root cause:

Leaving out 'status' reduces clarity and makes error handling inconsistent.

#3 Mixing Problem Details with other error formats in the same API.

Wrong approach:

Some endpoints return Problem Details JSON, others return XML error messages or plain text.

Correct approach:

All endpoints consistently return Problem Details JSON with 'application/problem+json' content type.

Root cause:

Inconsistent formats confuse clients and increase parsing complexity.

Key Takeaways

Problem Details (RFC 7807) is a standardized JSON format for API error responses that improves clarity and consistency.

It uses fixed fields like 'type', 'title', 'status', 'detail', and 'instance' to describe errors clearly.

The 'type' field is a URI identifier and does not have to point to a real webpage.

Extending the format with extra fields is allowed and helps provide richer error context.

Consistent use of Problem Details in APIs reduces bugs and improves client error handling.