Pagination with first and after in GraphQL - Deep Dive

Overview - Pagination with first and after

What is it?

Pagination with first and after is a way to get a part of a list of items step-by-step. Instead of getting all items at once, you ask for the first few items after a certain point. This helps when you have many items and want to load them bit by bit. It is common in GraphQL APIs to handle large lists efficiently.

Why it matters

Without pagination, loading large lists can be slow and use too much memory or data. Imagine scrolling a long list on your phone; if all items loaded at once, it would be slow and hard to use. Pagination with first and after solves this by loading only what you need, making apps faster and smoother.

Where it fits

Before learning this, you should understand basic GraphQL queries and how lists work. After this, you can learn about other pagination methods like offset-based or cursor-based pagination, and how to combine pagination with filtering and sorting.

Mental Model

Core Idea

Pagination with first and after fetches a limited number of items starting just after a known position, letting you move through a list step-by-step.

Think of it like...

It's like reading a book page by page: you start at the beginning or after a certain page, and read only a few pages at a time instead of the whole book.

List of items:
┌─────┬─────┬─────┬─────┬─────┬─────┐
│  1  │  2  │  3  │  4  │  5  │  6  │
└─────┴─────┴─────┴─────┴─────┴─────┘

Query: first=2, after=cursor for item 2
Result: items 3 and 4

Query: first=2, after=cursor for item 4
Result: items 5 and 6

Build-Up - 7 Steps

1

Foundation

Understanding basic pagination

Concept: Pagination means splitting a big list into smaller parts to load step-by-step.

Imagine you have 100 items but only want to see 10 at a time. Pagination lets you ask for the first 10, then the next 10, and so on. This avoids loading everything at once, which can be slow or use too much data.

Result

You get smaller chunks of the list, making loading faster and easier to handle.

Understanding pagination is key to managing large lists efficiently in any app or API.

2

Foundation

What are cursors in pagination?

Concept: Cursors are markers that point to a specific place in a list to continue from.

Instead of using page numbers, cursor pagination uses a unique identifier (cursor) for each item. When you ask for items after a cursor, you get the next items starting just after that marker.

Result

You can move through the list reliably even if items change, because cursors track exact positions.

Knowing cursors helps avoid problems with changing data that page numbers can cause.

3

Intermediate

Using 'first' to limit items returned

🤔Before reading on: do you think 'first' means the total items or the items after the cursor? Commit to your answer.

Concept: 'first' tells how many items to get starting after the cursor.

In GraphQL, you write a query with 'first' to say how many items you want. For example, first: 5 means get 5 items. If you also use 'after', it means get 5 items after that cursor.

Result

The query returns exactly the number of items requested, starting after the cursor if given.

Understanding 'first' lets you control how much data you get each time, improving performance and user experience.

4

Intermediate

How 'after' cursor controls pagination start

🤔Before reading on: do you think 'after' includes the cursor item or starts after it? Commit to your answer.

Concept: 'after' specifies the cursor after which to start fetching items.

'after' is a cursor string that marks the last item you saw. The query returns items starting just after this cursor. This way, you avoid repeating items and move forward in the list.

Result

You get the next set of items after the cursor, enabling smooth scrolling or loading more data.

Knowing how 'after' works prevents duplicate data and helps build seamless user interfaces.

5

Intermediate

Combining 'first' and 'after' in queries

Concept: Using both 'first' and 'after' together lets you fetch a slice of the list starting after a known point.

A typical GraphQL query looks like this: { items(first: 3, after: "cursor123") { edges { node { id name } cursor } pageInfo { hasNextPage endCursor } } } This asks for 3 items after the cursor 'cursor123'. The response includes items and info to continue pagination.

Result

You get a small list of items and a new cursor to fetch the next batch.

Combining 'first' and 'after' is the core of cursor-based pagination, enabling efficient data loading.

6

Advanced

Handling pageInfo for pagination control

🤔Before reading on: do you think pageInfo is optional or required for pagination? Commit to your answer.

Concept: pageInfo provides metadata about the pagination state, like if more items exist.

The pageInfo object in the response tells you: - hasNextPage: if more items are available - endCursor: the cursor of the last item returned This helps the client know if it can load more and where to start next.

Result

Clients can build 'Load More' buttons or infinite scroll knowing when to stop.

Using pageInfo prevents unnecessary queries and improves user experience by signaling list ends.

7

Expert

Why cursor pagination beats offset pagination

🤔Before reading on: do you think offset pagination or cursor pagination handles data changes better? Commit to your answer.

Concept: Cursor pagination is more reliable when data changes during pagination compared to offset pagination.

Offset pagination uses page numbers and skips, which can cause missing or duplicate items if the list changes (items added or removed). Cursor pagination uses stable cursors tied to items, so even if the list changes, you continue from the correct place without skipping or repeating items.

Result

Pagination stays consistent and accurate even with live data updates.

Understanding this helps build robust apps that handle real-world data changes gracefully.

Under the Hood

Cursor pagination works by encoding a unique position marker (cursor) for each item, often based on a unique ID or timestamp. When a query requests items after a cursor, the server decodes the cursor to find the exact position in the data. It then fetches the next 'first' items starting from that position. This avoids scanning from the start every time and handles data changes better than offset-based methods.

Why designed this way?

Cursor pagination was designed to solve problems with offset pagination, such as inconsistent results when data changes and performance issues with large offsets. By using cursors, the system can quickly jump to the right place in the data without counting or skipping many items. This design improves speed, accuracy, and user experience in APIs.

Client Query:
┌───────────────┐
│ first: 3      │
│ after: cursor │
└──────┬────────┘
       │
       ▼
Server:
┌─────────────────────────────┐
│ Decode cursor to position    │
│ Fetch next 3 items from pos  │
│ Return items + new endCursor │
└─────────────┬───────────────┘
              │
              ▼
Client receives items and pageInfo
Uses endCursor for next 'after' query

Myth Busters - 4 Common Misconceptions

Quick: Does 'after' include the item at the cursor or start after it? Commit to your answer.

Common Belief: 'after' includes the item at the cursor in the results.

Reality: 'after' starts fetching items strictly after the cursor, excluding the cursor item itself.

Why it matters: Including the cursor item causes duplicates in paginated results, confusing users and breaking UI logic.

Quick: Is 'first' the total number of items returned or the total items in the list? Commit to your answer.

Common Belief: 'first' means the total number of items in the entire list.

Reality: 'first' limits how many items are returned in this query, not the total list size.

Why it matters: Misunderstanding 'first' leads to wrong expectations and inefficient queries that fetch too much or too little data.

Quick: Can cursor pagination cause missing or duplicate items if data changes? Commit to your answer.

Common Belief: Cursor pagination can cause missing or duplicate items when data changes during pagination.

Reality: Cursor pagination is designed to avoid missing or duplicate items even if data changes, unlike offset pagination.

Why it matters: Believing cursor pagination is unreliable may lead developers to avoid it and use less efficient methods.

Quick: Is pageInfo optional in cursor pagination? Commit to your answer.

Common Belief: pageInfo is optional and not important for pagination.

Reality: pageInfo is essential to know if more items exist and to get the next cursor for continued pagination.

Why it matters: Ignoring pageInfo can cause infinite loading or stopping too early, harming user experience.

Expert Zone

1

Cursors should be opaque and stable; exposing raw IDs can leak internal data and cause security issues.

2

The choice of cursor encoding (e.g., base64 of ID or timestamp) affects performance and usability.

3

Handling deletions or insertions between paginated queries requires careful cursor design to avoid skipping or repeating items.

When NOT to use

Cursor pagination is not ideal when you need random access to arbitrary pages by number. In such cases, offset pagination or keyset pagination with sorting might be better. Also, if the dataset is small, simple pagination without cursors may suffice.

Production Patterns

In production, cursor pagination is used in APIs for social media feeds, product listings, and messaging apps. It is combined with filtering and sorting to provide smooth infinite scroll experiences. Servers often cache cursors and optimize queries to handle large datasets efficiently.

Connections

Offset Pagination

Alternative pagination method using page numbers and offsets.

Understanding offset pagination helps appreciate why cursor pagination is more reliable with changing data.

REST API Pagination

Pagination is a common pattern in REST APIs as well as GraphQL.

Knowing pagination in REST helps understand how GraphQL improves on it with cursors and flexible queries.

Book Reading

Both involve reading or accessing data in parts, sequentially.

Recognizing pagination as similar to reading pages helps grasp the concept intuitively and design user-friendly interfaces.

Common Pitfalls

#1 Using 'after' cursor but including the cursor item again in results.

Wrong approach:

{ items(first: 3, after: "cursor123") { edges { node { id } } } } // returns cursor123 item again

Correct approach:

{ items(first: 3, after: "cursor123") { edges { node { id } } } } // returns items strictly after cursor123

Root cause:

Misunderstanding that 'after' excludes the cursor item, causing duplicates.

#2 Ignoring pageInfo and not checking if more items exist.

Wrong approach:

Fetching items with first and after but never checking hasNextPage, causing infinite loading.

Correct approach:

Use pageInfo { hasNextPage endCursor } to know when to stop fetching more items.

Root cause:

Not using pagination metadata leads to poor user experience and wasted resources.

#3 Using offset pagination for large, frequently changing datasets.

Wrong approach:

Querying with offset and limit in GraphQL or database, causing inconsistent results.

Correct approach:

Use cursor pagination with first and after to handle data changes reliably.

Root cause:

Not knowing offset pagination's limitations with live data causes bugs and confusion.

Key Takeaways

Pagination with first and after lets you fetch small parts of a list step-by-step, improving performance and user experience.

Cursors mark exact positions in the list, making pagination reliable even when data changes.

'first' controls how many items to fetch, and 'after' tells where to start after a known item.

pageInfo provides essential info about more data availability and the next cursor to continue pagination.

Cursor pagination is preferred over offset pagination for large or changing datasets because it avoids duplicates and missing items.