I Want It All (Paginated Requests)

When you've queried the first thousand items and maybe you've selected data by a time window, and those little data puddles just aren't cutting it, sometimes you just want it all. Let's talk about paginated requests.

What is Pagination?

Letting people have an unlimited amount of data is a dangerous policy - you've probably noticed that Console let's you have the first 200 documents on the dashboard listing. Applications with very small data sets might not have to worry about this but the Legalesign platform absolutely does.

In compensation for this the platform allows you to get pages of data and lets you know if that is the last page or was a full page of the size you asked for (the maximum limit is 1000 rows).

You'll notice that there is a PageInfo type in the schema. Whenever you're running a connection meta information will be returned in this structure.

# A type that contains general purpose pagination information, see also Edges and https://graphql.org/learn/pagination/.
type PageInfo {
	# When paginating backward, the cursor to continue
	startCursor: String
	# When paginating forward, the cursor to continue
	endCursor: String
	# When paginating forwards, are there more items?
	hasNextPage: Boolean!
	# When paginating backwards, are there more items?
	hasPreviousPage: Boolean!
}

Making THAT Report

Let's use the most common scenario. You want the complete list of unarchived documents for your group so that you can take that information away and grind it up into reports.

info

When you make a paginated query, the Legalesign platform reverts to created order (rather than modified) and you'll need to warn it that you're asking for chunks of data in this order. You do this by passing "0" or "START" in the after connection parameter.

Let's create a query that gets our first 200 documents and warn it this is the start of many calls. Here's what that call will look like:

query listSentDocs {
  group(id: "Z3XXGVtbzE=") {
    documentConnection(
      archived: false
      after:"0"
      first: 200
    ) {
      documents {
        id
        name
        created
      }
      pageInfo {
        startCursor
        endCursor
        hasNextPage
        hasPreviousPage
      }
      totalCount
    }
  }
}

This will return a 200 document collection and if there is another page hasNextPage will be true. Your code can check this and generate another query using the endCursor as the next after parameter. Like this:

info

totalCount on the platform is the total of this page. There's a lot of debate between designers and architects whether this is right - however the sheer cost of telling you how many millions of records each query is part of makes this discussion moot. It's not ideal.

query listSentDocs {
  group(id: "Z3XXGVtbzE=") {
    documentConnection(
      archived: false
      after:"0ZG9jYDI66jMtMjc0MC0xMWYwLTk5MzItMDY5NzZlZmU0MzIx"
      first: 200
    ) {
      documents {
        id
        name
        created
      }
      pageInfo {
        startCursor
        endCursor
        hasNextPage
        hasPreviousPage
      }
      totalCount
    }
  }
}

And we'll keep iterating through calls until we finally get a pageInfo result where hasNextPage is false. You have a potentially very large data set at your disposal for analysis. It's probably time to save that away somewhere, then maybe have a coffee? Nice work.

warning

Some connections in the API haven't yet been upgraded for the latest features of our databases. These won't respond to after requests. Sorry, we're working on it! The same will be true of reverse pagination at the moment - less of a problem if you keep previous pages cached with a utility like TanStack.