Overview 

In v1.5, support for pagination was added to several open-ended” GET endpoints that were both highly used and tended to return the largest data sets. While a single open-ended GET call is efficient from the client’s perspective, this is not the case at scale. When many clients make open-ended GET calls, job queues can get backed up, leading to poor response time for all API users. Like time-sliced multi-threading, breaking large jobs into small pieces allows many simultaneous requests to be serviced while maintaining responsiveness.

Best Practices 

When paginating through a dataset that is constantly being updated, there is a risk that returned records will differ from the database. It is possible that records are altered during pagination shift position in the record set, causing them to be returned on more than one page, or in extreme cases not be returned at all.
Two new attributes: “dateAdded” and “dateEdited” have been added to resources returned from paginated calls. Not only do they enable highly targeted filtering, but they also can be used to reduce the risk of data accuracy issues described above.

Here are two commonly used methods:

1. Frozen Snapshot
To closely emulate the behavior of an open-ended GET call, use filtering to return only records added or updated before you made the first pagination call. Like a non-paginated open-ended call, anything added after pagination starts will not be included.

2. Sorting Oldest to Newest
To reduce the risk of excluding added records, use sorting to return the oldest records first (sorting on “dateAdded”), and any records added after paginated started should show up on the last page(s). It’s a best practice to never sort results from newest to oldest.
If you have any questions or feedback about this article, please contact your Customer Success Manager or Account Manager for further assistance.