Adding Pagination in APIs – Django REST Framework

Imagine you have huge amount of details in your database. Do you think that it is wise to retrieve all at once while making an HTTP GET request? Here comes the importance of the Django REST framework pagination feature. It facilitates splitting the large result set into individual pages of data for each HTTP request.

So when we make an HTTP request, we must specify the details for the specific pages that we want to retrieve, and it will be based on predefined pagination schemes. Apart from retrieving data as pages, it also provides information about the total number of data, the next page, and the previous one in the response section.

• PageNumberPagination
• LimitOffsetPagination
• CursorPagination

Note: You can refer The Browsable API section for Models, Serializers, and Views of Project used in the article

PageNumberPagination

The PageNumberPagination style accepts a single number page number in the request query parameters. To enable this pagination style globally, you can set rest_framework.pagination.PageNumberPagination class to DEFAULT_PAGINATION_CLASS and also set the PAGE_SIZE as desired. You can open the settings.py file and add the below configuration settings.

You can also modify the pagination style by overriding the attributes included in the PageNumberPagination class. Let’s look at the available attributes.

• django_paginator_class – The default is django.core.paginator.Paginator.
• page_size – It indicates the page size (numeric value). If set, this overrides the PAGE_SIZE setting. Defaults to the same value as the PAGE_SIZE settings key.
• page_query_param – The name of the query parameter (string value) to use for the pagination control.
• page_size_query_param – It indicates the name of a query parameter (string value) that allows the client to set the page size on a per-request basis. Defaults to None.
• max_page_size – It indicates the maximum allowable requested page size (numeric value). This attribute is only valid if page_size_query_param is also set.
• last_page_strings – It is used with the page_query_param to request the final page in the set. Defaults to (‘last’,)
• template – The name of a template to use when rendering pagination controls in the browsable API.

Let’s add a few more robot details to our database. The HTTPie commands are:

http POST :8000/robot/ name=”M-10iD/8L” robot_category=”Articulated Robots” currency=”USD” price=20000 manufacturer=”Fanuc” manufacturing_date=”2020-02-12 00:00:00+00:00″

http POST :8000/robot/ name=”SR-6iA” robot_category=”SCARA Robots” currency=”USD” price=10000 manufacturer=”Fanuc” manufacturing_date=”2020-02-12 00:00:00+00:00″

Now, let’s compose and send an HTTP GET request and analyze the paginated results.

http :8000/robot/

Output

HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 531
Content-Type: application/json
Date: Mon, 01 Feb 2021 05:53:29 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.7.5
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "count": 4,
    "next": "http://localhost:8000/robot/?page=2",
    "previous": null,
    "results": [
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2019-10-12T00:00:00Z",
            "name": "FANUC M-710ic/50",
            "price": 37000,
            "robot_category": "Articulated Robots",
            "url": "http://localhost:8000/robot/1/"
        },
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "ABB",
            "manufacturing_date": "2020-05-10T00:00:00Z",
            "name": "IRB 910SC",
            "price": 27000,
            "robot_category": "SCARA Robots",
            "url": "http://localhost:8000/robot/2/"
        }
    ]
}

Sharing the command prompt screenshot for your reference.

You can notice that the response looks different from the previous HTTP GET request. The response has the following keys:

• count: total number of resources on all pages
• next: link to the next page
• previous: link to the previous page
• results: an array of JSON representations of instances.

Let’s retrieve the results on page 2. The HTTPie command is

http :8000/robot/?page=2

Output

HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 516
Content-Type: application/json
Date: Mon, 01 Feb 2021 05:52:36 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.7.5
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "count": 4,
    "next": null,
    "previous": "http://localhost:8000/robot/",
    "results": [
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2020-02-12T00:00:00Z",
            "name": "M-10iD/8L",
            "price": 20000,
            "robot_category": "Articulated Robots",
            "url": "http://localhost:8000/robot/4/"
        },
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2020-02-12T00:00:00Z",
            "name": "SR-6iA",
            "price": 10000,
            "robot_category": "SCARA Robots",
            "url": "http://localhost:8000/robot/5/"
        }
    ]
}

Sharing the command prompt screenshot

LimitOffsetPagination

In LimitOffsetPagination style, client includes both a “limit” and an “offset” query parameter. The limit indicates the maximum number of items to return, same as that of the page_size. The offset indicates the starting position of the query w.r.t unpaginated items. To enable the LimitOffsetPagination style globally, you can set rest_framework.pagination.LimitOffsetPagination class to DEFAULT_PAGINATION_CLASS. The configuration as follows:

You can skip setting the PAGE_SIZE. If set, then the client can omit the limit query parameter. 

If you want to modify the pagination style, you can override the attributes of the LimitOffsetPagination class.

• default_limit – It indicates (numeric value) the limit. Defaults to the same value as the PAGE_SIZE settings key.
• limit_query_param – It indicates the name of the “limit” query parameter. Defaults to ‘limit’.
• offset_query_param – It indicates the name of the “offset” query parameter. Defaults to ‘offset’.
• max_limit – It indicates the maximum allowable limit that the client may request. Defaults to None.
• template – The template name to use when rendering pagination controls in the browsable API

The HTTPie command is

http :8000/robot/

Output

HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 541
Content-Type: application/json
Date: Mon, 01 Feb 2021 06:47:42 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.7.5
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "count": 4,
    "next": "http://localhost:8000/robot/?limit=2&offset=2",
    "previous": null,
    "results": [
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2019-10-12T00:00:00Z",
            "name": "FANUC M-710ic/50",
            "price": 37000,
            "robot_category": "Articulated Robots",
            "url": "http://localhost:8000/robot/1/"
        },
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "ABB",
            "manufacturing_date": "2020-05-10T00:00:00Z",
            "name": "IRB 910SC",
            "price": 27000,
            "robot_category": "SCARA Robots",
            "url": "http://localhost:8000/robot/2/"
        }
    ]
}

Let’s try another HTTPie command based on the next field value from the above output. The HTTPie command is 

http GET “:8000/robot/?limit=2&offset=2”

Output

HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 524
Content-Type: application/json
Date: Mon, 01 Feb 2021 06:52:35 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.7.5
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "count": 4,
    "next": null,
    "previous": "http://localhost:8000/robot/?limit=2",
    "results": [
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2020-02-12T00:00:00Z",
            "name": "M-10iD/8L",
            "price": 20000,
            "robot_category": "Articulated Robots",
            "url": "http://localhost:8000/robot/4/"
        },
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2020-02-12T00:00:00Z",
            "name": "SR-6iA",
            "price": 10000,
            "robot_category": "SCARA Robots",
            "url": "http://localhost:8000/robot/5/"
        }
    ]
}

Sharing the command prompt screenshot for your reference

Let’s try with limit=1 and offset=0. The HTTPie command is:

http GET “:8000/robot/?limit=1&offset=0”

Output

HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 325
Content-Type: application/json
Date: Mon, 01 Feb 2021 10:36:19 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.7.5
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "count": 4,
    "next": "http://localhost:8000/robot/?limit=1&offset=1",
    "previous": null,
    "results": [
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2019-10-12T00:00:00Z",
            "name": "FANUC M-710ic/50",
            "price": 37000,
            "robot_category": "Articulated Robots",
            "url": "http://localhost:8000/robot/1/"
        }
    ]
}

Sharing the command prompt screenshot

CursorPagination

The CursorPagination provides a cursor indicator to page through the result set. It provides only forward or reverse controls and doesn’t permit the client to navigate to arbitrary positions. The CursorPagination style assumes that there must be a created timestamp field on the model instance and it orders the results by ‘-created’. To enable the CursorPagination style you can mention rest_framework.pagination.CursorPagination class in DEFAULT_PAGINATION_CLASS.

Let’s do a look at the set of attributes that we can modify in CursorPagination class. They are as follows:

• page_size – It indicates the page size (numeric value). Defaults to the same value as the PAGE_SIZE settings key.
• cursor_query_param – It indicates the name of the “cursor” query parameter (string value). Defaults to ‘cursor’.
• ordering – This should be a string, or list of strings, indicating the field against which the cursor-based pagination will be applied. Defaults to -created. This value may also be overridden by using OrderingFilter on the view.
• template – The name of a template to use when rendering pagination controls in the browsable API.

Let’s see how to customize the CursorPagination class. Here we will override the ordering attribute. By default, it will order based on the created timestamp. Here, we will use the id field instead of the created field for ordering.

Let’s create a new file named custompagination.py file in the apps (robots) folder and add the below code

Here we override the ordering attribute provided by the CursorPagination class. Next, you can mention the customized class in the DEFAULT_PAGINATION_CLASS as shown below.

Let’s analyze the output. You can send the below HTTP command.

http :8000/robot/

Output

HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 526
Content-Type: application/json
Date: Mon, 01 Feb 2021 11:09:45 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.7.5
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "next": "http://localhost:8000/robot/?cursor=cD0y",
    "previous": null,
    "results": [
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2019-10-12T00:00:00Z",
            "name": "FANUC M-710ic/50",
            "price": 37000,
            "robot_category": "Articulated Robots",
            "url": "http://localhost:8000/robot/1/"
        },
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "ABB",
            "manufacturing_date": "2020-05-10T00:00:00Z",
            "name": "IRB 910SC",
            "price": 27000,
            "robot_category": "SCARA Robots",
            "url": "http://localhost:8000/robot/2/"
        }
    ]
}

Sharing the command prompt screenshot

Now, let’s compose an HTTP request based on the next value from the above output (cursor=cD0y). The HTTPie command is:

http :8000/robot/?cursor=cD0y

Output

HTTP/1.1 200 OK
Allow: GET, POST, HEAD, OPTIONS
Content-Length: 530
Content-Type: application/json
Date: Mon, 01 Feb 2021 11:10:38 GMT
Referrer-Policy: same-origin
Server: WSGIServer/0.2 CPython/3.7.5
Vary: Accept, Cookie
X-Content-Type-Options: nosniff
X-Frame-Options: DENY

{
    "next": null,
    "previous": "http://localhost:8000/robot/?cursor=cj0xJnA9NA%3D%3D",
    "results": [
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2020-02-12T00:00:00Z",
            "name": "M-10iD/8L",
            "price": 20000,
            "robot_category": "Articulated Robots",
            "url": "http://localhost:8000/robot/4/"
        },
        {
            "currency": "USD",
            "currency_name": "US Dollar",
            "manufacturer": "Fanuc",
            "manufacturing_date": "2020-02-12T00:00:00Z",
            "name": "SR-6iA",
            "price": 10000,
            "robot_category": "SCARA Robots",
            "url": "http://localhost:8000/robot/5/"
        }
    ]
}

Sharing the command prompt screenshot