✨ Support total parameter for tortoise ext

[waketzheng]

In our case, table with a lot of rows (about 10m at MySQL8), cost too much time to execute select count(*) from ... (about 2.2 seconds). Then we use ClickHouse to speed up this select which decrease it to be 0.5 seconds. So we need the paginate function to be able to accept the total parameter, code snippet is similar like bellow:

from contextlib import asynccontextmanager
from typing import Any, AsyncGenerator

import uvicorn
from fastapi import FastAPI
from pydantic import BaseModel
from tortoise import Model, fields
from tortoise.contrib.fastapi import RegisterTortoise

from fastapi_pagination import Page, add_pagination
from fastapi_pagination.ext.tortoise import paginate

cached_record_info = {"total": 0}


class Record(Model):
    id = fields.IntField(pk=True)
    created_at = fields.IntField()


class RecordOut(BaseModel):
    id: int | None = None
    created_at: int


@asynccontextmanager
async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
    async with RegisterTortoise(
        app,
        db_url="sqlite://:memory:",
        modules={"models": [__name__]},
        generate_schemas=True,
    ):
        yield


app = FastAPI(lifespan=lifespan)


class RecordQuery(BaseModel):
    start_time: int
    end_time: int


def select_count(sql: str) -> int:
    # execute SQL in ClickHouse
    return cached_record_info["total"]


@app.post("/records", response_model=RecordOut)
async def create_record(data: RecordOut) -> Any:
    obj = await Record.create(**data.dict(exclude_none=True))
    cached_record_info["total"] += 1
    return obj


@app.get("/records", response_model=Page[RecordOut])
async def record_list(start_time: int, end_time: int) -> Any:
    qs = Record.filter(created_at__gte=start_time, created_at__lte=end_time)
    total = select_count(qs.sql())
    return await paginate(qs, total=total)


add_pagination(app)


def main() -> None:
    uvicorn.run(app)


if __name__ == "__main__":
    main()

[codecov]

Codecov Report

Attention: Patch coverage is 0% with 2 lines in your changes missing coverage. Please review.

Project coverage is 91.05%. Comparing base (245519c) to head (713091d).
Report is 415 commits behind head on main.

Files with missing lines	Patch %	Lines
fastapi_pagination/ext/tortoise.py	0.00%	2 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1269      +/-   ##
==========================================
- Coverage   93.26%   91.05%   -2.21%     
==========================================
  Files          35       38       +3     
  Lines        1040     1331     +291     
==========================================
+ Hits          970     1212     +242     
- Misses         70      119      +49     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[uriyyo]

Looks great, it actually makes sense to add it also to other extensions cause it seems like a useful feature