Skip to content

Commit

Permalink
feat: Infinite Scroll for Backend Services (OData/GraphQL)
Browse files Browse the repository at this point in the history
  • Loading branch information
ghiscoding committed Aug 7, 2024
1 parent 585d4f7 commit 13e2697
Show file tree
Hide file tree
Showing 20 changed files with 1,841 additions and 122 deletions.
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,7 +39,7 @@ npm install angular-slickgrid
- [Bootstrap 5 (single Locale)](https://github.com/ghiscoding/angular-slickgrid-demos/tree/master/bootstrap5-demo-with-locales) / [examples repo](https://github.com/ghiscoding/angular-slickgrid-demos/tree/master/bootstrap5-demo-with-locales) - Code Sample with a single Locale (without `ngx-translate`)

#### Working Demo
For a complete set of working demos (over 30 examples), we strongly suggest you clone [Angular-Slickgrid Demos](https://github.com/ghiscoding/angular-slickgrid-demos) repository (instructions are provided in the demo repo). The repo provides multiple demos and they are updated for every new project release, so it is updated frequently and is also used as the GitHub live demo page for both the [Bootstrap 5 demo](https://ghiscoding.github.io/Angular-Slickgrid) and [Bootstrap 5 demo (single Locale)](https://ghiscoding.github.io/angular-slickgrid-demos).
For a complete set of working demos (40+ examples), we strongly suggest you clone [Angular-Slickgrid Demos](https://github.com/ghiscoding/angular-slickgrid-demos) repository (instructions are provided in the demo repo). The repo provides multiple demos and they are updated for every new project release, so it is updated frequently and is also used as the GitHub live demo page for both the [Bootstrap 5 demo](https://ghiscoding.github.io/Angular-Slickgrid) and [Bootstrap 5 demo (single Locale)](https://ghiscoding.github.io/angular-slickgrid-demos).

```sh
git clone https://github.com/ghiscoding/angular-slickgrid-demos
Expand Down
1 change: 1 addition & 0 deletions docs/TOC.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,6 +58,7 @@
* [Grid State & Presets](grid-functionalities/Grid-State-&-Preset.md)
* [Grouping & Aggregators](grid-functionalities/grouping-and-aggregators.md)
* [Header Menu & Header Buttons](grid-functionalities/Header-Menu-&-Header-Buttons.md)
* [Infinite Scroll](grid-functionalities/infinite-scroll.md)
* [Pinning (frozen) of Columns/Rows](grid-functionalities/frozen-columns-rows.md)
* [Providing data to the grid](grid-functionalities/providing-grid-data.md)
* [Row Detail](grid-functionalities/row-detail.md)
Expand Down
3 changes: 2 additions & 1 deletion docs/backend-services/GraphQL.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
- [Pagination](graphql/GraphQL-Pagination.md)
- [Sorting](graphql/GraphQL-Sorting.md)
- [Filtering](graphql/GraphQL-Filtering.md)
- [Infinite Scroll](../grid-functionalities/infinite-scroll.md#infinite-scroll-with-backend-services)

### Description
GraphQL Backend Service (for Pagination purposes) to get data from a backend server with the help of GraphQL.
Expand All @@ -14,7 +15,7 @@ GraphQL Backend Service (for Pagination purposes) to get data from a backend ser
[Demo Page](https://ghiscoding.github.io/Angular-Slickgrid/#/gridgraphql) / [Demo Component](https://github.com/ghiscoding/angular-slickgrid/blob/master/src/app/examples/grid-graphql.component.ts)

### Note
You can use it when you need to support **Pagination** (though you could disable Pagination if you wish), that is when your dataset is rather large and has typically more than 5k rows, with a GraphQL endpoint. If your dataset is small (less than 5k rows), then you might be better off with [regular grid](https://ghiscoding.github.io/Angular-Slickgrid/#/basic) with the "dataset.bind" property. SlickGrid can easily handle million of rows using a DataView object, but personally when the dataset is known to be large, I usually use a backend service (OData or GraphQL) and when it's small I go with a [regular grid](https://ghiscoding.github.io/Angular-Slickgrid/#/basic).
You can use it when you need to support **Pagination** (though you could disable Pagination if you wish), that is when your dataset is rather large and has typically more than 5k rows, with a GraphQL endpoint. If your dataset is small (less than 5k rows), then you might be better off with [regular grid](https://ghiscoding.github.io/Angular-Slickgrid/#/basic) with the "dataset.bind" property. SlickGrid can easily handle million of rows using a DataView object, but personally when the dataset is known to be large, I usually use a backend service (OData or GraphQL) and when it's small I go with a [regular grid](https://ghiscoding.github.io/Angular-Slickgrid/#/basic).

## Implementation
To connect a backend service into `Slickgrid-Universal`, you simply need to modify your `gridOptions` and add a declaration of `backendServiceApi`. See below for the signature and an example further down below.
Expand Down
1 change: 1 addition & 0 deletions docs/backend-services/OData.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,7 @@
- [Passing Extra Arguments](#passing-extra-arguments-to-the-query)
- [OData options](#odata-options)
- [Override the filter query](#override-the-filter-query)
- [Infinite Scroll](../grid-functionalities/infinite-scroll.md#infinite-scroll-with-backend-services)

### Description
OData Backend Service (for Pagination purposes) to get data from a backend server with the help of OData.
Expand Down
150 changes: 150 additions & 0 deletions docs/grid-functionalities/infinite-scroll.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
## Description

Infinite scrolling allows the grid to lazy-load rows from the server (or locally) when reaching the scroll bottom (end) position.
In its simplest form, the more the user scrolls down, the more rows will get loaded and appended to the in-memory dataset.

### Demo
[JSON Data - Demo Page](https://ghiscoding.github.io/Angular-Slickgrid/#/infinite-json) / [Demo ViewModel](https://github.com/ghiscoding/angular-slickgrid/blob/master/src/app/examples/grid-infinite-json.component.ts)

[OData Backend Service - Demo Page](https://ghiscoding.github.io/Angular-Slickgrid/#/infinite-odata) / [Demo ViewModel](https://github.com/ghiscoding/angular-slickgrid/blob/master/src/app/examples/grid-infinite-odata.component.ts)

[GraphQL Backend Service - Demo Page](https://ghiscoding.github.io/Angular-Slickgrid/#/infinite-graphql) / [Demo ViewModel](https://github.com/ghiscoding/angular-slickgrid/blob/master/src/app/examples/grid-infinite-graphql.component.ts)

> ![WARNING]
> Pagination Grid Preset (`presets.pagination`) is **not** supported with Infinite Scroll
## Infinite Scroll with JSON data

As describe above, when used with a local JSON dataset, it will add data to the in-memory dataset whenever we scroll to the bottom until we reach the end of the dataset (if ever).

#### Code Sample
When used with a local JSON dataset, the Infinite Scroll is a feature that must be implemented by yourself. You implement by subscribing to 1 main event (`onScroll`) and if you want to reset the data when Sorting then you'll also need to subscribe to the (`onSort`) event. So the idea is to have simple code in the `onScroll` event to detect when we reach the scroll end and then use the DataView `addItems()` to append data to the existing dataset (in-memory) and that's about it.

##### View
```html
<angular-slickgrid
gridId="grid2"
[columnDefinitions]="columnDefinitions"
[gridOptions]="gridOptions"
[dataset]="dataset"
(onAngularGridCreated)="angularGridReady($event.detail)"
(onScroll)="handleOnScroll($event.$detail.args)"
(onSort)="handleOnSort()">
</angular-slickgrid>
```

```ts
export class Example implements OnInit {
scrollEndCalled = false;

// add onScroll listener which will detect when we reach the scroll end
// if so, then append items to the dataset
handleOnScroll(event) {
const args = event.detail?.args;
const viewportElm = args.grid.getViewportNode();
if (
['mousewheel', 'scroll'].includes(args.triggeredBy || '')
&& !this.scrollEndCalled
&& viewportElm.scrollTop > 0
&& Math.ceil(viewportElm.offsetHeight + args.scrollTop) >= args.scrollHeight
) {
// onScroll end reached, add more items
// for demo purposes, we'll mock next subset of data at last id index + 1
const startIdx = this.angularGrid.dataView?.getItemCount() || 0;
const newItems = this.loadData(startIdx, FETCH_SIZE);
this.angularGrid.dataView?.addItems(newItems);
this.scrollEndCalled = false; //
}
}

// do we want to reset the dataset when Sorting?
// if answering Yes then use the code below
handleOnSort() {
if (this.shouldResetOnSort) {
const newData = this.loadData(0, FETCH_SIZE);
this.angularGrid.slickGrid?.scrollTo(0); // scroll back to top to avoid unwanted onScroll end triggered
this.angularGrid.dataView?.setItems(newData);
this.angularGrid.dataView?.reSort();
}
}
}
```

---

## Infinite Scroll with Backend Services

As describe above, when used with the Backend Service API, it will add data to the in-memory dataset whenever we scroll to the bottom. However there is one thing to note that might surprise you which is that even if Pagination is hidden in the UI, but the fact is that behind the scene that is exactly what it uses (mainly the Pagination Service `.goToNextPage()` to fetch the next set of data).

#### Code Sample
We'll use the OData Backend Service to demo Infinite Scroll with a Backend Service, however the implementation is similar for any Backend Services. The main difference with the Infinite Scroll implementation is around the `onProcess` and the callback that we use within (which is the `getCustomerCallback` in our use case). This callback will receive a data object that include the `infiniteScrollBottomHit` boolean property, this prop will be `true` only on the 2nd and more passes which will help us make a distinction between the first page load and any other subset of data to append to our in-memory dataset. With this property in mind, we'll assign the entire dataset on 1st pass with `this.dataset = data.value` (when `infiniteScrollBottomHit: false`) but for any other passes, we'll want to use the DataView `addItems()` to append data to the existing dataset (in-memory) and that's about it.

##### View
```html
<angular-slickgrid
gridId="grid2"
[columnDefinitions]="columnDefinitions"
[gridOptions]="gridOptions"
[dataset]="dataset"
(onAngularGridCreated)="angularGridReady($event.detail)">
</angular-slickgrid>
```

```ts
export class Example implements OnInit {
initializeGrid() {
this.columnDefinitions = [ /* ... */ ];

this.gridOptions = {
presets: {
// NOTE: pagination preset is NOT supported with infinite scroll
// filters: [{ columnId: 'gender', searchTerms: ['female'] }]
},
backendServiceApi: {
service: new GridOdataService(), // or any Backend Service
options: {
// enable infinite scroll via Boolean OR via { fetchSize: number }
infiniteScroll: { fetchSize: 30 }, // or use true, in that case it would use default size of 25

preProcess: () => {
this.displaySpinner(true);
},
process: (query) => this.getCustomerApiCall(query),
postProcess: (response) => {
this.displaySpinner(false);
this.getCustomerCallback(response);
},
// we could use local in-memory Filtering (please note that it only filters against what is currently loaded)
// that is when we want to avoid reloading the entire dataset every time
// useLocalFiltering: true,
} as OdataServiceApi,
};
}

// Web API call
getCustomerApiCall(odataQuery) {
return this.http.get(`/api/getCustomers?${odataQuery}`);
}

getCustomerCallback(data: { '@odata.count': number; infiniteScrollBottomHit: boolean; metrics: Metrics; query: string; value: any[]; }) {
// totalItems property needs to be filled for pagination to work correctly
const totalItemCount: number = data['@odata.count'];
this.metrics.totalItemCount = totalItemCount;

// even if we're not showing pagination, it is still used behind the scene to fetch next set of data (next page basically)
// once pagination totalItems is filled, we can update the dataset

// infinite scroll has an extra data property to determine if we hit an infinite scroll and there's still more data (in that case we need append data)
// or if we're on first data fetching (no scroll bottom ever occured yet)
if (!data.infiniteScrollBottomHit) {
// initial load not scroll hit yet, full dataset assignment
this.angularGrid.slickGrid?.scrollTo(0); // scroll back to top to avoid unwanted onScroll end triggered
this.dataset = data.value;
this.metrics.itemCount = data.value.length;
} else {
// scroll hit, for better perf we can simply use the DataView directly for better perf (which is better compare to replacing the entire dataset)
this.angularGrid.dataView?.addItems(data.value);
}
}
}
```
26 changes: 13 additions & 13 deletions package.json
Original file line number Diff line number Diff line change
Expand Up @@ -50,13 +50,13 @@
},
"dependencies": {
"@ngx-translate/core": "^15.0.0",
"@slickgrid-universal/common": "~5.4.0",
"@slickgrid-universal/custom-footer-component": "~5.4.0",
"@slickgrid-universal/empty-warning-component": "~5.4.0",
"@slickgrid-universal/event-pub-sub": "~5.4.0",
"@slickgrid-universal/pagination-component": "~5.4.0",
"@slickgrid-universal/row-detail-view-plugin": "~5.4.0",
"@slickgrid-universal/rxjs-observable": "~5.4.0",
"@slickgrid-universal/common": "~5.5.0",
"@slickgrid-universal/custom-footer-component": "~5.5.0",
"@slickgrid-universal/empty-warning-component": "~5.5.0",
"@slickgrid-universal/event-pub-sub": "~5.5.0",
"@slickgrid-universal/pagination-component": "~5.5.0",
"@slickgrid-universal/row-detail-view-plugin": "~5.5.0",
"@slickgrid-universal/rxjs-observable": "~5.5.0",
"dequal": "^2.0.3",
"rxjs": "^7.8.1"
},
Expand Down Expand Up @@ -86,12 +86,12 @@
"@ngx-translate/http-loader": "^8.0.0",
"@popperjs/core": "^2.11.8",
"@release-it/conventional-changelog": "^8.0.1",
"@slickgrid-universal/composite-editor-component": "~5.4.0",
"@slickgrid-universal/custom-tooltip-plugin": "~5.4.0",
"@slickgrid-universal/excel-export": "~5.4.0",
"@slickgrid-universal/graphql": "~5.4.0",
"@slickgrid-universal/odata": "~5.4.0",
"@slickgrid-universal/text-export": "~5.4.0",
"@slickgrid-universal/composite-editor-component": "~5.5.0",
"@slickgrid-universal/custom-tooltip-plugin": "~5.5.0",
"@slickgrid-universal/excel-export": "~5.5.0",
"@slickgrid-universal/graphql": "~5.5.0",
"@slickgrid-universal/odata": "~5.5.0",
"@slickgrid-universal/text-export": "~5.5.0",
"@types/dompurify": "^3.0.5",
"@types/fnando__sparkline": "^0.3.7",
"@types/jest": "^29.5.12",
Expand Down
4 changes: 4 additions & 0 deletions src/app/app-routing.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -18,6 +18,8 @@ import { GridFrozenComponent } from './examples/grid-frozen.component';
import { GridGraphqlComponent } from './examples/grid-graphql.component';
import { GridGraphqlWithoutPaginationComponent } from './examples/grid-graphql-nopage.component';
import { GridGroupingComponent } from './examples/grid-grouping.component';
import { GridInfiniteGraphqlComponent } from './examples/grid-infinite-graphql.component';
import { GridInfiniteOdataComponent } from './examples/grid-infinite-odata.component';
import { GridHeaderButtonComponent } from './examples/grid-headerbutton.component';
import { GridHeaderFooterComponent } from './examples/grid-header-footer.component';
import { GridHeaderMenuComponent } from './examples/grid-headermenu.component';
Expand Down Expand Up @@ -67,6 +69,8 @@ const routes: Routes = [
{ path: 'draggrouping', component: GridDraggableGroupingComponent },
{ path: 'grouping', component: GridGroupingComponent },
{ path: 'header-footer', component: GridHeaderFooterComponent },
{ path: 'infinite-graphql', component: GridInfiniteGraphqlComponent },
{ path: 'infinite-odata', component: GridInfiniteOdataComponent },
{ path: 'localization', component: GridLocalizationComponent },
{ path: 'clientside', component: GridClientSideComponent },
{ path: 'odata', component: GridOdataComponent },
Expand Down
10 changes: 10 additions & 0 deletions src/app/app.component.html
Original file line number Diff line number Diff line change
Expand Up @@ -176,6 +176,16 @@
37- Footer Totals Row
</a>
</li>
<li class="nav-item">
<a class="nav-link" routerLinkActive="active" [routerLink]="['/infinite-odata']">
38- Infinite Scroll with OData
</a>
</li>
<li class="nav-item">
<a class="nav-link" routerLinkActive="active" [routerLink]="['/infinite-graphql']">
39- Infinite Scroll with GraphQL
</a>
</li>
</ul>
</section>

Expand Down
4 changes: 4 additions & 0 deletions src/app/app.module.ts
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,8 @@ import { GridGraphqlWithoutPaginationComponent } from './examples/grid-graphql-n
import { GridGroupingComponent } from './examples/grid-grouping.component';
import { GridHeaderButtonComponent } from './examples/grid-headerbutton.component';
import { GridHeaderMenuComponent } from './examples/grid-headermenu.component';
import { GridInfiniteGraphqlComponent } from './examples/grid-infinite-graphql.component';
import { GridInfiniteOdataComponent } from './examples/grid-infinite-odata.component';
import { GridLocalizationComponent } from './examples/grid-localization.component';
import { GridMenuComponent } from './examples/grid-menu.component';
import { GridOdataComponent } from './examples/grid-odata.component';
Expand Down Expand Up @@ -118,6 +120,8 @@ export function appInitializerFactory(translate: TranslateService, injector: Inj
GridHeaderButtonComponent,
GridHeaderFooterComponent,
GridHeaderMenuComponent,
GridInfiniteGraphqlComponent,
GridInfiniteOdataComponent,
GridLocalizationComponent,
GridMenuComponent,
GridOdataComponent,
Expand Down
Loading

0 comments on commit 13e2697

Please sign in to comment.