[Dub] Missing Tons of Links (Only Showing Most Recent 100 Links) Pagination Issue

[abass]

Extension

https://www.raycast.com/dubinc/dub

Raycast Version

1.81.2

macOS Version

No response

Description

The extension takes the latest 100 links and doesn't handle pagination, so it does't actually show all links when searching which makes it less useful if you're actually using Dub quite heavily.

Steps To Reproduce

Have more than 100 links and search for a link in Dub.

Current Behaviour

No response

Expected Behaviour

No response

[raycastbot]

Thank you for opening this issue!

🔔 @quuu @jfkisafk @steven-tey @devkiran you might want to have a look.

💡 Author and Contributors commands

The author and contributors of dubinc/dub can trigger bot actions by commenting:

• @raycastbot close this issue Closes the issue.
• @raycastbot close as not planned Closes the issue as not planned.
• @raycastbot rename this issue to "Awesome new title" Renames the issue.
• @raycastbot reopen this issue Reopens the issue.
• @raycastbot assign me Assigns yourself to the issue.
• @raycastbot good first issue Adds the "Good first issue" label to the issue.
• @raycastbot keep this issue open Make sure the issue won't go stale and will be kept open by the bot.

[jfkisafk]

I don't think dub API supports pagination at the moment.

[steven-tey]

@jfkisafk we do support pagination! How would this work with Raycast's API though – e.g. is there any way we can run async filtering instead of Raycast's default local-first filtering?

Edit: Looks like Raycast's List API supports that!

[jfkisafk]

Oh sorry @steven-tey I do not know how I missed that. But essentially we have to move with typeahead search option here, utilizing the search parameter. That way we will keep this command safe from memory heap issues #10127 (comment) (by not loading all the links in the memory).

We can dynamically trigger typeahead in case of large workspaces (i.e. when a page cursor is present for the next page page of results). But if the no. of links is < 100 then we can just supply keywords and use filtering=true instead (as we are right now). This will save unnecessary calls to the API for smaller workspaces. Here is a recent example from "Search Documents" command in Linear extension that uses this architecture:

extensions/extensions/linear/src/components/docs/DocumentList.tsx

Lines 97 to 99 in c7857fd

	{...(!supportsDocTypeahead
	? { searchBarPlaceholder: "Filter by title, creator, project or initiative name" }
	: { onSearchTextChange: setQuery, searchBarPlaceholder: "Search by document title", throttle: true })}

extensions/extensions/linear/src/hooks/useDocuments.ts

Lines 5 to 18 in c7857fd

	export function useDocuments(query: string = "", entity: DocumentEntity = { projectId: "" }) {
	const { data, error, isLoading, mutate } = useCachedPromise(getDocuments, [query, entity], {
	failureToastOptions: { title: "Failed to load documents" },
	keepPreviousData: true,
	});
	return {
	docs: data?.docs,
	docsError: error,
	isLoadingDocs: (!data && !error) || isLoading,
	supportsDocTypeahead: query.trim().length > 0 || data?.hasMoreDocs,
	mutateDocs: mutate,
	};
	}

extensions/extensions/linear/src/api/documents.ts

Lines 83 to 86 in c7857fd

	const docs = sortBy(data?.documents.nodes ?? [], (doc) => doc.sortOrder ?? Infinity);
	const hasMoreDocs = !!data?.documents.pageInfo.hasNextPage;
	return { docs, hasMoreDocs };

[steven-tey]

@jfkisafk that looks perfect, thanks for sharing! Feel free to make a PR for this – we can get to it on our end once as well we finish a few other integrations!

[jfkisafk]

Delegated it to @srikirank. He will be working on this one!

[steven-tey]

Thank you @srikirank 🙏

[steven-tey]

@srikirank were you able to figure out a solution for this? 👀