performance issue with long list

[wybryk-ananasa]

I have list with around 500k records.
To select an item I do:
(completing-read "Select: " my-long-list nil 'require-match)
Typing in a mini-buffer is a little bit jerky.
Any ideas how can I improve performance?

[minad]

500k is at least one magnitude too much. Vertico is supposed to handle up to 50k candidates well in it's default configuration. We cannot do much better due to technical limitations of the underlying APIs.

There are two possibilities to handle such a large amount of candidates, via using a custom completion style, but these approaches will also hit limitations.

1. The completion style limits the number of returned candidates. This works only if candidates are presorted.
2. The completion style requires a minimal input length, such that fewer candidates make it through the filter.

For option 2 see this code:

(defun +orderless-limited-all-completions (string table pred point)
  ;; Refuse to compute candidates if input is too short
  (and (length> string 2)
       (or (orderless-all-completions string table pred point)
           ;; Fallback to prefix completion to support dynamic completion tables
           (completion-emacs21-all-completions string table pred point))))

(defun +orderless-limited-try-completion (string table pred point)
  ;; Refuse to compute candidates if input is too short
  (and (length< string 2)
       (or (orderless-try-completion string table pred point)
           ;; Fallback to prefix completion to support dynamic completion tables
           (completion-emacs21-try-completion string table pred point))))

(add-to-list 'completion-styles-alist
             '(+orderless-limited
               +orderless-limited-try-completion
               +orderless-limited-all-completions
               "Limited orderless completion style."))

(setq completion-styles '(+orderless-limited)
      completion-category-defaults nil
      completion-category-overrides nil)

See also the discussion at doomemacs/doomemacs#6622 (comment).