deduplicate is slow

[jovan-stojanovic]

Describe the bug

deduplicate function is very slow even on relatively small datasets.

Steps/Code to Reproduce

from skrub.datasets import make_deduplication_data
from skrub import deduplicate

duplicated = make_deduplication_data(examples=['black', 'white', 'red', 'blue', 'green'], entries_per_example=[500, 500, 500, 500, 500], prob_mistake_per_letter=0.3)

deduplicate(duplicated)

Expected Results

Faster results with parallelization, there are some for loops that are not optimal

Actual Results

Slow, takes ~4 minutes on my laptop for the example above.

Versions

Current unreleased version