Duplicate entries in search results

[BPerlakiH]

Based on the findings from: #979

We can have multiple search results, as indexed search and title search can return the same entries.
Additionally some entries will have duplicate URLs, one with and one without a trailing slash, eg:
zim://B2A69C3D-7852-400F-9A07-8986875DF683/solar.lowtechmagazine.com/tags/speed/ zim://B2A69C3D-7852-400F-9A07-8986875DF683/solar.lowtechmagazine.com/tags/speed
Whereas both will result in the same search result, and page linked.

Example on macOS:

[kelson42]

Here a general remark. The libzim provides two searches:

• Title suggestions
• Fulltext searches

Usually it is either one or the other. Only the Apple reader does somehow a mix. I'm not super found of this approach honestly as it creates a lot of new challenges.

Not against fixes this one obviously, but just want to share the information that this approach might disappear in the future.

[BPerlakiH]

@kelson42 I did some further investigation on this. There are more issues discovered, we have this:

kiwix-apple/SwiftUI/Model/SearchOperation/SearchOperation.mm

Lines 62 to 65 in 9093134

	if (archive.hasFulltextIndex()) {
	indexSearchArchives.push_back(archive);
	}
	titleSearchArchives.push_back(archive);

Now this indeed means we do search in both ways, as you wrote.

Currently, with the wikipedia copy I have, it throws an exception on indexed search:
DatabaseCorruptError: dir_end invalid in block 28240
Which has the following consequences, if I do change it as you suggested: to be either indexed or title search:

if (archive.hasFulltextIndex()) {
    indexSearchArchives.push_back(archive);
} else {
    titleSearchArchives.push_back(archive);
}

it won't give any results, since the indexed search fails, and we won't do the title search at all.

Additionally to this, I did found that we do the search on a set of archives, which is also not perfect:

• if it throws an exception on 1 archive from the set, we loose the results from the whole set!

I am updating the PR to do it one try / catch per archive. That way we can continue to get results even if one of the archives is "bad".