Rust Port: Part 2 - Dependencies

[fosskers]

This PR expands on #671 by rounding off much of the internals of -A.

This is a vast improvement over the Dependencies.hs Haskell version in both performance and accuracy.

[Morganamilo]

Keep in mind there's https://docs.rs/aur-depends/1.0.3/aur_depends/ I'm still waiting for if you're okay with this library's API before making an asyncless version.

The library is very high level so I could see you maybe not wanting to use it to do stuff your self/

[fosskers]

I'm not against it - I haven't looked through it yet actually. Forgive me if that's caused you a delay.

The current code I'm working through now in the resolve and resolve_one functions is an effort to outdo my past self's Haskell version of similar logic. It's also part experiment, attempting to answer the question "If I code in the obvious way using mostly std, does it come out alright?" (say compared to what the Haskell does).

[Morganamilo]

say compared to what the Haskell does

What does haskell do?

[fosskers]

I meant the Haskell version of dep resolution found here: https://github.com/fosskers/aura/blob/master/aura/lib/Aura/Dependencies.hs

[fosskers]

Side note: The AUR RPC really needs to support searching by provides. It's too bad that we need to resort to workarounds like what aur-depends does (and what I'm writing code for right now too). For pkgbase it just "does the right thing". Why not provides too?

I just hit the stupid edge-case introduced by this depending on this but with a slightly different name.

[fosskers]

And am I right that the AUR site itself supports multiple search terms, but the RPC does not?

[fosskers]

This patch got my hopes up, but I don't see an actual v6 supported here https://gitlab.archlinux.org/archlinux/aurweb/-/blob/master/doc/rpc.txt

[fosskers]

The trail continues: https://gitlab.archlinux.org/archlinux/aurweb/-/issues/136

[fosskers]

https://gitlab.archlinux.org/archlinux/aurweb/-/issues/306

[Morganamilo]

I've given up on searching for provides because I find my hack to be sufficient to be honest. Do note though that I only do the provider search for targets and missing packages. Doing it at every level is not ideal.

[Morganamilo]

Also I pushed a threaded version of aur-fetch using crossbeam https://github.com/Morganamilo/aur-fetch.rs/blob/master/src/fetch.rs#L121

[fosskers]

This is now comically fast and handles provides correctly.

[Morganamilo]

Running your own aur mirror? Seems a bit fragile.

[fosskers]

Luckily it's just a data mirror of the RPC specifically. Otherwise yes, that would be a nightmare. The benefit is that it's extremely fast, and gives me multi-term search and O(logn) "provides" lookups out of the box.

Go nuts: https://git.sr.ht/~fosskers/faur

I haven't announced it anywhere yet. I'll likely do so as a side-note when I release Aura 4.

[Morganamilo]

How does it work as a speedup then? Can it handle the traffic of many users?

[fosskers]

Because it doesn't use a database and holds all package data in memory, using Rust ref tricks to store HashMaps of &Package data for each kind of lookup you'd want to do (info, search, and provides). So you get O(logn) lookups for everything right out of RAM. This way my own tool can pound its API as hard as I want, without burdening the actual AUR.