anki: 2.1.15 -> 2.1.60 #221229

euank · 2023-03-14T22:02:42Z

Description of changes

This updates the native packaging of anki for their new ninja-based build-system.

It's somewhat fragile feeling since their build-system isn't exactly designed for nix, but it does appear to work!
The high-level approach is that we do stuff like create node_modules, download stuff, create a python environment with nix, then we mock out a couple of things they run with our own stub bash scripts.
The upstream build system is responsible for building rust code and python wheels, and the nix derivation is responsible for getting node_modules in place, vendoring cargo deps, and all the other usual stuff like setting buildinputs.

After getting the wheel, we do normal python application packaging stuff.

I'm also happy to report that ibus based IME works on this one, unlike anki-bin, which was my main motivation for tackling this in the first place, so that's awesome!

I haven't tested this on macOS at all since I don't have macOS hardware, so someone else will have to see what's broken there!

Fixes #78449, Fixes #112921

Helps with #147989

Things done

martinetd · 2023-03-14T23:55:41Z

Syncing doesn't work, returning a network error ("⁨error sending request for url (): error trying to connect: invalid URL, scheme is not http⁩", like it's using empty string for the sync server), and I'm really not sure why.

I'm using my own anki server and setting the SYNC_ENDPOINT="https://xyz.tld/sync/" env var manually when running the client, you can perhaps give that a spin. The default seems to be https://sync.ankiweb.net/sync/

with that said -- awesome work. I agree not using pyoxidizer makes more sense for nixos, and I'll try to get my sync server working with this (been using the pip version as the ankisyncd in nixpkgs is too old and updating it requires a new anki...) when I find the time to try :)

pkgs/top-level/all-packages.nix

pkgs/games/anki/no-version-check.patch

pkgs/games/anki/default.nix

euank · 2023-03-15T12:20:24Z

I figured out what was wrong with syncing - I forgot to add buildFeatures = [ "rusttls" ] to the rust bridge, which is where http requests happen... so it just couldn't do tls and gave a kinda bad error.

Easy enough to fix, and syncing seems happy now 😀

pkgs/games/anki/default.nix

euank · 2023-03-15T13:42:12Z

Back to draft. While it now launches and syncs, the review and stats pages are blank, and it produces some javascript errors.

Something's still wrong with the packaging I guess.

euank · 2023-03-16T10:48:39Z

the review and stats pages are blank

Found it! I was missing applying this patch: https://github.com/ankitects/anki/blob/2.1.60/ts/patches/protobufjs%2B7.2.1.patch

I missed that they had a postinstall in their package.json which handles that, and I had --ignore-scripts on our invocation.

After fixing that, it all seems to work!

euank · 2023-03-16T10:58:36Z

I've updated the top-level comment with a bit more detail. I think this is ready for review now!

I'd also appreciate if someone with a mac could see if it works there

euank · 2023-03-16T11:02:24Z

pkgs/games/anki/default.nix

 , CoreAudio
-  # This little flag adds a huge number of dependencies, but we assume that
-  # everyone wants Anki to draw plots with statistics by default.
-, plotsSupport ? true


this used to pull in matplotlib, but grepping around in the repo, it looks like they don't use that anymore, so we can drop this.

pkgs/games/anki/default.nix

GabrielDougherty · 2023-03-16T23:35:24Z

Result of nixpkgs-review pr 221229 run on x86_64-linux 1

1 package failed to build:

ankisyncd

2 packages built:

anki
mnemosyne

nessdoor · 2023-03-31T17:02:40Z

@euank sorry for getting back to you so late. Yes, the Qt6 modifications worked, and I'm now able to launch Anki without any issues.

If anything else comes up, I'll try to let you know as soon as possible.

martinetd · 2023-04-02T13:33:35Z

opened #224366 for ankisyncd rust package update (addition) and service update
(I wouldn't hold this back for the server side; won't be using the client on nixos so no real opinion but this LGTM at this point)

euank · 2023-04-03T10:47:15Z

Thanks @martinetd!

Since I personally think this PR's good to merge at this point (and it has a couple approvals), I'll ping the PRs already reviewed thread in a couple days if no one here objects or merges it before then.
Thanks again for the help testing this!

martinetd · 2023-04-05T08:19:43Z

@euank I tried rebuilding this after cherry-picking on master and I can no longer build

cargo-deps-vendor.tar.gz> source root is source
cargo-deps-vendor.tar.gz> patching sources
cargo-deps-vendor.tar.gz> configuring
cargo-deps-vendor.tar.gz> no configure script, doing nothing
cargo-deps-vendor.tar.gz> building
cargo-deps-vendor.tar.gz> source = "git+https://github.com/ankitects/rust-csv.git?rev=1c9d3aab6f79a7d815c69f925a46a4590c115f90#1c9d3aab6f79a7d815c69f925a46a4590c115f90"
cargo-deps-vendor.tar.gz> source = "git+https://github.com/ankitects/rust-csv.git?rev=1c9d3aab6f79a7d815c69f925a46a4590c115f90#1c9d3aab6f79a7d815c69f925a46a4590c115f90"
cargo-deps-vendor.tar.gz> source = "git+https://github.com/ankitects/linkcheck.git?rev=2f20798ce521cc594d510d4e417e76d5eac04d4b#2f20798ce521cc594d510d4e417e76d5eac04d4b"
cargo-deps-vendor.tar.gz> ERROR: The Cargo.lock contains git dependencies
cargo-deps-vendor.tar.gz> This is currently not supported in the fixed-output derivation fetcher.
cargo-deps-vendor.tar.gz> Use cargoLock.lockFile / importCargoLock instead.

It looks like you'll need to switch to lockFile with a Cargo.lock in nixpkgs like I did for anki-sync-server-rs in the other PR (I assume they turned on fixed-output derivation style building recently? I'm pretty sure I was able to build on this system on your branch before rebasing... will recheck and update if not.

euank · 2023-04-05T10:46:36Z

Yup, indeed, it looks like since #221716 landed on master, I need to switch over as it says.

I'll give that a go now

Updating anki in the next commit will break it otherwise.

euank · 2023-04-05T11:35:20Z

$ maintainers/scripts/convert-to-import-cargo-lock.sh anki
converting anki (1/1)
thread 'main' panicked at 'package should contain cargoHash/cargoSha256', src/main.rs:176:52

The automated tool couldn't get all of it, but it was easy enough to do the rest by hand.

Updated, verified it builds and launches after converting it from fetchCargoTarball to importCargoLock

nixos-discourse · 2023-04-06T07:11:52Z

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/893

oxalica

Builds and works on plasma/wayland (through Xwayland).

SuperSandro2000 · 2023-04-06T23:25:47Z

pkgs/servers/ankisyncd/default.nix

@@ -11,7 +10,8 @@ python3.pkgs.buildPythonApplication rec {
    owner = "ankicommunity";
    repo = "anki-sync-server";
    rev = version;
-    sha256 = "196xhd6vzp1ncr3ahz0bv0gp1ap2s37j8v48dwmvaywzayakqdab";
+    hash = "sha256-RXrdJGJ+HMSpDGQBuVPPqsh3+uwAgE6f7ZJ0yFRMI8I=";
+    fetchSubmodules = true;


2.4.0 no longer requires submodules https://github.com/ankicommunity/anki-sync-server/tree/v2.4.0

Thanks for the review!

anki-sync-server 2.4.0 still requires anki from pypi, so it's not going to be a straightforward change (and the update isn't as simple as bumping the version number)

The current discussion seems to favor just getting rid of this flavor of ankisyncd in favor of the rust one (being discussed in #224366 ) so I'd say to just not touch ankisyncd here more than we have to for now.

Yup, as discussed in #224366 and previously in this pr, I don't think it's worth updating it (nor is it easy to do so).
I'm just trying to leave it functioning at all since this PR would otherwise break it, and then we can deal with it separately

pkgs/games/anki/default.nix

This redoes all the packaging for their new build-system. It feels a bit fragile, but in practice it works. Basically, we build most of it in nix, write some wrapper scripts to mock out stuff we just did in nix, and then call thier build system to make a wheel

Thanks to Lucio for pointing this one out, appreciated

mightyiam · 2023-04-12T03:58:36Z

Thank you!

euank mentioned this pull request Mar 14, 2023

anki: upgrade to latest release #78449

Closed

euank force-pushed the anki-native branch from 2b789c1 to ae14d0b Compare March 14, 2023 22:17

ofborg bot requested review from oxij and Profpatsch March 14, 2023 22:44

ofborg bot added 10.rebuild-darwin: 1-10 10.rebuild-linux: 1-10 labels Mar 14, 2023

Atemu reviewed Mar 15, 2023

View reviewed changes

pkgs/top-level/all-packages.nix Outdated Show resolved Hide resolved

euank commented Mar 15, 2023

View reviewed changes

pkgs/games/anki/no-version-check.patch Show resolved Hide resolved

euank commented Mar 15, 2023

View reviewed changes

pkgs/games/anki/default.nix Show resolved Hide resolved

euank commented Mar 15, 2023

View reviewed changes

pkgs/games/anki/default.nix Outdated Show resolved Hide resolved

euank force-pushed the anki-native branch from d42b549 to 77cca10 Compare March 15, 2023 13:21

euank marked this pull request as ready for review March 15, 2023 13:21

euank force-pushed the anki-native branch from 77cca10 to ec59e6b Compare March 15, 2023 13:23

euank commented Mar 15, 2023

View reviewed changes

pkgs/games/anki/default.nix Show resolved Hide resolved

euank marked this pull request as draft March 15, 2023 13:41

ofborg bot added the 11.by: package-maintainer label Mar 15, 2023

euank force-pushed the anki-native branch 3 times, most recently from bd4a249 to f5d0635 Compare March 16, 2023 10:47

euank marked this pull request as ready for review March 16, 2023 10:48

euank force-pushed the anki-native branch from f5d0635 to 12597d9 Compare March 16, 2023 10:49

euank commented Mar 16, 2023

View reviewed changes

euank mentioned this pull request Mar 16, 2023

anki: 2.1.15 -> 2.1.40 (fixes #112849) [DRAFT for others to pick up] #112921

Closed

10 tasks

Atemu reviewed Mar 16, 2023

View reviewed changes

pkgs/games/anki/default.nix Outdated Show resolved Hide resolved

kurnevsky approved these changes Apr 3, 2023

View reviewed changes

euank mentioned this pull request Apr 4, 2023

ankisyncd-rs: add package for anki-sync-server-rs #224366

Merged

9 tasks

ankisyncd: use bundled anki version

5451b89

Updating anki in the next commit will break it otherwise.

euank force-pushed the anki-native branch from 0888fc4 to 17c79a7 Compare April 5, 2023 11:33

euank mentioned this pull request Apr 6, 2023

Anki does not pick up gtk theme #126630

Open

oxalica approved these changes Apr 6, 2023

View reviewed changes

SuperSandro2000 reviewed Apr 6, 2023

View reviewed changes

pkgs/games/anki/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Apr 6, 2023

View reviewed changes

pkgs/games/anki/default.nix Outdated Show resolved Hide resolved

SuperSandro2000 reviewed Apr 6, 2023

View reviewed changes

pkgs/games/anki/default.nix Outdated Show resolved Hide resolved

euank force-pushed the anki-native branch from 17c79a7 to 8d22773 Compare April 7, 2023 13:05

anki: 2.1.15 -> 2.1.60

903f78e

This redoes all the packaging for their new build-system. It feels a bit fragile, but in practice it works. Basically, we build most of it in nix, write some wrapper scripts to mock out stuff we just did in nix, and then call thier build system to make a wheel

euank requested a review from SuperSandro2000 April 9, 2023 11:02

anki: remove an unused symlink

9ce0e05

Thanks to Lucio for pointing this one out, appreciated

euank force-pushed the anki-native branch from 8d22773 to 9ce0e05 Compare April 10, 2023 09:41

SuperSandro2000 merged commit 0754f14 into NixOS:master Apr 11, 2023

This was referenced Apr 12, 2023

Anki 2.1 addons not working #158388

Closed

anki: "Segmentation fault: 11" on macOS #40956

Open

anki: 2.1.60 -> 2.1.61 #225882

Merged

sbruder mentioned this pull request Apr 16, 2023

anki: allow using wayland on linux #226426

Merged

12 tasks

sbruder mentioned this pull request May 1, 2023

yuzu: update, switch to qt6, clean up, etc #228872

Merged

12 tasks

Atemu mentioned this pull request May 16, 2023

Is Anki too hard to build? (Or, make life easier for packagers.) ankitects/anki#1378

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

anki: 2.1.15 -> 2.1.60 #221229

anki: 2.1.15 -> 2.1.60 #221229

euank commented Mar 14, 2023 •

edited

Loading

martinetd commented Mar 14, 2023

euank commented Mar 15, 2023

euank commented Mar 15, 2023

euank commented Mar 16, 2023

euank commented Mar 16, 2023

euank Mar 16, 2023

GabrielDougherty commented Mar 16, 2023

nessdoor commented Mar 31, 2023

martinetd commented Apr 2, 2023

euank commented Apr 3, 2023 •

edited

Loading

martinetd commented Apr 5, 2023

euank commented Apr 5, 2023

euank commented Apr 5, 2023 •

edited

Loading

nixos-discourse commented Apr 6, 2023

oxalica left a comment

SuperSandro2000 Apr 6, 2023

martinetd Apr 7, 2023

euank Apr 7, 2023

mightyiam commented Apr 12, 2023

anki: 2.1.15 -> 2.1.60 #221229

anki: 2.1.15 -> 2.1.60 #221229

Conversation

euank commented Mar 14, 2023 • edited Loading

Description of changes

Things done

martinetd commented Mar 14, 2023

euank commented Mar 15, 2023

euank commented Mar 15, 2023

euank commented Mar 16, 2023

euank commented Mar 16, 2023

euank Mar 16, 2023

Choose a reason for hiding this comment

GabrielDougherty commented Mar 16, 2023

nessdoor commented Mar 31, 2023

martinetd commented Apr 2, 2023

euank commented Apr 3, 2023 • edited Loading

martinetd commented Apr 5, 2023

euank commented Apr 5, 2023

euank commented Apr 5, 2023 • edited Loading

nixos-discourse commented Apr 6, 2023

oxalica left a comment

Choose a reason for hiding this comment

SuperSandro2000 Apr 6, 2023

Choose a reason for hiding this comment

martinetd Apr 7, 2023

Choose a reason for hiding this comment

euank Apr 7, 2023

Choose a reason for hiding this comment

mightyiam commented Apr 12, 2023

euank commented Mar 14, 2023 •

edited

Loading

euank commented Apr 3, 2023 •

edited

Loading

euank commented Apr 5, 2023 •

edited

Loading