FIX key formating divergence when inspecting plugin dictionary. #11708

fcharras · 2023-12-15T16:11:04Z

What happens is that str(conftestpath) is not necessarily equal to mod.__file__, which is the key that is really used in self._name2plugin. For instance I've noticed that on windows, the module.__file__ string for some modules is sometimes small cased (c:\\users...), while str(conftestpath) is not (C:\\Users...). For my particular reproducer of #9765, it causes some plugins loaded from conftest to be double loaded, since the preliminary lookup fails because the key is wrong. And double-loading leads to a crash.

This might not be a good fix if there are other direct calls to self.register in other areas of the code base that would risk to format the keys differently when it's crafted from paths to conftest files, but I've not found any evidences of that.

fcharras · 2023-12-15T16:15:00Z

Also fixes scikit-learn/scikit-learn#27806

fcharras · 2023-12-15T16:47:36Z

If adopted it should be backported to latest supported versions, since this bug has been reported in 7.1.x for the first time.

bluetech · 2023-12-16T12:52:04Z

Thanks for tracking down this sneaky problem @fcharras.

My main initial request is: can you add a new, standalone test which reproduces the issue on Windows (i.e. fails before the fix, and passes after)?

And what happens if you run pytest like PYTHONCASEOK=1 pytest ...?

lesteve · 2023-12-18T07:37:24Z

My main initial request is: can you add a new, standalone test which reproduces the issue on Windows (i.e. fails before the fix, and passes after)?

Would it be enough to do a simple unit test, i.e. something like registering a plugin with upper-case, querying the plugin dict with lower-case and making sure that it is found?

Would you rather have something closer to an integration test that looks like the original bug #9765. The latter is probably significantly harder to put together.

And what happens if you run pytest like PYTHONCASEOK=1 pytest ...?

The issue is still there with PYTHONCASEOK=1.

jgb · 2023-12-18T10:03:47Z

@fcharras @bluetech I can confirm that this PR fixes the issue on Linux!

RonnyPfannschmidt · 2023-12-18T12:41:56Z

src/_pytest/config/__init__.py

@@ -744,9 +746,9 @@ def consider_pluginarg(self, arg: str) -> None:
                    del self._name2plugin["pytest_" + name]
            self.import_plugin(arg, consider_entry_points=True)

-    def consider_conftest(self, conftestmodule: types.ModuleType) -> None:
+    def consider_conftest(self, conftestmodule: types.ModuleType, name: str) -> None:


strictly speaking this one is a "breaking api change" do we have any idea on the impact - i suspect its zero, but i'm not certain

Would it fix to have name default to None and use conftestmodule.__file__ (maybe normalized ) if so ?

Or, another way I considered would be to run mod = import_path(conftestpath, mode=importmode, root=rootpath) first, then using mod.__file__ to pass the existing check rather than str(conftestpath): that works too, with the advantage of not changing the api, and not altering the keys in name2plugins compared to current. The downside is that import_path is now also called for plugins that are already registered (which might not be important since the module should be already imported so the import_module will do nothing ? but I wasn't sure of that).

The first way sounds practical,

I'd recommend normalizing

I'd defer implementation, in Case there's next to no impact it should go into 8.x as is and the backport would trigger a deprecation warning plus use none

fcharras · 2023-12-19T14:21:31Z

Adressed comment + best-effort attempt at non-regression tests. (see #11725 )

bluetech · 2023-12-23T20:19:04Z

I left a comment on the issue: #9765 (comment)

Would you rather have something closer to an integration test that looks like the original bug #9765. The latter is probably significantly harder to put together.

Basically yes -- doesn't need to be an integration test in pytest test suite, but just an end-to-end reproduction. The unit tests in this PR are useful to prevent regressions, but not for understanding the issue because they already pass the two differing paths as inputs.

The issue is still there with PYTHONCASEOK=1.

Thanks for testing. This is on Windows right?

So it may indicate that the issue also in your case is not merely with case but also normalization.

@fcharras It will be helpful if you could split the change to two commits: one which does the normpath without the normcase, and a second with adds the normcase. Then we can check exactly which of the two fixes the issue in each situation.

bluetech · 2023-12-23T20:19:48Z

src/_pytest/config/__init__.py

@@ -639,7 +639,11 @@ def _rget_with_confmod(
    def _importconftest(
        self, conftestpath: Path, importmode: Union[str, ImportMode], rootpath: Path
    ) -> types.ModuleType:
-        existing = self.get_plugin(str(conftestpath))
+        conftest_registration_name = str(


The str here is not necessary I believe.

lesteve · 2023-12-23T22:57:08Z

I put together a reproducer repo that reproduces (Windows-only) https://github.com/lesteve/pytest-issue-9765-reproducer.

For some reason you need the combination of the three following things in our case for the issue to appear:

a confest.py inside the package
a plugin inside the package that is early-loaded with pytest -p
install the package in editable mode with python setup.py develop or its pip equivalent (pip install -e . --no-use-pep517)

Not sure how easy this is to turn the repo into a pytest test.

The issue is still there with PYTHONCASEOK=1.

Thanks for testing. This is on Windows right?

Yes

So it may indicate that the issue also in your case is not merely with case but also normalization.

In the case we looked at together with @fcharras (Windows-only), I think we only need os.path.normcase and not os.path.normpath.

fcharras · 2023-12-23T23:52:36Z

So it may indicate that the issue also in your case is not merely with case but also normalization.

It seems like a user is experiencing the issue on Linux with normalization, see #9765 (comment) , but we don't know how to reproduce.

It's likely that I don't know the pytest code base enough in depth in order to propose an accurate fix. If this PR did target the root of the issue, that would be nice, but I don't have a big picture.

What I understand of the issue is that there is a dict (name2plugin) that maps keys to python modules, sometimes it's written into, sometimes it's read into. Reads will work if the keys when writing are consistent with the keys when reading. This PR points out that the crashes occur because the key is not derived from the inputs in the same way in at least two places, and proposes to apply standard normalization to fix. I assumed that paths that resolves to the same file should map to the same module. I want to believe that's a safe assumption 🤞 . And that the right thing to do would be to scan the code base and apply the same kind of normalization wherever it holds.

However, I discovered that it is indeed possible to import a same module several times by registering it with different paths, it sounds crazy but maybe there are valid usecases with that that you want to support ? It's beyond what I am able to do as far as crafting reproducers go or thinking about exotic use-cases.

But since all tests pass on this branch already and that it seems to cover a private area of the codebase (_pytest ?) I thought it might be a fairly minor change after all.

lesteve · 2024-01-08T12:56:13Z

I have added a PR on @fcharras fork to add a test based on my reproducer repo in fcharras#1 (a PR on top of this PR). For some reason I don't understand the error when running this additional test is not exactly the same (ImportPathMismatchError rather than AssertionError with the assert mod not in mods) but seems close enough (note the capital C: vs small c: in the paths):

ImportError while loading conftest 'C:\msys64\tmp\pytest-of-rjxQE\pytest-32\test_issue_97650\my_package\conftest.py'.
_pytest.pathlib.ImportPathMismatchError: ('my_package.conftest', 'c:\\msys64\\tmp\\pytest-of-rjxqe\\pytest-22\\test_issue_97650\\my_package\\conftest.py', WindowsPath('C:/msys64/tmp/pytest-of-rjxQE/pytest-32/test_issue_97650/my_package/conftest.py'))

My personal take on this PR:

we should only try to fix the issue on Windows with os.path.normcase. The issue on Windows seems the most common one as reported in Regression in 7.1 ? Windows CI started to fail with "assert mod not in mods" since 7.1 #9765.
we should not try to fix the Linux issue with os.path.normpath in this PR, which can always be done in a separate PR. My feeling is also that the issue on Linux is more of a edge-case: non-standard code layout + relative PYTHONPATH=.. with probably straight-forward work-arounds (use a more standard layout of use PYTHONPATH with a absolute path)

RonnyPfannschmidt · 2024-01-09T09:24:52Z

testing/acceptance_test.py

+        )
+    )
+
+    pytester.run(sys.executable, "setup.py", "develop")


setup.py develop has been deprecated due to numerous issues with it

does the issue only happen with legacy editables, or does it also happen with modern editables?

I know but this is the we originally had the issue in scikit-learn, and we manage to put together a reproducer. This only happens with legacy editable install python setup.py develop or its pip equivalent pip install --no-use-pep517 -e .

Note that other projects (scipy see #9765 (comment) and jdaviz #9765 (comment) for now but I could magine others) seem to have a very closely related problems in pytest 8 release candidate, due to normcase path difference on Windows. This does not seem to involve legacy editable installs, so this is a more general issue than this.

Note although this is somewhat deprecated, we still recommend to use pip install --no-use-pep517 for scikit-learn contributors. The incremental builds do not seem to work with the non-legacy installs so you rebuild everything from scratch. If you really want to know more, look at for example scikit-learn/scikit-learn#27960

RonnyPfannschmidt · 2024-01-09T09:26:12Z

testing/acceptance_test.py

+        """
+                            )
+    )
+    Path("pytest.ini").write_text(


those paths are relative to cwd

if there is a need to create a python package and doing a editable install of it,
i'm under the impression we need to rework how things are called a little

lesteve · 2024-01-09T18:56:31Z

OK in 31a42915bbcf6d59a1236624549f8bdd2723e95aI managed to reproduce the assert mod not in mods failure in the CI (finally!) https://github.com/pytest-dev/pytest/actions/runs/7465328064/job/20314253960?pr=11708#step:6:440

In my last commit, everything should be green, except coverage maybe (this is because the except clause in my additional test is not covered, I think that is fine, but I can also remove it if needed).

I tried to simplify the code diff trying to follow "solution 1." in #9765 (comment).

About the added tests:

I left the unit test @fcharras added
I added a test that is translated from the reproducer I posted above. Do let me know in case if you see some possible improvements! One thing I have noticed is that it would be nice to uninstall my_package once the test finishes. I have seen some unintended side effects when running the tests locally because of this.

bluetech · 2024-01-09T22:11:53Z

Thanks for the update @lesteve!

This solution has the problem described under "Solution 2":

Problem with this approach: the conftestpath will get out of sync with _dirpath2confmods which I think will cause issues; the code assumes they're consistent.

Further, I think we should completely avoid dependence on __file__, since we already know it can be subtly different from the conftestpath. So we should do Solution 1 in addition to Solution 3 (I can do the Solution 3 part in a separate PR).

Regarding the regression test which does setup.py develop: it's helpful that we have it, however I don't think we can merge it:

It modifies the environment which breaks test isolation
It relies on deprecated setuptools functionality

Unless we can make the test independent of setuptools stuff, we can go without a regression test for now.

lesteve · 2024-01-10T07:30:45Z

This solution has the problem described under "Solution 2":

Rereading your summary post, I think indeed you are right in the sense that I kept using mod.__file__ but what I changed is to only used normcase because I think the Linux issue is less important.

In my last commit I tried to do something closer to what you had in mind in solution 1.

The questions I have:

about the breaking API question, it is not clear whether FIX key formating divergence when inspecting plugin dictionary. #11708 (comment) was only about Pytest internal usage or also includes usages by other projects. Using this Github search I manage to find at least one project, CatBoost that seems to use consider_conftest:
https://github.com/catboost/catboost/blob/1f5857906cea28a210961c6a816884e305f063de/library/python/pytest/plugins/conftests.py#L49. There are also plenty of not important matches (e.g. venv with pytest checked into the git repo or what seems like copied-and-pasted code from pytest). Let me know if this is OK to break consider_conftest or issue a warning or something else. For now I have issued a warning with a not so great message.

I kind of think the registration key should be normalized, but that means you need to normalize it too when querying the dict. If I understand you correctly you think it is fine to not normalize it and use str(conftestpath). If we don't normalize we also have to remove the "simple" unit-test from @fcharras but we can probably find an alternative, I need to look at this in more details

I get the worry about using deprecated functionality, but my feeling is the following: this is used for a good reason (see #11708 (comment)) by a well-managed project like scikit-learn, so there are chances that it is used in the wild by a number of other projects. I would also expect the setuptools deprecation to take a while (years?) before the warning is turned into an error, but I did not find anything precise about this.

Also this is the only realistic test we have so far, personally I would lean towards including it, rather than flying blind. This would be completely fine to remove it when it becomes too much of a maintenance burden e.g. python setup.py develop becomes an error.

I agree about test isolation but I suppose this could be worked around by removing my_package after the test or if you think it is necessary creating and using a venv inside this test.

lesteve · 2024-01-10T10:20:24Z

@jgb by the way if you want take the time to put together a reproducer for your issue on Linux and even better a pytest test (but that's not the hardest part), that would be super useful 🙏.

As you might have seen from the discussion, the only reproducer is on Windows and it is under discussion whether it is going to be included in this PR or not because it is using deprecated setuptools functionality.

lesteve · 2024-01-10T10:29:38Z

@bluetech in my last commit b816b3a I tried to follow your "Solution 1" approach using str(conftestpath) for registration.

nicoddemus · 2024-01-10T10:54:36Z

src/_pytest/config/__init__.py

+        if registration_name is None:
+            warnings.warn(
+                "'consider_conftest' was called with registration_name=None, this will become an error in the future'",
+                PytestDeprecationWarning,
+            )
+            # Keep Pytest < 8 previous behaviour
+            registration_name = conftestmodule.__file__


I read the discussion, but to be honest I don't think we need to deprecate receiving None here.

@RonnyPfannschmidt what issue do you see that could be a problem accepting None as registration_name, and fallback to the previous behavior, which is what has been used for ages now?

If we feel strongly that the new parameter is needed, giving that it seems like it would break very little plugins that we know of and the "fix" is trivial, I would rather we just break the API then, but I'm not convinced that we need to, accepting None and fallback to the previous behavior seems acceptable to me.

So my preference would be (in order):

Use None as default for registration_name and fallback to the previous behavior.

Break the API and make registration_name mandatory.

I think we should go for 2, that is break without deprecation:

The function is not documented in the API Reference and is marked :private:

Searching my corpus of pytest plugins finds no matches

Searching github finds no matches (other than pytest itself and copies of it)

Sounds good to me. 👍

nicoddemus · 2024-01-10T10:59:04Z

src/_pytest/config/__init__.py

@@ -634,7 +635,8 @@ def _rget_with_confmod(
    def _importconftest(
        self, conftestpath: Path, importmode: Union[str, ImportMode], rootpath: Path
    ) -> types.ModuleType:
-        existing = self.get_plugin(str(conftestpath))
+        conftestpath_str = str(conftestpath)


I think we should create a function:

def get_conftest_plugin_name(p: Path) -> str: return str(p)

and reuse that in both places here. This makes it standout more, plus we can document the issue (and reference the issue number) in the get_conftest_plugin_name docstring.

Sorry, I am not sure I understood what you meant by "reuse that in both places here" since so for now I only renamed the variable to be more explicit confestpath_str -> conftest_plugin_name.

Maybe you were suggesting something like this?

existing = self.get_plugin(get_conftest_plugin_name(conftestpath) ... self.consider_conftest(mod, registration_name=get_conftest_plugin_name(conftestpath))

Both places are in the same function, so it feels slightly weird to call the helper function twice but maybe that is indeed what you wanted, anticipating further use of the get_conftest_plugin_name in other places of the code in the future?

My intention is that having a function we make this issue more explicit, is easy to miss the significance of that tiny str call and why it is so important in the first place. Also the function lets us add more documentation/context.

But nevermind, given this is only local to the function I think the variable name is enough to convey the importance of that str call and it being reused in the self.consider_conftest.

bluetech · 2024-01-10T18:58:32Z

src/_pytest/config/__init__.py

+        if registration_name is None:
+            warnings.warn(
+                "'consider_conftest' was called with registration_name=None, this will become an error in the future'",
+                PytestDeprecationWarning,
+            )
+            # Keep Pytest < 8 previous behaviour
+            registration_name = conftestmodule.__file__


I think we should go for 2, that is break without deprecation:

The function is not documented in the API Reference and is marked :private:

Searching my corpus of pytest plugins finds no matches

Searching github finds no matches (other than pytest itself and copies of it)

bluetech · 2024-01-10T19:05:40Z

testing/test_pluginmanager.py

+            config.pluginmanager._name2plugin[str(conftest_upper_case)] is mod_uppercase
+        )
+
+        # No str(conftestpath) normalization so conftest should be imported


I'll note that this is not the behavior we want -- we want them to be the same. But as we agreed I will add the normalization in a separate PR.

bluetech · 2024-01-10T19:07:00Z

testing/acceptance_test.py

@@ -1390,3 +1392,63 @@ def test_boo(self):
    )
    result = pytester.runpytest_subprocess()
    result.stdout.fnmatch_lines("*1 passed*")
+
+
+def test_issue_9765(pytester: Pytester) -> None:


Suggested change

def test_issue_9765(pytester: Pytester) -> None:

@pytest.mark.skip(reason="Test is not isolated")

def test_issue_9765(pytester: Pytester) -> None:

I skipped the test for now, but I may consider doing a follow-up PR to add venv creation and activation in this test, via python -m venv + activate, would you be OK with this?

venv creation is quite heavy and slow, and the setuptools thing still remains, so I'd still prefer it skipped

lesteve · 2024-01-13T04:57:37Z

I think we should go for 2, that is break without deprecation:

I went for the API break in my last commit. As I mentioned above in #11708 (comment) I found at least one project (CatBoost) that was using consider_conftest (I haven't looked at why) but since this is marked private I guess that's fine.

I haven't added a changelog, but feel free to push into this branch as you see fit. I am pretty sure @fcharras would not mind either 😉.

nicoddemus · 2024-01-13T12:30:48Z

Thanks!

I haven't added a changelog, but feel free to push into this branch as you see fit

Done. 👍

nicoddemus · 2024-01-13T12:31:25Z

I think this warrants a backport to 8.x

Fix #27806. Co-authored-by: Loïc Estève <[email protected]> Co-authored-by: Ran Benita <[email protected]> Co-authored-by: Bruno Oliveira <[email protected]>

bluetech · 2024-01-13T19:30:16Z

I've squashed the commits to one with some minor cleanups to the tests, and added a commit with the "solution 3" normalization (under my own name so as not to implicate @fcharras and @lesteve in it 😄 ). If all of the tests pass, I will merge & backport it.

(Later) the tests fail so I'll do it in a separate PR anyway...

bluetech · 2024-01-13T19:46:16Z

Thanks everyone for the hard work here!

lesteve · 2024-01-15T05:54:55Z

Great to see this one merged 🎉, thanks for steering this PR to completion!

nicoddemus · 2024-01-15T10:14:48Z

Should we backport this to 8.x?

lesteve · 2024-01-15T10:33:27Z

It has been backported I think see #11810

What would be great, at one point, is to do a second release candidate (8.0.0rc2) to make sure that the issue is fixed for projects that bumped into this and reported in #9765 (scipy, jdaviz, ewokscore, maybe others ...)

bluetech · 2024-01-15T11:25:25Z

I am planning to do 8.0.0rc2 sometime this week.

lesteve · 2024-01-15T12:46:12Z

I am planning to do 8.0.0rc2 sometime this week.

Sounds great, I don't think this is in any way urgent (e.g. today vs Friday does not make a huge difference), but it would be certainly be very useful to double-check that it fixes the issue for the affected projects.

nicoddemus · 2024-01-15T12:58:42Z

It has been backported I think see #11810

Oops missed that

fcharras · 2024-01-15T15:47:39Z

Thank you for contributors and maintainer that took the PR much farther than I initiated and making the merge happen ❤️ @jgb from our understanding now it's likely that despite reverting part of the initial pr (the "normpath" call) the PR fixed the issue on linux too.

jgb · 2024-01-16T12:32:23Z

@fcharras I just tested #11810 on top of pytest v7.4.4 and I can confirm that this does fix the issue for me on linux!

Thanks to everyone for getting this solved after all, I was afraid I would remain stuck on pytest v7.0.1 until the end of days ;-)

fcharras mentioned this pull request Dec 15, 2023

Regression in 7.1 ? Windows CI started to fail with "assert mod not in mods" since 7.1 #9765

Closed

4 tasks

fcharras force-pushed the FIX/crash_during_conftest_collection branch from 1ff7cda to 44c0319 Compare December 15, 2023 16:43

RonnyPfannschmidt reviewed Dec 18, 2023

View reviewed changes

fcharras mentioned this pull request Dec 19, 2023

Adding a non regression test for #9765 #11725

Closed

bluetech self-requested a review December 23, 2023 08:22

bluetech reviewed Dec 23, 2023

View reviewed changes

RonnyPfannschmidt reviewed Jan 9, 2024

View reviewed changes

lesteve force-pushed the FIX/crash_during_conftest_collection branch 3 times, most recently from ba9f654 to 31a4291 Compare January 9, 2024 18:26

pllim added a commit to pllim/pytest that referenced this pull request Jan 9, 2024

Manually apply pytest-dev#11708

22949f2

This was referenced Jan 10, 2024

DO NOT MERGE: Debug pytest 8 compatibility (rc_testing) spacetelescope/jdaviz#2654

Closed

Improve assert mod not in mods error message #11795

Merged

lesteve force-pushed the FIX/crash_during_conftest_collection branch from ff303fc to b940999 Compare January 10, 2024 07:25

lesteve force-pushed the FIX/crash_during_conftest_collection branch 2 times, most recently from 126b151 to 6c3cac3 Compare January 10, 2024 09:20

lesteve force-pushed the FIX/crash_during_conftest_collection branch from 6c3cac3 to b816b3a Compare January 10, 2024 09:48

nicoddemus reviewed Jan 10, 2024

View reviewed changes

bluetech reviewed Jan 10, 2024

View reviewed changes

nicoddemus approved these changes Jan 13, 2024

View reviewed changes

Fix assert mod not in mods crash

a7c2549

Fix #27806. Co-authored-by: Loïc Estève <[email protected]> Co-authored-by: Ran Benita <[email protected]> Co-authored-by: Bruno Oliveira <[email protected]>

bluetech force-pushed the FIX/crash_during_conftest_collection branch from facd721 to 203889b Compare January 13, 2024 19:24

bluetech force-pushed the FIX/crash_during_conftest_collection branch from 203889b to a7c2549 Compare January 13, 2024 19:30

bluetech approved these changes Jan 13, 2024

View reviewed changes

bluetech merged commit e403bbf into pytest-dev:main Jan 13, 2024
23 of 24 checks passed

bluetech added the backport 8.0.x label Jan 13, 2024

github-actions bot mentioned this pull request Jan 13, 2024

[8.0.x] FIX key formating divergence when inspecting plugin dictionary. #11810

Merged

lesteve mentioned this pull request Jan 15, 2024

Main and 8.0.x (not released yet): cannot find fixture when using --pyargs #11816

Closed

4 tasks

fcharras deleted the FIX/crash_during_conftest_collection branch January 15, 2024 12:19

woutdenolf mentioned this pull request Jan 15, 2024

module __file__ attribute is not the canonical path #11821

Closed

lesteve mentioned this pull request May 30, 2024

BUG: pytest error when loading conftest (seemingly platform-specific) scikit-learn/scikit-learn#27806

Closed

	def test_issue_9765(pytester: Pytester) -> None:
	@pytest.mark.skip(reason="Test is not isolated")
	def test_issue_9765(pytester: Pytester) -> None:

FIX key formating divergence when inspecting plugin dictionary. #11708

FIX key formating divergence when inspecting plugin dictionary. #11708

Conversation

fcharras commented Dec 15, 2023 • edited Loading

fcharras commented Dec 15, 2023

fcharras commented Dec 15, 2023

bluetech commented Dec 16, 2023

lesteve commented Dec 18, 2023

jgb commented Dec 18, 2023

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fcharras commented Dec 19, 2023

bluetech commented Dec 23, 2023

Choose a reason for hiding this comment

lesteve commented Dec 23, 2023 • edited Loading

fcharras commented Dec 23, 2023 • edited Loading

lesteve commented Jan 8, 2024 • edited Loading

Choose a reason for hiding this comment

lesteve Jan 9, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lesteve commented Jan 9, 2024 • edited Loading

bluetech commented Jan 9, 2024

lesteve commented Jan 10, 2024 • edited Loading

lesteve commented Jan 10, 2024

lesteve commented Jan 10, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nicoddemus Jan 13, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

lesteve commented Jan 13, 2024 • edited Loading

nicoddemus commented Jan 13, 2024

nicoddemus commented Jan 13, 2024

bluetech commented Jan 13, 2024

bluetech commented Jan 13, 2024

lesteve commented Jan 15, 2024

nicoddemus commented Jan 15, 2024

lesteve commented Jan 15, 2024 • edited Loading

bluetech commented Jan 15, 2024

lesteve commented Jan 15, 2024

nicoddemus commented Jan 15, 2024

fcharras commented Jan 15, 2024

jgb commented Jan 16, 2024

fcharras commented Dec 15, 2023 •

edited

Loading

lesteve commented Dec 23, 2023 •

edited

Loading

fcharras commented Dec 23, 2023 •

edited

Loading

lesteve commented Jan 8, 2024 •

edited

Loading

lesteve Jan 9, 2024 •

edited

Loading

lesteve commented Jan 9, 2024 •

edited

Loading

lesteve commented Jan 10, 2024 •

edited

Loading

nicoddemus Jan 13, 2024 •

edited

Loading

lesteve commented Jan 13, 2024 •

edited

Loading

lesteve commented Jan 15, 2024 •

edited

Loading