Make more tests robust and independent

[glatterf42]

Unfortunately, despite our recent improvements in #543, we still see some apparent flakiness. Its main source seems to be the setup-graphviz action on the windows runners, which sometimes fails like this:

C:\ProgramData\Chocolatey\bin\choco.exe install graphviz
Chocolatey v2.3.0
Installing the following packages:
graphviz
By installing, you accept licenses for the packages.
Downloading package from source 'https://community.chocolatey.org/api/v2/'
[NuGet] Error downloading 'Graphviz.12.1.1 : chocolatey-core.extension [1.3.5, )' from 'https://community.chocolatey.org/api/v2/package/Graphviz/12.1.1'.
[NuGet] Response status code does not indicate success: 503 (Service Unavailable: Back-end server is at capacity).
[NuGet] Error downloading 'Graphviz.12.1.1 : chocolatey-core.extension [1.3.5, )' from 'https://community.chocolatey.org/api/v2/package/Graphviz/12.1.1'.
[NuGet] Response status code does not indicate success: 503 (Service Unavailable: Back-end server is at capacity).
Graphviz not installed. An error occurred during installation:
 Error downloading 'Graphviz.12.1.1 : chocolatey-core.extension [1.3.5, )' from 'https://community.chocolatey.org/api/v2/package/Graphviz/12.1.1'.
Graphviz package files install failed with exit code 1. Performing other installation steps.
The install of Graphviz was NOT successful.
Graphviz not installed. An error occurred during installation:
 Error downloading 'Graphviz.12.1.1 : chocolatey-core.extension [1.3.5, )' from 'https://community.chocolatey.org/api/v2/package/Graphviz/12.1.1'.

Chocolatey installed 0/1 packages. 1 packages failed.
 See the log for details (C:\ProgramData\chocolatey\logs\chocolatey.log).

Failures
 - Graphviz (exited 1) - Graphviz not installed. An error occurred during installation:
 Error downloading 'Graphviz.12.1.1 : chocolatey-core.extension [1.3.5, )' from 'https://community.chocolatey.org/api/v2/package/Graphviz/12.1.1'.

In addition, there was a new flaky test today:

test_connect_message::AssertionError

      ____________________________ test_connect_message _____________________________
[gw0] win32 -- Python 3.9.13 C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe

capfd = <_pytest.capture.CaptureFixture object at 0x000001C064A052B0>
caplog = <_pytest.logging.LogCaptureFixture object at 0x000001C065035A30>
request = <FixtureRequest for <Function test_connect_message>>

    def test_connect_message(capfd, caplog, request):
        msg = (
            f"connected to database 'jdbc:hsqldb:mem://{request.node.name}_0' "
            "(user: ixmp)..."
        )
    
        ixmp.Platform(
            backend="jdbc",
            driver="hsqldb",
            url=f"jdbc:hsqldb:mem://{request.node.name}_0",
            log_level="INFO",
        )
    
        # Java code via JPype does not log to the standard Python logger
        assert not any(msg in m for m in caplog.messages)
    
        # NB cannot inspect capfd here. Depending on the order in which the tests were run,
        #    a previous run may have left the Java log level higher than INFO, in which
        #    case the Java Platform object would not write to stderr before set_log_level()
        #    in the above call. Try again now that the level is INFO:
        msg = (
            f"connected to database 'jdbc:hsqldb:mem://{request.node.name}_1' "
            "(user: ixmp)..."
        )
        ixmp.Platform(
            backend="jdbc",
            driver="hsqldb",
            url=f"jdbc:hsqldb:mem://{request.node.name}_1",
        )
    
        # Instead, log messages are printed to stdout
        captured = capfd.readouterr()
>       assert msg in captured.out
E       assert "connected to database 'jdbc:hsqldb:mem://test_connect_message_1' (user: ixmp)..." in ''
E        +  where '' = CaptureResult(out='', err="2024-09-26 05:51:51,552  INFO at.ac.iiasa.ixmp.Platform:182 - closed the connection to database 'jdbc:hsqldb:mem://test_connect_message_0'\r\n2024-09-26 05:51:51,708  INFO at.ac.iiasa.ixmp.Platform:165 - Welcome to the IX modeling platform!\r\n2024-09-26 05:51:51,708  INFO at.ac.iiasa.ixmp.Platform:166 -  connected to database 'jdbc:hsqldb:mem://test_connect_message_1' (user: ixmp)...\r\n").out

ixmp\tests\backend\test_jdbc.py:277: AssertionError
------------------------------ Captured log call ------------------------------
INFO     ixmp.backend.jdbc:jdbc.py:288 launching ixmp.Platform connected to jdbc:hsqldb:mem://test_connect_message_1

Which almost seems like out and err got mixed up in the CaptureResult. If this originates from one test changing the logging level without changing it back again, we should figure out a way to change the level just for this one test, without affecting the whole environment.

[glatterf42]

• test_R_transport::CellTimeoutError

  FAILED ixmp/tests/test_tutorials.py::test_R_transport@Darwin3.8.18-1 - nbclient.exceptions.CellTimeoutError: A cell timed out while it was being executed, after 30 seconds.
  The message was: Cell execution timed out.
  Here is a preview of the cell contents:
  -------------------
  # Load reticulate, used to access the Python API from R
  library(reticulate)
  
  # Import ixmp and message_ix, just as in Python
  ixmp <- import("ixmp")
  -------------------

  This reappeared today on macos-13-py3.8, unfortunately.

[glatterf42]

The same error at test_connect_message happened today on Windows-latest-py3.11, though with slightly different error and captured output:

test_connect_message::AssertionError-2

>       assert msg in captured.out
E       assert "connected to database 'jdbc:hsqldb:mem://test_connect_message_1' (user: ixmp)..." in ''
E        +  where '' = CaptureResult(out='', err='').out

ixmp\tests\backend\test_jdbc.py:277: AssertionError
 ---------------------------- Captured stdout call -----------------------------
2024-09-30 05:17:54,257  INFO at.ac.iiasa.ixmp.Platform:182 - closed the connection to database 'jdbc:hsqldb:mem://test_connect_message_0'

2024-09-30 05:17:54,445  INFO at.ac.iiasa.ixmp.Platform:165 - Welcome to the IX modeling platform!

2024-09-30 05:17:54,445  INFO at.ac.iiasa.ixmp.Platform:166 -  connected to database 'jdbc:hsqldb:mem://test_connect_message_1' (user: ixmp)...

------------------------------ Captured log call ------------------------------
INFO     ixmp.backend.jdbc:jdbc.py:288 launching ixmp.Platform connected to jdbc:hsqldb:mem://test_connect_message_1

[khaeru]

Thanks for continuing to track these.

I'll remove the bug label, because I think we should reserve that for cases when we confirm that the code per se does not do what it is expected or advertised to do in user or downstream applications. These flaky tests do pass, meaning the code itself and the tests work as expected/advertised, but only under certain conditions specifically in the GitHub Actions environment do they occasionally fail.

If particular users report failures of the code in real-world usage that correspond to the flaky tests on GHA, then those are definitely bugs and we can open separate issues for each one.

[glatterf42]

• test_close_increased_logging::AssertionError

  ________________________ test_close_increased_logging _________________________
  [gw1] win32 -- Python 3.9.13 C:\hostedtoolcache\windows\Python\3.9.13\x64\python.exe
  
  test_mp_f = <ixmp.core.platform.Platform object at 0x0000015C13BA03A0>
  capfd = <_pytest.capture.CaptureFixture object at 0x0000015C13BA0EB0>
  
      def test_close_increased_logging(test_mp_f, capfd):
          """Platform.close_db() doesn't throw needless exceptions."""
          # Use the session-scoped fixture to avoid affecting other tests in this file
          mp = test_mp_f
    
          # Close once
          mp.close_db()
    
          # Set higher log level INFO
          level = mp.get_log_level()
          mp.set_log_level(logging.INFO)
    
          # Close again, once already closed
          # With logging.INFO, a message is printed
          mp.close_db()
          captured = capfd.readouterr()
          msg = "Database connection could not be closed or was already closed"
          try:
  >           assert msg in captured.out
  E           AssertionError: assert 'Database connection could not be closed or was already closed' in ''
  E            +  where '' = CaptureResult(out='', err='').out
  
  ixmp\tests\backend\test_jdbc.py:83: AssertionError

[glatterf42]

• test_default_version::AssertionError

  ______________________ TestScenario.test_default_version ______________________
  [gw1] win32 -- Python 3.12.7 C:\hostedtoolcache\windows\Python\3.12.7\x64\python.exe
  
  self = <ixmp.tests.core.test_scenario.TestScenario object at 0x0000023A4553EC60>
  mp = <ixmp.core.platform.Platform object at 0x0000023A454B05F0>
  
      def test_default_version(self, mp):
          scen = ixmp.Scenario(mp, **models["dantzig"])
  >       assert scen.version == 2
  E       assert 5 == 2
  E        +  where 5 = <ixmp.core.scenario.Scenario object at 0x0000023A518462D0>.version
  
  ixmp\tests\core\test_scenario.py:82: AssertionError
  ---------------------------- Captured stdout setup ----------------------------
  --- Warning: The GAMS version [43.4.1] differs from the API version [24.8.3].
  
  --- Warning: The GAMS version [43.4.1] differs from the API version [24.8.3].

• test_from_url::AssertionError

  _________________________ TestScenario.test_from_url __________________________
  [gw1] win32 -- Python 3.12.7 C:\hostedtoolcache\windows\Python\3.12.7\x64\python.exe
  
  self = <ixmp.tests.core.test_scenario.TestScenario object at 0x0000023A4553EF00>
  mp = <ixmp.core.platform.Platform object at 0x0000023A50746FF0>
  caplog = <_pytest.logging.LogCaptureFixture object at 0x0000023A506D43E0>
  
      def test_from_url(self, mp, caplog):
          url = f"ixmp://{mp.name}/Douglas Adams/Hitchhiker"
    
          # Default version is loaded
          scen, mp = ixmp.Scenario.from_url(url)
  >       assert scen.version == 1
  E       assert 2 == 1
  E        +  where 2 = <ixmp.core.scenario.Scenario object at 0x0000023A506D7BF0>.version
  
  ixmp\tests\core\test_scenario.py:89: AssertionError

• test_add_timeseries_with_extra_col[TimeSeries-long]::AssertionError

  _____ TestTimeSeries.test_add_timeseries_with_extra_col[TimeSeries-long] ______
  [gw1] win32 -- Python 3.12.7 C:\hostedtoolcache\windows\Python\3.12.7\x64\python.exe
  
  self = <ixmp.tests.core.test_timeseries.TestTimeSeries object at 0x0000023A45580E00>
  caplog = <_pytest.logging.LogCaptureFixture object at 0x0000023A507EED20>
  ts = <ixmp.core.timeseries.TimeSeries object at 0x0000023A507ED190>
  format = 'long'
  
      @pytest.mark.parametrize("format", ["long", "wide"])
      def test_add_timeseries_with_extra_col(self, caplog, ts, format):
          _data = DATA[0].copy()
          _data["climate_model"] = [0, 1]
          data = _data if format == "long" else wide(_data)
    
          # check that extra column wasn't dropped by `wide(_data)
          assert "climate_model" in data.columns
    
          # Data added
          ts.add_timeseries(data)
          # TODO: add check that warning message is displayed
          ts.commit("")
  >       assert [
              "Dropped extra column(s) ['climate_model'] from data"
          ] == caplog.messages
  E       assert ["Dropped ext...'] from data"] == []
  E         
  E         Left contains one more item: "Dropped extra column(s) ['climate_model'] from data"
  E         
  E         Full diff:
  E         - []
  E         + [
  E         +     "Dropped extra column(s) ['climate_model'] from data",
  E         + ]
  
  ixmp\tests\core\test_timeseries.py:423: AssertionError

• test_add_timeseries_with_extra_col[Scenario-long]::AssertionError

  ```python ______ TestTimeSeries.test_add_timeseries_with_extra_col[Scenario-long] _______ [gw1] win32 -- Python 3.12.7 C:\hostedtoolcache\windows\Python\3.12.7\x64\python.exe

  self = <ixmp.tests.core.test_timeseries.TestTimeSeries object at 0x0000023A45580FE0>
  caplog = <_pytest.logging.LogCaptureFixture object at 0x0000023A518F1C70>
  ts = <ixmp.core.scenario.Scenario object at 0x0000023A518F1850>, format = 'long'

  @pytest.mark.parametrize("format", ["long", "wide"])
  def test_add_timeseries_with_extra_col(self, caplog, ts, format):
      _data = DATA[0].copy()
      _data["climate_model"] = [0, 1]
      data = _data if format == "long" else wide(_data)
  
      # check that extra column wasn't dropped by `wide(_data)
      assert "climate_model" in data.columns
  
      # Data added
      ts.add_timeseries(data)
      # TODO: add check that warning message is displayed
      ts.commit("")
  

    assert [
  

          "Dropped extra column(s) ['climate_model'] from data"
      ] == caplog.messages
  

  E assert ["Dropped ext...'] from data"] == []
  E
  E Left contains one more item: "Dropped extra column(s) ['climate_model'] from data"
  E
  E Full diff:
  E - []
  E + [
  E + "Dropped extra column(s) ['climate_model'] from data",
  E + ]

  ixmp\tests\core\test_timeseries.py:423: AssertionError

  </details>
  

• test_add_timeseries_with_extra_col[Scenario-wide]::AssertionError

  ______ TestTimeSeries.test_add_timeseries_with_extra_col[Scenario-wide] _______
  [gw1] win32 -- Python 3.12.7 C:\hostedtoolcache\windows\Python\3.12.7\x64\python.exe
  
  self = <ixmp.tests.core.test_timeseries.TestTimeSeries object at 0x0000023A455810A0>
  caplog = <_pytest.logging.LogCaptureFixture object at 0x0000023A518F2B40>
  ts = <ixmp.core.scenario.Scenario object at 0x0000023A518F2C60>, format = 'wide'
  
      @pytest.mark.parametrize("format", ["long", "wide"])
      def test_add_timeseries_with_extra_col(self, caplog, ts, format):
          _data = DATA[0].copy()
          _data["climate_model"] = [0, 1]
          data = _data if format == "long" else wide(_data)
    
          # check that extra column wasn't dropped by `wide(_data)
          assert "climate_model" in data.columns
    
          # Data added
          ts.add_timeseries(data)
          # TODO: add check that warning message is displayed
          ts.commit("")
  >       assert [
              "Dropped extra column(s) ['climate_model'] from data"
          ] == caplog.messages
  E       assert ["Dropped ext...'] from data"] == []
  E         
  E         Left contains one more item: "Dropped extra column(s) ['climate_model'] from data"
  E         
  E         Full diff:
  E         - []
  E         + [
  E         +     "Dropped extra column(s) ['climate_model'] from data",
  E         + ]
  
  ixmp\tests\core\test_timeseries.py:423: AssertionError

• test_store_ts[attrseries]::IndexError

  __________________________ test_store_ts[attrseries] __________________________
  [gw1] win32 -- Python 3.12.7 C:\hostedtoolcache\windows\Python\3.12.7\x64\python.exe
  
  request = <FixtureRequest for <Function test_store_ts[attrseries]>>
  caplog = <_pytest.logging.LogCaptureFixture object at 0x0000023A518F7FE0>
  test_mp = <ixmp.core.platform.Platform object at 0x0000023A454B05F0>
  
      def test_store_ts(request, caplog, test_mp) -> None:
          # Computer and target scenario
          c = Computer()
     
          # Target scenario
          model_name = __name__
          scenario_name = "test scenario"
          scen = Scenario(test_mp, model_name, scenario_name, version="new")
          scen.commit("Empty scenario")
          c.add("target", scen)
    
          # Add test data to the Computer: a pd.DataFrame
          input_1 = test_data[0].assign(variable="Foo")
          c.add("input 1", input_1)
    
          # A pyam.IamDataFrame
          input_2 = test_data[2050].assign(variable="Bar")
          c.add("input 2", pyam.IamDataFrame(input_2))
    
          # Expected results: same as input, but with the `model` and `scenario` columns
          # filled automatically.
          expected_1 = input_1.assign(model=model_name, scenario=scenario_name)
          expected_2 = input_2.assign(model=model_name, scenario=scenario_name)
    
          # Task to update the scenario with the data
          c.add("test 1", store_ts, "target", "input 1", "input 2")
    
          # Scenario starts empty of time series data
          assert 0 == len(scen.timeseries())
    
          # The computation runs successfully
          c.get("test 1")
    
          # All rows from both inputs are present
          assert len(input_1) + len(input_2) == len(scen.timeseries())
    
          # Input is stored exactly
          assert_frame_equal(expected_1, scen.timeseries(variable="Foo"))
          assert_frame_equal(expected_2, scen.timeseries(variable="Bar"))
    
          # Data with an unregistered region name
          c.add("input 3", test_data[0].assign(variable="Foo", region="Moon"))
          c.add("test 2", store_ts, "target", "input 3")
    
          # Succeeds with default strict=False
          caplog.clear()
          c.get("test 2")
    
          # A message is logged
  >       r = caplog.record_tuples[-1]
  E       IndexError: list index out of range
  
  ixmp\tests\report\test_operator.py:213: IndexError

[glatterf42]

• test_platform_copy::AssertionError

    ______________________________ test_platform_copy ______________________________
  [gw3] darwin -- Python 3.9.20 /Users/runner/hostedtoolcache/Python/3.9.20/x64/bin/python
  
  ixmp_cli = <ixmp.testing.ixmp_cli.<locals>.Runner object at 0x15e23b640>
  tmp_path = PosixPath('/private/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/pytest-of-runner/pytest-0/popen-gw3/test_platform_copy0')
  request = <FixtureRequest for <Function test_platform_copy>>
  
      def test_platform_copy(ixmp_cli, tmp_path, request):
          """Test 'platform' command."""
          test_specific_name = request.node.name
    
          def call(*args, exit_0=True):
              result = ixmp_cli.invoke(["platform"] + list(map(str, args)))
              assert not exit_0 or result.exit_code == 0, result.output
              return result
    
          # Add some temporary platform configuration
          call(
              "add",
              f"p1-{test_specific_name}",
              "jdbc",
              "oracle",
              "HOSTNAME",
              "USER",
              "PASSWORD",
          )
          call(
              "add",
              f"p2-{test_specific_name}",
              "jdbc",
              "hsqldb",
              tmp_path.joinpath(f"p2-{test_specific_name}"),
          )
          # Force connection to p2 so that files are created
          ixmp_cli.invoke([f"--platform=p2-{test_specific_name}", "list"])
    
          # Dry-run produces expected output
          r = call("copy", f"p2-{test_specific_name}", f"p3-{test_specific_name}")
          assert re.search(
              f"Copy .*p2-{test_specific_name}.script → .*p3-{test_specific_name}.script",
              r.output,
          )
          with pytest.raises(ValueError):
              # New platform configuration is not saved
              ixmp.config.get_platform_info(f"p3-{test_specific_name}")
    
          # --go actually copies files, saves new platform config
  >       r = call("copy", "--go", f"p2-{test_specific_name}", f"p3-{test_specific_name}")
  
  ixmp/tests/test_cli.py:208: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  exit_0 = True
  args = ('copy', '--go', 'p2-test_platform_copy', 'p3-test_platform_copy')
  result = <Result FileNotFoundError(2, 'No such file or directory')>
  @py_assert1 = ['not True', "1\n{1 = <Result FileNotFoundError(2, 'No such file or directory')>.exit_code\n} == 0"]
  
      def call(*args, exit_0=True):
          result = ixmp_cli.invoke(["platform"] + list(map(str, args)))
  >       assert not exit_0 or result.exit_code == 0, result.output
  E       AssertionError: Copy /private/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/pytest-of-runner/pytest-0/popen-gw3/test_platform_copy0/p2-test_platform_copy.script.new → /private/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/pytest-of-runner/pytest-0/popen-gw3/test_platform_copy0/p3-test_platform_copy.new
  E         Copy /private/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/pytest-of-runner/pytest-0/popen-gw3/test_platform_copy0/p2-test_platform_copy.script → /private/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/pytest-of-runner/pytest-0/popen-gw3/test_platform_copy0/p3-test_platform_copy.script
  E         Copy /private/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/pytest-of-runner/pytest-0/popen-gw3/test_platform_copy0/p2-test_platform_copy.properties.new → /private/var/folders/nj/wh528zms06j9t8y7bmlvpmjm0000gn/T/pytest-of-runner/pytest-0/popen-gw3/test_platform_copy0/p3-test_platform_copy.new
  E         
  E       assert (not True or 1 == 0)
  E        +  where 1 = <Result FileNotFoundError(2, 'No such file or directory')>.exit_code
  
  ixmp/tests/test_cli.py:174: AssertionError

[glatterf42]

• setup of test_check_single_model_access and test_check_multi_model_access::ConnectionRefusedError

  _______________ ERROR at setup of test_check_single_model_access _______________
  [gw3] darwin -- Python 3.9.20 /Users/runner/hostedtoolcache/Python/3.9.20/x64/bin/python
  
  server = None
  
      @pytest.fixture(scope="function")
      def mock(server):
          # Create the mock server
  >       httpmock = HTTPMock("localhost", 8000)
  
  ixmp/tests/test_access.py:43: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/http.py:49: in __init__
      super(HTTPMock, self).__init__(host, port, timeout, name)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/__init__.py:102: in __init__
      self.pretender_details = self._request_mock_access()
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/__init__.py:152: in _request_mock_access
      response, data = self.boss_access.http('POST',
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/__init__.py:39: in http
      response = self._get_response(method, *args, **kwargs)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/__init__.py:27: in _get_response
      self.connection.request(method=method, *args, **kwargs)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:1285: in request
      self._send_request(method, url, body, headers, encode_chunked)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:1331: in _send_request
      self.endheaders(body, encode_chunked=encode_chunked)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:1280: in endheaders
      self._send_output(message_body, encode_chunked=encode_chunked)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:1040: in _send_output
      self.send(msg)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:980: in send
      self.connect()
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:946: in connect
      self.sock = self._create_connection(
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/socket.py:856: in create_connection
      raise err
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  address = ('localhost', 8000), timeout = <object object at 0x1097e53a0>
  source_address = None
  
      def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                            source_address=None):
          """Connect to *address* and return the socket object.
    
          Convenience function.  Connect to *address* (a 2-tuple ``(host,
          port)``) and return the socket object.  Passing the optional
          *timeout* parameter will set the timeout on the socket instance
          before attempting to connect.  If no *timeout* is supplied, the
          global default timeout setting returned by :func:`getdefaulttimeout`
          is used.  If *source_address* is set it must be a tuple of (host, port)
          for the socket to bind as a source address before making the connection.
          A host of '' or port 0 tells the OS to use the default.
          """
    
          host, port = address
          err = None
          for res in getaddrinfo(host, port, 0, SOCK_STREAM):
              af, socktype, proto, canonname, sa = res
              sock = None
              try:
                  sock = socket(af, socktype, proto)
                  if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                      sock.settimeout(timeout)
                  if source_address:
                      sock.bind(source_address)
  >               sock.connect(sa)
  E               ConnectionRefusedError: [Errno 61] Connection refused
  
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/socket.py:844: ConnectionRefusedError
  ---------------------------- Captured stderr setup -----------------------------
  2024-11-06 05:10:16,187 DEBUG:pretenders.server.server:44: Setting pretender timeout: 120
  2024-11-06 05:10:16,187 DEBUG:pretenders.server.server:16: Starting maintainer process
  _______________ ERROR at setup of test_check_multi_model_access ________________
  [gw3] darwin -- Python 3.9.20 /Users/runner/hostedtoolcache/Python/3.9.20/x64/bin/python
  
  server = None
  
      @pytest.fixture(scope="function")
      def mock(server):
          # Create the mock server
  >       httpmock = HTTPMock("localhost", 8000)
  
  ixmp/tests/test_access.py:43: 
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/http.py:49: in __init__
      super(HTTPMock, self).__init__(host, port, timeout, name)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/__init__.py:102: in __init__
      self.pretender_details = self._request_mock_access()
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/__init__.py:152: in _request_mock_access
      response, data = self.boss_access.http('POST',
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/__init__.py:39: in http
      response = self._get_response(method, *args, **kwargs)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/site-packages/pretenders/client/__init__.py:27: in _get_response
      self.connection.request(method=method, *args, **kwargs)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:1285: in request
      self._send_request(method, url, body, headers, encode_chunked)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:1331: in _send_request
      self.endheaders(body, encode_chunked=encode_chunked)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:1280: in endheaders
      self._send_output(message_body, encode_chunked=encode_chunked)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:1040: in _send_output
      self.send(msg)
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:980: in send
      self.connect()
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/http/client.py:946: in connect
      self.sock = self._create_connection(
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/socket.py:856: in create_connection
      raise err
  _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
  
  address = ('localhost', 8000), timeout = <object object at 0x1097e53a0>
  source_address = None
  
      def create_connection(address, timeout=_GLOBAL_DEFAULT_TIMEOUT,
                            source_address=None):
          """Connect to *address* and return the socket object.
    
          Convenience function.  Connect to *address* (a 2-tuple ``(host,
          port)``) and return the socket object.  Passing the optional
          *timeout* parameter will set the timeout on the socket instance
          before attempting to connect.  If no *timeout* is supplied, the
          global default timeout setting returned by :func:`getdefaulttimeout`
          is used.  If *source_address* is set it must be a tuple of (host, port)
          for the socket to bind as a source address before making the connection.
          A host of '' or port 0 tells the OS to use the default.
          """
    
          host, port = address
          err = None
          for res in getaddrinfo(host, port, 0, SOCK_STREAM):
              af, socktype, proto, canonname, sa = res
              sock = None
              try:
                  sock = socket(af, socktype, proto)
                  if timeout is not _GLOBAL_DEFAULT_TIMEOUT:
                      sock.settimeout(timeout)
                  if source_address:
                      sock.bind(source_address)
  >               sock.connect(sa)
  E               ConnectionRefusedError: [Errno 61] Connection refused
  
  ../../../hostedtoolcache/Python/3.9.20/x64/lib/python3.9/socket.py:844: ConnectionRefusedError

[glatterf42]

• test_del_ts::AssertionError

  _________________________________ test_del_ts _________________________________
  [gw0] win32 -- Python 3.12.7 C:\hostedtoolcache\windows\Python\3.12.7\x64\python.exe
  
  request = <FixtureRequest for <Function test_del_ts>>
  
      @pytest.mark.flaky(
          reruns=5,
          rerun_delay=2,
          condition="GITHUB_ACTIONS" in os.environ and platform.system() == "Windows",
          reason="Flaky; see iiasa/ixmp#543",
      )
      def test_del_ts(request):
          mp = ixmp.Platform(
              backend="jdbc",
              driver="hsqldb",
              url="jdbc:hsqldb:mem:test_del_ts",
          )
    
          backend: ixmp.backend.jdbc.JDBCBackend = mp._backend  # type: ignore
    
          # Number of Java objects referenced by the JDBCBackend
          N_obj = len(backend.jindex)
    
          # Create a list of some Scenario objects
          N = 8
          scenarios = [make_dantzig(mp, request=request)]
          for i in range(1, N):
              scenarios.append(scenarios[0].clone(scenario=f"{request.node.name} clone {i}"))
    
          # Number of referenced objects has increased by 8
          assert len(backend.jindex) == N_obj + N
    
          # Pop and free the objects
          for i in range(N):
              s = scenarios.pop(0)
    
              message = "\n".join(map(str, gc.get_referrers(s)))
    
              # The variable 's' is the only reference to this Scenario object
              assert 1 == getrefcount(s) - 1, (message, gc.garbage)
    
              # ID of the Scenario object
              s_id = id(s)
    
              # Underlying Java object
              s_jobj = backend.jindex[s]
    
              # Now delete the Scenario object
              # del s # should work, but doesn't always resolve to s.__del__()
              backend.del_ts(s)
    
              # Number of referenced objects decreases by 1
  >           assert len(backend.jindex) == N_obj + N - (i + 1)
  E           assert 7 == ((8 + 8) - (0 + 1))
  E            +  where 7 = len(<WeakKeyDictionary at 0x1c6f48ff410>)
  E            +    where <WeakKeyDictionary at 0x1c6f48ff410> = <ixmp.backend.jdbc.JDBCBackend object at 0x000001C69DF1D760>.jindex
  
  ixmp\tests\backend\test_jdbc.py:431: AssertionError