Rebuild for openmpi 5, add wget dep, fix ftp.unidata refs, increase ctest timeout

[xylar]

Checklist

• Used a personal fork of the feedstock to propose changes
• Bumped the build number (if the version is unchanged)
• Re-rendered with the latest conda-smithy (Use the phrase @conda-forge-admin, please rerender in a comment in this PR for automated rerendering)
• Ensured the license file is being packaged.

• rebuild openmpi5 and applies updated env vars
• add wget dep needed for testing scripts
• fix now-defunct ftp.unidata.ucar.edu references in testing
• increase default ctest timeout to hopefully help slow mac builds

closes #193

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

I do have some suggestions for making it better though...

For recipe:

• This recipe is using a compiler, which now requires adding a build dependence on {{ stdlib("c") }} as well. Note that this rule applies to each output of the recipe using a compiler. For further details, please see META: {{ stdlib("c") }} migration conda-forge.github.io#2102.

[xylar]

It seems like this got missed in:
https://conda-forge.org/status/migration/openmpi5
Does anyone have time to report this to the bot folks? I'm a bit too pressed for time to figure out where to report it right now.

[conda-forge-webservices]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[xylar]

@conda-forge-admin, please rerender

[xylar]

Needed by conda-forge/netcdf4-feedstock#163 and presumably others

[xylar]

@conda-forge/libnetcdf, anyone able to investigate the failing tests? They seem unrelated to OpenMPI 5 and rather to be due to missing files or something:

 53/246 Test  #47: ncdump_tst_grp_spec ...................   Passed    0.05 sec
test 48
        Start  48: ncdump_tst_nccopy5

48: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_nccopy5.sh tst_nccopy5.sh "
48: Working Directory: $SRC_DIR/build-shared/ncdump
48: Test timeout computed to be: 1500
48: 
48: *** Creating chunkable test file tmp_nc5_base.nc...
48: ok.
48: *** Tests successful!
48: *** Test nccopy -c with per-variable chunking; classic->enhanced
4: Could not ftp AMSR_E_L2_Rain_V10_200905312326_A.hdf.gz
4: $SRC_DIR/build-shared/hdf4_test/run_get_hdf4_files.sh: line 47: return: can only `return' from a function or sourced script
 54/246 Test   #4: hdf4_test_run_get_hdf4_files ..........***Failed   20.84 sec

Getting HDF4 sample files from Unidata FTP site...
Getting HDF4 test files AMSR_E_L2_Rain_V10_200905312326_A.hdf AMSR_E_L3_DailyLand_V06_20020619.hdf     MYD29.A2009152.0000.005.2009153124331.hdf MYD29.A2002185.0000.005.2007160150627.hdf     MOD29.A2000055.0005.005.2006267200024.hdf
$SRC_DIR/build-shared/hdf4_test/run_get_hdf4_files.sh: line 22: wget: command not found
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: ftp.unidata.ucar.edu; Unknown error
wget failed: try 1
$SRC_DIR/build-shared/hdf4_test/run_get_hdf4_files.sh: line 22: wget: command not found
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: ftp.unidata.ucar.edu; Unknown error
wget failed: try 2
$SRC_DIR/build-shared/hdf4_test/run_get_hdf4_files.sh: line 22: wget: command not found
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: ftp.unidata.ucar.edu; Unknown error
wget failed: try 3
$SRC_DIR/build-shared/hdf4_test/run_get_hdf4_files.sh: line 22: wget: command not found
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed

  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0curl: (6) Could not resolve host: ftp.unidata.ucar.edu; Unknown error
wget failed: try 4
Could not ftp AMSR_E_L2_Rain_V10_200905312326_A.hdf.gz
$SRC_DIR/build-shared/hdf4_test/run_get_hdf4_files.sh: line 47: return: can only `return' from a function or sourced script

test 49
        Start  49: ncdump_tst_inttags4

49: Test command: /usr/bin/bash "-c" "export srcdir=$SRC_DIR/ncdump;export TOPSRCDIR=$SRC_DIR;$SRC_DIR/build-shared/ncdump/tst_inttags4.sh tst_inttags4.sh "
49: Working Directory: $SRC_DIR/build-shared/ncdump
49: Test timeout computed to be: 1500
49: *** Test netcdf-4 integer constant suffixes
49: *** creating inttags4.nc from inttags4.cdl...
49: *** creating tst_inttags4.cdl from inttags4.nc...
49: *** comparing tst_inttags4.cdl to ref_inttags4.nc...
 55/246 Test  #49: ncdump_tst_inttags4 ...................   Passed    0.05 sec

[akrherz]

Appears to be a simple wget: command not found , will push a change now

[akrherz]

Eh, no perms to push @xylar

[xylar]

@akrherz, it doesn't look like you're a maintainer of this feedstock. I can add you as a collaborator on my fork.

[xylar]

Oh, I'm really sick this week and it's clear that my head is just too foggy. command not found! That should have been a great clue! I was so stuck on the URLs themselves being the issue.

[xylar]

Thanks @akrherz!

[akrherz]

Now the error are updates Unidata did

Could not resolve host: ftp.unidata.ucar.edu; Unknown error

They are dangerously close to a new libnetcdf release, so maybe wait a bit more?

[xylar]

They are dangerously close to a new libnetcdf release, so maybe wait a bit more?

Okay, but that'll put the whole OpenMPI migration on hold, right?

[xylar]

Thanks @akrherz! That seems to be doing the trick!

[akrherz]

@xylar , I don't see an openmpi migration applied here? I see this PR is green, but I think this needs updated. Doing so now.

[akrherz]

@conda-forge-admin please rerender

[akrherz]

serenity now, transient CDN error

HTTP 502 BAD GATEWAY for url

@conda-forge-admin please restart ci

[xylar]

@akrherz, good catch! I added the migrator file locally but forgot to git add it.

[xylar]

Okay, I see. We need to migrate libpnetcdf first. Also missed by the bot???

[xylar]

I've seen the openssh thing be necessary on other feedstocks. I assumed it was always required, so was surprised it wasn't here, too.

[xylar]

https://github.com/conda-forge/netcdf4-feedstock/blob/69b6fd7f657124ecf78064422f6013c0ace5e16a/recipe/meta.yaml#L76
https://github.com/conda-forge/netcdf-cxx4-feedstock/blob/74022cfdfbefec51cdcf1c9f78c9c5d49b1dcae9/recipe/meta.yaml#L70
https://github.com/conda-forge/esmf-feedstock/blob/d457f25a89d2d143c60fc32889496a473c36226b/recipe/meta.yaml#L94

We seem to typically add it to run, rather than build I see.

[akrherz]

We seem to typically add it to run, rather than build I see.

Fair. Am going to hope to avoid it completely, I have the next commit teed up, awaiting a green mpiopenmpi job first :)

[xylar]

No dice :-(

[akrherz]

No dice :-(

If this doesn't work, will try your openssh in run dep.

[akrherz]

This doesn't make sense to me as the testing is done within the build phase, so if this somehow works....

[xylar]

This doesn't make sense to me as the testing is done within the build phase, so if this somehow works....

No, right, I was just pointing out that it was probably needed for a different purpose in those other repos, not that moving to run made sense here. Sorry for the confusion.

[akrherz]

Sorry for the confusion.

At least putting it in run confirmed what I assumed should be happening, lets try host, for funzies.

[akrherz]

Holy cow, I need to review this more and back out some debugging before somebody thinks about merging... @xylar feel free to do that, or I will get to it in a couple of hours

[akrherz]

my debugging triggered an unrelated failure

 which: no ssh in ($BUILD_PREFIX/bin:$PREFIX/bin:/opt/conda/condabin:$BUILD_PREFIX/bin:$PREFIX/bin:/opt/conda/bin:/opt/conda/condabin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/home/conda/bin)

now I think openssh needs to be in build , unwinded some debugging 👀

[akrherz]

If this gets green, I then what to check if we can remove openssh dep and just use OMPI_MCA_plm_ssh_agent=false. It continues to be very confusing with issues and conflating plm and pml. I'll have much putzing done in a few hours and can then move this forward, hopefully.

[akrherz]

osx_64 failure may be transient timeout

Test #174: nc_test4_run_par_test .................***Timeout 1500.22 sec

[xylar]

@akrherz, I think it's better to stick with this PR than to move to #193. It just makes things more confusing as far as I'm concerned and I think the history here is useful. If we merge that one, folks won't necessarily know to look here for the discussion. With 2 PRs open, it's not clear where folks should contribute. I updated the name of this one to basically match yours.

I have brought most of those commits over here (skipping the one that conflicted with @minrk's) and applied @minrk's commit after them.

[xylar]

Assuming all the tests pass, I'll merge this later this afternoon (European time).

[xylar]

Okay, let's keep this moving along. Thanks so much @akrherz for working so hard on this and @minrk for your contribution.