[vcpkg_find_fortran] Update libiconv and zlib hash #19545

JackBoosY · 2021-08-13T06:20:13Z

Fixes #18876.

Tested my changes with lapack-reference with resuilt: success.

cenit · 2021-08-13T06:23:50Z

@JackBoosY i tagged only one line but there was more

also mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst was changed without fixing its hash!!

JackBoosY · 2021-08-13T06:27:58Z

@cenit I forgot this, thanks for notice.

cenit · 2021-08-13T06:29:46Z

my only concern is that ci was not noticing the problem. why? no port being tested? is it difficult to make one working to check problems like this in the future?

JackBoosY · 2021-08-13T06:30:58Z

Starting package 107/271: lapack-reference:x86-windows
Building package lapack-reference[blas-select,core,noblas]:x86-windows...
-- Using D:/downloads/Reference-LAPACK-lapack-v3.8.0.tar.gz
-- Extracting source D:/downloads/Reference-LAPACK-lapack-v3.8.0.tar.gz
-- Using source at D:/buildtrees/lapack-reference/src/v3.8.0-954f10683f.clean
-- The Fortran compiler identification is unknown
-- No Fortran compiler found on the PATH. Using MinGW gfortran!
-- Using msys root at D:/downloads/tools/msys2/51cfb821b1485bc1
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Using msys root at D:/downloads/tools/msys2/aa5af7b2aa7e90e8
-- Fixing pkgconfig file: D:/packages/lapack-reference_x86-windows/lib/pkgconfig/blas.pc
-- Fixing pkgconfig file: D:/packages/lapack-reference_x86-windows/lib/pkgconfig/lapack.pc
-- Fixing pkgconfig file: D:/packages/lapack-reference_x86-windows/debug/lib/pkgconfig/blas.pc
-- Fixing pkgconfig file: D:/packages/lapack-reference_x86-windows/debug/lib/pkgconfig/lapack.pc
-- Installing: D:/packages/lapack-reference_x86-windows/share/lapack-reference/copyright
-- Performing post-build validation
-- Performing post-build validation done
Uploaded binaries to 1 HTTP remotes.
Building package lapack-reference[blas-select,core,noblas]:x86-windows... done
Installing package lapack-reference[blas-select,core,noblas]:x86-windows...
Installing package lapack-reference[blas-select,core,noblas]:x86-windows... done
Elapsed time for package lapack-reference:x86-windows: 53.15 s

It seems that our ci system didn't catch this error.

JackBoosY · 2021-08-13T06:31:52Z

cc @BillyONeal do you know why our ci didn't trigger this changes?

BillyONeal · 2021-08-13T06:33:41Z

cc @BillyONeal do you know why our ci didn't trigger this changes?

I don't understand; it says that the builds are running?

JackBoosY · 2021-08-13T06:35:46Z

@BillyONeal In #18876, the update of msys components should trigger our pipeline test to download these new components and detect their hash, but this is not the case.

The expect result should be:

PS F:\vcpkg> .\vcpkg.exe install lapack-reference
Computing installation plan...
The following packages will be built and installed:
    lapack-reference[blas-select,core,noblas]:x86-windows -> 3.8.0#5
Detecting compiler hash for triplet x86-windows...
Could not locate cached archive: C:\Users\usr\AppData\Local\vcpkg\archives\af\afe00567fe7be33cc31626c28a6cfa3a1d4c94c5841f9b26fd9f290545602459.zip
Starting package 1/1: lapack-reference:x86-windows
Building package lapack-reference[blas-select,core,noblas]:x86-windows...
-- Using F:/vcpkg/downloads/Reference-LAPACK-lapack-v3.8.0.tar.gz
-- Cleaning sources at F:/vcpkg/buildtrees/lapack-reference/src/v3.8.0-954f10683f.clean. Use --editable to skip cleaning for the packages you specify.
-- Extracting source F:/vcpkg/downloads/Reference-LAPACK-lapack-v3.8.0.tar.gz
-- Using source at F:/vcpkg/buildtrees/lapack-reference/src/v3.8.0-954f10683f.clean
-- The Fortran compiler identification is unknown
-- No Fortran compiler found on the PATH. Using MinGW gfortran!
[DEBUG] Feature flag 'binarycaching' unset
[DEBUG] Feature flag 'manifests' = off
[DEBUG] Feature flag 'compilertracking' unset
[DEBUG] Feature flag 'registries' unset
[DEBUG] Feature flag 'versions' unset
[DEBUG] Downloading https://repo.msys2.org/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst
[DEBUG] Downloading https://www2.futureware.at/~nickoe/msys2-mirror/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst
[DEBUG] Downloading https://mirror.yandex.ru/mirrors/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst
[DEBUG] Downloading https://mirrors.tuna.tsinghua.edu.cn/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst
[DEBUG] Downloading https://mirrors.ustc.edu.cn/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst
[DEBUG] Downloading https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst
[DEBUG] Downloading https://mirror.selfnet.de/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst
[DEBUG] Downloading https://mirrors.sjtug.sjtu.edu.cn/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst
Error: Failed to download from mirror set:
File does not have the expected hash:
             url : [ https://repo.msys2.org/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst ]
       File path : [ F:\vcpkg\downloads\msys-mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst.5916.part ]
   Expected hash : [ 459850a8c42b1d497268736629ef713beee70cd0d3161d02c7a9fad08aca4560f4e17ba02d5cabda8a19d7c614f7e0ef5a6ec13afd91dd3004057139a5469c8f ]
     Actual hash : [ 46bbf0f28d9faf047221def19e6f94b9556fc6c951cad9c4fb657fde9d15303b9cb64aad11eaa57892cde49eafb43fbe2ec6da6b612449a20ae49dc8233e945b ]

File does not have the expected hash:
             url : [ https://www2.futureware.at/~nickoe/msys2-mirror/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst ]
       File path : [ F:\vcpkg\downloads\msys-mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst.5916.part ]
   Expected hash : [ 459850a8c42b1d497268736629ef713beee70cd0d3161d02c7a9fad08aca4560f4e17ba02d5cabda8a19d7c614f7e0ef5a6ec13afd91dd3004057139a5469c8f ]
     Actual hash : [ 46bbf0f28d9faf047221def19e6f94b9556fc6c951cad9c4fb657fde9d15303b9cb64aad11eaa57892cde49eafb43fbe2ec6da6b612449a20ae49dc8233e945b ]

File does not have the expected hash:
             url : [ https://mirror.yandex.ru/mirrors/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst ]
       File path : [ F:\vcpkg\downloads\msys-mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst.5916.part ]
   Expected hash : [ 459850a8c42b1d497268736629ef713beee70cd0d3161d02c7a9fad08aca4560f4e17ba02d5cabda8a19d7c614f7e0ef5a6ec13afd91dd3004057139a5469c8f ]
     Actual hash : [ 46bbf0f28d9faf047221def19e6f94b9556fc6c951cad9c4fb657fde9d15303b9cb64aad11eaa57892cde49eafb43fbe2ec6da6b612449a20ae49dc8233e945b ]

File does not have the expected hash:
             url : [ https://mirrors.tuna.tsinghua.edu.cn/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst ]
       File path : [ F:\vcpkg\downloads\msys-mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst.5916.part ]
   Expected hash : [ 459850a8c42b1d497268736629ef713beee70cd0d3161d02c7a9fad08aca4560f4e17ba02d5cabda8a19d7c614f7e0ef5a6ec13afd91dd3004057139a5469c8f ]
     Actual hash : [ 46bbf0f28d9faf047221def19e6f94b9556fc6c951cad9c4fb657fde9d15303b9cb64aad11eaa57892cde49eafb43fbe2ec6da6b612449a20ae49dc8233e945b ]

File does not have the expected hash:
             url : [ https://mirrors.ustc.edu.cn/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst ]
       File path : [ F:\vcpkg\downloads\msys-mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst.5916.part ]
   Expected hash : [ 459850a8c42b1d497268736629ef713beee70cd0d3161d02c7a9fad08aca4560f4e17ba02d5cabda8a19d7c614f7e0ef5a6ec13afd91dd3004057139a5469c8f ]
     Actual hash : [ 46bbf0f28d9faf047221def19e6f94b9556fc6c951cad9c4fb657fde9d15303b9cb64aad11eaa57892cde49eafb43fbe2ec6da6b612449a20ae49dc8233e945b ]

https://mirror.bit.edu.cn/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst: WinHttpSendRequest() failed: 12007
File does not have the expected hash:
             url : [ https://mirror.selfnet.de/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst ]
       File path : [ F:\vcpkg\downloads\msys-mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst.5916.part ]
   Expected hash : [ 459850a8c42b1d497268736629ef713beee70cd0d3161d02c7a9fad08aca4560f4e17ba02d5cabda8a19d7c614f7e0ef5a6ec13afd91dd3004057139a5469c8f ]
     Actual hash : [ 46bbf0f28d9faf047221def19e6f94b9556fc6c951cad9c4fb657fde9d15303b9cb64aad11eaa57892cde49eafb43fbe2ec6da6b612449a20ae49dc8233e945b ]

File does not have the expected hash:
             url : [ https://mirrors.sjtug.sjtu.edu.cn/msys2/mingw/i686/mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst ]
       File path : [ F:\vcpkg\downloads\msys-mingw-w64-i686-zlib-1.2.11-8-any.pkg.tar.zst.5916.part ]
   Expected hash : [ 459850a8c42b1d497268736629ef713beee70cd0d3161d02c7a9fad08aca4560f4e17ba02d5cabda8a19d7c614f7e0ef5a6ec13afd91dd3004057139a5469c8f ]
     Actual hash : [ 46bbf0f28d9faf047221def19e6f94b9556fc6c951cad9c4fb657fde9d15303b9cb64aad11eaa57892cde49eafb43fbe2ec6da6b612449a20ae49dc8233e945b ]


[DEBUG] C:\a\1\s\src\vcpkg\base\downloads.cpp(648)
[DEBUG] Exiting after 10831810 us (10809760 us)

CMake Error at scripts/cmake/vcpkg_download_distfile.cmake:251 (message):

      Failed to download file.
      If you use a proxy, please check your proxy setting. Possible causes are:

      1. You are actually using an HTTP proxy, but setting HTTPS_PROXY variable
         to `https://address:port`. This is not correct, because `https://` prefix
         claims the proxy is an HTTPS proxy, while your proxy (v2ray, shadowsocksr
         , etc..) is an HTTP proxy. Try setting `http://address:port` to both
         HTTP_PROXY and HTTPS_PROXY instead.

      2. You are using Fiddler. Currently a bug (https://github.com/microsoft/vcpkg/issues/17752)
         will set HTTPS_PROXY to `https://fiddler_address:port` which lead to problem 1 above.
         Workaround is open Windows 10 Settings App, and search for Proxy Configuration page,
         Change `http=address:port;https=address:port` to `address`, and fill the port number.

      3. You proxy's remote server is out of service.

      In future vcpkg releases, if you are using Windows, you no longer need to set
      HTTP(S)_PROXY environment variables. Vcpkg will simply apply Windows IE Proxy
      Settings set by your proxy software. See (https://github.com/microsoft/vcpkg-tool/pull/49)
      and (https://github.com/microsoft/vcpkg-tool/pull/77)

      Otherwise, please submit an issue at https://github.com/Microsoft/vcpkg/issues

Call Stack (most recent call first):
  scripts/cmake/vcpkg_acquire_msys.cmake:84 (vcpkg_download_distfile)
  scripts/cmake/vcpkg_acquire_msys.cmake:174 (z_vcpkg_acquire_msys_download_package)
  scripts/cmake/vcpkg_find_fortran.cmake:30 (vcpkg_acquire_msys)
  ports/lapack-reference/portfile.cmake:50 (vcpkg_find_fortran)
  scripts/ports.cmake:141 (include)


Error: Building package lapack-reference:x86-windows failed with: BUILD_FAILED

cenit · 2021-08-13T09:07:50Z

@JackBoosY @NancyLi1013 shall we not merge this PR until we have found out why ci was not detecting the issue?

JackBoosY · 2021-08-13T09:28:55Z

Starting package 107/271: lapack-reference:x86-windows
Building package lapack-reference[blas-select,core,noblas]:x86-windows...
-- Downloading https://github.com/Reference-LAPACK/lapack/archive/v3.8.0.tar.gz -> Reference-LAPACK-lapack-v3.8.0.tar.gz...
-- Extracting source D:/downloads/Reference-LAPACK-lapack-v3.8.0.tar.gz
-- Using source at D:/buildtrees/lapack-reference/src/v3.8.0-954f10683f.clean
-- The Fortran compiler identification is unknown
-- No Fortran compiler found on the PATH. Using MinGW gfortran!
-- Using msys root at D:/downloads/tools/msys2/6d8be427ceb450c6
-- Configuring x86-windows
-- Building x86-windows-dbg
-- Building x86-windows-rel
-- Using msys root at D:/downloads/tools/msys2/aa5af7b2aa7e90e8
-- Fixing pkgconfig file: D:/packages/lapack-reference_x86-windows/lib/pkgconfig/blas.pc
-- Fixing pkgconfig file: D:/packages/lapack-reference_x86-windows/lib/pkgconfig/lapack.pc
-- Fixing pkgconfig file: D:/packages/lapack-reference_x86-windows/debug/lib/pkgconfig/blas.pc
-- Fixing pkgconfig file: D:/packages/lapack-reference_x86-windows/debug/lib/pkgconfig/lapack.pc
-- Installing: D:/packages/lapack-reference_x86-windows/share/lapack-reference/copyright
-- Performing post-build validation
-- Performing post-build validation done
Uploaded binaries to 1 HTTP remotes.
Building package lapack-reference[blas-select,core,noblas]:x86-windows... done
Installing package lapack-reference[blas-select,core,noblas]:x86-windows...
Installing package lapack-reference[blas-select,core,noblas]:x86-windows... done
Elapsed time for package lapack-reference:x86-windows: 53.34 s

The path D:/downloads/tools/msys2/aa5af7b2aa7e90e8 is correct now, whether to merge this PR depends on @BillyONeal 's decision.

ras0219 · 2021-08-14T05:34:25Z

The reason the CI didn't detect this issue is that we've added an asset cache (https://github.com/microsoft/vcpkg/blob/master/docs/users/assetcaching.md). This has greatly improved stability of the CI, however it means that it will only check the URL for a given hash once -- after the asset with that hash has been downloaded a single time, we forever more fetch it from the asset cache instead (by hash).

We need to find a solution moving forward to check changed references while still maintaining the asset cache.

[vcpkg_find_fortran] Update libiconv hash

32cd47c

JackBoosY added info:internal This PR or Issue was filed by the vcpkg team. category:tool-update The issue is with build tool or build script, which requires update or should be executed correctly labels Aug 13, 2021

JackBoosY assigned NancyLi1013 Aug 13, 2021

JackBoosY mentioned this pull request Aug 13, 2021

[find_fortran] update msys components libiconv and zlib #18876

Merged

JackBoosY requested a review from NancyLi1013 August 13, 2021 06:21

github-actions bot approved these changes Aug 13, 2021

View reviewed changes

Fix more hash

e1478f5

JackBoosY changed the title ~~[vcpkg_find_fortran] Update libiconv hash~~ [vcpkg_find_fortran] Update libiconv and zlib hash Aug 13, 2021

github-actions bot approved these changes Aug 13, 2021

View reviewed changes

NancyLi1013 approved these changes Aug 13, 2021

View reviewed changes

NancyLi1013 added the info:reviewed Pull Request changes follow basic guidelines label Aug 13, 2021

ras0219-msft merged commit 6bc4362 into microsoft:master Aug 14, 2021

JackBoosY deleted the dev/jack/update-mingw-libiconv-hash branch August 16, 2021 02:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[vcpkg_find_fortran] Update libiconv and zlib hash #19545

[vcpkg_find_fortran] Update libiconv and zlib hash #19545

JackBoosY commented Aug 13, 2021 •

edited

Loading

cenit commented Aug 13, 2021

JackBoosY commented Aug 13, 2021

cenit commented Aug 13, 2021

JackBoosY commented Aug 13, 2021

JackBoosY commented Aug 13, 2021

BillyONeal commented Aug 13, 2021

JackBoosY commented Aug 13, 2021 •

edited

Loading

cenit commented Aug 13, 2021

JackBoosY commented Aug 13, 2021

ras0219 commented Aug 14, 2021 •

edited

Loading

[vcpkg_find_fortran] Update libiconv and zlib hash #19545

[vcpkg_find_fortran] Update libiconv and zlib hash #19545

Conversation

JackBoosY commented Aug 13, 2021 • edited Loading

cenit commented Aug 13, 2021

JackBoosY commented Aug 13, 2021

cenit commented Aug 13, 2021

JackBoosY commented Aug 13, 2021

JackBoosY commented Aug 13, 2021

BillyONeal commented Aug 13, 2021

JackBoosY commented Aug 13, 2021 • edited Loading

cenit commented Aug 13, 2021

JackBoosY commented Aug 13, 2021

ras0219 commented Aug 14, 2021 • edited Loading

JackBoosY commented Aug 13, 2021 •

edited

Loading

JackBoosY commented Aug 13, 2021 •

edited

Loading

ras0219 commented Aug 14, 2021 •

edited

Loading