Are Big Sur installers working? #86
Comments
|
@blacktop This happens with an empty content subdir, and does not appear to have anything to do with the downloads themselves. |
craig65535
changed the title
|
Building a Catalina installer (e.g. |
|
I’ve built Big Sur installers for each release without issue. Haven’t tried in the last week or so. |
|
@gregneagle I've built them before, too, so maybe something changed. Taking a look at my install.log... The .dist file that is being installed contains a reference to InstallAssistant.pkg. That pkg file contains Scripts/postinstall.zsh which calls Scripts/postinstall_actions/link_shared_support.zsh. When run, the latter tries to find SharedSupport.dmg in the nonsense path /private/tmp/dmg.y9IuqGApplications/Install macOS Big Sur.app/Contents/SharedSupport/SharedSupport.dmg. Note the lack of / before Applications. Maybe the script in InstallAssistant.pkg is now broken and will only work when the target is / and not something like /private/tmp/dmg.XXXXXX? |
|
Could also be different behaviour with /usr/sbin/installer on 11.2.3. |
Interesting, I seem to recall a block of code in your installer where you rename the /private/tmp/dmg.y9IuqGApplications to /private/tmp/dmg.y9IuqG/Applications here - https://github.com/munki/macadmin-scripts/blob/main/installinstallmacos.py#L228 So maybe Apple got lazy and now just tries to use the "mistake" /private/tmp/dmg.y9IuqGApplications instead of fixing it? |
|
@blacktop It definitely seems to be a related issue. In this case the script is half-correct - writing to the correct path, but then reading from the wrong one. |
|
I found something interesting: running the script from a different directory, e.g. ~/devel/macadmin-scripts, works. It fails when running from ~/Downloads, e.g. ~/Downloads/macadmin-scripts-main. I noticed the same for ~/Documents. I think this is a SIP thing? It looks like the script will need to be be updated to refuse to run from certain protected folders. |
|
Whoa! that's very interesting. I've also been running out of my Downloads folder, I'll try again in a different one |
|
Or you could read the Read Me :-) https://github.com/munki/macadmin-scripts#important-note-for-catalina |
|
@gregneagle I noticed that there was a check for the current directory, but it wasn't working for me (my script was in a subdir under Downloads, not Downloads itself). #87 should address that, and also make the warning more noticeable to users. |
|
The most recent Big Sur update (11.3.1) is now giving me the error:
|
|
Can you be more specific about what you are trying to do? ./installinstallmacos.py
Downloading https://swscan.apple.com/content/catalogs/others/index-11-10.15-10.14-10.13-10.12-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog...
Downloading http://swcdn.apple.com/content/downloads/20/55/001-51042-A_2EJTJOSUC2/rsvf13iphg5lvcqcysqcarv8cvddq8igek/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/20/55/001-51042-A_2EJTJOSUC2/rsvf13iphg5lvcqcysqcarv8cvddq8igek/001-51042.English.dist...
No server metadata for 071-08935
Downloading https://swdist.apple.com/content/downloads/37/23/071-08935-A_VI70PU5ZSV/dxl5i8eyczg8zje0wiks3r03r91euyi9sa/071-08935.English.dist...
Downloading http://swcdn.apple.com/content/downloads/59/18/001-15219-A_EE9FN7UHJA/efuz3h0zlmx4qlinf4bxlguigo9jvhcr55/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/59/18/001-15219-A_EE9FN7UHJA/efuz3h0zlmx4qlinf4bxlguigo9jvhcr55/001-15219.English.dist...
Downloading http://swcdn.apple.com/content/downloads/34/54/041-88800-A_HLMBDM42FL/anrmoj880qkj0lbybqm0c3830p70nawjrv/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/34/54/041-88800-A_HLMBDM42FL/anrmoj880qkj0lbybqm0c3830p70nawjrv/041-88800.English.dist...
Downloading http://swcdn.apple.com/content/downloads/41/23/061-26578-A_FF0C6M87LS/ylvl0phx6uy3i444qt9p57n8wz0xnylmd9/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/41/23/061-26578-A_FF0C6M87LS/ylvl0phx6uy3i444qt9p57n8wz0xnylmd9/061-26578.English.dist...
Downloading http://swcdn.apple.com/content/downloads/61/24/001-57224/0o1xetys5n505gocjsil5qo0p4kg939elb/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/61/24/001-57224/0o1xetys5n505gocjsil5qo0p4kg939elb/001-57224.English.dist...
Downloading http://swcdn.apple.com/content/downloads/23/45/001-36735-A_IRCLJ5YYKF/z8vka6xfykydycui5b7atpc52btxmbffdn/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/23/45/001-36735-A_IRCLJ5YYKF/z8vka6xfykydycui5b7atpc52btxmbffdn/001-36735.English.dist...
Downloading http://swcdn.apple.com/content/downloads/08/44/041-90855-A_452WLVQACP/mzfvoi59bljh3w4w4lvwenfk0r1a2f7212/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/08/44/041-90855-A_452WLVQACP/mzfvoi59bljh3w4w4lvwenfk0r1a2f7212/041-90855.English.dist...
Downloading http://swcdn.apple.com/content/downloads/06/50/041-91758-A_M8T44LH2AW/b5r4og05fhbgatve4agwy4kgkzv07mdid9/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/06/50/041-91758-A_M8T44LH2AW/b5r4og05fhbgatve4agwy4kgkzv07mdid9/041-91758.English.dist...
Downloading http://swcdn.apple.com/content/downloads/01/28/061-86291-A_JPEIWIOZES/enpozvvbmj3mj2dhulhevlt8b429qd5kw0/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/01/28/061-86291-A_JPEIWIOZES/enpozvvbmj3mj2dhulhevlt8b429qd5kw0/061-86291.English.dist...
No server metadata for 071-29172
Downloading https://swdist.apple.com/content/downloads/40/42/071-29172-A_50FL5BT4I6/5w0xm8c31yigcf2iov4n4mlsvnw1upzo2q/071-29172.English.dist...
Downloading http://swcdn.apple.com/content/downloads/26/37/001-68446/r1dbqtmf3mtpikjnd04cq31p4jk91dceh8/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/26/37/001-68446/r1dbqtmf3mtpikjnd04cq31p4jk91dceh8/001-68446.English.dist...
No server metadata for 071-32414
Downloading https://swdist.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/071-32414.English.dist...
Downloading http://swcdn.apple.com/content/downloads/10/06/001-36801-A_9MBEQ97Z9M/672a1q4u8978ize3s15ygivp2dqwmk10qb/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/10/06/001-36801-A_9MBEQ97Z9M/672a1q4u8978ize3s15ygivp2dqwmk10qb/001-36801.English.dist...
Downloading http://swcdn.apple.com/content/downloads/17/32/061-26589-A_8GJTCGY9PC/25fhcu905eta7wau7aoafu8rvdm7k1j4el/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/17/32/061-26589-A_8GJTCGY9PC/25fhcu905eta7wau7aoafu8rvdm7k1j4el/061-26589.English.dist...
Downloading http://swcdn.apple.com/content/downloads/37/41/001-04366/a6boefh8b6bee4k2x6v5lwyfjr7od2ica3/InstallAssistantAuto.smd...
Downloading https://swdist.apple.com/content/downloads/37/41/001-04366/a6boefh8b6bee4k2x6v5lwyfjr7od2ica3/001-04366.English.dist...
No server metadata for 071-05432
Downloading https://swdist.apple.com/content/downloads/00/60/071-05432-A_QOY2QE0UMR/puuz6c0epc7o0ozyovvi6tjxhzpf6uf04s/071-05432.English.dist...
No server metadata for 071-14766
Downloading https://swdist.apple.com/content/downloads/12/32/071-14766-A_Q2H6ELXGVG/zx8saim8tei7fezrmvu4vuab80m0e8a5ll/071-14766.English.dist...
# ProductID Version Build Post Date Title
1 001-15219 10.15.5 19F2200 2020-06-15 macOS Catalina
2 001-68446 10.15.7 19H15 2020-11-11 macOS Catalina
3 001-36801 10.15.6 19G2021 2020-08-12 macOS Catalina
4 071-32414 11.3.1 20E241 2021-05-03 macOS Big Sur
5 001-04366 10.15.4 19E2269 2020-05-04 macOS Catalina
6 071-08935 11.2.2 20D80 2021-03-01 macOS Big Sur
7 061-86291 10.15.3 19D2064 2020-03-23 macOS Catalina
8 041-91758 10.13.6 17G66 2019-10-19 macOS High Sierra
9 041-88800 10.14.4 18E2034 2019-10-23 macOS Mojave
10 071-14766 11.2.3 20D91 2021-03-08 macOS Big Sur
11 061-26589 10.14.6 18G103 2019-10-14 macOS Mojave
12 071-05432 11.2.1 20D75 2021-02-15 macOS Big Sur
13 001-51042 10.15.7 19H2 2020-09-24 macOS Catalina
14 001-36735 10.15.6 19G2006 2020-08-06 macOS Catalina
15 001-57224 10.15.7 19H4 2020-10-27 macOS Catalina
16 041-90855 10.13.5 17F66a 2019-10-23 Install macOS High Sierra Beta
17 061-26578 10.14.5 18F2059 2019-10-14 macOS Mojave
18 071-29172 11.3 20E232 2021-04-26 macOS Big Sur
Choose a product to download (1-18): 4
Downloading http://swcdn.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/MajorOSInfo.pkg...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 2571k 100 2571k 0 0 41.1M 0 --:--:-- --:--:-- --:--:-- 41.8M
Downloading https://swdist.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/MajorOSInfo.pkm...
Downloading http://swcdn.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/InstallAssistant.pkg...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 11.5G 100 11.5G 0 0 105M 0 0:01:52 0:01:52 --:--:-- 112M
Downloading https://swdist.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/InstallAssistant.pkm...
Downloading http://swcdn.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/UpdateBrain.zip...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1921k 100 1921k 0 0 1542k 0 0:00:01 0:00:01 --:--:-- 1542k
Downloading http://swcdn.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/BuildManifest.plist...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 35251 100 35251 0 0 124k 0 --:--:-- --:--:-- --:--:-- 124k
Downloading http://swcdn.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/Info.plist...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1683 100 1683 0 0 54290 0 --:--:-- --:--:-- --:--:-- 54290
Downloading http://swcdn.apple.com/content/downloads/16/05/071-32414-A_33YH16R8HF/28ment5qr993c0l9i1qbh4fuknli9wzglp/InstallInfo.plist...
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 188 100 188 0 0 6482 0 --:--:-- --:--:-- --:--:-- 6482
Making empty sparseimage...
installer: Package name is macOS Big Sur
installer: Installing at base path /private/tmp/dmg.7mftYx
installer: The install was successful.
*********************************************************
*** Working around a very dumb Apple bug in a package ***
*** postinstall script that fails to correctly target ***
*** the Install macOS.app when installed to a volume ***
*** other than the current boot volume. ***
*** Please file feedback with Apple! ***
*********************************************************
Product downloaded and installed to /Users/Shared/macadmin-scripts-main/Install_macOS_11.3.1-20E241.sparseimage
Making read-only compressed disk image containing Install macOS Big Sur.app...
...................................................................................................................................
created: /Users/Shared/macadmin-scripts-main/Install_macOS_11.3.1-20E241.dmg
Disk image created at: ./Install_macOS_11.3.1-20E241.dmg |
|
416 is "range not satisfiable" so that sounds like an error attempting to resume a partial download. I'd clear your local content directory (in the same directory as the installinstallmacos.py tool) and try again. |
|
I'm closing this because Big Sur installers are working and always have been -- there are no issues with the script itself. External issues are not predicable or on me to fix :-) |
That seems to have fixed my issue, not sure what went wrong. Thanks though! |
|
@evprkr In most cases, HTTP error 416 just means the file is already up-to-date, and no new download is necessary. The message is logged only because it's output by curl. @gregneagle This error could be suppressed if it's causing confusion. Basically, stderr could be collected from curl and then only printed if it's not error 416. |
|
Sounds like a fragile bug if "In most cases, HTTP error 416 just means the file is already up-to-date". What about the other cases? And how do we tell them apart? |
|
I agree it's not ideal, but I'm not sure it's really a bug. If we're resuming, our request contains:
The server returns 416 ("Range not satisfiable") if it can't satisfy the range condition. This could happen if the file is 100 bytes, we already have all 100 bytes, and the script asks for Range: bytes 100-. It could also happen in edge cases, like the user appending junk bytes to the end of a downloaded file. The server returns 412 ("Precondition failed") if it can't satisfy the date condition. This could happen if the file was updated server-side. In this case, we delete the file and download again. Beyond suppressing the error message, another fix for the 416 error would be to not attempt to re-download if we are sure we already have the file. We would need to track successful downloads in a file (maybe a zero-byte file for each downloaded file), and if that file exists, issue a HEAD request with no range (just a date condition) instead of a GET request. If that HEAD with no range returns 200, take no action. If it returns 412, delete both the download and the zero-byte file, and download again. @gregneagle is that worth pursuing? The current behaviour is less complex. |
|
Even less complex would be to not attempt to resume partial downloads. If the current code, which does attempt to resume partial downloads, causes too much confusion, or more importantly, too many questions/support requests/GitHub issues, it needs to change, even if that means removing or disabling that feature. I've made my work available for others to use, but I don't have to set things up to make it a chore for me... |
|
So I guess that's 3 options -
It's your call, of course, but there is a usability advantage to having resume functionality when the downloads are measured in GB. The feature (with my previous PR) also seems to work well, outside of logging this error message. |
|
A problem with suppressing the 416 error is that it isn't always because you have a complete file. |
|
I think that unless the user is sabotaging their own downloaded content, that is what it means. (Unless there's something I haven't thought of) |
|
And there's the problem: I'm expected to support software whose behavior I don't completely understand (and it sounds like possibly you don't, either). I'd rather hoped curl itself would handle partial download resumption without a lot of poorly-understood logic having to be written around it. |
|
The key thing is that we have a script that is asking to resume an already completed file. That wouldn't happen in a typical web browser situation. Maybe the solution is to act more like a browser and download to That's easier than what I laid out above. I can look into making this change. |
|
@gregneagle PTAL at #90 |
I have a 2008 (Intel) Mac Mini running macOS 11.2.3.
I was trying to download/create a Big Sur installer and was unable to download either 11.2.3 or 11.2.1. Here is the log for 11.2.1:
install.log:
Two observations:
Install macOS Big Sur.app/Contents/SharedSupport/SharedSupport.dmgdoes not exist, but seems to be expected by postinstall.zsh.The text was updated successfully, but these errors were encountered: