Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

MacOS and nix #7

Open
PedroRegisPOAR opened this issue Jul 19, 2022 · 1 comment
Open

MacOS and nix #7

PedroRegisPOAR opened this issue Jul 19, 2022 · 1 comment

Comments

@PedroRegisPOAR
Copy link
Contributor

PedroRegisPOAR commented Jul 19, 2022 •
edited
Loading

Notes about nix and MacOS and AMD and ARM

Some background about AMD vs ARM

Take a look at:

UTM

It worked, but still missing a lot of documentation!

Why?

I don't think this is an issue that should be fixed on our end, its more of a homebrew install issue, and homebrew specifically states that some packages may break if not installed in their default location
containers/podman#12161 (comment)

I try to install as many packages as possible through Nix, since it’s the only way to synchronize packages across all of NixOS, Arch Linux and MacOS.
From: https://www.fbrs.io/nix-overlays/

The main issue we found with using Docker locally is that docker filesharing is extremely slow, especially on MacOS. The interwebs have plenty of resources to address this problem but these approaches simply mitigate rather than resolve the underlying performance issue.

Docker performance is pretty bad for Rails development but it’s even worse for front-end apps that require a gazillion files to be loaded and written (cough cough webpack). Poor Docker performance usually leads developers to give up on Docker for their frontend - and return to a painfully slow backend development process.
https://tech.precisionnutrition.com/dev-tooling/2020/05/03/speedy-dev-envs-with-nix.html

https://www.slant.co/topics/511/~best-mac-package-managers
https://news.ycombinator.com/item?id=27825420

Setting up nix-darwin home-manager and flakes on M1 Macbook Pro OSX Monterey 12.0.1
Discontinuing support for macOS Monterey

Mitchell is a Co-founder and CTO of HashiCorp, creators of popular tools such as Terraform, Vault, Vagrant, Packer, and more.

List:

TODO: watch it
this Did someone say virtual is slow?, start=76&end=181
+
Walkthrough of Nix Install and Setup on MacOS
+
This Linux PC Runs macOS Faster Than a Real Mac

TODO:

The sandbox

Really watch this:

See https://discourse.nixos.org/t/nixos-virtual-machine-on-apple-m1-arm64/15167 and https://mac.getutm.app/

NixOS/nixpkgs#95903 (comment)

Bugs, many bugs

> # This works on my computer (host: Ubuntu 14.04.1, guest: MINIX 3.2.1). It's
> # not my fault if it doesn't work on yours.
https://gist.github.com/ashaindlin/ebf807cd14b82066058e#file-vm-sh-L19-L20

I tried 10 code editors, start=0&end=19

In my case after copying of the executable the bug that I wanted to debug stopped happening ..(
https://stackoverflow.com/questions/31045575/how-to-trace-system-calls-of-a-program-in-mac-os-x#comment80637364_46799054

how a silent computer bug ended 6 lives

Bugs and more bugs:

Take an look at this:

If you remove the first word from the string "hello world", what should the result be? This is the story of how we discovered that the answer could be your root password!
https://lock.cmpxchg8b.com/zenbleed.html

List:

Domen Kožar - Nix Is Going Mainstream (SoN2022 - public lecture series), start=480&end=516

The NixOS "shebang" and the GNU/Linux kernel:

https://www.youtube.com/embed/QkcjCRE8Vvs?start=673&end=807&version=3
+
https://discussion.fedoraproject.org/t/disabling-selinux-breaks-docker-overlayfs/20232

MWEs + how to never write bug, start=264&end=294

Unfortunately there are really hard/impossible to reproduce bugs:
how to never write bug, start=294&end=319

I too faced similar issues of stuttering. This was because of the input overflowed error which came up during the streaming. If the chunk size of every communication is reduced to 512bytes the communication is coming out well as per my experience.
Ref.: https://stackoverflow.com/a/36214862/9577149

If nothing works:

Specific to javascript, but valid for many other situations:

This kind of bug is scary:

TODO: test is

nix \
--extra-experimental-features 'nix-command flakes ca-derivations' \
realisation \
info \
github:NixOS/nixpkgs/58c85835512b0db938600b6fe13cc3e3dc4b364e#hello
this path will be fetched (0.02 MiB download, 0.11 MiB unpacked):
  /nix/store/c4n3i1nfhvcim30s4ij502cw84yl4vdn-hello-2.12.1
error: unexpected end-of-file
nix shell nixpkgs#nix-info --command nix-info --markdown
  • system: "aarch64-darwin"
  • host os: Darwin 22.3.0, macOS 13.2.1
  • multi-user?: yes
  • sandbox: no
  • version: nix-env (Nix) 2.10.2
  • channels(root): "nixpkgs"
  • nixpkgs: /nix/var/nix/profiles/per-user/root/channels/nixpkgs
{"description":"A collection of packages for the Nix package manager","lastModified":1688392541,"locked":{"lastModified":1688392541,"narHash":"sha256-lHrKvEkCPTUO+7tPfjIcb7Trk6k31rz18vkyqmkeJfY=","owner":"NixOS","repo":"nixpkgs","rev":"ea4c80b39be4c09702b0cb3b42eab59e2ba4f24b","type":"github"},"locks":{"nodes":{"root":{}},"root":"root","version":7},"original":{"id":"nixpkgs","type":"indirect"},"originalUrl":"flake:nixpkgs","path":"/nix/store/gmdhl9qfaic5765lxw2wj6hb2ifjmhjd-source","resolved":{"lastModified":1688392541,"narHash":"sha256-lHrKvEkCPTUO+7tPfjIcb7Trk6k31rz18vkyqmkeJfY=","owner":"NixOS","repo":"nixpkgs","rev":"ea4c80b39be4c09702b0cb3b42eab59e2ba4f24b","type":"github"},"resolvedUrl":"github:NixOS/nixpkgs/ea4c80b39be4c09702b0cb3b42eab59e2ba4f24b","revision":"ea4c80b39be4c09702b0cb3b42eab59e2ba4f24b","url":"github:NixOS/nixpkgs/ea4c80b39be4c09702b0cb3b42eab59e2ba4f24b"}
nix --extra-experimental-features 'nix-command flakes ca-derivations' realisation info github:NixOS/nixpkgs/nixpkgs-unstable#python3
these 5 paths will be fetched (13.08 MiB download, 68.12 MiB unpacked):
  /nix/store/61s4l8b46g3bg9ymcji3bmjv8kzk2sms-readline-8.2p1
  /nix/store/bpz8rh8pnlm6zsh64xjk2c7hgc71f99p-gdbm-1.23
  /nix/store/f67hia73n1rf333qkwk294g44vkr6rn7-openssl-3.0.10
  /nix/store/fd7wwy1bzzpg68pd20gxb02rgwdaa41w-libxcrypt-4.4.36
  /nix/store/g5cm6iik6p4k39cj9k7a6sg2p09hl7wf-python3-3.10.12
error: unexpected end-of-file

The rosetta thing

List:

The qemu-user-static thing

What name put here?

macOS to NixOS the Purely Functional Linux Distribution by Daniel Britten

Franz Pletz: NixOS, start=913&end=928, about homebrew vs nix

https://discourse.nixos.org/t/nixos-on-the-new-apple-macbook-m1/15715/2

Remember Mac uses zsh (since when?):

utmapp/UTM#2688 (comment)

MacOS Simple KVM

Part 1:

git clone https://github.com/adisbladis/macOS-Simple-KVM.git \
&& cd macOS-Simple-KVM \
&& git checkout de4114741e5dd0079d04a4d56d275659d0579c31 \
&& (direnv allow || nix-shell)

Part 2:

./jumpstart.sh --mojave

Part 3:

test -f MyDisk.qcow2 || qemu-img create -f qcow2 MyDisk.qcow2 64G

./basic.sh
test -f MyDisk.qcow2 || qemu-img create -f qcow2 MyDisk.qcow2 64G

HEADLESS=1 MEM=1G CPUS=2 SYSTEM_DISK=MyDisk.qcow2 ./headless.sh

About the installer

TODO: try to test that
NixOS/nix#3616 (comment)

qemu

TODO: it documents part of history: https://stackoverflow.com/a/66887947
TODO: https://www.willhaley.com/blog/debian-arm-qemu/

# nix flake metadata --no-write-lock-file 'github:Mic92/nixos-aarch64-images'
# github:Mic92/nixos-aarch64-images/c29e786f552a5c38e1dcdceb552412ad488e45d3
nix \
build \
--no-write-lock-file \
'github:Mic92/nixos-aarch64-images#aarch64Image'
nix \
build \
--no-write-lock-file \
--override-input nixpkgs github:nixos/nixpkgs/nixpkgs-unstable \
'github:Mic92/nixos-aarch64-images#aarch64Image'
nix profile install nixpkgs#qemu_full
curl -O http://ftp.us.debian.org/debian/dists/stable/main/installer-armhf/current/images/cdrom/initrd.gz \
  && curl -O http://ftp.us.debian.org/debian/dists/stable/main/installer-armhf/current/images/cdrom/vmlinuz

curl -O -L https://cdimage.debian.org/debian-cd/current/armhf/iso-dvd/debian-11.1.0-armhf-DVD-1.iso
qemu-img create -f qcow2 debian-arm.sda.qcow2 100G
qemu-system-arm \
  -m 4G \
  -machine type=virt \
  -cpu cortex-a7 \
  -smp 4 \
  -initrd "./initrd.gz" \
  -kernel "./vmlinuz" \
  -append "console=ttyAMA0" \
  -drive file="./debian-11.1.0-armhf-DVD-1.iso",id=cdrom,if=none,media=cdrom \
    -device virtio-scsi-device \
    -device scsi-cd,drive=cdrom \
  -drive file="./debian-arm.sda.qcow2",id=hd,if=none,media=disk \
    -device virtio-scsi-device \
    -device scsi-hd,drive=hd \
  -netdev user,id=net0,hostfwd=tcp::5555-:22 \
    -device virtio-net-device,netdev=net0 \
  -nographic

https://www.willhaley.com/blog/debian-arm-qemu/

file $(nix --system aarch64-darwin build --no-link --print-out-paths --print-build-logs github:NixOS/nixpkgs/nixpkgs-unstable#qemu)/bin/qemu-system-aarch64

nix --system aarch64-darwin run github:NixOS/nixpkgs/nixpkgs-unstable#qemu -- --version

qemu + NixOS => build-vm

Excelent: Running qemu-system-aarch64 with -accel hvf on aarch64-darwin

Copying files like .img .qcow2

https://serverfault.com/a/670227
https://stackoverflow.com/questions/41733410/remove-qcow2-image-files-holes
@PedroRegisPOAR
Copy link
Contributor Author

PedroRegisPOAR commented Mar 18, 2023 •
edited
Loading

The nixpkgs#darwin.builder

Main references:

nix show-derivation nixpkgs#darwin.builder
nix show-derivation --system aarch64-darwin nixpkgs#darwin.builder | jq '.[]."inputDrvs"' -r
nix build -L --system aarch64-darwin nixpkgs#darwin.builder
nix \
--option allowed-impure-host-deps \
'/bin/sh /usr/lib/libSystem.B.dylib /usr/lib/system/libunc.dylib /dev/zero /dev/random /dev/urandom' \
build \
-L \
--system aarch64-darwin \
nixpkgs#darwin.builder
file /bin/sh
/bin/sh: Mach-O universal binary with 2 architectures: [x86_64:Mach-O 64-bit executable x86_64] [arm64e:Mach-O 64-bit executable arm64e] 
/bin/sh (for architecture x86_64):      Mach-O 64-bit executable x86_64                                                                                      
/bin/sh (for architecture arm64e):      Mach-O 64-bit executable arm64e

TODO: Name it

nix profile install nixpkgs#qemu_kvm nixpkgs#podman nixpkgs#socat
podman machine stop; \
podman machine rm --force; \
podman --log-level=trace machine init --memory=3072 --cpus=8 \
&& podman --log-level=trace machine start
socat -u OPEN:/dev/null UNIX-CONNECT:"${XDG_RUNTIME_DIR}"/tmp/podman-run-1000/podman/podman-machine-default_ready.sock
curl -v -s -X GET --unix-socket /tmp/podman-run-1000/podman/podman-machine-default_ready.sock "http:///libpod/containers/json"
qemu-system-x86_64 \
-m 3072 \
-smp 4 \
-fw_cfg name=opt/com.coreos/config,file=/home/vagrant/.config/containers/podman/machine/qemu/podman-machine-default.ign \
-qmp unix:/tmp/podman-run-$(id -u)/podman/qmp_podman-machine-default.sock,server=on,wait=off \
-netdev socket,id=vlan,fd=3 \
-device virtio-net-pci,netdev=vlan,mac=5a:94:ef:e4:0c:ee \
-device virtio-serial \
-chardev socket,path=/tmp/podman-run-$(id -u)/podman/podman-machine-default_ready.sock,server=on,wait=off,id=apodman-machine-default_ready \
-device virtserialport,chardev=apodman-machine-default_ready,name=org.fedoraproject.port.0 \
-pidfile /tmp/podman-run-$(id -u)/podman/podman-machine-default_vm.pid \
-accel kvm \
-cpu host \
-virtfs local,path=/home/vagrant,mount_tag=vol0,security_model=none \
-drive if=virtio,file=/home/vagrant/.local/share/containers/podman/machine/qemu/podman-machine-default_fedora-coreos-37.20230303.2.0-qemu.x86_64.qcow2 \
 -nographic &
podman system service --time=0 unix:///tmp/podman.sock &
curl -s --unix-socket /tmp/podman.sock http://d/v1.0.0/libpod/info

Refs.:

NixOS build-vm

--memory=3072 --cpus=8

podman machine stop; \
podman machine rm --force; \
podman --log-level=trace machine init --image-path $(show-coreos-path) \
&& podman --log-level=trace machine start

Alpine

podman machine stop; \
podman machine rm --force; \
podman --log-level=trace machine init \
&& sleep 30 \
&& podman --log-level=trace machine start
podman machine stop; \
podman machine rm --force; \
podman --log-level=trace machine init --memory=3072 --cpus=8 \
&& sleep 30 \
&& podman --log-level=trace machine start
mkdir -p ~/.config/containers
cat << 'EOF' >> ~/.config/containers/policy.json
{
    "default": [
        {
            "type": "insecureAcceptAnything"
        }
    ],
    "transports":
        {
            "docker-daemon":
                {
                    "": [{"type":"insecureAcceptAnything"}]
                }
        }
}
EOF

mkdir -p ~/.config/containers
cat << 'EOF' >> ~/.config/containers/registries.conf
[registries.search]
registries = ['docker.io']
[registries.block]
registries = []
EOF

Troubleshooting

podman machine list
podman machine inspect
cat /home/nixuser/.config/containers/storage.conf

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@PedroRegisPOAR