-
Notifications
You must be signed in to change notification settings - Fork 270
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Crash on M1 chip instantly even using docker #1362
Comments
Hi @chapulina , |
Hi @brolin-botsync, Things are ready to provide, see here: osrf/homebrew-simulation#1628 I have this M1 few weeks ago and I'm so addicted with this computer! Super fast! So, keep that in mind..I joined Mac world at the same time I got this M1. Maybe you know how to do this. I will list them all below. See here: I understand that this is outdated for M1 now. It's for Mac with intel. This M1's location is now Another thing, see here in Formula. Full Formula file: I tried this too, but didn't work. It could because I'm inexperience with this area. Do you know how to do this? If you do know how to set them up, can you teach me how? Apparently, docker in container couldn't run gazebo on M1 due to chip difference even if you set container as linux. So, to reproduce this has to be on local first, right? If we can make it work on local, then we can find a way in container immediately. Let me know if it worked for you. Once it worked for you, tell us the secret too! ;) I know this is a little too complicated but I think it's because of homebrew situation. |
@Kakcalu13 and @brolin-botsync - most of the changes required to support Ignition on macOS using Metal have now been merged into the main branches of various repos. The outstanding PR for macOS is in ignition-gazebo (this PR runs fine on macOS but there are some Linux/OpenGL threading compatibility issues to resolve for GPU enabled sensors). I should update or remove the wiki notes you refer to above - they have mostly been superseded now. For Metal support you should use the Garden branch and will need to build from source. I'd be happy to help you both get a native installation of ignition gazebo running on macOS - it would help figure out what the platform specific installation notes should look like. Starting point is to install Garden and test that the examples in ignition-rendering are working with the ogre2 metal flags (e.g. I have not had much luck running ignition using virtualised versions of Ubuntu with accelerated graphics post Ignition Edifice. The main problem is a lack of OpenGL 4.3+ support in the hypervisor drivers. For example VMWare fusion will only support OpenGL 4.1 and that is not sufficient for Ogre2.2. I have not looked at running docker machines with hardware acceleration for a while, but suspect you may run into a similar issue there. Software rendering may work, but that will be an order of magnitude slower for interesting rendering environments. |
Thank you so much for your time, @srmainwaring! This is actually helpful information! I will update you with the status on virtualised version part after I'm able to run gazebo on native. This would be good way for us to work on it and share the result. I've been trying to install Citadel/Fortress on M1 but I will start with Garden. I will keep you guys posted along with steps or issues. |
The key step is getting ignition-rendering ( |
@Kakcalu13 I am at the same spot as you are. I bought a mac book just 2 days back. I am also new to the mac environment. Its like fish out of water. I am still finding my way around the OS. I am trying to run ROS and Gazebo using docker on the machine and I am finding it exhausting. Any help is appreciated from either you or @srmainwaring. I will also try the suggestion given by @srmainwaring. I will keep you posted on my improvements. |
@brolin-botsync Join my boat, I have more space here. :P I'm currently building one. Let us know on your end. I will post more information on my end |
After spend like 10 minutes, I eventually realized I was in ign-rendering7 by default. So here is what I ran after removed ign-rendering7 branch:
This instantly gave me an error: So I followed exact same as the link. Same result. but I didn't have this issue when I was using ign-rendering7 earlier (just saying So I assumed I might need to follow garden from the docs. This didn't go well either. Did I started somewhere wrong? |
There are few errors which can be resolved on my own end. I just gotta download few things. I will post the result shortly! |
@Kakcalu13 not quite sure what the status of you build is. In your colcon workspace have you managed to build the main branch of ignition-rendering and dependencies using: $ cd ~/ign_ws
$ colcon build --merge-install --cmake-args -DCMAKE_BUILD_TYPE=RelWithDebInfo -DCMAKE_MACOSX_RPATH=FALSE -DCMAKE_INSTALL_NAME_DIR=$(pwd)/install/lib After calling $ cd ~/ign_ws
$ source ./install/setup.zsh you should then be able to run $ cd ~/ign_ws/src/ign-rendering/examples/simple_demo
$ mkdir build && cd build
$ cmake ..
$ make -j4
$ ./simple_demo ogre2 metal Your cmake log files appear to be reporting missing Xcode framework libraries etc. This can happen if you upgrade Xcode or the command line tools or brew and don't do a proper clean of all of your build and install directories (i.e. the CMakeCache is stale and subsequent builds get corrupted). |
ok - you're jumping straight to getting the ignition-gazebo metal PR built! I'd still recommend going step by step through the libraries, testing at each stage, as it will be easier to try figure out what's going wrong if gazebo doesn't run from the get go. |
Sorry! I just never had any luck with gazebo on M1. I'm dying to see what it look like on M1 since few weeks ago lol :'(
I think this is the problem here. I don't think I'm on the same page. First of all, I was following ign-rendering5 instead of latest. This is the result in previous messages. So scratch those messages. So, I followed: here instead of This part didn't go through. The error log:
|
Looks like you need to modify a couple of steps from the install instructions (or the env variables got wiped)
Modify the paths below for an M1 machine # [email protected]
export CMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}:/usr/local/opt/[email protected]
export DYLD_FALLBACK_LIBRARY_PATH=${DYLD_FALLBACK_LIBRARY_PATH}:/usr/local/opt/[email protected]/lib:/usr/local/opt/octomap/local
export PKG_CONFIG_PATH=${PKG_CONFIG_PATH}:/usr/local/opt/[email protected]/lib/pkgconfig
# qt5
export CMAKE_PREFIX_PATH=${CMAKE_PREFIX_PATH}:/usr/local/opt/qt@5
brew install gdal
$ cd ~/ign_ws/src/ign-common
$ git fetch
$ git pull
|
Unfortunately, that didn't work :( I modified the path like this:
So, I decided to start all over and started from scratch:
The output:
Lastly, Returns error. :S
I tried this 2 times. Eventually, I noticed that X-code isn't installed by default on M1. I'm not sure how much it impacts on everything. SO after download the x-code, I re-do steps above and other message where I posted steps with cmake. Both of em didn't go through, still. The only thing it changed is how it halts the program really fast after I installed xcode. my brain is fried so I'll try again in few hours or tomorrow. This is EXACTLY how I see M1 right now This is different from linux, tbh. As for m1, there is a lot of stuff involves with pathing, library and setting. Linux do have it, but they are more accessible in some way? Maybe I'm wrong on this. I will try again tomorrow anyway 😎 BTW, thank you so much for your time! I truly appreciate it so much! I hope I am able to do the same thing for you in future! |
I think colcon should show some error msgs on why ign-gui failed. Some differences I saw between your step 9 and mine are these libraries:
The paths setup in step 10 looks the same as mine. I had one extra export which I'm not sure if it's actually needed
|
You will need Xcode and the matching Command line tools (which need to be installed from the Apple developer downloads pages). It looks like you managed to get |
Good morning, gazebo engineers! Round 2 for me!
I think those made my colcon build go smooth without getting halt immediately.
colcon-buld-error.txt As for the ign-gui:
it's on the last page of log
Ngl, this got me excited for a second lol
I think we are really really close in ign-rendering tho |
This question came to my mind, which M1 did you have? M1 PRO, M1 max or just M1? I'm using just M1 |
@Kakcalu13 can you try exporting the following before running the export OGRE2_RESOURCE_PATH=/opt/homebrew/opt/ogre2.2/lib/OGRE-2.2/OGRE (double check I've got that path right). The failure is macOS not resolving an rpath. It's trying a couple of paths from |
I had to re-do the steps due to experiment and tried another ways. The error persists. FYI, colcon build failed too. Just letting you know.
I got this after I run this:
Those error seems like they needed colcon build first. Any suggestion? 😅 EDIT one: I had to delete the whole directory of workspace and re-do the workspace then start all over with all steps include iche033's input. |
Hi, I hope it's ok for me to bump this issue :P I'm kinda dying to try this on M1 👼 |
In addition to using the ogre2.2.rb patch in osrf/homebrew-simulation#1823, can you try adding |
Hi @iche033! Sorry for the late reply. So, apparently I already left it on using your new fix. See here: I ran It returns error
First one returns the same thing but the second one...it's literally THIS close to open ign gazebo but then it crashed immediately. The error log:
|
yes you'll need that but what I mean is also adding it into ogre2.2.rb file here: running |
Oops, I was looking for ignition to be on website and then I found this. Let me test it again, gimme 10 minutes or less |
Seems like I already have it in my ogre2.2.rb. I ran colcon build in the root of workspace (~/workspace)
which returns error. See here: was colcon build supposed to be in the root of workspace? Just wanted to confirm this. BTW I still would love to have ign on m1! I even dream about it last night! lol |
@srmainwaring @iche033 Command failed: 2
'/Applications/Xcode.app/Contents/Developer/usr/bin/make'
See also
/Users/lapyx/ign-workspace/build/ignition-physics6/test/FAKE_INSTALL-prefix/src/FAKE_INSTALL-stamp/FAKE_INSTALL-build-*.log
make[2]: *** [test/FAKE_INSTALL-prefix/src/FAKE_INSTALL-stamp/FAKE_INSTALL-build] Error 1
make[1]: *** [test/CMakeFiles/FAKE_INSTALL.dir/all] Error 2
make: *** [all] Error 2
make: INTERNAL: Exiting with 12 jobserver tokens available; should be 10!
---
Failed <<< ignition-physics6 [53.3s, exited with code 2]
Summary: 13 packages finished [7min 58s]
1 package failed: ignition-physics6
14 packages had stderr output: ignition-cmake3 ignition-common5 ignition-fuel_tools8 ignition-gui7 ignition-math7 ignition-msgs9 ignition-physics6 ignition-plugin2 ignition-rendering7 ignition-sensors7 ignition-tools2 ignition-transport12 ignition-utils2 sdformat13
2 packages not processed Have you experienced any issue like this before? Below is the colcon graph I got from the latest garden yaml file, and the log files are also attached.
FAKE_INSTALL-build-err.log As you can read from the file, the error complains Could this be related to disabling SIP? |
I was able to reproduce the issue with ign-physics. Could be due to recent changes in Garden. This patch is probably not a proper fix but it got ign-physics to build on my mac M1 machine: diff --git a/CMakeLists.txt b/CMakeLists.txt
index 0645982..fc958e8 100644
--- a/CMakeLists.txt
+++ b/CMakeLists.txt
@@ -63,6 +63,8 @@ ign_find_package(EIGEN3 REQUIRED)
ign_find_package(sdformat13
REQUIRED_BY sdf dartsim tpe bullet)
+unset(TINYXML2_LIBRARIES)
+
#--------------------------------------
# Find dartsim for the dartsim plugin wrapper
ign_find_package(DART |
That worked! I had more issues to be addressed though
I had to replace https://github.com/gazebosim/gz-launch/blob/8c78b6c937943ba1650e887c3a82dbfaea6c154f/src/vendor/backward.hpp#L3937 with
I had to create a symlink of |
oh looks like that's also fixed upstream in the backward repo: https://github.com/bombela/backward-cpp/blob/master/backward.hpp#L4220-L4225. |
Is this solved? Error is always:
|
Hi Tadteo! Not yet, but you can start with Garden though. See here: osrf/homebrew-simulation#1628 (comment) I still haven't had a chance to test it. Hopefully soon for me! If it worked for you, feel free to post it. It's important to know that it worked for you so others can try it out too |
After spend time with Gazebo on M1 using docker (Specifically using ubuntu 20.04 on docker)..I get the same output as above. Full log:
I assume building from gazebo on local inside M1 would find the root cause in Ubuntu 20.04 Docker and solve the issue right away, but this seems like it is not the case anymore. It's mostly from the Ruby issue. |
Created an issue since this seems more Ruby issue? https://bugs.ruby-lang.org/issues/18807 Here is the reproduce to error immediately:
This zip focus on load gazebo using I will post updates on here as well |
@Kakcalu13 wondering how to best progress with the macOS help here. There seem to be two different objectives in this issue and it might help to focus on one of them.
For item 1, we know it works for Intel based macs on Big Sur 11.6.2, and most likely Monterey (although I have not verified the latter). I am fairly sure a couple of the OSRF team have built Garden for a M1 machine and managed to get it working. You may need to use a couple of open PRs for gz-gui and gz-sim to get all the features. For item 2, I am not aware of how to get hardware acceleration working on a mac with Docker for applications requiring OpenGL 4.2+ (Gazebo Sim or otherwise). The docker based hardware support is for devices with Nvidia graphics cards (so not applicable to modern macs), alternatively you can use software rendering (but it is very slow, and it seems like a sub-optimal use of a M1 machine to do that). Which of 1 and 2 would you like to solve? If 1, we can probably help. If 2, then I think you need to verify that you have a method to get hardware accelerated GUI applications working with Docker on macOS, but would not recommend starting with Gazebo Sim, but choose a simpler application with fewer dependencies. I'd be very interested if you do get it working as I currently do not have a virtualised OpenGL 4.2+ environment for macOS and it would be extremely useful to have a solution for this. |
My apologies. I realize this does seem like it have two objectives. This issue is for the item 2 only as the item 1 is on osrf/homebrew-simulation#1628. That issue focus on Gazebo Sim natively only. The reason why we were talking about Gazebo Sim natively on here because if we were able to build from the source then we can certainly build it on Docker using network mode host. Unfortunately, it doesn't seem like it can be the case due to multiple issues and heavy slow outside of Gazebo situation. Would you like me to close this and create an issue to focus on docker and hardware accelerated GUI applications only? Maybe it can make things more clear? Side note, I will start with verify the method to get hardware accelerated GUI applications working with Docker on MacOS M1 now. |
No - good to keep this open and thanks for clarifying.
I'm missing something here. How does building Gazebo natively on macOS help with building Gazebo on an Ubuntu Docker machine and vice-versa? I can see how a network mode host would let you share IP addresses between applications running natively on macOS with those running in a docker container - but the toolchains for building are completely different?
I really think this is the essential starting point. |
~~Because there's somewhere missing with the package. Unlike other machine, this M1 integrated GPU and CPU in one
I didn't know Garden on Docker is out now. Thanks!!! Still middle of OpenGL accelerated applications. I will post updates on here once I get it up P.S. if you have a LITTLE spare of time, do you know the answer for ign-gui on osrf/homebrew-simulation#1628 (comment)? 👼 if no, all good! |
I don't know about that. One of the ArduPilot devs used a nightly build of Garden on an Ubuntu Jammy machine, so I guess the packages are available if you wanted to roll your own docker file. |
Unfortunately, I'm really out of ideas. Maybe this is good time for me to take a break for a bit then resume this in few days. So, basically..I decided to check openGL on docker on ubuntu and ubuntu natively vs docker on m1 and m1 ubuntu. Ubuntu natively: So, this led me to believe that both are exactly same which as it should be! So I ran tests on M1/Ubuntu using Both made the exact same result. See here: Screen.Recording.2022-05-26.at.3.07.32.PM.movAs you can see that this is a simple test for acceleration GUI just fine. So I tried the gui by git clone gui only and run examples from standalone/scene_provider. For some reason, it just didn't work with m1.
It wouldn't work if you install sudo apt lib ignition-msgs8 either. I could be wrong but I think maybe some packages are being denied during install.. Also, According to Ruby issue I created this morning, Gazebo calls runGui in I need to look into somewhere to make sure everything in GUI is working properly. It's the GUI prevents the program to run. I know I've said this already, but I strongly believe that GUI is the issue at this point. It is just that it seems there's some missing packages within GUI. As for the Garden on docker. It sounds like it's all ready for Jammy, is that correct? DO you know if anyone with m1 managed to install it using Jammy? For some reason, I have no luck with Garden on docker either. I honestly think my newbie knowledge on Mac contributes issue too. I'm a linux guy, I almost never use M1 until last February. Oh well, I will resume this later. 🖖 |
It's officially working, everyone!!! See here on M1 where I spinned my robot: ezgif.com-gif-maker.movON M1! It's surprisingly faster and almost same speed as my local linux (it lags on docker on linux) Here is the solution: Here is what I used for docker: That's it!!! Looks like amd64 is NOT supported on Gazebo/M1. I believe we can build Gazebo using arm64 only Thank you, everyone! This helped me a ton! Hope it helps you a lot too, guys! See you around in Gazebo community! |
Environment
OS Version: macOS Monterey with Apple M1 chip using Docker (Ubuntu 20.04)
Source or binary build: Citadel binary build
If this is a GUI or sensor rendering bug, describe your GPU and rendering system. Otherwise delete this section.
GUI Bug
Full log after launch gazebo such as
ign gazebo --verbose
log.txt
Ran
ign gazebo -v 3
and it has the same as above. Ranign gazebo -v 4
and it showed 4 extra blue line. Here log while it was inside middle log like above first log:log 2.txt
I tried to do it locally and this results the exact same as this issue
So that leaves me two options: Pull gazebo from gazebo website and run docker using feagi.yml
So, I ran feagi.yml following this steps:
docker-compose -f feagi.yml up
The error looks exact same as log.txt (That's actually where I copied all from)
So, that leaves me the last option.
docker pull gazebo
(This is from gazebo's docker hub)docker run gazebo
You can see the error instantly too.
The full log (I can't get more than that)
So this seems like IGN_IP issue, GPU and ogre altogether.
system_profiler SPDisplaysDataType
. Copy the output here.Description
ign gazebo
Addition information: I ran --platform = linux/arm64 too or anything related to this command. Everything else worked except Gazebo.
The text was updated successfully, but these errors were encountered: