Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Installation Steps #583

Open
MuffinsThaCat opened this issue Feb 9, 2024 · 28 comments
Open

Installation Steps #583

MuffinsThaCat opened this issue Feb 9, 2024 · 28 comments

Comments

@MuffinsThaCat
Copy link

MuffinsThaCat commented Feb 9, 2024

The step we're following to install are here : https://github.com/edgelesssys/marblerun/blob/master/BUILD.md#build

/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make: *** [Makefile:91: all] Error 2```
@daniel-weisse
Copy link
Member

Hi there,
Can you please provide the following information:

  • Your OS, e.g. from cat /etc/os-release
  • Go version, e.g. from go version
  • EdgelessRT version, e.g. from apt show edglessrt
  • The git commit you are trying to build from, e.g. from git rev-parse HEAD
  • Are you to following the Docker instructions or the local building instructions ?

@dhruvinparikh
Copy link

Hi there, Can you please provide the following information:

  • Your OS, e.g. from cat /etc/os-release
  • Go version, e.g. from go version
  • EdgelessRT version, e.g. from apt show edglessrt
  • The git commit you are trying to build from, e.g. from git rev-parse HEAD
  • Are you to following the Docker instructions or the local building instructions ?
  • cat /etc/os-release
PRETTY_NAME="Ubuntu 22.04.3 LTS"
NAME="Ubuntu"
VERSION_ID="22.04"
VERSION="22.04.3 LTS (Jammy Jellyfish)"
VERSION_CODENAME=jammy
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=jammy
  • go version
go version go1.21.2 linux/arm64
  • apt show edglessrt
N: Unable to locate package edglessrt
N: Unable to locate package edglessrt
E: No packages found
  • git rev-parse HEAD
c33669b10382afad721d2dd534504dbcfff125df
  • Having issues setting up environment itself. But I am able to build using docker instructions with no issues.

@daniel-weisse
Copy link
Member

Hi @dhruvinparikh
Please follow the installation instructions for EdglessRT to set up the prerequisites for building MarbleRun.

@MuffinsThaCat
Copy link
Author

It was installed but not sure why it isn't showing up in apt, I think was the issue. Looking into it. Will update

@MuffinsThaCat
Copy link
Author

MuffinsThaCat commented Feb 13, 2024

When we build the Coordinator control plane and Marble test applications:

mkdir build
cd build
cmake ..
make

I am stuck at make with the following error:

runtime/cgo

gcc_arm64.S: Assembler messages:
gcc_arm64.S:30: Error: no such instruction: stp x29,x30,[sp,' gcc_arm64.S:34: Error: too many memory references for mov'
gcc_arm64.S:36: Error: no such instruction: stp x19,x20,[sp,' gcc_arm64.S:39: Error: no such instruction: stp x21,x22,[sp,'
gcc_arm64.S:42: Error: no such instruction: stp x23,x24,[sp,' gcc_arm64.S:45: Error: no such instruction: stp x25,x26,[sp,'
gcc_arm64.S:48: Error: no such instruction: stp x27,x28,[sp,' gcc_arm64.S:52: Error: too many memory references for mov'
gcc_arm64.S:53: Error: too many memory references for mov' gcc_arm64.S:54: Error: too many memory references for mov'
gcc_arm64.S:56: Error: no such instruction: blr x20' gcc_arm64.S:57: Error: no such instruction: blr x19'
gcc_arm64.S:59: Error: no such instruction: ldp x27,x28,[sp,' gcc_arm64.S:62: Error: no such instruction: ldp x25,x26,[sp,'
gcc_arm64.S:65: Error: no such instruction: ldp x23,x24,[sp,' gcc_arm64.S:68: Error: no such instruction: ldp x21,x22,[sp,'
gcc_arm64.S:71: Error: no such instruction: ldp x19,x20,[sp,' gcc_arm64.S:74: Error: no such instruction: ldp x29,x30,[sp],'
[ 0%] Built target coordinator-noenclave
[ 0%] Built target coordinatorlib
Consolidate compiler generated dependencies of target coordinator-enclave
[ 11%] Linking CXX executable coordinator-enclave
[ 22%] Built target coordinator-enclave
[ 33%] Built target signing-key
[ 44%] Generating coordinator-config.json
/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

@daniel-weisse
Copy link
Member

gcc_arm64.S: Assembler messages:

Are you trying to build MarbleRun on an ARM host?
Or do you have the GOARCH environment variable set to arm64?

In the second case, unset the GOARCH env variable, and try again.
In the first case, this will not work out of the box.
You might be successful if you just export GOARCH=amd64, but will likely also have to tell the build script to use a toolchain that targets amd64 hosts (using the CC env variable).
But I'm not confident enough to tell you that this will work, as cross compiling for different architectures can be a bit tricky, especially if CGO is involved.

@MuffinsThaCat
Copy link
Author

MuffinsThaCat commented Feb 16, 2024

uname -a
Linux d1cc01526652 6.6.12-linuxkit #1 SMP Fri Jan 19 08:53:17 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux

I tried following unset GOARCH make but still same error

[ 0%] Built target coordinator-noenclave
[ 0%] Built target coordinatorlib
[ 22%] Built target coordinator-enclave
[ 33%] Built target signing-key
[ 44%] Generating coordinator-config.json
/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

@thomasten
Copy link
Member

thomasten commented Feb 16, 2024

Hi, can you please provide the full terminal output of all these commands

. /opt/edgelessrt/share/openenclave/openenclaverc
mkdir build && cd build
cmake ..
OE_LOG_LEVEL=INFO make VERBOSE=1
ls -l

@MuffinsThaCat
Copy link
Author

root@d1cc01526652:/com.docker.devenvironments.code/marblerun# mkdir build && cd build
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# cmake ..
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
CMake Error at CMakeLists.txt:4 (find_package):
Could not find a package configuration file provided by "OpenEnclave" with
any of the following names:

 OpenEnclaveConfig.cmake
 openenclave-config.cmake

Add the installation prefix of "OpenEnclave" to CMAKE_PREFIX_PATH or set
"OpenEnclave_DIR" to a directory containing one of the above files. If
"OpenEnclave" provides a separate development package or SDK, be sure it
has been installed.

-- Configuring incomplete, errors occurred!
See also "/com.docker.devenvironments.code/marblerun/build/CMakeFiles/CMakeOutput.log".
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# OE_LOG_LEVEL=INFO make VERBOSE=1
make: *** No targets specified and no makefile found. Stop.
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# ls -l
total 16
-rw-r--r-- 1 root root 14292 Feb 16 14:34 CMakeCache.txt
drwxr-xr-x 6 root root 192 Feb 16 14:34 CMakeFiles
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build#

@thomasten
Copy link
Member

updated my comment to include the source command

@MuffinsThaCat
Copy link
Author

Works, thanks!

@MuffinsThaCat
Copy link
Author

Actually sorry, need to unclose this for a second. There was an error at the end we missed.

Updated terminal log
root@d1cc01526652:/com.docker.devenvironments.code/marblerun# mkdir build && cd build
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# cmake ..
-- The C compiler identification is GNU 11.4.0
-- The CXX compiler identification is GNU 11.4.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /usr/bin/cc - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /usr/bin/c++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Looking for pthread.h
-- Looking for pthread.h - found
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
-- Looking for crypto library - found
-- Looking for dl library - found
CMake Warning at /opt/edgelessrt/lib/openenclave/cmake/openenclave-config.cmake:122 (message):
-- Looking for sgx_dcap_ql library - not found. Attestations based on
quotes would not function without the quote provider.
Call Stack (most recent call first):
CMakeLists.txt:4 (find_package)
-- Performing Test OE_SPECTRE_MITIGATION_C_FLAGS_SUPPORTED
-- Performing Test OE_SPECTRE_MITIGATION_C_FLAGS_SUPPORTED - Failed
-- Performing Test OE_SPECTRE_MITIGATION_CXX_FLAGS_SUPPORTED
-- Performing Test OE_SPECTRE_MITIGATION_CXX_FLAGS_SUPPORTED - Failed
-- Configuring done
-- Generating done
-- Build files have been written to: /com.docker.devenvironments.code/marblerun/build
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# OE_LOG_LEVEL=INFO make VERBOSE=1
/usr/bin/cmake -S/com.docker.devenvironments.code/marblerun -B/com.docker.devenvironments.code/marblerun/build --check-build-system CMakeFiles/Makefile.cmake 0
/usr/bin/cmake -E cmake_progress_start /com.docker.devenvironments.code/marblerun/build/CMakeFiles /com.docker.devenvironments.code/marblerun/build//CMakeFiles/progress.marks
make -f CMakeFiles/Makefile2 all
make[1]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/coordinator-noenclave.dir/build.make CMakeFiles/coordinator-noenclave.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/coordinator-noenclave.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/coordinator-noenclave.dir/build.make CMakeFiles/coordinator-noenclave.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/cmd/coordinator && /usr/bin/cmake -P /com.docker.devenvironments.code/marblerun/build_with_version.cmake go 1.4.0 /com.docker.devenvironments.code/marblerun/build/coordinator-noenclave main
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
[ 0%] Built target coordinator-noenclave
make -f CMakeFiles/coordinatorlib.dir/build.make CMakeFiles/coordinatorlib.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/coordinatorlib.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/coordinatorlib.dir/build.make CMakeFiles/coordinatorlib.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/cmd/coordinator && /usr/bin/cmake -P /com.docker.devenvironments.code/marblerun/build_with_version.cmake ertgo 1.4.0 /com.docker.devenvironments.code/marblerun/build/libcoordinator.a main
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
[ 0%] Built target coordinatorlib
make -f CMakeFiles/coordinator-enclave.dir/build.make CMakeFiles/coordinator-enclave.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/coordinator-enclave.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/coordinator-enclave.dir/build.make CMakeFiles/coordinator-enclave.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
[ 11%] Building C object CMakeFiles/coordinator-enclave.dir/enclave/main.c.o
/usr/bin/cc -DOE_BUILD_ENCLAVE -isystem /opt/edgelessrt/include -isystem /opt/edgelessrt/include/openenclave/3rdparty/libc -isystem /opt/edgelessrt/include/openenclave/3rdparty -g -m64 -fPIE -nostdinc -fstack-protector-strong -fvisibility=hidden -fno-omit-frame-pointer -ffunction-sections -fdata-sections -ftls-model=local-exec -MD -MT CMakeFiles/coordinator-enclave.dir/enclave/main.c.o -MF CMakeFiles/coordinator-enclave.dir/enclave/main.c.o.d -o CMakeFiles/coordinator-enclave.dir/enclave/main.c.o -c /com.docker.devenvironments.code/marblerun/enclave/main.c
[ 22%] Linking CXX executable coordinator-enclave
/usr/bin/cmake -E cmake_link_script CMakeFiles/coordinator-enclave.dir/link.txt --verbose=1
/usr/bin/c++ -g CMakeFiles/coordinator-enclave.dir/enclave/main.c.o /opt/edgelessrt/lib/openenclave/enclave/objects-Release/ertmeshentry/meshentry.cpp.o /opt/edgelessrt/lib/openenclave/enclave/objects-Release/ertmeshentry/emain_t.c.o -o coordinator-enclave /opt/edgelessrt/lib/openenclave/enclave/liboeenclave.a libcoordinator.a /opt/edgelessrt/lib/openenclave/enclave/libertttls.a /opt/edgelessrt/lib/openenclave/enclave/libertlibc.a -static -Wl,--eh-frame-hdr -Wl,-u,ert_syscall -lstdc++ /opt/edgelessrt/lib/openenclave/enclave/libertlibunwind.a /opt/edgelessrt/lib/openenclave/enclave/liboecryptombedtls.a /opt/edgelessrt/lib/openenclave/enclave/libmbedtls.a /opt/edgelessrt/lib/openenclave/enclave/libmbedx509.a /opt/edgelessrt/lib/openenclave/enclave/libmbedcrypto.a /opt/edgelessrt/lib/openenclave/enclave/liboehostepoll.a /opt/edgelessrt/lib/openenclave/enclave/liboehostfs.a /opt/edgelessrt/lib/openenclave/enclave/liboehostresolver.a /opt/edgelessrt/lib/openenclave/enclave/liboehostsock.a /opt/edgelessrt/lib/openenclave/enclave/liboelibc.a /opt/edgelessrt/lib/openenclave/enclave/liboesyscall.a /opt/edgelessrt/lib/openenclave/enclave/liboelibc.a /opt/edgelessrt/lib/openenclave/enclave/liboesyscall.a /opt/edgelessrt/lib/openenclave/enclave/liboecore.a -nostdlib -nodefaultlibs -nostartfiles -Wl,--no-undefined,-Bstatic,-Bsymbolic,--export-dynamic,-pie,--build-id -Wl,-z,noexecstack -Wl,-z,now -Wl,-gc-sections
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
[ 22%] Built target coordinator-enclave
make -f CMakeFiles/signing-key.dir/build.make CMakeFiles/signing-key.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/signing-key.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/signing-key.dir/build.make CMakeFiles/signing-key.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
[ 33%] Generating private.pem, public.pem
openssl genrsa -out private.pem -3 3072
openssl rsa -in private.pem -pubout -out public.pem
writing RSA key
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
[ 33%] Built target signing-key
make -f CMakeFiles/sign-coordinator.dir/build.make CMakeFiles/sign-coordinator.dir/depend
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
cd /com.docker.devenvironments.code/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build /com.docker.devenvironments.code/marblerun/build/CMakeFiles/sign-coordinator.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make -f CMakeFiles/sign-coordinator.dir/build.make CMakeFiles/sign-coordinator.dir/build
make[2]: Entering directory '/com.docker.devenvironments.code/marblerun/build'
[ 44%] Generating coordinator-config.json
/opt/edgelessrt/bin/oesign eradump -e coordinator-enclave.signed > coordinator-config.json
/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[2]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make[1]: Leaving directory '/com.docker.devenvironments.code/marblerun/build'
make: *** [Makefile:91: all] Error 2
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build# ls -l
total 76048
-rw-r--r-- 1 root root 14994 Feb 16 17:41 CMakeCache.txt
drwxr-xr-x 26 root root 832 Feb 16 17:41 CMakeFiles
-rw-r--r-- 1 root root 1665 Feb 16 17:41 cmake_install.cmake
-rw-r--r-- 1 root root 107 Feb 16 17:41 coordinator.conf
-rwxr-xr-x 1 root root 34408432 Feb 16 17:41 coordinator-enclave
-rw-r--r-- 1 root root 43300922 Feb 16 17:41 libcoordinator.a
-rw-r--r-- 1 root root 1758 Feb 16 17:41 libcoordinator.h
-rw-r--r-- 1 root root 11399 Feb 16 17:41 Makefile
-rw------- 1 root root 2484 Feb 16 17:41 private.pem
-rw-r--r-- 1 root root 621 Feb 16 17:41 public.pem
root@d1cc01526652:/com.docker.devenvironments.code/marblerun/build#

The error: ' make: *** [Makefile:91: all] Error 2 '

It looks like we're close but something may be missing.

@thomasten
Copy link
Member

It seems it doesn't (even try to) sign the coordinator-enclave binary. We have [ 22%] Built target coordinator-enclave and [ 44%] Generating coordinator-config.json. But between these two, Generating coordinator-enclave.signed is missing. What cmake version is this?

@thomasten thomasten reopened this Mar 20, 2024
@dhruvinparikh
Copy link

It seems it doesn't (even try to) sign the coordinator-enclave binary. We have [ 22%] Built target coordinator-enclave and [ 44%] Generating coordinator-config.json. But between these two, Generating coordinator-enclave.signed is missing. What cmake version is this?

cmake version 3.22.1

@thomasten
Copy link
Member

Is the git working tree clean or did you change any file?

After getting this error, try what happens if you run
oesign sign -e coordinator-enclave -c coordinator.conf -k private.pem

@dhruvinparikh
Copy link

oesign sign -e coordinator-enclave -c coordinator.conf -k private.pem

Created coordinator-enclave.signed

@dhruvinparikh
Copy link

dhruvinparikh commented Mar 25, 2024

  • The issue was due to go PATH variable was reset somehow i guess.

Following are the last few lines after running make , let me know if this expected?

make[2]: Leaving directory '/home/ubuntu/marblerun/build'
[100%] Built target sign-marble-test
make  -f CMakeFiles/premain-libos.dir/build.make CMakeFiles/premain-libos.dir/depend
make[2]: Entering directory '/home/ubuntu/marblerun/build'
cd /home/ubuntu/marblerun/build && /usr/bin/cmake -E cmake_depends "Unix Makefiles" /home/ubuntu/marblerun /home/ubuntu/marblerun /home/ubuntu/marblerun/build /home/ubuntu/marblerun/build /home/ubuntu/marblerun/build/CMakeFiles/premain-libos.dir/DependInfo.cmake --color=
make[2]: Leaving directory '/home/ubuntu/marblerun/build'
make  -f CMakeFiles/premain-libos.dir/build.make CMakeFiles/premain-libos.dir/build
make[2]: Entering directory '/home/ubuntu/marblerun/build'
cd /home/ubuntu/marblerun/cmd/premain-libos && ertgo build -buildmode=pie -buildvcs=false -o /home/ubuntu/marblerun/build
make[2]: Leaving directory '/home/ubuntu/marblerun/build'
[100%] Built target premain-libos
make[1]: Leaving directory '/home/ubuntu/marblerun/build'
/usr/bin/cmake -E cmake_progress_start /home/ubuntu/marblerun/build/CMakeFiles 0

@dhruvinparikh
Copy link

dhruvinparikh commented Mar 25, 2024

Inside build directory. Ran erthost coordinator-enclave.signed

getting following messages on console

OE_SIMULATION=1 erthost coordinator-enclave.signed
[erthost] running in simulation mode
[erthost] loading enclave ...
[erthost] entering enclave ...
[meshentry] invoking premain
[meshentry] invoking main
{"level":"info","ts":1711388126.5188923,"caller":"coordinator/run.go:40","msg":"Starting coordinator","version":"1.4.1","commit":"c916af2d1592717d4e455e22fc663a0cc700a111"}
{"level":"info","ts":1711388126.5188923,"caller":"coordinator/run.go:73","msg":"Creating the Core object"}
{"level":"info","ts":1711388126.5228922,"caller":"core/core.go:124","msg":"Loading state"}
{"level":"info","ts":1711388126.5228922,"caller":"core/core.go:162","msg":"No sealed state found. Proceeding with new state."}
{"level":"info","ts":1711388126.5228922,"caller":"core/core.go:289","msg":"Generating quote"}
ERROR: can't get report in simulation mode (oe_result_t=OE_UNSUPPORTED) [openenclave-src/enclave/sgx/report.c:oe_get_report_v2:190]
{"level":"warn","ts":1711388126.5228922,"caller":"core/core.go:293","msg":"Failed to get quote. Proceeding in simulation mode.","error":"OE_UNSUPPORTED"}
{"level":"info","ts":1711388126.5228922,"caller":"coordinator/run.go:102","msg":"Starting the client server"}
{"level":"info","ts":1711388126.5228922,"caller":"coordinator/run.go:111","msg":"Starting the marble server"}
{"level":"info","ts":1711388126.5228922,"caller":"server/server.go:121","msg":"Starting client https server","address":":4433"}
{"level":"info","ts":1711388126.5228922,"caller":"grpclog/component.go:36","msg":"[core][Server #1] Server created","system":"grpc","grpc_log":true}
{"level":"info","ts":1711388126.5228922,"caller":"grpclog/component.go:36","msg":"[core][Server #1 ListenSocket #2] ListenSocket created","system":"grpc","grpc_log":true}
{"level":"info","ts":1711388126.5228922,"caller":"coordinator/run.go:123","msg":"Started gRPC server","grpcAddr":"[::]:2001"}

@thomasten
Copy link
Member

make and coordinator output both look good

@MuffinsThaCat
Copy link
Author

@thomasten wondering if you can take a look at this. Wondering what exit code 128 is and if this is an issue that is affecting our build?

[+] Building 5.3s (6/10)                                                                                                                                                      docker:default => [internal] load build definition from Dockerfile                                                                                                                                    0.0s => => transferring dockerfile: 2.87kB                                                                                                                                                  0.0s => [internal] load metadata for ghcr.io/edgelesssys/marblerun/build-base:v1.4.1                                                                                                        0.1s => [internal] load .dockerignore                                                                                                                                                       0.0s => => transferring context: 2B                                                                                                                                                         0.0s => [build 1/5] FROM ghcr.io/edgelesssys/marblerun/build-base:v1.4.1@sha256:f00b8652db5ccc88832df06450cd10e6d2846934c72675a929666cac331852de                                            0.0s => CACHED [build 2/5] RUN apt-get install -y --no-install-recommends   build-essential   ca-certificates   clang-11   cmake   git   libssl-dev   ninja-build   wget                    0.0s => ERROR [build 3/5] RUN wget -qO- https://go.dev/dl/go1.21.8.linux-amd64.tar.gz \| tar -C /usr/local -xz   && git clone -b v0.4.3 --depth=1 https://github.com/edgelesssys/edgelessrt  5.1s------ > [build 3/5] RUN wget -qO- https://go.dev/dl/go1.21.8.linux-amd64.tar.gz \| tar -C /usr/local -xz   && git clone -b v0.4.3 --depth=1 https://github.com/edgelesssys/edgelessrt   && git clone -b v1.5.0 --depth=1 https://github.com/edgelesssys/marblerun   && mkdir ertbuild mrbuild:3.415 Cloning into 'edgelessrt'...3.791 Note: switching to 'b49e0175599ad3deae43eb69a52e1784d1a12f20'.3.7913.791 You are in 'detached HEAD' state. You can look around, make experimental3.791 changes and commit them, and you can discard any commits you make in this3.791 state without impacting any branches by switching back to a branch.3.7913.791 If you want to create a new branch to retain commits you create, you may3.791 do so (now or later) by using -c with the switch command. Example:3.7913.791   git switch -c 3.7913.791 Or undo this operation with:3.7913.791   git switch -3.7913.791 Turn off this advice by setting config variable advice.detachedHead to false3.7913.821 Cloning into 'marblerun'...3.958 warning: Could not find remote branch v1.5.0 to clone.3.958 fatal: Remote branch v1.5.0 not found in upstream origin------Dockerfile:17--------------------  16 \|     ARG goversion=1.21.8  17 \| >>> RUN wget -qO- https://go.dev/dl/go${goversion}.linux-amd64.tar.gz \| tar -C /usr/local -xz \  18 \| >>>   && git clone -b $erttag --depth=1 https://github.com/edgelesssys/edgelessrt \  19 \| >>>   && git clone -b $mrtag --depth=1 https://github.com/edgelesssys/marblerun \  20 \| >>>   && mkdir ertbuild mrbuild  21 \|--------------------ERROR: failed to solve: process "/bin/sh -c wget -qO- https://go.dev/dl/go${goversion}.linux-amd64.tar.gz \| tar -C /usr/local -xz   && git clone -b $erttag --depth=1 https://github.com/edgelesssys/edgelessrt   && git clone -b $mrtag --depth=1 https://github.com/edgelesssys/marblerun   && mkdir ertbuild mrbuild" did not complete successfully: exit code: 128 --

@thomasten
Copy link
Member

The dockerfiles in the master branch are for an upcoming release. You can use the ones at tag v1.4.1: https://github.com/edgelesssys/marblerun/tree/v1.4.1/dockerfiles

@MuffinsThaCat
Copy link
Author

I think we're right near the end then: when we run make we get this error:

0%] Built target coordinator-noenclave
[ 0%] Built target coordinatorlib
Consolidate compiler generated dependencies of target coordinator-enclave
[ 11%] Linking CXX executable coordinator-enclave
[ 22%] Built target coordinator-enclave
[ 33%] Built target signing-key
[ 44%] Generating coordinator-config.json
/opt/edgelessrt/bin/oesign ERROR: Failed to load coordinator-enclave.signed as ELF64
make[2]: *** [CMakeFiles/sign-coordinator.dir/build.make:74: coordinator-config.json] Error 1
make[2]: *** Deleting file 'coordinator-config.json'
make[1]: *** [CMakeFiles/Makefile2:207: CMakeFiles/sign-coordinator.dir/all] Error 2
make: *** [Makefile:91: all] Error 2

@MuffinsThaCat
Copy link
Author

Here are the full steps we are following and where we get stuck:

Ego installation

sudo mkdir -p /etc/apt/keyrings
wget -qO- https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo tee /etc/apt/keyrings/intel-sgx-keyring.asc > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/intel-sgx-keyring.asc arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/intel-sgx.list
sudo apt update
EGO_DEB=ego_1.5.2_amd64_ubuntu-$(lsb_release -rs).deb
wget https://github.com/edgelesssys/ego/releases/download/v1.5.2/$EGO_DEB
sudo apt install ./$EGO_DEB build-essential libssl-dev

Verify ego installation

https://github.com/edgelesssys/ego.git
cd ./ego/samples/helloworld./
ego-go build
ego sign helloworld

OE_SIMULATION=1 ego run helloworld

Install edgelessrt

sudo mkdir -p /etc/apt/keyrings
wget -qO- https://download.01.org/intel-sgx/sgx_repo/ubuntu/intel-sgx-deb.key | sudo tee /etc/apt/keyrings/intel-sgx-keyring.asc > /dev/null
echo "deb [signed-by=/etc/apt/keyrings/intel-sgx-keyring.asc arch=amd64] https://download.01.org/intel-sgx/sgx_repo/ubuntu $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/intel-sgx.list

sudo apt update
ERT_DEB=edgelessrt_0.4.3_amd64_ubuntu-$(lsb_release -rs).deb
wget https://github.com/edgelesssys/edgelessrt/releases/download/v0.4.3/$ERT_DEB
sudo apt install ./$ERT_DEB build-essential cmake libssl-dev

. /opt/edgelessrt/share/openenclave/openenclaverc

Check edglessrt installation

apt show edglessrt

git clone https://github.com/edgelesssys/edgelessrt.git
cd ./edgelessrt/samples/helloworld/
mkdir build
cd ./build
cmake ..

make

OE_SIMULATION=1 erthost enclave.signed

Install go

cd ~
curl -OL https://golang.org/dl/go1.22.3.linux-amd64.tar.gz
sha256sum go1.22.3.linux-amd64.tar.gz
sudo tar -C /usr/local -xvf go1.22.3.linux-amd64.tar.gz
sudo nano ~/.profile
add the following information to the end of your file:
export PATH=$PATH:/usr/local/go/bin
source ~/.profile
go version

Install Docker

sudo apt-get update
sudo apt-get install ca-certificates curl
sudo install -m 0755 -d /etc/apt/keyrings
sudo curl -fsSL https://download.docker.com/linux/ubuntu/gpg -o /etc/apt/keyrings/docker.asc
sudo chmod a+r /etc/apt/keyrings/docker.asc
echo
"deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.asc] https://download.docker.com/linux/ubuntu
$(. /etc/os-release && echo "$VERSION_CODENAME") stable" |
sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt-get update
sudo apt-get install docker-ce docker-ce-cli containerd.io docker-buildx-plugin docker-compose-plugin
sudo docker run hello-world
sudo usermod -aG docker ubuntu
Log out and log back in

Build marble run

git clone --branch v1.4.1 https://github.com/edgelesssys/marblerun.git
cd ./marblerun
openssl genrsa -out private.pem -3 3072
export DOCKER_BUILDKIT=1
docker build --secret id=signingkey,src=private.pem --target export -o. - < dockerfiles/Dockerfile.coordinator
docker build -o. - < dockerfiles/Dockerfile.cli
Mkdir build
Cd build
. /opt/edgelessrt/share/openenclave/openenclaverc
Cmake ..
Make

That's when we get the above error

@thomasten
Copy link
Member

The docker build and the direct build in your host environment are alternatives. You don't need both. From your commands, I'd assume that the direct build on the host fails. Let's ignore this for a moment and look at the docker build. So when running

docker build --secret id=signingkey,src=private.pem --target export -o. - < dockerfiles/Dockerfile.coordinator

Does this command succeed? Does it create the binaries in the current working dir?

@MuffinsThaCat
Copy link
Author

That worked. But still getting the same error with make.

@thomasten
Copy link
Member

The problem seems to be that this commit can't be built with latest EdgelessRT and Go. You can use EdgelessRT 0.4.2 and Go 1.20.14 or build latest master of MarbleRun instead.

@MuffinsThaCat
Copy link
Author

MuffinsThaCat commented May 29, 2024

Still the same error.

The config is :
marblerun v1.4.1
EdgelessRT 0.4.2
Go 1.20.14

Are make and Docker build alternatives?

If docker build works then how can i run “OE_SIMULATION=1 erthost build/coordinator-enclave.signed” , is that something needs to run inside container?

@thomasten
Copy link
Member

Regarding the build error, I can't reproduce and am out of ideas. If the docker build works on your machine, executing the commands from the docker file on the Ubuntu 20.04 or 22.04 host should also work.

Regarding the binary, after the docker command I mentioned above successfully completed, coordinator-enclave.signed should be created in the current working dir.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants