Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reproducible build for version 0.9.1 #2470

Closed
xrviv opened this issue Oct 17, 2024 · 11 comments
Closed

Reproducible build for version 0.9.1 #2470

xrviv opened this issue Oct 17, 2024 · 11 comments
Labels
Reproducible builds
Milestone
v0.9.2

Comments

@xrviv
Copy link

xrviv commented Oct 17, 2024

Hello team Zeus ⚡️,

Danny here from WalletScrutiny.

We ran several tests:

  1. We used our test script together with an app-specific script for Zeus, however the build failed. We documented our efforts here.
  2. Since our current application-specific script is failing, we decided to use a custom script based off the instructions provided by Zeus here. This is currently a work in progress, but our tentative verdict for this is the results are coming off as non-verifiable.
  • We have successfully integrated a new script that's based off the instructions noted above together with our test script.
  • This is the new script:
#!/bin/bash

# Script: app.zeusln.zeus.sh
# Version: 1.1.7

# Variables passed from test.sh's metadata extraction process
appId=$appId  # Retrieved by test.sh from the APK metadata
versionName=$versionName  # Retrieved by test.sh from the APK metadata
versionCode=$versionCode  # Retrieved by test.sh from the APK metadata
signer=$(getSigner "$downloadedApk")  # Retrieved from test.sh for the official APK

# Zeus-specific variables
repo=https://github.com/ZeusLN/zeus
branch_or_tag="v$versionName"  # Dynamically use versionName from test.sh
build_dir="zeus"

# APK URLs based on architecture
declare -A apk_urls=(
    ["arm64"]="https://zeusln.com/zeus-$branch_or_tag-arm64-v8a.apk"
    ["armeabi"]="https://zeusln.com/zeus-$branch_or_tag-armeabi-v7a.apk"
    ["x86"]="https://zeusln.com/zeus-$branch_or_tag-x86.apk"
    ["x86_64"]="https://zeusln.com/zeus-$branch_or_tag-x86_64.apk"
    ["universal"]="https://zeusln.com/zeus-$branch_or_tag-universal.apk"
)

# Step 1: Clone the Zeus repository for the correct version
echo "Cloning the Zeus repository for version $versionName..."
git clone --depth 1 --branch $branch_or_tag $repo $build_dir

# Step 2: Change to the cloned directory
cd $build_dir

# Step 3: Run the build script
echo "Running the build script for version $versionName..."
./build.sh || { echo "Build failed"; exit 1; }

# Step 4: Find the generated APKs
apk_dir="android/app/build/outputs/apk/release/"
declare -A built_apks
built_apks["arm64"]="$(find "$apk_dir" -name "*arm64-v8a.apk")"
built_apks["armeabi"]="$(find "$apk_dir" -name "*armeabi-v7a.apk")"
built_apks["x86"]="$(find "$apk_dir" -name "*x86.apk")"
built_apks["x86_64"]="$(find "$apk_dir" -name "*x86_64.apk")"
built_apks["universal"]="$(find "$apk_dir" -name "*universal.apk")"

# Step 5: Download, extract, and compare each APK
for arch in "${!apk_urls[@]}"; do
    official_apk="zeus-$branch_or_tag-$arch.apk"
    official_apk_url="${apk_urls[$arch]}"
    
    echo "Downloading the official APK for $arch..."
    wget -O "$official_apk" "$official_apk_url"
    
    if [ -f "${built_apks[$arch]}" ]; then
        built_hash=$(sha256sum "${built_apks[$arch]}" | awk '{ print $1 }')
        official_hash=$(sha256sum "$official_apk" | awk '{ print $1 }')

        # Unzip both APKs for content comparison
        built_unzip_dir="built_$arch"
        official_unzip_dir="official_$arch"
        
        mkdir -p "$built_unzip_dir" "$official_unzip_dir"
        unzip -q "${built_apks[$arch]}" -d "$built_unzip_dir"
        unzip -q "$official_apk" -d "$official_unzip_dir"

        # Exclude META-INF (signing-related files) and compare directories
        diff_result=$(diff -r --exclude=META-INF "$built_unzip_dir" "$official_unzip_dir")

        # Determine verdict based on diff
        if [ -z "$diff_result" ]; then
            verdict="reproducible"
        else
            verdict="nonverifiable"
        fi

        # Output comparison results for each architecture
        echo "Architecture: $arch"
        echo "Built APK: ${built_apks[$arch]} - SHA256: $built_hash"
        echo "Official APK: $official_apk - SHA256: $official_hash"

        # Output full details
        echo "===== Begin Results ====="
        echo "appId:          $appId"
        echo "signer:         $signer"
        echo "apkVersionName: $versionName"
        echo "apkVersionCode: $versionCode"
        echo "architecture:   $arch"
        echo "verdict:        $verdict"
        echo "appHash:        $(sha256sum "$downloadedApk" | awk '{print $1;}')"
        echo "commit:         $(git log -n 1 --pretty=oneline | awk '{print $1}')"
        echo "Diff:           $(echo "$diff_result" | grep -v 'Only in META-INF')"  # Exclude META-INF files from diff
        echo "===== End Results ====="

        # Clean up extracted folders
        rm -rf "$built_unzip_dir" "$official_unzip_dir"

    else
        echo "Built APK for $arch not found."
    fi
done

# Step 6: Cleanup (optional)
echo "Cleaning up..."
rm -rf $build_dir

# Version 1.1.7 - Added diffing of APK contents, excluding signing-related files, and incorporated diff results into the output.

These are the results:
Asciicast recording

BUILD SUCCESSFUL in 8m 59s
1399 actionable tasks: 1399 executed


********************************
**** APKs and SHA256 Hashes
********************************

8611193f43ce066dd7e1ffd610dfb5547ac1ab123320c1726c1c2b9e6c96a2e9  android/app/build/outputs/apk/release/zeus-arm64-v8a.apk
c196870cd0159a987c61dfc1b6a5fd8e16b91d7052a6dfd9119f05d7a6b9e525  android/app/build/outputs/apk/release/zeus-armeabi-v7a.apk
ab91974de1ee847e13d9ef236c4a22b77336e2554921cde3cb7ca2ecc8925e39  android/app/build/outputs/apk/release/zeus-universal.apk
6143cfae8944e37cbeed3d8caa64cb7ae42c67c07f3e357b394f64dadc60343c  android/app/build/outputs/apk/release/zeus-x86.apk
62a98bfe7ca1e0c0e810994594c2f65e5101768765e6824e271c64353a52b908  android/app/build/outputs/apk/release/zeus-x86_64.apk


Downloading the official APK for armeabi...
--2024-10-17 06:43:18--  https://zeusln.com/zeus-v0.9.1-armeabi-v7a.apk
Resolving zeusln.com (zeusln.com)... 139.144.53.218
Connecting to zeusln.com (zeusln.com)|139.144.53.218|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 57447868 (55M) [application/octet-stream]
Saving to: ‘zeus-v0.9.1-armeabi.apk’

zeus-v0.9.1-armeabi.apk                                   100%[===================================================================================================================================>]  54.79M  22.4MB/s    in 2.4s    

2024-10-17 06:43:21 (22.4 MB/s) - ‘zeus-v0.9.1-armeabi.apk’ saved [57447868/57447868]

Architecture: armeabi
Built APK: android/app/build/outputs/apk/release/zeus-armeabi-v7a.apk - SHA256: c196870cd0159a987c61dfc1b6a5fd8e16b91d7052a6dfd9119f05d7a6b9e525
Official APK: zeus-v0.9.1-armeabi.apk - SHA256: 50e263b6f01bcfd6c9226b94f462d2e315f3117797be4c72c9fc8b72ee6246ae
===== Begin Results =====
appId:          app.zeusln.zeus
signer:         cbcc8ccfbf89c002b5fed484a59f5f2a6f5c8ad30a1934f36af2c9fcdec6b359
apkVersionName: 0.9.1
apkVersionCode: 93001
architecture:   armeabi
verdict:        nonverifiable
appHash:        77f43884148dfbc70deb06bf7ad881b40cee8e0ac7a56d6cb31118093b41087b
commit:         773c51b84b064ec500b983dcd77f83734bf53350
Diff:           Binary files built_armeabi/assets/index.android.bundle and official_armeabi/assets/index.android.bundle differ
===== End Results =====
Downloading the official APK for x86...
--2024-10-17 06:43:25--  https://zeusln.com/zeus-v0.9.1-x86.apk
Resolving zeusln.com (zeusln.com)... 139.144.53.218
Connecting to zeusln.com (zeusln.com)|139.144.53.218|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 60871716 (58M) [application/octet-stream]
Saving to: ‘zeus-v0.9.1-x86.apk’

zeus-v0.9.1-x86.apk                                       100%[===================================================================================================================================>]  58.05M  23.4MB/s    in 2.5s    

2024-10-17 06:43:28 (23.4 MB/s) - ‘zeus-v0.9.1-x86.apk’ saved [60871716/60871716]

Architecture: x86
Built APK: android/app/build/outputs/apk/release/zeus-x86.apk - SHA256: 6143cfae8944e37cbeed3d8caa64cb7ae42c67c07f3e357b394f64dadc60343c
Official APK: zeus-v0.9.1-x86.apk - SHA256: 11f212816c35a8a61d3e0a2ad826820040b47603d1904da5827529ca2a2364eb
===== Begin Results =====
appId:          app.zeusln.zeus
signer:         cbcc8ccfbf89c002b5fed484a59f5f2a6f5c8ad30a1934f36af2c9fcdec6b359
apkVersionName: 0.9.1
apkVersionCode: 93001
architecture:   x86
verdict:        nonverifiable
appHash:        77f43884148dfbc70deb06bf7ad881b40cee8e0ac7a56d6cb31118093b41087b
commit:         773c51b84b064ec500b983dcd77f83734bf53350
Diff:           Binary files built_x86/assets/index.android.bundle and official_x86/assets/index.android.bundle differ
===== End Results =====
Downloading the official APK for universal...
--2024-10-17 06:43:31--  https://zeusln.com/zeus-v0.9.1-universal.apk
Resolving zeusln.com (zeusln.com)... 139.144.53.218
Connecting to zeusln.com (zeusln.com)|139.144.53.218|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 153621554 (147M) [application/octet-stream]
Saving to: ‘zeus-v0.9.1-universal.apk’

zeus-v0.9.1-universal.apk                                 100%[===================================================================================================================================>] 146.50M  28.3MB/s    in 6.0s    

2024-10-17 06:43:38 (24.5 MB/s) - ‘zeus-v0.9.1-universal.apk’ saved [153621554/153621554]

Architecture: universal
Built APK: android/app/build/outputs/apk/release/zeus-universal.apk - SHA256: ab91974de1ee847e13d9ef236c4a22b77336e2554921cde3cb7ca2ecc8925e39
Official APK: zeus-v0.9.1-universal.apk - SHA256: 8bbcbfb2370e0c37d60b3d3bc45895c1846db57387e8f86e65a5665a8318e175
===== Begin Results =====
appId:          app.zeusln.zeus
signer:         cbcc8ccfbf89c002b5fed484a59f5f2a6f5c8ad30a1934f36af2c9fcdec6b359
apkVersionName: 0.9.1
apkVersionCode: 93001
architecture:   universal
verdict:        nonverifiable
appHash:        77f43884148dfbc70deb06bf7ad881b40cee8e0ac7a56d6cb31118093b41087b
commit:         773c51b84b064ec500b983dcd77f83734bf53350
Diff:           Binary files built_universal/assets/index.android.bundle and official_universal/assets/index.android.bundle differ
===== End Results =====
Downloading the official APK for x86_64...
--2024-10-17 06:43:47--  https://zeusln.com/zeus-v0.9.1-x86_64.apk
Resolving zeusln.com (zeusln.com)... 139.144.53.218
Connecting to zeusln.com (zeusln.com)|139.144.53.218|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 60449981 (58M) [application/octet-stream]
Saving to: ‘zeus-v0.9.1-x86_64.apk’

zeus-v0.9.1-x86_64.apk                                    100%[===================================================================================================================================>]  57.65M  20.7MB/s    in 2.8s    

2024-10-17 06:43:51 (20.7 MB/s) - ‘zeus-v0.9.1-x86_64.apk’ saved [60449981/60449981]

Architecture: x86_64
Built APK: android/app/build/outputs/apk/release/zeus-x86_64.apk - SHA256: 62a98bfe7ca1e0c0e810994594c2f65e5101768765e6824e271c64353a52b908
Official APK: zeus-v0.9.1-x86_64.apk - SHA256: ac50e91058ae3ca9c6db2286373a50920b804eebd3d91900a431ff5cf0b3c30f
===== Begin Results =====
appId:          app.zeusln.zeus
signer:         cbcc8ccfbf89c002b5fed484a59f5f2a6f5c8ad30a1934f36af2c9fcdec6b359
apkVersionName: 0.9.1
apkVersionCode: 93001
architecture:   x86_64
verdict:        nonverifiable
appHash:        77f43884148dfbc70deb06bf7ad881b40cee8e0ac7a56d6cb31118093b41087b
commit:         773c51b84b064ec500b983dcd77f83734bf53350
Diff:           Binary files built_x86_64/assets/index.android.bundle and official_x86_64/assets/index.android.bundle differ
===== End Results =====
Downloading the official APK for arm64...
--2024-10-17 06:43:54--  https://zeusln.com/zeus-v0.9.1-arm64-v8a.apk
Resolving zeusln.com (zeusln.com)... 139.144.53.218
Connecting to zeusln.com (zeusln.com)|139.144.53.218|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 58488150 (56M) [application/octet-stream]
Saving to: ‘zeus-v0.9.1-arm64.apk’

zeus-v0.9.1-arm64.apk                                     100%[===================================================================================================================================>]  55.78M  16.3MB/s    in 3.4s    

2024-10-17 06:43:58 (16.3 MB/s) - ‘zeus-v0.9.1-arm64.apk’ saved [58488150/58488150]

Architecture: arm64
Built APK: android/app/build/outputs/apk/release/zeus-arm64-v8a.apk - SHA256: 8611193f43ce066dd7e1ffd610dfb5547ac1ab123320c1726c1c2b9e6c96a2e9
Official APK: zeus-v0.9.1-arm64.apk - SHA256: 2117420b310310f4dddc0345a5faff30e44ed1cdb0242c9161df8f32122f6c9e
===== Begin Results =====
appId:          app.zeusln.zeus
signer:         cbcc8ccfbf89c002b5fed484a59f5f2a6f5c8ad30a1934f36af2c9fcdec6b359
apkVersionName: 0.9.1
apkVersionCode: 93001
architecture:   arm64
verdict:        nonverifiable
appHash:        77f43884148dfbc70deb06bf7ad881b40cee8e0ac7a56d6cb31118093b41087b
commit:         773c51b84b064ec500b983dcd77f83734bf53350
Diff:           Binary files built_arm64/assets/index.android.bundle and official_arm64/assets/index.android.bundle differ
===== End Results =====
@xrviv
Copy link
Author

xrviv commented Oct 17, 2024

The manual build results are consistent with the results generated by the new script.

Main diff is index.android.bundle + signing differences

Manual build results:

armeabi_v7a

$ diff -r built-zeus-armeabi_v7a/ official-armeabi_v7a/
Binary files built-zeus-armeabi_v7a/assets/index.android.bundle and official-armeabi_v7a/assets/index.android.bundle differ
Only in official-armeabi_v7a/META-INF: MANIFEST.MF
Only in official-armeabi_v7a/META-INF: ZEUS-KEY.RSA
Only in official-armeabi_v7a/META-INF: ZEUS-KEY.SF

arm64_v8a

$ diff -r built-arm64-v8a/ official-arm64-v8a/
Binary files built-arm64-v8a/assets/index.android.bundle and official-arm64-v8a/assets/index.android.bundle differ
Only in official-arm64-v8a/META-INF: MANIFEST.MF
Only in official-arm64-v8a/META-INF: ZEUS-KEY.RSA
Only in official-arm64-v8a/META-INF: ZEUS-KEY.SF

universal

$ diff -r extracted/built-zeus-universal/ extracted/official-zeus-universal/
Binary files extracted/built-zeus-universal/assets/index.android.bundle and extracted/official-zeus-universal/assets/index.android.bundle differ
Only in extracted/official-zeus-universal/META-INF: MANIFEST.MF
Only in extracted/official-zeus-universal/META-INF: ZEUS-KEY.RSA
Only in extracted/official-zeus-universal/META-INF: ZEUS-KEY.SF

x86_64

$ diff -r extracted/built-zeus-x86_64/ extracted/official-zeus-x86_64/
Binary files extracted/built-zeus-x86_64/assets/index.android.bundle and extracted/official-zeus-x86_64/assets/index.android.bundle differ
Only in extracted/official-zeus-x86_64/META-INF: MANIFEST.MF
Only in extracted/official-zeus-x86_64/META-INF: ZEUS-KEY.RSA
Only in extracted/official-zeus-x86_64/META-INF: ZEUS-KEY.SF

x86

 diff -r extracted/built-zeus-x86 extracted/official-zeus-x86
Binary files extracted/built-zeus-x86/assets/index.android.bundle and extracted/official-zeus-x86/assets/index.android.bundle differ
Only in extracted/official-zeus-x86/META-INF: MANIFEST.MF
Only in extracted/official-zeus-x86/META-INF: ZEUS-KEY.RSA
Only in extracted/official-zeus-x86/META-INF: ZEUS-KEY.SF

@kaloudis
Copy link
Contributor

Getting the same hashes as you for v0.9.1 and the commit that precedes the merge commit.

********************************
**** APKs and SHA256 Hashes
********************************

8611193f43ce066dd7e1ffd610dfb5547ac1ab123320c1726c1c2b9e6c96a2e9  android/app/build/outputs/apk/release/zeus-arm64-v8a.apk
c196870cd0159a987c61dfc1b6a5fd8e16b91d7052a6dfd9119f05d7a6b9e525  android/app/build/outputs/apk/release/zeus-armeabi-v7a.apk
ab91974de1ee847e13d9ef236c4a22b77336e2554921cde3cb7ca2ecc8925e39  android/app/build/outputs/apk/release/zeus-universal.apk
6143cfae8944e37cbeed3d8caa64cb7ae42c67c07f3e357b394f64dadc60343c  android/app/build/outputs/apk/release/zeus-x86.apk
62a98bfe7ca1e0c0e810994594c2f65e5101768765e6824e271c64353a52b908  android/app/build/outputs/apk/release/zeus-x86_64.apk

I'm lead to believe we may have just had a fluke with the build we used for release (have seen this happen before). I think the path forward would be for us just to just be more judicious and run multiple build processes, before cutting it.

@kaloudis
Copy link
Contributor

Thanks for reporting.

FWIW v0.9.2 will be out sooner than later so we should be able to remedy the issue in the coming weeks. Completely understand if v0.9.1 is marked as not reproducible.

@kaloudis kaloudis mentioned this issue Oct 27, 2024
Merged
@kaloudis kaloudis added this to the v0.9.2 milestone Oct 27, 2024
@kaloudis
Copy link
Contributor

kaloudis commented Nov 2, 2024

GitHub workflows have been updated to build the Android binaries reproducibly and spit out the hashes when commits are pushed to master branch #2510

@kaloudis
Copy link
Contributor

kaloudis commented Nov 2, 2024

Build artifacts added here #2511

@kaloudis
Copy link
Contributor

kaloudis commented Nov 4, 2024

New build and verification process used in the v0.9.2-beta1 release. Will leave this issue open until we get this resolved for the v0.9.2 general release.

@xrviv
Copy link
Author

xrviv commented Nov 7, 2024

Taking a look! Thank you for your attention to this!

@kaloudis
Copy link
Contributor

@xrviv v0.9.2 should be reproducible now! Let us know if you have issues. Please re-open the issue or create a new one if you have issues.

We greatly appreciate the testing and reporting.

@xrviv
Copy link
Author

xrviv commented Nov 14, 2024

Hi, sorry, for the late reply. I've been stuck with another build. Trying this now!

@xrviv
Copy link
Author

xrviv commented Nov 14, 2024

Confirming that v0.9.2 is reproducible

@kaloudis
Copy link
Contributor

Confirming that v0.9.2 is reproducible

Great news. Thanks for testing and thanks again for reporting.

I am confident that the changes we made to our build procedures and script runners will ensure that things stay on track.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Reproducible builds
Projects
None yet
Milestone
v0.9.2
Development

No branches or pull requests
2 participants
@kaloudis @xrviv