Support OpenBSD/adJ

[vtamara]

Context and scope

I was following the Subnet architecture course that required installing avalanche-cli, then I tried to do it on my platform that is OpenBSD/adJ 7.4 but it didn't work.

After some experiments that I described at ava-labs/avalanche-cli#1498 I was told by @arturrez to open this request here.

Discussion and alternatives

I was able to generate build/avalanchego by:

1. Adding support for OpenBSD to utils/storage as described by this commit vtamara@07de44d explained by syscall: Statfs_t on OpenBSD with f_ golang/go#47958
2. Modifying my downloaded version of [email protected] (used by avalanchego and avalanche-cli) to backport the support for OpenBSD added in version v1.1.0. Implementing the changes for vfs/disk_usage_openbsd.go and vfs/disk_usage_unix.go described at cockroachdb/pebble@v1.0.0...v1.1.0#diff-e80071db294208cc80f86aa6d7915a187eaa64a03a17ec7eec3a2d96bda5dbfb

However when I run build/avalanchego , it produces a segmentatio violation:... The segmentation violation was produced by supranational/blst, I opened an issue and the author improved that library and now build/avalanchego runs without issue.

In my humble opinion it is good for Avalanchego to:

1. Support more platforms, and in particular OpenBSD/adJ that is well known for its security.
2. Use an updated version of pebble that supports OpenBSD/adJ (greater than v1.1.0) or to fork the version v0.0.0-20230209160836-829675f94811 currently used by Avalanchego to support more platforms, in particular to backport the support for OpenBSD added in version v1.1.0
3. Reproduce and further investigate the segmentation violation produced during my experiments on OpenBSD/adJ 7.4, because one possibility is that it is pointing to an avalanchego implementation bug triggered by OpenBSD security features. Upgrade blst to version 0.3.12 as requested at Update blst to 0.3.12 #3079

Open questions

• Is there a bounty program to discover security flaws in avalanche?
• Is there any grant program to port avalanchego/avalanche-cli to a different platorm like a secure and updated OpenBSD/adJ 7.4?
• Do you envision running Avalanche nodes in platforms different to the current supported ones and particularly OpenBSD/adJ?
• Should I open a PR with vtamara@07de44d ?

[StephenButtolph]

Hi! Thanks for the detailed post.

Is there a bounty program to discover security flaws in avalanche?

Yes. See: https://github.com/ava-labs/avalanchego/blob/master/SECURITY.md

Is there any grant program to port avalanchego/avalanche-cli to a different platform like a secure and updated OpenBSD/adJ 7.4?

I'm not aware of any grant programs for this.

Do you envision running Avalanche nodes in platforms different to the current supported ones and particularly OpenBSD/adJ?

We are yet to expand our supported platforms. If there is sufficient demand for the level of effort then I think it's reasonable. If all that OpenBSD takes to support is supporting the disk usage method and updating pebble (which we'll want to do anyways once coreth updates) then I'm very willing to do that.

Should I open a PR with vtamara@07de44d ?

Please feel free to open a PR for this.

[vtamara]

@StephenButtolph I wanted to inform:

1. The segmentation fault was produced by supranational/blst (used by avalanchego). I reported the problem and the author found an initial solution that he published in the branch dot-asm/blst.

2. Using dot-asm/blst (in avalanchego and coreth both ported to OpenBSD/adJ 7.4) I could compile and test avalanchego v1.10.11. Running ./scripts/build_test.sh produced -race is not supported on openbsd/amd64, removing that option all the tests passed except one sporadically fails (sometimes it passes):

ok      github.com/ava-labs/avalanchego/indexer 0.131s  coverage: 13.3% of statements                      
-test.shuffle 1710270324100367331                                                                                                                               
PASS                                                                                                                                                            
panic: runtime error: invalid memory address or nil pointer dereference                                                                                         
[signal SIGSEGV: segmentation violation code=0x1 addr=0x18 pc=0x60ee32]                                                                                         
                                                                                                                                                                
goroutine 8 [running]:                                                                                                                                          
github.com/ava-labs/avalanchego/ipcs/socket.(*Socket).Send(0xc000016240, {0xc000180000, 0xf4249, 0xf6000})                 
        /home/vtamara/comp/go/avalanchego/ipcs/socket/socket.go:109 +0x5d2                                                                                      
github.com/ava-labs/avalanchego/ipcs/socket.TestSocketSendAndReceive.func1()                                                                                    
        /home/vtamara/comp/go/avalanchego/ipcs/socket/socket_test.go:35 +0x37                                                                                   
created by github.com/ava-labs/avalanchego/ipcs/socket.TestSocketSendAndReceive in goroutine 6                             
        /home/vtamara/comp/go/avalanchego/ipcs/socket/socket_test.go:33 +0x305                                                                                  
FAIL    github.com/ava-labs/avalanchego/ipcs/socket     0.086s                                                                                                  
        github.com/ava-labs/avalanchego/nat             coverage: 0.0% of statements        

3. Before dot-asm/blst, using a computer where the unmodifed blst worked, I could port and use avalanchego and other tools of ava-labs to complete some tasks of the course "Customizing the EVM" of Avalanche Academy as I described at https://github.com/vtamara/avalanchego/wiki/A-local-environment-in-OpenBSD-adJ-7.4-for-Avalanche

4. Two simple patches I sent to two ava-labs projects were already merged: Bash more portable coreth#510 Shebang that works in more systems. Advances #1498 avalanche-cli#1539

[vtamara]

@StephenButtolph I wanted to inform:

1. The fix for supranational/bslt was already commited by its author in the master branch (commit 1 and commit 2 ) and the issue was closed --I guess soon the author will release a new version.

2. Other simple patch I sent to an ava-labs project was merged: Bash more portable.  avalanche-network-runner#707

[vtamara]

@StephenButtolph I see something called Avalanche Community Grants Program open now at https://forum.avax.network/c/avalanche-grants/11 Do you recommend me to apply with the open source porting of several ava-labs tools to OpenBSD/adJ?

[vtamara]

Blessings @StephenButtolph and community, since the effort of porting to OpenBSD has brought improvements in security I wanted you to notice that I applied for funding in the "Avalanche Community Grants Quadratic Funding Round One" to do a deeper audit of the sources of the fundamental tools of Avalanche, complete the porting and test more fully. Could you please vote for the project at: https://explorer.gitcoin.co/#/round/43114/4/28 ?

[vtamara]

Thanks to God, Avalanche Foundation and donors I received some funding to keep auditing and porting, see https://forum.avax.network/t/announcing-round-1-results/2376

Today, building the master branch plus the changes needed for OpenBSD/adJ and running scripts/build_test.sh I noticed that all the tests passed except 9:

% ./scripts/build_test.sh
...
ok      github.com/ava-labs/avalanchego/snow/networking/benchlist       0.403s  coverage: 10.5% of statements               
ok      github.com/ava-labs/avalanchego/snow/networking/handler 0.096s  coverage: 17.3% of statements                       
        github.com/ava-labs/avalanchego/snow/snowtest           coverage: 0.0% of statements                                                                    
# github.com/ava-labs/avalanchego/tests/fixture/tmpnet                                                                                                          
tests/fixture/tmpnet/node_process.go:118:2: undefined: configureDetachedProcess                                                                                 
FAIL    github.com/ava-labs/avalanchego/tests/antithesis [build failed]                                                                                         
FAIL    github.com/ava-labs/avalanchego/tests/antithesis/avalanchego/gencomposeconfig [build failed]                                                            
FAIL    github.com/ava-labs/avalanchego/tests/antithesis/avalanchego [build failed]                                                                             
        github.com/ava-labs/avalanchego/tests           coverage: 0.0% of statements                                                                            
FAIL    github.com/ava-labs/avalanchego/tests/antithesis/xsvm [build failed]                                                                                    
FAIL    github.com/ava-labs/avalanchego/tests/antithesis/xsvm/gencomposeconfig [build failed]                                                                   
FAIL    github.com/ava-labs/avalanchego/tests/fixture/e2e [build failed]                                                                                        
FAIL    github.com/ava-labs/avalanchego/tests/fixture/tmpnet [build failed]                                                                                     
FAIL    github.com/ava-labs/avalanchego/tests/fixture/tmpnet/cmd [build failed]                                                                                 
FAIL    github.com/ava-labs/avalanchego/tests/fixture/subnet [build failed]
        github.com/ava-labs/avalanchego/trace           coverage: 0.0% of statements                                
        github.com/ava-labs/avalanchego/utils/formatting/address                coverage: 0.0% of statements
        github.com/ava-labs/avalanchego/utils/hashing           coverage: 0.0% of statements                                                        
ok      github.com/ava-labs/avalanchego/snow/networking/router  6.727s  coverage: 13.5% of statements                
ok      github.com/ava-labs/avalanchego/snow/networking/sender  0.233s  coverage: 16.3% of statements
...

I noticed that the issue is with the recent file tests/fixture/tmpnet/detached_process_default.go that has build constraints linux || darwin and uses the Setsid attribute of syscall.SysProcAttr --field not available in Windows.
Since the function setsid is part of POSIX 1003.1-2008 (see https://pubs.opengroup.org/onlinepubs/9699919799/functions/setsid.html) I think it is fine to suppose that in other unix platforms this attribute is available, and at least the following change works in OpenBSD/adJ because there are no failures in tests after applying it:

--- a/tests/fixture/tmpnet/detached_process_default.go
+++ b/tests/fixture/tmpnet/detached_process_default.go
@@ -1,7 +1,7 @@
 // Copyright (C) 2019-2024, Ava Labs, Inc. All rights reserved.
 // See the file LICENSE for licensing terms.
 
-//go:build linux || darwin
+//go:build linux || darwin || unix

That is why I'm adding this change to the PR #2809

[github-actions]

This issue has become stale because it has been open 60 days with no activity. Adding the lifecycle/frozen label will cause this issue to ignore lifecycle events.

[vtamara]

After merging #2809 (thank you @StephenButtolph ), what is missing to support OpenBSD/adJ fully is:

1. Upgrade the library blst as requested at Update blst to 0.3.12 #3079
2. Smaller issues with scripts: -race is not supported in tests, ggrep should be used in OpenBSD/adJ where GNU grep is needed.