Onboarding a Linux client from source

[mrngm]

Note: I ran this on Ubuntu 19.10 (which ships Go 1.12), using the latest version of Go available:

$ go version
go version go1.16.5 linux/amd64

What to compile?

• The README doesn't specify what exactly to build
  • cmd/ contains ca_fingerprint, create_auth_token, create_ca_pair, and register, but no notion of a client
  • client/ contains a main package, but only after inspecting the source (although there are no standards regarding repository layout, cmd/ typically contains "buildable" packages)
• The release notes refer to the Linux client
  • publish/linux/ contains a systemd service file referring to a discovery binary
  • discovery/ also contains a main package, but it's unclear whether this is necessary to run the client (the README suggests this is not the case)

Just trying stuff

mrngm@pepparkaka:~/scm/rufs2$ git status
HEAD detached at v2.0
nothing to commit, working tree clean

mrngm@pepparkaka:~/scm/rufs2$ go build
no Go files in /home/mrngm/scm/rufs2

Let's try and build the client/ subdir:

mrngm@pepparkaka:~/scm/rufs2/client$ go build
go: downloading github.com/pkg/browser v0.0.0-20210115035449-ce105d075bb4
go: downloading gopkg.in/yaml.v2 v2.4.0
go: downloading github.com/Jille/dfr v1.0.0
go: downloading google.golang.org/grpc v1.37.0
go: downloading bazil.org/fuse v0.0.0-20200407214033-5883e5a4b512
go: downloading github.com/yookoala/realpath v1.0.0
go: downloading github.com/Jille/convreq v1.1.0
go: downloading github.com/golang/protobuf v1.5.2
go: downloading google.golang.org/protobuf v1.26.0
go: downloading github.com/Jille/errchain v0.0.0-20160829115253-c62feead6f25
go: downloading golang.org/x/net v0.0.0-20200625001655-4c5254603344
go: downloading google.golang.org/genproto v0.0.0-20200526211855-cb27e3aa2013
go: downloading github.com/gorilla/mux v1.8.0
go: downloading github.com/gorilla/schema v1.2.0
go: downloading golang.org/x/sys v0.0.0-20210309074719-68d13333faf2
go: downloading golang.org/x/text v0.3.2

Success!

Running the client

mrngm@pepparkaka:~/scm/rufs2/client$ ./client 
20:40:29.694081 client.go:43: starting rufs (development build)
20:40:29.694425 client.go:46: failed to load configuration: open /home/mrngm/.rufs2/config.yaml: no such file or directory
20:40:29.694610 contentserver.go:52: content server listening on addr :12010.
20:40:29.694800 web.go:61: web server listening on addr :12011.
/usr/bin/xdg-open: 870: www-browser: not found
/usr/bin/xdg-open: 870: links2: not found
/usr/bin/xdg-open: 870: elinks: not found
/usr/bin/xdg-open: 870: links: not found
/usr/bin/xdg-open: 870: lynx: not found
/usr/bin/xdg-open: 870: w3m: not found
xdg-open: no method available for opening 'http://127.0.0.1:12011/'
20:40:29.722914 client.go:76: no circles configured - visit http://127.0.0.1:12011/ to start rufs configuration.

See #27. I would not depend on a browser being installed, but this may be a different experience per platform and per user.

Headless experience

./client -h gives me enough options to make sure a LAN client can access the web interface.

$ ./client -http_allow_hosts '2a10:3781:x/64'

And connecting on another LAN client through IPv6 works!

• Circle address and CA certificate were not provided, but found afterwards
• 20:50:55.569838 connectivity.go:103: Connected to RuFS. My endpoints: [[2a10:3781:x]:12010]
• Adding a share folder is easy
• All files you share, and files shared with you, should now be visible in your rufs directory.
  • I'm not sure what I provided here (probably nothing, so some default). It would be good to show the user what was configured.

What's next?

Apparently, my client's connected, but I have no idea where I can find the other clients in the circle. Navigating to http://[ipv6-address]:12011/ brings me back to the initial configuration screen. Ah, the config.yaml contains an empty mountpoint, that might be a hint.

More configuration

mrngm@pepparkaka:~/.rufs2$ cat config.yaml 
circles:
- name: rufs.x
  shares:
  - local: /home/mrngm/rufs2-shares/movies/
    remote: movies
mountpoint: "/home/mrngm/rufs2-shares/theirs"

(starting the client again)

mrngm@pepparkaka:~$ mount | grep theirs
rufs on /home/mrngm/rufs2-shares/theirs type fuse.rufs (ro,nosuid,nodev,relatime,user_id=1000,group_id=1000)

Success!

Retrieving files from theirs/

• It's useful to have rufs-warnings.txt inside the directory, but perhaps some prefix would be useful for sorting (i.e. 000- or zzz-) to make sure it's visible
• Some subdirectories also give warnings for some peers (similar to Don't fanout readdir RPCs to clients for which you know they don't have the share #40). NotFound shouldn't be listed as warning here:

*** RUFS encountered some issues showing this directory: ***
[snip]
failed to readdir on peer [email protected], ignoring: rpc error: code = NotFound desc = file "series/subdir" not found
failed to readdir on peer [email protected], ignoring: rpc error: code = NotFound desc = share series not found

and RPCing to the local client seems unneccessary (although for the purpose of showing all files in the circle's share, this might be needed)

Input/output errors when transferring?

I used rsync to view progress, so perhaps this doesn't show up using cp.

mrngm@pepparkaka:~/rufs2-shares/theirs/series/F/S01$ rsync -P F.mkv /tmp/
F.mkv
    404,131,599 100%  153.11MB/s    0:00:02 (xfr#1, to-chk=0/1)
rsync: read errors mapping "/home/mrngm/rufs2-shares/theirs/series/F/S01/F.mkv": Input/output error (5)
WARNING: F.mkv failed verification -- update discarded (will try again).
F.mkv
    404,131,599 100%  125.38MB/s    0:00:03 (xfr#2, to-chk=0/1)
rsync: read errors mapping "/home/mrngm/rufs2-shares/theirs/series/F/S01/F.mkv": Input/output error (5)
ERROR: F.mkv failed verification -- update discarded.
rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1207) [sender=3.1.3]
mrngm@pepparkaka:~/rufs2-shares/theirs/series/F/S01$ sha256sum F.mkv /tmp/F.mkv 
sha256sum: F.mkv: Input/output error
b978506fac28463b57e83684b84b4eeac8c93ccd3fb9aa29badf96b095eab02c  /tmp/F.mkv

And it seems F.mkv was transferred successfully (tested on another machine that has video output)