Newbie problem building chat example as standalone

[futurechimp]

I've had some problems getting the libp2p chat example running. I'm going to document things here so that if others have the same problems there's maybe a pointer to help them.

If anybody can let me know how to get around the problems I'm having in a less hacky way, I promise a PR to the docs if that will help :).

First, I installed as per the instructions in the project README:

> go get -d github.com/libp2p/go-libp2p/...
> cd $GOPATH/src/github.com/libp2p/go-libp2p
> make
> make deps

I was able to easily build the chat example from within its place in the libp2p project:

> go build ./examples/chat

This worked happily as advertised.

Next step: copy it verbatim into a separate directory in a new project and get it working on its own, so I could start to modify it for my own purposes.

Initially I had a few gx related problems:

 main.go:49:2: cannot find package "gx/ipfs/QmXZUgiMTgYGEKazHbxYfghCWzvwWfZwBYNHRPEWh4D54y/go-libp2p/p2p/host/basic"

As noted by people in IRC this was caused by PATH problems. I had some conflicting entries in ~/.bashrc and ~/.bash_profile. Check your PATH closely if you get errors like this.

Once those were fixed, a new error appeared:

$ go build
# constructiveproof.com/experiments/chat-libp2p
./main.go:187:23: cannot use network (type *swarm.Network) as type "gx/ipfs/QmRQX1yaPQFynWkByKcQTPpy3uC21oXZ5X32KEcLZnefz8/go-libp2p-net".Network in argument to basichost.New:
	*swarm.Network does not implement "gx/ipfs/QmRQX1yaPQFynWkByKcQTPpy3uC21oXZ5X32KEcLZnefz8/go-libp2p-net".Network (wrong type for ClosePeer method)
		have ClosePeer("github.com/libp2p/go-libp2p-peer".ID) error
		want ClosePeer("gx/ipfs/QmZoWKhxUmZ2seW4BzX6fJkNR8hh9PsGModr7q171yq2SS/go-libp2p-peer".ID) error
./main.go:193:25: cannot use handleStream (type func("github.com/libp2p/go-libp2p-net".Stream)) as type "gx/ipfs/QmRQX1yaPQFynWkByKcQTPpy3uC21oXZ5X32KEcLZnefz8/go-libp2p-net".StreamHandler in argument to host.SetStreamHandler
./main.go:204:31: cannot use host (type *basichost.BasicHost) as type "github.com/libp2p/go-libp2p-host".Host in argument to addAddrToPeerstore:
	*basichost.BasicHost does not implement "github.com/libp2p/go-libp2p-host".Host (wrong type for Addrs method)
		have Addrs() []"gx/ipfs/QmWWQ2Txc2c6tqjsBpzg5Ar652cHPGNsQQp2SejkNmkUMb/go-multiaddr".Multiaddr
		want Addrs() []"github.com/multiformats/go-multiaddr".Multiaddr
./main.go:214:46: cannot use peerID (type "github.com/libp2p/go-libp2p-peer".ID) as type "gx/ipfs/QmZoWKhxUmZ2seW4BzX6fJkNR8hh9PsGModr7q171yq2SS/go-libp2p-peer".ID in argument to host.NewStream

It appears to me that whatever version of "github.com/libp2p/go-libp2p-net" (and others) needed by the example is not being found by go build.

I have gotten the build to work by applying a suggestion I saw in another ticket: gx-go rewrite. This didn't work at first, but I reasoned that the working example code in the main project must be getting version information from somewhere inside the libp2p codebase.

I copied go-libp2p's package.json into my new project, ran gx-go rewrite again, and it converted most "normal" go paths to gx paths, allowing the example code to build without errors.

The final set of imports that worked are:

import (
	"bufio"
	"context"
	"crypto/rand"
	"flag"
	"fmt"
	"io"
	"log"
	mrand "math/rand"
	"os"

	"github.com/libp2p/go-libp2p/p2p/host/basic"
	"gx/ipfs/QmRQX1yaPQFynWkByKcQTPpy3uC21oXZ5X32KEcLZnefz8/go-libp2p-net"
	"gx/ipfs/QmWPKovoJxF2bfiYLQK6xPeFvMJW8hX6RFRuA9bv6cURha/go-libp2p-host"
	"gx/ipfs/QmWWQ2Txc2c6tqjsBpzg5Ar652cHPGNsQQp2SejkNmkUMb/go-multiaddr"
	"gx/ipfs/QmXauCuJzmzapetmC6W4TuDJLL1yFFrVzSHoWv8YdbmnxH/go-libp2p-peerstore"
	"gx/ipfs/QmYJ3st4ZBibtmecPq9VkX3Xn2DGcJfJru7Mzd9U3VUfzE/go-libp2p-swarm"
	"gx/ipfs/QmZoWKhxUmZ2seW4BzX6fJkNR8hh9PsGModr7q171yq2SS/go-libp2p-peer"
	"gx/ipfs/QmaPbCnUMBohSGo3KnxEa2bHqyJVVeEEcwtqJAYxerieBo/go-libp2p-crypto"
)

Is this gx-go rewrite trick the normal way to do things when you want to write your own project incorporating libp2p? If so, I'm happy to discuss a bit to clarify and then offer a PR for it.

[futurechimp]

I think I'm starting to understand what's going on: reading the Makefile, make deps ensures that gx-go is installed. Then it calls out to gx-go rewrite.

That causes all of the go import paths in the go-libp2p project to be replaced with their equivalent gx import paths (presumably to ensure that all versions match up in some way?) and the file builds.

So in order to use go-libp2p inside my own code, I needed to run gx-go rewrite to get the imports rewritten, or get hit by version inconsistencies (?).

[upperwal]

If you want to build "chat" out of libp2p project you can try the following

go get -d github.com/libp2p/go-libp2p
wget https://raw.githubusercontent.com/libp2p/go-libp2p/master/examples/chat/chat.go
go build chat.go

This should build it without any problem.

[florianlenz]

@futurechimp you are right, you have to use gx-go to rewrite your import path's.

[futurechimp]

@upperwal I get the same errors as previously:

$ go build chat.go
# command-line-arguments
./chat.go:187:23: cannot use network (type *swarm.Network) as type "gx/ipfs/QmRQX1yaPQFynWkByKcQTPpy3uC21oXZ5X32KEcLZnefz8/go-libp2p-net".Network in argument to basichost.New:
	*swarm.Network does not implement "gx/ipfs/QmRQX1yaPQFynWkByKcQTPpy3uC21oXZ5X32KEcLZnefz8/go-libp2p-net".Network (wrong type for ClosePeer method)

As a guess, that looks like possible version drift between the example code and whatever version go get pulls in.

@florianlenz cool, thanks for the confirmation, I think it's starting to make more sense now.

If a PR to the README would help, I am very happy to do that, explaining that to use libp2p in your own code you need to:

• have a current gx package.json in place
• run gx-go rewrite to rewrite import paths

Alternately, if the fix is to somehow get things to Just Work in code using go get, consider this a bug report :). Thanks for the help so far.

[upperwal]

@futurechimp I see. It looks like that. I don't know if go get or gx is the culprit. gx documents that

It is highly recommended that you set your GOPATH to a temporary directory when running import. This ensures that your current go packages are not affected, and also that fresh versions of the packages in question are pulled down.

Although gx is very useful for projects, it is not necessary to build (standalone) this example (I am building it without gx).

Apart from this if you happen to use gx for package imports then I guess you will need gx-go rewrite to rewrite go paths without which your dependencies won't be resolved.

If you are using gx in your entire project then ignore the following else just try to change your GOPATH to a fresh directory, go get libp2p and compile the chat example again (without path rewrite).

[futurechimp]

Ah, strange. I had actually deleted nearly everything in src/go so that this didn't happen, but I must've somehow gotten something outdated in there.

export GOPATH=/tmp/go
go get -d github.com/libp2p/go-libp2p
wget https://raw.githubusercontent.com/libp2p/go-libp2p/master/examples/chat/chat.go
go build chat.go

^^ that worked. I don't currently understand how, as I deleted everything in $HOME/go/src except code I'd made myself. Anyway, sorry for the noise, I should have followed the instructions more exactly - I had assumed that because it was my first time downloading libp2p source, initial versions would be up to date with examples.

[upperwal]

as I deleted everything in $HOME/go/src except code I'd made myself

Just a guess. With this all the outdated libraries are also gone. So when you did go get it downloaded the updated master branch and your dependencies got fixed.

plus you also did

export GOPATH=/tmp/go

which I guess is empty at first. So again go get fetches the updated code.

[futurechimp]

At that point, the dependencies were still broken I think, that's what was confusing. I'm not sure which combination of mistakes I was making caused that though.

Incorporating the echo example code into my actual project, I've now also fought my way through a new set of dependency problems as well - go get retrieves latest code and works, while dep ensure attempts to grab semver dependency tags and fails (there are no semver tags in most libp2p repos from what I can see).

In order to get going and try to write some code, I've committed a bit of a sin against dependency management for the moment. This is the minimal list of libs to temporarily ignore in order to get a dep versioned project to work:

ignored = [
  "github.com/agl/ed25519",
  "github.com/btcsuite/btcd",
  "github.com/ipfs/go-ipfs-util",
  "github.com/ipfs/go-log",
  "github.com/libp2p/go-libp2p-peerstore",
  "github.com/libp2p/go-libp2p-transport",
  "github.com/libp2p/go-maddr-filter",
  "github.com/libp2p/go-libp2p",
  "github.com/libp2p/go-libp2p-crypto",
  "github.com/libp2p/go-libp2p-host",
  "github.com/libp2p/go-libp2p-interface-connmgr",
  "github.com/libp2p/go-libp2p-net",
  "github.com/libp2p/go-libp2p-peer",
  "github.com/whyrusleeping/go-logging",
  "github.com/multiformats/go-multiaddr",
  "github.com/multiformats/go-multiaddr-net",
  "github.com/whyrusleeping/mafmt",
]

Thanks for your help @upperwal, it's really helpful when getting into a new library for the first time!

[upperwal]

👍