Ability to connect to existing node cluster

[mrfelton]

Is your feature request related to a problem? Please describe.
I have an existing cluster of lnd nodes defined in docker-compose files and would like to be able to connect the Polar UI to it so that I can visualise the node cluster and easily move money between nodes.

Describe the solution you'd like
Ability to connect Polar to my existing lnd/bitcoins node cluster.

[jamaljsr]

Hey @mrfelton, thanks for the suggestion.

Just for clarity, how do you imagine this working from a UX perspective? My initial thoughts are that when you create a new network, you add some "external" nodes by supplying their API credentials (ex: host, cert, macaroon, rpc login). With the network created, Polar would just connect to all the nodes and display them in the UI and allow you to interact with them. Some features would not work in this scenario, such as the terminal and log viewer, since they rely on connecting to the docker container. Is this what you had in mind?

I am curious to ask why you need to continue using your own cluster versus creating a new one in Polar? What features are missing? I know you've been doing LN development long before Polar existed, but my hopes for the project was to replace those hand-made local networks with ones you can easily create and destroy. With the new Custom Nodes feature, I think you may be able to migrate your cluster into Polar. Is that not possible?

[mrfelton]

Hi @jamaljsr

My initial thoughts are that when you create a new network, you add some "external" nodes by supplying their API credentials

I think thats pretty much it - the ability to connect to external nodes so that you can use the UI as a visualisation tool and as an easy way to manage the channels between the nodes.

I am curious to ask why you need to continue using your own cluster versus creating a new one in Polar

We have a node cluster defined in docker-compose that we use as part of a development stack for our apps. Multiple members of the team develop against this stack and so this approach allows them to do a quick spin up of the cluster that's needed for local development purposes.

Also, we spin up the exact same cluster in our CI environments and run our integration tests against it.

Another point is that we run cluster using the same docker images that we use in production so that our local environment matches the production stack as closely as possible.

We currently have our own scripts that we use to setup initial channel configurations after running docker-compose up to bring up the cluster but I've also been looking at Simverse as a way to provide a better cli experience for this aspect.

So really this is about needing the ability to have a fully scripted environment spin up process and wanting to leverage Polar on top of that as an additional developer tool.

[jamaljsr]

Thanks for the additional info. That does clear it up for me and I see why Polar, as it stands today, wouldn't work for your use case.

I think this is a good idea and is worth implementing. It's going to take a bit of work because right now the UI layer expects there to be a Polar-managed docker-compose network living underneath it. There will need to be a distinction between networks that Polar manages and external networks that are user managed. The UI would be different for the latter case, maybe with connect/disconnect buttons instead of start/stop.

I do have a couple more questions:

1. Currently, Polar will automatically mine blocks when necessary. Such as opening & closing channels or depositing funds. Would you want it to continue to do this or would you only connect the LN nodes and mine blocks manually yourself?
2. Would you ever want to have the internal nodes communicate with the external nodes? This may be challenging, especially since they are running in two different docker networks.

[mrfelton]

Currently, Polar will automatically mine blocks when necessary. Such as opening & closing channels or depositing funds. Would you want it to continue to do this or would you only connect the LN nodes and mine blocks manually yourself?

Our stack includes a bitcoind node so this would also be an externally managed node (at least with how our setup works today). We generally mine the blocks manually as and when needed but from a Polar perspective I think it would certainly be useful to be able to make the bitcoin node automatically mine blocks when needed (when opening channels for example) and leverage the "quick mine" button to be able to do that manually.

Would you ever want to have the internal nodes communicate with the external nodes? This may be challenging, especially since they are running in two different docker networks.

I hadn't envisaged being able to add internal nodes to an externally managed cluster, but I can see how it might be useful, for example to extend the predefined/external network with some new nodes in order to test more complex routing scenarios - but this is probably more of an advanced feature.

[jamaljsr]

Cool. Thanks for the additional details. This is helpful. The overall idea is pretty clear to me now.

[sagarpattasi7]

sorry that I'm posting this out here . May I know how could I use this tool for research purpose.
I would be very grateful for your suggestions .

[jamaljsr]

@sagarpattasi7 Yes, it would have been better to open a new issue for your question instead of asking here. It’s ok.

To answer your question, this project is open source with an MIT license. You can use it however you choose in your research.

[Kixunil]

I'd love to integrate polar into my project and this feature is required for smooth integration. I'd also like to request being able to configure it automatically (ideally by placing a file somewhere in /etc on the host OS).

Being able to communicate with the internal nodes would be very valuable for me.