Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Memory Leak With Go-Libp2p #2841

Closed
nisdas opened this issue Jun 19, 2024 · 9 comments · Fixed by #2852
Closed

Memory Leak With Go-Libp2p #2841

nisdas opened this issue Jun 19, 2024 · 9 comments · Fixed by #2852

Comments

@nisdas
Copy link

nisdas commented Jun 19, 2024

In Prysm, we have support for running Prysm nodes both with the Quic transport and the TCP transport. Recently we updated our version to be v0.35.0. This has been working for the most part, however while profiling long-running nodes running we have come across what appears to be a memory leak. Over the course of a few weeks the heap occupied by a prysm node has grown rather than stayed constant size. If you look at the heap profiles below, you can see the large amount of memory allocated by the respective quic and tcp dialers.

image
image

The above 2 screenshots capture the heatmaps occupied by QUIC connections. If you take a look, it is sizeable amount occupied by the pre-setup step of quic connections:

github.com/quic-go/quic-go.(*connection).preSetup
external/com_github_quic_go_quic_go/connection.go

  Total:    698.35MB   734.85MB (flat, cum) 12.86%
    452            .          .           	} 
    453            .          .           	return s 
    454            .          .           } 
    455            .          .            
    456            .          .           func (s *connection) preSetup() { 
    457          5MB        9MB           	s.initialStream = newCryptoStream() 
    458          1MB        8MB           	s.handshakeStream = newCryptoStream() 
    459       9.50MB     9.50MB           	s.sendQueue = newSendQueue(s.conn) 
    460       3.50MB     3.50MB           	s.retransmissionQueue = newRetransmissionQueue() 
    461          1MB        1MB           	s.frameParser = *wire.NewFrameParser(s.config.EnableDatagrams) 
    462       1.50MB     1.50MB           	s.rttStats = &utils.RTTStats{} 
    463       2.50MB     2.50MB           	s.connFlowController = flowcontrol.NewConnectionFlowController( 
    464            .          .           		protocol.ByteCount(s.config.InitialConnectionReceiveWindow), 
    465            .          .           		protocol.ByteCount(s.config.MaxConnectionReceiveWindow), 
    466     512.01kB   512.01kB           		s.onHasConnectionWindowUpdate, 
    467     512.01kB   512.01kB           		func(size protocol.ByteCount) bool { 
    468            .          .           			if s.config.AllowConnectionWindowIncrease == nil { 
    469            .          .           				return true 
    470            .          .           			} 
    471            .          .           			return s.config.AllowConnectionWindowIncrease(s, uint64(size)) 
    472            .          .           		}, 
    473            .          .           		s.rttStats, 
    474            .          .           		s.logger, 
    475            .          .           	) 
    476       2.50MB     2.50MB           	s.earlyConnReadyChan = make(chan struct{}) 
    477          4MB       28MB           	s.streamsMap = newStreamsMap( 
    478            .          .           		s.ctx, 
    479            .          .           		s, 
    480     512.01kB   512.01kB           		s.newFlowController, 
    481            .          .           		uint64(s.config.MaxIncomingStreams), 
    482            .          .           		uint64(s.config.MaxIncomingUniStreams), 
    483            .          .           		s.perspective, 
    484            .          .           	) 
    485          2MB        2MB           	s.framer = newFramer(s.streamsMap) 
    486     647.84MB   647.84MB           	s.receivedPackets = make(chan receivedPacket, protocol.MaxConnUnprocessedPackets) 
    487       3.50MB     3.50MB           	s.closeChan = make(chan closeError, 1) 
    488       1.50MB     1.50MB           	s.sendingScheduled = make(chan struct{}, 1) 
    489            .     1.50MB           	s.handshakeCtx, s.handshakeCtxCancel = context.WithCancel(context.Background()) 
    490            .          .            
    491            .          .           	now := time.Now() 
    492            .          .           	s.lastPacketReceivedTime = now 
    493            .          .           	s.creationTime = now 
    494            .          .            
    495       2.50MB     2.50MB           	s.windowUpdateQueue = newWindowUpdateQueue(s.streamsMap, s.connFlowController, s.framer.QueueControlFrame) 
    496          9MB        9MB           	s.datagramQueue = newDatagramQueue(s.scheduleSending, s.logger) 
    497            .          .           	s.connState.Version = s.version 
    498            .          .           } 
    499            .          .            
    500            .          .           // run the connection main loop 
    501            .          .           func (s *connection) run() error { 

I am not very familiar with the internals of quic-go but it appears these streams/connections are not being appropriately garbage collected by go-libp2p ? You have old connections somehow still living on the heap.

Finally we did a profile differential for the same node of its current state and its state a few days ago:

image

With this heatmap it becomes obvious where the memory leak is coming from, it does appear the new release has introduced either a regression or new bug for old connections.

Version Information
v0.35.0
@MarcoPolo
Copy link
Collaborator

Just to be clear, you did not see this issue on a prior version of go-libp2p? Which version was that?

@master255

This comment was marked as off-topic.

@MarcoPolo
Copy link
Collaborator

Are you running runtime.GC() before capturing the profile? The dangling memory from the connection may not have been GC'd yet. When a quic connection is active it handles incoming packets by registering the conn in a map. When a conn is closed the value in the map is overwritten, but the conn is still on the heap waiting to be cleaned up by the GC.

If I try to repro this issue without doing runtime.GC() when capturing the heap profile I see a similar flamegraph:
Screenshot 2024-06-19 at 3 54 15 PM

However, if I call runtime.GC() to ensure I'm cleaning up any pending items I don't see anything out of the ordinary.

Here's my test code:
https://gist.github.com/MarcoPolo/faca66afeef81edf84e880148d22a601

@marten-seemann
Copy link
Contributor

Are you sure it's a memory leak? How many QUIC connections are you handling? An easy way to check is using the Swarm Prometheus dashboard.

protocol.MaxConnUnprocessedPackets is 256. receivedPacket weighs 112 bytes, so this is about ~ 29 kB per QUIC connection. The 600 MB you're seeing would therefore correspond to roughly 2000 QUIC connection. Pre-allocating this much memory is not ideal, and we'll most likely refactor this logic when implementing support for GRO.

@nisdas
Copy link
Author

nisdas commented Jun 20, 2024

Yeap, we have updated from v0.33.1 @MarcoPolo . This wasn't an issue before on v0.33.1 , possibly something was refactored/changed with the dialer/swarm in either v0.34.x or v0.35.x ? I can try downgrading to v0.34.x to see if its still an issue there.

runtime.GC() is called by the go runtime periodically as this is a live node rather than an isolated test. This node has been running for 3 weeks which means the runtime isn't clearing this out for some reason. Do you have nodes for other network running on go-libp2p reporting any similar issues ?

@marten-seemann Does very much appear to be on our end, my live node that was running has again had an increased heap from yesterday of about 50mb. At any one time there are only 15-20 active quic connections on my node. It has a maximum peer count of 70.

# HELP libp2p_swarm_connections_closed_total Connections Closed
# TYPE libp2p_swarm_connections_closed_total counter
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 1
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 120908
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 225525
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 362962
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 80
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32158
# HELP libp2p_swarm_connections_opened_total Connections Opened
# TYPE libp2p_swarm_connections_opened_total counter
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 1
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 120925
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 225533
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 362993
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 80
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32171
# HELP libp2p_swarm_dial_errors_total Dial Error
# TYPE libp2p_swarm_dial_errors_total counter
libp2p_swarm_dial_errors_total{error="canceled: concurrent dial successful",ip_version="ip4",transport="tcp"} 84201
libp2p_swarm_dial_errors_total{error="canceled: concurrent dial successful",ip_version="ip6",transport="tcp"} 17
libp2p_swarm_dial_errors_total{error="connection refused",ip_version="ip4",transport="tcp"} 37979
libp2p_swarm_dial_errors_total{error="deadline",ip_version="ip4",transport="quic-v1"} 29
libp2p_swarm_dial_errors_total{error="deadline",ip_version="ip4",transport="tcp"} 210743
libp2p_swarm_dial_errors_total{error="deadline",ip_version="ip6",transport="tcp"} 11
libp2p_swarm_dial_errors_total{error="other",ip_version="ip4",transport="quic-v1"} 347816
libp2p_swarm_dial_errors_total{error="other",ip_version="ip4",transport="tcp"} 216050
libp2p_swarm_dial_errors_total{error="other",ip_version="ip6",transport="quic-v1"} 65
libp2p_swarm_dial_errors_total{error="other",ip_version="ip6",transport="tcp"} 1
libp2p_swarm_dial_errors_total{error="timeout",ip_version="ip4",transport="quic-v1"} 112813
libp2p_swarm_dial_errors_total{error="timeout",ip_version="ip4",transport="tcp"} 62900
libp2p_swarm_dial_errors_total{error="timeout",ip_version="ip6",transport="quic-v1"} 1
libp2p_swarm_dial_errors_total{error="timeout",ip_version="ip6",transport="tcp"} 4

Swarm metrics from my node

@MarcoPolo
Copy link
Collaborator

I can try downgrading to v0.34.x to see if its still an issue there.

v0.34 and v0.35 are very similar. There was just a small breaking change. I wouldn't expect it to behave differently.

@MarcoPolo
Copy link
Collaborator

Could you share repro steps? Maybe I can try with Kubo. Do you have a sense of the connection churn per hour? e.g. could you share the metrics of a node an hour apart? (or even 4 measurements 15 min apart)

I wonder if you disconnected the node from the internet for a bit (1 minute?) and ran runtime.GC() if you would still see the issue.

@nisdas
Copy link
Author

nisdas commented Jun 20, 2024

Could you share repro steps? Maybe I can try with Kubo. Do you have a sense of the connection churn per hour? e.g. could you share the metrics of a node an hour apart? (or even 4 measurements 15 min apart)

I can share the metrics in a bit, for Kubo just running this with both TCP and QUIC transports enabled for an extended period of time should show something. These are the options we currently use https://github.com/prysmaticlabs/prysm/blob/develop/beacon-chain/p2p/options.go#L63

@nisdas
Copy link
Author

nisdas commented Jun 20, 2024

Some connection metrics over 1 hour at 15 min intervals

Measurement 1:

# TYPE libp2p_swarm_connections_closed_total counter
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 2
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 121470
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 226686
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 364902
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 81
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32332
# HELP libp2p_swarm_connections_opened_total Connections Opened
# TYPE libp2p_swarm_connections_opened_total counter
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 2
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 121494
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 226694
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 364926
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 81
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32345

Measurement 2:
 libp2p_swarm_connections_closed_total Connections Closed
# TYPE libp2p_swarm_connections_closed_total counter
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 2
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 121518
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 226789
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 365107
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 81
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32342
# HELP libp2p_swarm_connections_opened_total Connections Opened
# TYPE libp2p_swarm_connections_opened_total counter
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 2
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 121540
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 226797
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 365129
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 81
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32356

Measurement 3:
libp2p_swarm_connections_closed_total Connections Closed
# TYPE libp2p_swarm_connections_closed_total counter
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 2
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 121573
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 226915
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 365254
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 81
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32364
# HELP libp2p_swarm_connections_opened_total Connections Opened
# TYPE libp2p_swarm_connections_opened_total counter
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 2
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 121591
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 226925
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 365280
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 81
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32376

Measurement 4:

 libp2p_swarm_connections_closed_total Connections Closed
# TYPE libp2p_swarm_connections_closed_total counter
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 2
libp2p_swarm_connections_closed_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 121626
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 227011
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 365424
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 81
libp2p_swarm_connections_closed_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32375
# HELP libp2p_swarm_connections_opened_total Connections Opened
# TYPE libp2p_swarm_connections_opened_total counter
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 13
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 2
libp2p_swarm_connections_opened_total{dir="inbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 6
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="",security="",transport="quic-v1"} 121645
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/mplex/6.7.0",security="/noise",transport="tcp"} 227021
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 365453
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="",security="",transport="quic-v1"} 33
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="false",ip_version="ip6",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 81
libp2p_swarm_connections_opened_total{dir="outbound",early_muxer="true",ip_version="ip4",muxer="/yamux/1.0.0",security="/noise",transport="tcp"} 32385

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants