Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

pprof: quotes in labels break graphviz output #101523

Open
tbg opened this issue Apr 14, 2023 · 2 comments
Open

pprof: quotes in labels break graphviz output #101523

tbg opened this issue Apr 14, 2023 · 2 comments
Labels
A-observability-inf C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-testcluster Issues found or occurred on a test cluster, i.e. a long-running internal cluster T-observability

Comments

@tbg
Copy link
Member

tbg commented Apr 14, 2023

Describe the problem

Make sure graphviz is installed. Download this profile, which was taken using the 23.1 test cluster, mv pprof.pb.gz{.txt,}, then go tool pprof -http :6060 pprof.pb.gz. Then try to get the "Graph" view to work; it will instead print

Could not execute dot; may need to install graphviz.

but really the problem is that what pprof feeds to dot is not valid input:

Serving web UI on http://localhost:6060
Error: <stdin>: syntax error in line 43 near '{'
Failed to execute dot. Is Graphviz installed?
exit status 1

pprof.pb.gz.txt

To Reproduce

Output of go tool pprof -dot pprof.pb.gz.txt > pprof.dot is attached:
pprof.dot.txt

and this reproduces:

$ dot < pprof.dot.txt
Error: <stdin>: syntax error in line 44 near '{'

That line is the range_str label, which evidently isn't getting escaped properly:

N36_0 [label = "range_str:12419/2:/Table/136/1/"{NHCH-…-PWN-a"}" id="N36_0" fontsize=8 shape=box3d tooltip="0.01s"]

Short term, we should properly escape the range_str label, but really pprof should properly escape the tags, which would be an upstream contribution (which would then take a while to trickle into a Go distribution we'll eventually upgrade to). Arguably this is an upstream bug, will file it as such.

Jira issue: CRDB-26989

@tbg tbg added C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. T-observability-inf labels Apr 14, 2023
@tbg tbg changed the title pprof: tripped up by some label we put in pprof: quotes in labels break graphviz output Apr 14, 2023
@tbg
Copy link
Member Author

tbg commented Apr 14, 2023

Filed google/pprof#769 upstream.

We should quick-fix on our end by avoiding the quotes, though.

tbg added a commit to tbg/cockroach that referenced this issue Apr 14, 2023
```
 range_str: Total 1.6s
            1.6s (  100%): 62/1
```

Touches cockroachdb#101523.

Epic: none
Release note: None
craig bot pushed a commit that referenced this issue Apr 17, 2023
101524: kvserver: avoid quote in range_str profiler tag r=pavelkalinnikov a=tbg

```
 range_str: Total 1.6s
            1.6s (  100%): 62/1
```

Touches #101523.

Epic: none
Release note: None


Co-authored-by: Tobias Grieger <[email protected]>
blathers-crl bot pushed a commit that referenced this issue Apr 17, 2023
```
 range_str: Total 1.6s
            1.6s (  100%): 62/1
```

Touches #101523.

Epic: none
Release note: None
@tbg tbg added the O-testcluster Issues found or occurred on a test cluster, i.e. a long-running internal cluster label Apr 21, 2023
@cucaroach
Copy link
Contributor

cucaroach commented May 17, 2023

Hacky work around:

git clone github.com:/cucaroach/pprof
cd pprof
go build -o crpprof

// usage, if you don't want to see pprof labels in svg...
PPROF_OMIT_NODELETS=y crpprof ...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
A-observability-inf C-bug Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior. O-testcluster Issues found or occurred on a test cluster, i.e. a long-running internal cluster T-observability
Projects
None yet
Development

No branches or pull requests

2 participants