Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Reducing Binary Sizes in Packages #12768

Closed
andrewkroh opened this issue Jul 3, 2019 · 3 comments · Fixed by #13258
Closed

Reducing Binary Sizes in Packages #12768

andrewkroh opened this issue Jul 3, 2019 · 3 comments · Fixed by #13258
Assignees
@mikemadden42
Labels
discuss Issue needs further discussion. Packaging

Comments

@andrewkroh
Copy link
Member

andrewkroh commented Jul 3, 2019 •
edited
Loading

I ran a few tests to see what options we have for shrinking our binary sizes. We might want to consider making some changes.

Filebeat Size (linux/amd64) Build Options
85MB (current) golang-crossbuild:1.12.4-main-debian7, GCC 4.7.2
57MB golang-crossbuild:1.12.4-main, GCC 6.3.0* (provides zlib compressed debug symbols)
42MB golang-crossbuild:1.12.4-main, GCC 6.3.0, -ldflags "-w -s" (stripped)
15MB golang-crossbuild:1.12.4-main, GCC 6.3.0, -ldflags "-w -s" (stripped), UPX

* GCC 6.3.0 is part of the debian 9 golang-crossbuild images so it has a newer glibc requirement.
@andrewkroh andrewkroh added discuss Issue needs further discussion. Packaging labels Jul 3, 2019
@urso
Copy link

urso commented Jul 3, 2019

Nice.

For reference: golang/go#26074

Given that DWARF is redundant info and only required for external tools (like debuggers or third party profilers), I’m fine with stripping by default.

Would be nice to know if builds are truly reproducible. Just in case we have to deal with actual core dumps.

@mikemadden42
Copy link
Contributor

There's a related issue regarding Debian 7 being EOL.
#11750

According to the following doc, Debian 8 is also EOL unless we're considering LTS. Maybe it's a good time to make the switch to Debian 9?

https://wiki.debian.org/DebianReleases

@andrewkroh
Copy link
Member Author

andrewkroh commented Aug 15, 2019 •
edited
Loading

I did a comparison of all packages by adding -ldflags "-s" to the build. This test was based on e8786fe. The packages reduced in average size by 42.10% and the Beat binaries reduced an average of 54.99%.

Unstripped (MiB) Stripped (MiB) Diff (MiB) Diff (%) Filename
58.85 58.85 0.00 0.00 auditbeat-darwin-amd64
87.04 45.04 42.00 63.59 auditbeat-linux-386
103.07 51.27 51.80 67.12 auditbeat-linux-amd64
71.32 35.20 36.12 67.82 auditbeat-windows-386.exe
85.20 40.64 44.56 70.82 auditbeat-windows-amd64.exe
29.78 18.29 11.49 47.79 auditbeat-8.0.0-amd64.deb
17.37 17.36 0.00 0.01 auditbeat-8.0.0-darwin-x86_64.tar.gz
28.39 17.24 11.16 48.91 auditbeat-8.0.0-i386.deb
28.04 17.05 11.00 48.78 auditbeat-8.0.0-i686.rpm
130.94 107.95 22.99 19.25 auditbeat-8.0.0-linux-amd64.docker.tar.gz
29.52 18.02 11.50 48.37 auditbeat-8.0.0-linux-x86_64.tar.gz
28.61 17.19 11.42 49.89 auditbeat-8.0.0-linux-x86.tar.gz
23.89 13.99 9.90 52.29 auditbeat-8.0.0-windows-x86_64.zip
22.90 13.21 9.68 53.65 auditbeat-8.0.0-windows-x86.zip
29.23 18.01 11.22 47.48 auditbeat-8.0.0-x86_64.rpm
27.70 16.97 10.73 48.03 auditbeat-oss-8.0.0-amd64.deb
16.20 16.20 0.00 0.01 auditbeat-oss-8.0.0-darwin-x86_64.tar.gz
26.47 16.03 10.43 49.09 auditbeat-oss-8.0.0-i386.deb
26.14 15.86 10.28 48.95 auditbeat-oss-8.0.0-i686.rpm
126.77 105.30 21.47 18.50 auditbeat-oss-8.0.0-linux-amd64.docker.tar.gz
27.43 16.69 10.74 48.68 auditbeat-oss-8.0.0-linux-x86_64.tar.gz
26.64 15.96 10.68 50.16 auditbeat-oss-8.0.0-linux-x86.tar.gz
22.41 13.07 9.35 52.70 auditbeat-oss-8.0.0-windows-x86_64.zip
21.52 12.37 9.14 53.96 auditbeat-oss-8.0.0-windows-x86.zip
27.20 16.72 10.48 47.73 auditbeat-oss-8.0.0-x86_64.rpm
36.63 22.91 13.73 46.11 filebeat-8.0.0-amd64.deb
22.25 22.24 0.00 0.01 filebeat-8.0.0-darwin-x86_64.tar.gz
34.98 21.63 13.35 47.17 filebeat-8.0.0-i386.deb
34.54 21.40 13.14 46.99 filebeat-8.0.0-i686.rpm
144.94 117.48 27.46 20.93 filebeat-8.0.0-linux-amd64.docker.tar.gz
36.51 22.78 13.73 46.32 filebeat-8.0.0-linux-x86_64.tar.gz
35.45 21.78 13.67 47.79 filebeat-8.0.0-linux-x86.tar.gz
30.05 18.14 11.91 49.42 filebeat-8.0.0-windows-x86_64.zip
28.86 17.19 11.67 50.70 filebeat-8.0.0-windows-x86.zip
35.96 22.57 13.39 45.76 filebeat-8.0.0-x86_64.rpm
30.65 19.13 11.52 46.29 filebeat-oss-8.0.0-amd64.deb
18.61 18.62 0.00 0.01 filebeat-oss-8.0.0-darwin-x86_64.tar.gz
29.33 18.12 11.21 47.27 filebeat-oss-8.0.0-i386.deb
28.98 17.93 11.05 47.09 filebeat-oss-8.0.0-i686.rpm
132.92 109.85 23.07 19.00 filebeat-oss-8.0.0-linux-amd64.docker.tar.gz
30.50 18.97 11.53 46.62 filebeat-oss-8.0.0-linux-x86_64.tar.gz
29.67 18.19 11.48 47.98 filebeat-oss-8.0.0-linux-x86.tar.gz
25.43 15.31 10.12 49.69 filebeat-oss-8.0.0-windows-x86_64.zip
24.46 14.55 9.91 50.80 filebeat-oss-8.0.0-windows-x86.zip
30.10 18.85 11.25 45.97 filebeat-oss-8.0.0-x86_64.rpm
49.47 37.73 11.74 26.93 functionbeat-8.0.0-darwin-x86_64.tar.gz
62.97 38.26 24.71 48.83 functionbeat-8.0.0-linux-x86_64.tar.gz
61.89 37.24 24.65 49.73 functionbeat-8.0.0-linux-x86.tar.gz
56.93 33.78 23.14 51.03 functionbeat-8.0.0-windows-x86_64.zip
55.75 32.84 22.91 51.73 functionbeat-8.0.0-windows-x86.zip
28.72 17.64 11.08 47.81 heartbeat-8.0.0-amd64.deb
17.00 17.00 -0.01 0.03 heartbeat-8.0.0-darwin-x86_64.tar.gz
27.37 16.61 10.77 48.96 heartbeat-8.0.0-i386.deb
27.03 16.43 10.61 48.82 heartbeat-8.0.0-i686.rpm
128.81 106.63 22.18 18.84 heartbeat-8.0.0-linux-amd64.docker.tar.gz
28.45 17.36 11.09 48.41 heartbeat-8.0.0-linux-x86_64.tar.gz
27.58 16.55 11.02 49.96 heartbeat-8.0.0-linux-x86.tar.gz
23.44 13.71 9.73 52.40 heartbeat-8.0.0-windows-x86_64.zip
22.45 12.93 9.52 53.79 heartbeat-8.0.0-windows-x86.zip
28.19 17.36 10.82 47.53 heartbeat-8.0.0-x86_64.rpm
27.17 16.64 10.53 48.08 heartbeat-oss-8.0.0-amd64.deb
16.05 16.05 0.00 0.00 heartbeat-oss-8.0.0-darwin-x86_64.tar.gz
25.96 15.72 10.24 49.13 heartbeat-oss-8.0.0-i386.deb
25.64 15.55 10.09 49.00 heartbeat-oss-8.0.0-i686.rpm
125.70 104.62 21.08 18.31 heartbeat-oss-8.0.0-linux-amd64.docker.tar.gz
26.89 16.35 10.54 48.75 heartbeat-oss-8.0.0-linux-x86_64.tar.gz
26.13 15.64 10.48 50.18 heartbeat-oss-8.0.0-linux-x86.tar.gz
22.24 12.96 9.28 52.74 heartbeat-oss-8.0.0-windows-x86_64.zip
21.33 12.26 9.07 54.01 heartbeat-oss-8.0.0-windows-x86.zip
26.67 16.38 10.29 47.78 heartbeat-oss-8.0.0-x86_64.rpm
30.67 18.87 11.79 47.61 journalbeat-8.0.0-amd64.deb
29.22 17.82 11.40 48.49 journalbeat-8.0.0-i386.deb
28.85 17.62 11.23 48.33 journalbeat-8.0.0-i686.rpm
132.96 109.33 23.63 19.50 journalbeat-8.0.0-linux-amd64.docker.tar.gz
30.52 18.72 11.80 47.94 journalbeat-8.0.0-linux-x86_64.tar.gz
29.55 17.87 11.68 49.25 journalbeat-8.0.0-linux-x86.tar.gz
30.11 18.59 11.52 47.31 journalbeat-8.0.0-x86_64.rpm
29.16 17.91 11.25 47.78 journalbeat-oss-8.0.0-amd64.deb
27.82 16.95 10.87 48.57 journalbeat-oss-8.0.0-i386.deb
27.47 16.76 10.71 48.41 journalbeat-oss-8.0.0-i686.rpm
129.92 107.41 22.51 18.97 journalbeat-oss-8.0.0-linux-amd64.docker.tar.gz
29.00 17.75 11.25 48.14 journalbeat-oss-8.0.0-linux-x86_64.tar.gz
28.11 16.98 11.13 49.35 journalbeat-oss-8.0.0-linux-x86.tar.gz
28.62 17.64 10.98 47.47 journalbeat-oss-8.0.0-x86_64.rpm
48.97 30.14 18.83 47.61 metricbeat-8.0.0-amd64.deb
28.83 28.83 0.00 0.01 metricbeat-8.0.0-darwin-x86_64.tar.gz
46.65 28.43 18.22 48.53 metricbeat-8.0.0-i386.deb
46.03 28.09 17.93 48.39 metricbeat-8.0.0-i686.rpm
169.56 131.92 37.64 24.97 metricbeat-8.0.0-linux-amd64.docker.tar.gz
48.82 30.00 18.82 47.76 metricbeat-8.0.0-linux-x86_64.tar.gz
47.49 28.77 18.72 49.10 metricbeat-8.0.0-linux-x86.tar.gz
37.90 22.05 15.84 52.85 metricbeat-8.0.0-windows-x86_64.zip
36.47 20.97 15.51 53.99 metricbeat-8.0.0-windows-x86.zip
48.03 29.64 18.39 47.35 metricbeat-8.0.0-x86_64.rpm
39.61 24.39 15.22 47.56 metricbeat-oss-8.0.0-amd64.deb
23.42 23.42 0.00 0.01 metricbeat-oss-8.0.0-darwin-x86_64.tar.gz
37.80 23.03 14.77 48.57 metricbeat-oss-8.0.0-i386.deb
37.30 22.76 14.54 48.43 metricbeat-oss-8.0.0-i686.rpm
150.69 120.23 30.45 22.48 metricbeat-oss-8.0.0-linux-amd64.docker.tar.gz
39.39 24.17 15.23 47.91 metricbeat-oss-8.0.0-linux-x86_64.tar.gz
38.26 23.12 15.15 49.36 metricbeat-oss-8.0.0-linux-x86.tar.gz
31.09 18.15 12.93 52.54 metricbeat-oss-8.0.0-windows-x86_64.zip
29.87 17.21 12.67 53.80 metricbeat-oss-8.0.0-windows-x86.zip
38.87 24.00 14.87 47.31 metricbeat-oss-8.0.0-x86_64.rpm
30.69 18.96 11.73 47.25 packetbeat-8.0.0-amd64.deb
18.18 18.18 0.00 0.00 packetbeat-8.0.0-darwin-x86_64.tar.gz
29.09 17.86 11.23 47.85 packetbeat-8.0.0-i386.deb
28.72 17.66 11.06 47.70 packetbeat-8.0.0-i686.rpm
132.78 109.31 23.47 19.39 packetbeat-8.0.0-linux-amd64.docker.tar.gz
30.44 18.70 11.74 47.79 packetbeat-8.0.0-linux-x86_64.tar.gz
29.32 17.82 11.50 48.80 packetbeat-8.0.0-linux-x86.tar.gz
24.71 14.68 10.03 50.93 packetbeat-8.0.0-windows-x86_64.zip
23.69 13.86 9.83 52.37 packetbeat-8.0.0-windows-x86.zip
30.13 18.68 11.46 46.94 packetbeat-8.0.0-x86_64.rpm
29.16 17.98 11.18 47.44 packetbeat-oss-8.0.0-amd64.deb
17.23 17.23 0.00 0.01 packetbeat-oss-8.0.0-darwin-x86_64.tar.gz
27.68 16.98 10.70 47.92 packetbeat-oss-8.0.0-i386.deb
27.33 16.79 10.54 47.79 packetbeat-oss-8.0.0-i686.rpm
129.71 107.33 22.38 18.88 packetbeat-oss-8.0.0-linux-amd64.docker.tar.gz
28.90 17.71 11.19 48.03 packetbeat-oss-8.0.0-linux-x86_64.tar.gz
27.87 16.91 10.96 48.95 packetbeat-oss-8.0.0-linux-x86.tar.gz
23.52 13.94 9.58 51.14 packetbeat-oss-8.0.0-windows-x86_64.zip
22.57 13.18 9.38 52.48 packetbeat-oss-8.0.0-windows-x86.zip
28.63 17.71 10.92 47.13 packetbeat-oss-8.0.0-x86_64.rpm
25.63 15.27 10.35 50.63 winlogbeat-8.0.0-windows-x86_64.zip
24.60 14.46 10.14 51.89 winlogbeat-8.0.0-windows-x86.zip
24.40 14.51 9.90 50.87 winlogbeat-oss-8.0.0-windows-x86_64.zip
23.46 13.77 9.69 52.06 winlogbeat-oss-8.0.0-windows-x86.zip
65.74 65.74 0.00 0.00 filebeat-darwin-amd64
95.33 50.41 44.91 61.63 filebeat-linux-386
112.35 57.01 55.34 65.35 filebeat-linux-amd64
79.06 40.17 38.89 65.24 filebeat-windows-386.exe
93.90 45.97 47.93 68.54 filebeat-windows-amd64.exe
58.28 58.28 0.00 0.00 heartbeat-darwin-amd64
85.50 44.27 41.23 63.55 heartbeat-linux-386
101.23 50.33 50.90 67.16 heartbeat-linux-amd64
70.72 34.93 35.79 67.75 heartbeat-windows-386.exe
84.46 40.28 44.18 70.84 heartbeat-windows-amd64.exe
91.87 48.13 43.74 62.49 journalbeat-linux-386
108.73 54.55 54.19 66.37 journalbeat-linux-amd64
93.76 93.76 0.00 0.00 metricbeat-darwin-amd64
136.30 69.53 66.78 64.89 metricbeat-linux-386
159.32 79.14 80.18 67.25 metricbeat-linux-amd64
110.09 53.15 56.94 69.77 metricbeat-windows-386.exe
129.62 61.44 68.17 71.37 metricbeat-windows-amd64.exe
61.90 61.90 0.00 0.00 packetbeat-darwin-amd64
90.59 47.36 43.23 62.68 packetbeat-linux-386
107.57 53.85 53.71 66.55 packetbeat-linux-amd64
74.33 37.09 37.24 66.84 packetbeat-windows-386.exe
88.56 42.85 45.71 69.58 packetbeat-windows-amd64.exe
76.89 38.81 38.08 65.83 winlogbeat-windows-386.exe
91.44 44.47 46.97 69.11 winlogbeat-windows-amd64.exe
64.41 64.41 0.00 0.00 auditbeat-darwin-amd64
94.60 49.11 45.49 63.31 auditbeat-linux-386
112.23 56.36 55.87 66.28 auditbeat-linux-amd64
76.86 38.13 38.72 67.35 auditbeat-windows-386.exe
92.09 44.50 47.59 69.68 auditbeat-windows-amd64.exe
83.15 83.15 0.00 0.00 filebeat-darwin-amd64
119.06 62.68 56.39 62.05 filebeat-linux-386
139.84 71.40 68.44 64.80 filebeat-linux-amd64
97.54 49.35 48.19 65.61 filebeat-windows-386.exe
115.54 57.08 58.47 67.74 filebeat-windows-amd64.exe
78.72 78.72 0.00 0.00 functionbeat-darwin-amd64
114.77 58.46 56.32 65.02 functionbeat-linux-386
135.05 67.16 67.89 67.15 functionbeat-linux-amd64
93.30 45.31 47.98 69.23 functionbeat-windows-386.exe
110.86 53.04 57.82 70.55 functionbeat-windows-amd64.exe
67.42 67.42 0.00 0.00 aws-darwin-amd64 (functionbeat)
99.85 50.65 49.20 65.38 aws-linux-386 (functionbeat)
117.58 57.81 59.77 68.15 aws-linux-amd64 (functionbeat)
81.68 39.41 42.27 69.82 aws-windows-386.exe (functionbeat)
97.05 45.71 51.34 71.93 aws-windows-amd64.exe (functionbeat)
63.24 63.24 0.00 0.00 heartbeat-darwin-amd64
91.63 47.60 44.03 63.25 heartbeat-linux-386
108.70 54.58 54.12 66.29 heartbeat-linux-amd64
75.60 37.53 38.07 67.30 heartbeat-windows-386.exe
90.54 43.74 46.80 69.70 heartbeat-windows-amd64.exe
97.97 51.43 46.54 62.31 journalbeat-linux-386
116.17 58.76 57.41 65.64 journalbeat-linux-amd64
126.13 126.13 0.00 0.00 metricbeat-darwin-amd64
180.81 92.05 88.77 65.07 metricbeat-linux-386
210.45 105.27 105.18 66.63 metricbeat-linux-amd64
143.98 69.50 74.48 69.77 metricbeat-windows-386.exe
168.97 81.08 87.89 70.30 metricbeat-windows-amd64.exe
66.86 66.86 0.00 0.00 packetbeat-darwin-amd64
96.73 50.69 46.04 62.46 packetbeat-linux-386
115.04 58.10 56.93 65.77 packetbeat-linux-amd64
79.21 39.69 39.52 66.47 packetbeat-windows-386.exe
94.64 46.31 48.33 68.57 packetbeat-windows-amd64.exe
81.77 41.40 40.36 65.54 winlogbeat-windows-386.exe
97.52 47.94 49.58 68.17 winlogbeat-windows-amd64.exe
Average 23.47 46.69

andrewkroh added a commit to andrewkroh/beats that referenced this issue Aug 15, 2019
@andrewkroh
Strip debug symbols from binaries
7ab10b8 
The debug symbols are only required for external tools (like debuggers or third party profilers). This does not affect Go stack traces or pprof.

Closes elastic#12768
@andrewkroh andrewkroh mentioned this issue Aug 15, 2019
Merged
andrewkroh added a commit that referenced this issue Aug 16, 2019
@andrewkroh
Strip debug symbols from binaries (#13258)
d5d755d 
The debug symbols are only required for external tools (like debuggers or third party profilers). This does not affect Go stack traces or pprof.

Closes #12768
@andrewkroh andrewkroh mentioned this issue Nov 15, 2019
Closed
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@mikemadden42 mikemadden42

Labels
discuss Issue needs further discussion. Packaging
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Strip debug symbols from binaries andrewkroh/beats
3 participants
@urso @andrewkroh @mikemadden42