kibana 6.3.0

[ddillinger]

Created with brew bump-formula-pr.

[jasontedor]

@ilovezfs @MikeMcQuaid Sorry for the ping, but we need some help here. We have changed the licensing on our artifacts here so they do not carry a Apache 2.0 license anymore starting with 6.3.0. We want to work with Homebrew to allow distribution in an appropriate way. Is there someone that can reach out to us to start a conversation? Until we can sort that out though, we want to be conservative and ask that any PRs bumping the formula to 6.3.0 not be merged. I am happy to have a conversation here, or I can be reached at my first name at elastic.co.

[commitay]

@jasontedor Does the license change also apply to the versioned formulae? [email protected] and [email protected] both have a new version 5.6.10?

[jasontedor]

@commitay No, it only applies to 6.3.0 and later versions. All existing and future maintenance releases in the 5.6 series will continue to be under the same Apache 2.0 license as before. The 6.0, 6.1, and 6.2 series are also Apache 2.0 licensed. The change for the default distribution starts with the 6.3 series.

[ilovezfs]

What is the new license?

[ilovezfs]

OK, I see https://github.com/elastic/kibana/blob/master/licenses/ELASTIC-LICENSE.txt and https://github.com/elastic/kibana/blob/master/LICENSE.txt

Is there anything in the formula that is installing stuff from x-pack?

[ilovezfs]

@jasontedor my recommendation is that upstream provide a source tarball that doesn't have the x-pack components at all so there are no misunderstandings. If that's not something upstream is willing to do, we can rm_rf "x-pack" at the beginning of the install method.

[mistydemeo]

I agree with @ilovezfs; it would be great to get a copy of the tarballs with only the Apache 2 components and not the Elastic-licensed x-pack components.

[MikeMcQuaid]

Also agree with @ilovezfs here.

[JCount]

I agree with @ilovezfs on this matter as well.

[ilovezfs]

@jasontedor @ddillinger any update here?

[jasontedor]

Heya, sorry, I’m on vacation since last Friday until next Monday so my replies will be terse and with high latency.

Briefly, the x-pack folder is integrated into our build scripts for some of the projects (if not all, I can’t check right now) so removing it at build time or providing a source tarball without the x-pack folder would not be possible at this time.

[ilovezfs]

OK, we'll leave kibana, etc. at 6.2.x in Homebrew until upstream is able to provide an open source tarball and build script for 6.3.x. Thanks for the updated information @jasontedor

[ilovezfs]

Thanks for the PR nonetheless @ddillinger

[ilovezfs]

if not all, I can’t check right now

@jasontedor when you are able to check, of course please feel free to comment on any of the other PRs if you determine Elastic license is not an issue for them (assuming using 6.3.x for some of the packages doesn't interfere with using 6.2.x for others).

[jrosengren]

Apache 2.0 licensed downloads (at least for metricbeat) appear to be available here: https://www.elastic.co/downloads/beats/metricbeat-oss

Are you able to use the "-oss" versions of the beats or is there still a question about whether those are fully Apache 2.0 compatible?

[ilovezfs]

@jrosengren those should be fine! @jasontedor please correct me if that's not the case.

[ilovezfs]

@jrosengren I think the problem there is that there's no way for us to build those from source unless I'm not seeing it. So actually the situation is unchanged.

[felixbuenemann]

@jasontedor Would removing the bin/x-pack and modules/x-pack directories from the elasticsearch tar-ball be enough to keep only the Apache 2.0 licensed parts?

[jrosengren]

It seems somewhat ridiculous that this hasn't been resolved yet. What are we waiting for to be able to include Metricbeat 6.3.x in Homebrew?

[MikeMcQuaid]

@jrosengren The ability to build a package from source with an open source license.

[jrosengren]

Elastic publishes -oss docker images that presumably only contain the open source bits, so it seems like it should be possible for Homebrew to build similarly open source bits.

https://raw.githubusercontent.com/elastic/beats/99824df163863478d27688206e18ec27bc1b3ac5/LICENSE.txt

Based on the text in the LICENSE.txt, should it be possible to just download the source, run the build, and only use the -oss package? Or does Homebrew have a requirement to not download any non-open source bits?

[mschneid]

@MikeMcQuaid As I have seen, Elastic offers OSS downloads for the new 6.4.0 series:

• Kibana 6.4.0
• Elasticsearch 6.4.0
• Logstash 6.4.0
• Filebeat 6.4.0
• Metricbeat 6.4.0
• Auditbeat 6.4.0
• Packetbeat 6.4.0

[MikeMcQuaid]

Based on the text in the LICENSE.txt, should it be possible to just download the source, run the build, and only use the -oss package? Or does Homebrew have a requirement to not download any non-open source bits?

If there's a guaranteed way of building only the OSS version; yeh we can do that, I don't think we're saying we can't download it.

[felixbuenemann]

@mschneid At least some of those packages contain platform specific tar-balls, so since they contain pre-built binaries I don't think they can be used for homebrew packages.

[chrmoritz]

At least kibana (not sure about the other ones) seems to have an option -oss in their build script to build only the OSS version now.

I was able to build the OSS version after rm_rf "x-pack" with the upgraded formula in chrmoritz@1f31903, which also follows upstream by switching from npm to yarn and upgrades the bundled node to the 8.x release line (which should fix the 10.14 bottle #31741).

But I don't know if (and how) we could do similar OSS-only builds for the other elastic formulae.

[MikeMcQuaid]

@chrmoritz Sounds like a good call for a PR 👍

[chrmoritz]

@MikeMcQuaid The main problem is, that this PR would be blocked on upgrading elasticsearch to 6.4.0 (as [email protected] isn't compatible with [email protected]), which itself is blocked on upgrading most (all?) other elastic formulae to 6.4.0.

I have no idea if (and how) building only the OSS part is possible for these other formulae as they are all using quite different build systems. But if you want I could open this as another do-not-merge PR to start the effort of upgrading the elastic formulae to 6.4.0 and hope that someone else, who is more familar with these build systems, will chime in and continue the work.

[MikeMcQuaid]

I have no idea if (and how) building only the OSS part is possible for these other formulae as they are all using quite different build systems. But if you want I could open this as another do-not-merge PR to start the effort of upgrading the elastic formulae to 6.4.0 and hope that someone else, who is more familar with these build systems, will chime in and continue the work.

This sounds good 👍

[chrmoritz]

I've currently kibana, elasticsearch and logstash upgraded to 6.4.0 OSS.

Still need help for the go based:

• all require a new build tool mage: https://github.com/magefile/mage (mage 1.4.0 (new formula) #31956)
• the beats formulae upstream mono repo
  • packetbeat: formula
  • metricbeat: formula
  • heartbeat: formula
  • filebeat: formula
  • auditbeat: formula
• apm-server: formula, upstream repo

Edit: I give up here for now, as I can't find a way to build only the OSS part after removing x-pack for the go based formulae.

But according their their compatibility table these other updates are not strictly needed for shipping kibana and elasticsearch 6.4.0 and considering their high popularity vs the low popularity of the *beats formulas it might be worth shipping them anyway.