Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Elasticsearch build and Gradle plugins should support Gradle configuration cache #57918

Open
16 tasks done
breskeby opened this issue Jun 10, 2020 · 2 comments
Open
16 tasks done
Assignees
Labels
:Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team

Comments

@breskeby
Copy link
Contributor

breskeby commented Jun 10, 2020

The gradle configuration cache is an incubating feature that significantly improves gradle configuration time by applying cache mechanisms to the gradle build configuration.
The functionality is described in detail at https://docs.gradle.org/nightly/userguide/configuration_cache.html
In order to support this configuration cache some requirements must be met by plugins and build scripts that are described here: https://docs.gradle.org/nightly/userguide/configuration_cache.html#requirements

Tasks

  1. :Delivery/Build :Delivery/Packaging >enhancement Team:Delivery v8.12.0
    breskeby
  2. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
  3. :Delivery/Build >enhancement Team:Delivery test-update-serverless v8.12.0
    breskeby
  4. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
  5. :Delivery/Build >enhancement Team:Delivery auto-merge-without-approval v8.12.0
    breskeby
  6. :Delivery/Build >enhancement Team:Delivery buildkite-opt-in v8.12.0
    breskeby
  7. :Delivery/Build >enhancement Team:Delivery buildkite-opt-in v8.12.0
    breskeby
  8. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
  9. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
  10. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
  11. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
  12. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
  13. :Delivery/Build >enhancement Team:Delivery auto-merge-without-approval v8.12.0
    breskeby
  14. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
  15. :Delivery/Build >non-issue Team:Delivery v8.12.0
    breskeby
  16. :Delivery/Build >enhancement Team:Delivery v8.12.0
    breskeby
@breskeby breskeby added :Delivery/Build Build or test infrastructure Team:Core/Infra Meta label for core/infra team labels Jun 10, 2020
@breskeby breskeby self-assigned this Jun 10, 2020
@elasticmachine
Copy link
Collaborator

Pinging @elastic/es-core-infra (:Core/Infra/Build)

breskeby added a commit to breskeby/elasticsearch that referenced this issue Jul 17, 2020
- avoid using extra plugin
- avoid extra task listener (should be avoided related to elastic#57918 )
- keep all logic in the listener
breskeby added a commit that referenced this issue Jul 21, 2020
* Simplify test error reporting

- avoid using extra plugin
- avoid extra task listener (should be avoided related to #57918 )
- keep all logic in the listener
breskeby added a commit to breskeby/elasticsearch that referenced this issue Jul 21, 2020
* Simplify test error reporting

- avoid using extra plugin
- avoid extra task listener (should be avoided related to elastic#57918 )
- keep all logic in the listener
breskeby added a commit to breskeby/elasticsearch that referenced this issue Jul 21, 2020
* Simplify test error reporting

- avoid using extra plugin
- avoid extra task listener (should be avoided related to elastic#57918 )
- keep all logic in the listener
breskeby added a commit to breskeby/elasticsearch that referenced this issue Jul 21, 2020
* Simplify test error reporting

- avoid using extra plugin
- avoid extra task listener (should be avoided related to elastic#57918 )
- keep all logic in the listener
breskeby added a commit that referenced this issue Jul 21, 2020
* Simplify test error reporting

- avoid using extra plugin
- avoid extra task listener (should be avoided related to #57918 )
- keep all logic in the listener
breskeby added a commit that referenced this issue Jul 21, 2020
* Simplify test error reporting

- avoid using extra plugin
- avoid extra task listener (should be avoided related to #57918 )
- keep all logic in the listener
breskeby added a commit that referenced this issue Jul 21, 2020
* Simplify test error reporting

- avoid using extra plugin
- avoid extra task listener (should be avoided related to #57918 )
- keep all logic in the listener
breskeby added a commit that referenced this issue Oct 1, 2020
Referencing a project instance during task execution is discouraged by
Gradle and should be avoided. E.g. It is incompatible with Gradles
incubating configuration cache. Instead there are services available to handle
archive and filesystem operations in task actions.

Brings us one step closer to #57918
breskeby added a commit to breskeby/elasticsearch that referenced this issue Oct 2, 2020
Referencing a project instance during task execution is discouraged by
Gradle and should be avoided. E.g. It is incompatible with Gradles
incubating configuration cache. Instead there are services available to handle
archive and filesystem operations in task actions.

Brings us one step closer to elastic#57918
breskeby added a commit that referenced this issue Oct 5, 2020
Referencing a project instance during task execution is discouraged by
Gradle and should be avoided. E.g. It is incompatible with Gradles
incubating configuration cache. Instead there are services available to handle
archive and filesystem operations in task actions.

Brings us one step closer to #57918
@mark-vieira mark-vieira added Team:Delivery Meta label for Delivery team and removed Team:Core/Infra Meta label for core/infra team labels Nov 11, 2020
breskeby added a commit to breskeby/elasticsearch that referenced this issue Feb 22, 2022
usage of getProject() within a task should be avoided as it is not compatible
with gradle configuration cache. Related to elastic#57918
breskeby added a commit to breskeby/elasticsearch that referenced this issue Jun 9, 2022
This fixes references to project that makes the plugin incompatible with Gradle
configuration cache

Related to elastic#57918
breskeby added a commit that referenced this issue Jun 13, 2022
This fixes references to project that makes the plugin incompatible with Gradle
configuration cache. We also remove custom xpackProject utility:

using xpackProject in certain situations can break configure configuration cache compatibility as it uses a mutual project object under the hood that is discouraged to use in some use cases (e.g. at execution time)

It always breaks compatibility with --configure-on-demand

using xpackProject uses the project of the :x-pack project. referencing other project objects from other subproject should avoided where possible to decouple (sub project configurations). There's a good explanation of why we want to decouple our project configurations as much as possible here: https://docs.gradle.org/current/userguide/multi_project_configuration_and_execution.html#sec:decoupled_projects

it adds little value over default out of the box gradle api (just use project(':x-pack:someProject') instead of xpackProject('someProject') Also in some occasions its even shorter. e.g. when this is used as xpackProject('someProject').path instead of just passing :x-pack:someProject

I'll try to put a bit more context in the PR description in the future to make the motivation behind these kind of changes more clear upfront

Related to #57918
breskeby added a commit to breskeby/elasticsearch that referenced this issue Jun 27, 2022
This is intended to help us getting closer to elastic#57918 by implicitly testing our build logic
configuration-cache support. Plugin and Task tests can be marked as configuration cache compatible
now and we will always run then with configuration cache enabled.

By default, gradle will fail the build if configuration cache problems have been detected during
build execution. That should be in general better then adding explicit tests for testing configuration
cache compatibility per Test class
elasticsearchmachine pushed a commit that referenced this issue Jun 27, 2022
)

This is intended to help us getting closer to #57918 by implicitly
testing our build logic configuration-cache support. Plugin and Task
tests can be marked as configuration cache compatible now and we will
always run then with configuration cache enabled.

By default, gradle will fail the build if configuration cache problems
have been detected during build execution. That should be in general
better then adding explicit tests for testing configuration cache
compatibility per Test class
breskeby added a commit to breskeby/elasticsearch that referenced this issue Jul 4, 2022
TaskExecutionListener are deprecated in Gradle and we want to move away from them
to be able to take advantage of the gradle configuration cache (see elastic#57918)

This removes the listener used for shutting down test clusters. We want to remove more
but replacing logic in beforeActions is not as straight forward yet with Gradle 7.4
breskeby added a commit that referenced this issue Jul 11, 2022
This changes the LoggedExec task to be configuration cache compatible. We changed the implementation
to use `ExecOperations` instead of extending `Exec` task. As double checked with the Gradle team this task
is not planned to be made configuration cache compatible out of the box anytime soon.

This is part of the effort on #57918
breskeby added a commit to breskeby/elasticsearch that referenced this issue Jul 15, 2022
…stic#88047)

This is intended to help us getting closer to elastic#57918 by implicitly
testing our build logic configuration-cache support. Plugin and Task
tests can be marked as configuration cache compatible now and we will
always run then with configuration cache enabled.

By default, gradle will fail the build if configuration cache problems
have been detected during build execution. That should be in general
better then adding explicit tests for testing configuration cache
compatibility per Test class
# Conflicts:
#	build-tools-internal/src/integTest/groovy/org/elasticsearch/gradle/fixtures/AbstractGradleInternalPluginFuncTest.groovy
#	build-tools-internal/src/integTest/groovy/org/elasticsearch/gradle/internal/precommit/LicenseHeadersPrecommitPluginFuncTest.groovy
#	build-tools-internal/src/integTest/groovy/org/elasticsearch/gradle/internal/precommit/TestingConventionsPrecommitPluginFuncTest.groovy
breskeby added a commit that referenced this issue Jul 15, 2022
) (#88569)

This is intended to help us getting closer to #57918 by implicitly
testing our build logic configuration-cache support. Plugin and Task
tests can be marked as configuration cache compatible now and we will
always run then with configuration cache enabled.

By default, gradle will fail the build if configuration cache problems
have been detected during build execution. That should be in general
better then adding explicit tests for testing configuration cache
compatibility per Test class
@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-delivery (Team:Delivery)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
:Delivery/Build Build or test infrastructure Team:Delivery Meta label for Delivery team
Projects
None yet
Development

No branches or pull requests

4 participants