Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python tests: compatibility and stability improvements #31588

Merged
merged 1 commit into from
May 12, 2022

Conversation

belimawr
Copy link
Contributor

@belimawr belimawr commented May 11, 2022

What does this PR do?

This PR improves the compatibility and stability of Python tests
for Filebeat. It addresses two main probmes:

  1. Some deprecated TLS functions are now raising deprecation
    exceptions (since Python 3.10.0)
  2. Some tests, on failure, leave Filebeat running and bound to port
    8080, this creates a cascading failure of the subsequent tests that
    require the same port to be available.

The main changes are:

  • Ensuring Filebeat process is always terminated, even if the test
    fails
  • Using the replacement methods/objects for the deprecated TLS
    functions.

Why is it important?

It make the tests less flaky and enables support for newer versions of Python by removing the use of deprecated functions from Python 2.7.x

Checklist

  • My code follows the style guidelines of this project
  • [ ] I have commented my code, particularly in hard-to-understand areas
  • [ ] I have made corresponding changes to the documentation
  • [ ] I have made corresponding change to the default configuration files
  • [ ] I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in CHANGELOG.next.asciidoc or CHANGELOG-developer.next.asciidoc.

## Author's Checklist

How to test this PR locally

On a environment with Python >= 3.9.x (like Arch Linux), run the Python tests from Filebeat:

cd filebeat
mage -v PythonUnitTest

They should run without any issue.

Run tests from filebeat/tests/system/test_syslog.py or filebeat/tests/system/test_tcp_tls.py and force a failure or interrupt them, after they finish there should be not dangling Filebeat process.

## Related issues
## Use cases
## Screenshots
## Logs

@botelastic botelastic bot added the needs_team Indicates that the issue/PR needs a Team:* label label May 11, 2022
@belimawr belimawr added backport-skip Skip notification from the automated backport with mergify Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team labels May 11, 2022
@botelastic botelastic bot removed the needs_team Indicates that the issue/PR needs a Team:* label label May 11, 2022
@belimawr belimawr changed the title [WIP] Fix python tests on Python >=3.9.x Fix python tests on Python >=3.9.x May 11, 2022
@belimawr belimawr added the bug label May 11, 2022
@belimawr belimawr force-pushed the fix-python-tests-for-python-3.9 branch 2 times, most recently from 38b34e7 to cc557e1 Compare May 11, 2022 09:38
@elasticmachine
Copy link
Collaborator

elasticmachine commented May 11, 2022

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS
Pipeline View Test View Changes Artifacts preview preview

Expand to view the summary

Build stats

  • Start Time: 2022-05-12T09:33:00.957+0000

  • Duration: 81 min 23 sec

Test stats 🧪

Test Results
Failed 0
Passed 22556
Skipped 1938
Total 24494

💚 Flaky test report

Tests succeeded.

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

  • /test : Re-trigger the build.

  • /package : Generate the packages and run the E2E tests.

  • /beats-tester : Run the installation tests with beats-tester.

  • run elasticsearch-ci/docs : Re-trigger the docs validation. (use unformatted text in the comment!)

@belimawr belimawr changed the title Fix python tests on Python >=3.9.x Fix python tests on Python >=3.10.10 on Linux May 11, 2022
@belimawr belimawr marked this pull request as ready for review May 11, 2022 16:24
@belimawr belimawr requested a review from a team as a code owner May 11, 2022 16:24
@belimawr belimawr requested review from rdner and kvch and removed request for a team May 11, 2022 16:24
@elasticmachine
Copy link
Collaborator

Pinging @elastic/elastic-agent-data-plane (Team:Elastic-Agent-Data-Plane)

@belimawr belimawr added the Filebeat Filebeat label May 11, 2022
@belimawr belimawr force-pushed the fix-python-tests-for-python-3.9 branch 3 times, most recently from 2b81140 to ff8f60f Compare May 12, 2022 07:26
@belimawr belimawr changed the title Fix python tests on Python >=3.10.10 on Linux python tests: compatibility and stability improvements May 12, 2022
@belimawr belimawr force-pushed the fix-python-tests-for-python-3.9 branch from ff8f60f to ba3a8c2 Compare May 12, 2022 07:31

self.wait_until(lambda: self.log_contains(
"Start accepting connections"))

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

context = ssl.create_default_context(ssl.Purpose.CLIENT_AUTH)
context = ssl.create_default_context(purpose=ssl.Purpose.SERVER_AUTH)
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I keep the ssl.Purpose.CLIENT_AUTH the test fails with ssl.SSLError: Cannot create a client socket with a PROTOCOL_TLS_SERVER context (_ssl.c:801)

@belimawr belimawr requested a review from cmacknz May 12, 2022 07:53
@belimawr
Copy link
Contributor Author

@cmacknz could you review it again? I refactored it to cover more cases and improved the documentation.

This commit improves the compatibility and stability of Python tests
for Filebeat. It addresses two main probmes:
1. Some deprecated TLS functions are now raising deprecation
exceptions (since Python 3.10.0)
2. Some tests, on failure, leave Filebeat running and bound to port
8080, this creates a cascading failure of the subsequent tests that
require the same port to be available.

The main changes are:
 - Ensuring Filebeat process is always terminated, even if the test
 fails
 - Using the replacement methods/objects for the deprecated TLS
 functions.
@belimawr belimawr force-pushed the fix-python-tests-for-python-3.9 branch from 0147c47 to 4989ce0 Compare May 12, 2022 09:32
@belimawr belimawr merged commit 682045a into elastic:main May 12, 2022
@belimawr belimawr deleted the fix-python-tests-for-python-3.9 branch May 12, 2022 14:11
chrisberkhout pushed a commit that referenced this pull request Jun 1, 2023
This commit improves the compatibility and stability of Python tests
for Filebeat. It addresses two main probmes:
1. Some deprecated TLS functions are now raising deprecation
exceptions (since Python 3.10.0)
2. Some tests, on failure, leave Filebeat running and bound to port
8080, this creates a cascading failure of the subsequent tests that
require the same port to be available.

The main changes are:
 - Ensuring Filebeat process is always terminated, even if the test
 fails
 - Using the replacement methods/objects for the deprecated TLS
 functions.
@cmacknz cmacknz added the backport-7.17 Automated backport to the 7.17 branch with mergify label Jun 21, 2023
mergify bot pushed a commit that referenced this pull request Jun 21, 2023
This commit improves the compatibility and stability of Python tests
for Filebeat. It addresses two main probmes:
1. Some deprecated TLS functions are now raising deprecation
exceptions (since Python 3.10.0)
2. Some tests, on failure, leave Filebeat running and bound to port
8080, this creates a cascading failure of the subsequent tests that
require the same port to be available.

The main changes are:
 - Ensuring Filebeat process is always terminated, even if the test
 fails
 - Using the replacement methods/objects for the deprecated TLS
 functions.

(cherry picked from commit 682045a)

# Conflicts:
#	libbeat/tests/system/beat/beat.py
cmacknz added a commit that referenced this pull request Jun 21, 2023
…olang Debian base image (#35840)

* ci: use ubuntu:22 workers (#34315)

(cherry picked from commit a5f15b6)

# Conflicts:
#	.ci/beats-tester.groovy
#	.ci/build-docker-images.groovy
#	.ci/heartbeat-synthetics.groovy
#	x-pack/filebeat/Jenkinsfile.yml
#	x-pack/metricbeat/Jenkinsfile.yml

* remove unexisting files in 7.17 when backporting another PR

* fix merge conflicts

* Upgrade to Go 1.19.10 and handle changes in golang Debian base image (#35780)

* Explicitly install netcat-openbsd everywhere.

The plain netcat package appears to have been removed in the latest
golang:1.19.10 base image.

* Setup a virtualenv in Docker containers.

This fixes the PEP668 error about "externally managed environments"
which prevents modifying the system python installation.

* Revert "Revert "[Automation] Bump Golang version to 1.19.10 (#35751)" (#35784)"

This reverts commit 27b1799.

* Reword CHANGELOG.next.asciidoc

Co-authored-by: subham sarkar <[email protected]>

* Improve PEP668 comments.

* Install latest PyYAML in test containers.

* Add check to see if already in a venv.

* Upgrade pytest and py.

Remove dependency on pytest-otel as it prevented upgrading pytest.

* Check for pre-set virtualenv earlier.

* Always create a fresh virtualenv in Docker.

Remove duplicate dependencies from the docker files.

* Fix metricbeat integration tests.

* Force install Python dependencies for Metricbeat.

---------

Co-authored-by: subham sarkar <[email protected]>
(cherry picked from commit 62374dd)

# Conflicts:
#	dev-tools/mage/pytest.go
#	libbeat/docs/version.asciidoc
#	libbeat/tests/system/requirements.txt
#	libbeat/tests/system/requirements_aix.txt
#	metricbeat/Dockerfile

* Remove extra entry in CHANGELOG.next.asciidoc

* Fix conflict in version.asciidoc

* Fix conflict in requirements.txt

* Fix conflict in requirements_aix.txt

* Fix conflict in metricbeat Dockerfile

* Fix conflict in pytest.go

* Add missing force create virtual env option.

* python tests: compatibility and stability improvements (#31588)

This commit improves the compatibility and stability of Python tests
for Filebeat. It addresses two main probmes:
1. Some deprecated TLS functions are now raising deprecation
exceptions (since Python 3.10.0)
2. Some tests, on failure, leave Filebeat running and bound to port
8080, this creates a cascading failure of the subsequent tests that
require the same port to be available.

The main changes are:
 - Ensuring Filebeat process is always terminated, even if the test
 fails
 - Using the replacement methods/objects for the deprecated TLS
 functions.

(cherry picked from commit 682045a)

# Conflicts:
#	libbeat/tests/system/beat/beat.py

* Remove duplicate entries in CHANGELOG-developer.next.asciidoc

* Resolve conflicts in beat.py

---------

Co-authored-by: Victor Martinez <[email protected]>
Co-authored-by: Craig MacKenzie <[email protected]>
Co-authored-by: Tiago Queiroz <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-7.17 Automated backport to the 7.17 branch with mergify backport-skip Skip notification from the automated backport with mergify bug Filebeat Filebeat Team:Elastic-Agent-Data-Plane Label for the Agent Data Plane team
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants