Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Failing test: APM API Integration tests (trial).x-pack/test/apm_api_integration/tests/index·ts - APM API tests correlations/failed_transactions.spec.ts trial 8.0.0 "before all" hook: runBefore in "8.0.0" #176544

Closed
kibanamachine opened this issue Feb 8, 2024 · 5 comments · Fixed by #187444 or #188836
Assignees
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team

Comments

@kibanamachine
Copy link
Contributor

kibanamachine commented Feb 8, 2024

A test failed on a tracked branch

AggregateError: 
    Error: Bulk doc failure [operation=index]:
      doc: {"@timestamp":"2020-08-26T11:27:12.868Z","agent":{"name":"python","version":"5.8.1"},"client":{"ip":"10.52.6.50"},"cloud":{"availability_zone":"europe-west1-c","instance":{"id":"6890738630140009699","name":"gke-dev-next-oblt-dev-next-oblt-pool-404d7f0c-qmhb"},"machine":{"type":"projects/8560181848/machineTypes/n1-standard-4"},"project":{"id":"8560181848","name":"elastic-observability"},"provider":"gcp","region":"europe-west1"},"container":{"id":"a81887ae6d1077b68f8680035c0df55753a6144c1aa03cab45bc785c10761a70"},"ecs":{"version":"1.5.0"},"event":{"ingested":"2020-08-26T11:27:20.647673Z","outcome":"success"},"host":{"architecture":"x86_64","ip":"10.52.6.49","os":{"platform":"linux"}},"http":{"request":{"env":{"REMOTE_ADDR":"10.52.6.50","SERVER_NAME":"0.0.0.0","SERVER_PORT":"3000"},"headers":{"Connection":["close"],"Content-Length":["0"],"Elastic-Apm-Traceparent":["00-190d10e860196576e11060edc2a60baf-2b65ed4567cf2802-01"],"Host":["opbeans:3000"],"Traceparent":["00-190d10e860196576e11060edc2a60baf-2b65ed4567cf2802-01"],"User-Agent":["http.rb/4.4.1"]},"method":"get","socket":{"encrypted":false,"remote_address":"10.52.6.50"}},"response":{"headers":{"Content-Length":["128"],"Content-Type":["application/json"],"Vary":["Cookie"],"X-Content-Type-Options":["nosniff"],"X-Frame-Options":["DENY"]},"status_code":200}},"kubernetes":{"pod":{"name":"opbeans-python-6b8757c7-zb2hn","uid":"8dedd6b1-e741-11ea-819e-42010a84004a"}},"labels":{"customer_tier":"$","request_id":"6a8be21e-63b1-4283-9f47-5e4e6aa825e2","served_from_cache":false},"observer":{"ephemeral_id":"a6feea29-be28-45f1-9ba9-58ebcfd959d9","hostname":"apm-apm-server-74f49577df-mhfml","id":"60768289-3c4e-4d8b-ae70-122a0ef437d1","type":"apm-server","version":"8.0.0","version_major":8},"parent":{"id":"2b65ed4567cf2802"},"process":{"args":["/app/venv/bin/gunicorn","--workers","4","--bind","0.0.0.0:3000","opbeans.wsgi"],"pid":35,"ppid":24},"processor":{"event":"transaction","name":"transaction"},"service":{"environment":"production","framework":{"name":"django","version":"3.0.7"},"language":{"name":"python","version":"3.6.12"},"name":"opbeans-python","node":{"name":"a81887ae6d1077b68f8680035c0df55753a6144c1aa03cab45bc785c10761a70"},"runtime":{"name":"CPython","version":"3.6.12"},"version":"2020-08-26 02:09:21"},"source":{"ip":"10.52.6.50"},"timestamp":{"us":1598441232868836},"trace":{"id":"190d10e860196576e11060edc2a60baf"},"transaction":{"duration":{"us":126723},"id":"7f25786cfb619929","name":"GET opbeans.views.stats","result":"HTTP 2xx","sampled":true,"span_count":{"dropped":0,"started":6},"type":"request"},"url":{"domain":"opbeans","full":"http://opbeans:3000/api/stats","path":"/api/stats","port":3000,"scheme":"http"},"user":{"email":"[email protected]","id":"4","name":"ooneal"},"user_agent":{"device":{"name":"Generic Feature Phone"},"name":"Other","original":"http.rb/4.4.1"}}
      error: {"type":"illegal_argument_exception","reason":"mapper [labels.served_from_cache] cannot be changed from type [keyword] to [boolean]"}
        at Array.map (<anonymous>)
        at indexDocs (index_doc_records_stream.ts:64:13)
        at processTicksAndRejections (node:internal/process/task_queues:95:5)
        at Writable.writev [as _writev] (index_doc_records_stream.ts:88:9)
    at indexDocs (index_doc_records_stream.ts:64:13)
    at processTicksAndRejections (node:internal/process/task_queues:95:5)
    at Writable.writev [as _writev] (index_doc_records_stream.ts:88:9)

First failure: CI Build - main

@kibanamachine kibanamachine added the failed-test A test failure on a tracked branch, potentially flaky-test label Feb 8, 2024
@botelastic botelastic bot added the needs-team Issues missing a team label label Feb 8, 2024
@kibanamachine kibanamachine added the Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team label Feb 8, 2024
@elasticmachine
Copy link
Contributor

Pinging @elastic/obs-ux-infra_services-team (Team:obs-ux-infra_services)

@botelastic botelastic bot removed the needs-team Issues missing a team label label Feb 8, 2024
@kibanamachine
Copy link
Contributor Author

New failure: CI Build - main

@mistic
Copy link
Member

mistic commented Feb 15, 2024

Skipped.

main: 8b4c655

@cauemarcondes cauemarcondes self-assigned this Feb 15, 2024
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this issue Feb 15, 2024
CoenWarmer pushed a commit to CoenWarmer/kibana that referenced this issue Feb 15, 2024
awahab07 pushed a commit to awahab07/kibana that referenced this issue Feb 19, 2024
fkanout pushed a commit to fkanout/kibana that referenced this issue Mar 4, 2024
walterra added a commit that referenced this issue Jul 3, 2024
…187444)

## Summary

Fixes #176544.
Fixes #187421.
Fixes #176119.
Fixes #176425.
Fixes #175855.
Fixes #175911.
Fixes #176780.

Follow up to #186182.

Reenables and stabilizes APM correlations API integration tests.

Review hint: View with the `w=1` flag to ignore whitespace changes:
https://github.com/elastic/kibana/pull/187444/files?w=1

### Checklist

- [x] [Unit or functional
tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)
were updated or added to match the most common scenarios
- [x] [Flaky Test
Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was
used on any tests changed
- [x] This was checked for breaking API changes and was [labeled
appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process)
@kibanamachine kibanamachine reopened this Jul 19, 2024
@kibanamachine
Copy link
Contributor Author

New failure: kibana-on-merge - 8.15

@walterra
Copy link
Contributor

This looks again like the test is failing during setup not during the test itself.

@crespocarlos crespocarlos self-assigned this Jul 22, 2024
crespocarlos added a commit that referenced this issue Jul 24, 2024
fixes [#176425](#176425)
fixes [#176544](#176544)

## Summary

The problem happens intermittently, I've managed to reproduce it a few
times though. I suspect that since the es_archive `8.0.0` does not have
a type definition for the `label.served_from_cache` field, it leaves ES
to determine what's the appropriate type, therefore causing ingestion to
sometimes fail.

The `apm_8.0.0` has a type definition for this field for `transaction`
index
https://github.com/elastic/kibana/blob/main/x-pack/test/apm_api_integration/common/fixtures/es_archiver/apm_8.0.0/mappings.json#L18824,
so I followed the same configuration.

Co-authored-by: Elastic Machine <[email protected]>
kibanamachine pushed a commit to kibanamachine/kibana that referenced this issue Jul 24, 2024
fixes [elastic#176425](elastic#176425)
fixes [elastic#176544](elastic#176544)

## Summary

The problem happens intermittently, I've managed to reproduce it a few
times though. I suspect that since the es_archive `8.0.0` does not have
a type definition for the `label.served_from_cache` field, it leaves ES
to determine what's the appropriate type, therefore causing ingestion to
sometimes fail.

The `apm_8.0.0` has a type definition for this field for `transaction`
index
https://github.com/elastic/kibana/blob/main/x-pack/test/apm_api_integration/common/fixtures/es_archiver/apm_8.0.0/mappings.json#L18824,
so I followed the same configuration.

Co-authored-by: Elastic Machine <[email protected]>
(cherry picked from commit 850aa69)
kibanamachine added a commit that referenced this issue Jul 24, 2024
…189061)

# Backport

This will backport the following commits from `main` to `8.15`:
- [[APM] Fix &#x60;labels.served_from_cache&#x60; type
(#188836)](#188836)

<!--- Backport version: 9.4.3 -->

### Questions ?
Please refer to the [Backport tool
documentation](https://github.com/sqren/backport)

<!--BACKPORT [{"author":{"name":"Carlos
Crespo","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-07-24T12:08:52Z","message":"[APM]
Fix `labels.served_from_cache` type (#188836)\n\nfixes
[#176425](https://github.com/elastic/kibana/issues/176425)\r\nfixes
[#176544](https://github.com/elastic/kibana/issues/176544)\r\n\r\n##
Summary\r\n\r\nThe problem happens intermittently, I've managed to
reproduce it a few\r\ntimes though. I suspect that since the es_archive
`8.0.0` does not have\r\na type definition for the
`label.served_from_cache` field, it leaves ES\r\nto determine what's the
appropriate type, therefore causing ingestion to\r\nsometimes
fail.\r\n\r\nThe `apm_8.0.0` has a type definition for this field for
`transaction`\r\nindex\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/apm_api_integration/common/fixtures/es_archiver/apm_8.0.0/mappings.json#L18824,\r\nso
I followed the same configuration.\r\n\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"850aa6964898adbe4659a748b9b4ebc59e54366b","branchLabelMapping":{"^v8.16.0$":"main","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","backport:prev-minor","ci:project-deploy-observability","Team:obs-ux-infra_services","v8.16.0"],"title":"[APM]
Fix `labels.served_from_cache`
type","number":188836,"url":"https://github.com/elastic/kibana/pull/188836","mergeCommit":{"message":"[APM]
Fix `labels.served_from_cache` type (#188836)\n\nfixes
[#176425](https://github.com/elastic/kibana/issues/176425)\r\nfixes
[#176544](https://github.com/elastic/kibana/issues/176544)\r\n\r\n##
Summary\r\n\r\nThe problem happens intermittently, I've managed to
reproduce it a few\r\ntimes though. I suspect that since the es_archive
`8.0.0` does not have\r\na type definition for the
`label.served_from_cache` field, it leaves ES\r\nto determine what's the
appropriate type, therefore causing ingestion to\r\nsometimes
fail.\r\n\r\nThe `apm_8.0.0` has a type definition for this field for
`transaction`\r\nindex\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/apm_api_integration/common/fixtures/es_archiver/apm_8.0.0/mappings.json#L18824,\r\nso
I followed the same configuration.\r\n\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"850aa6964898adbe4659a748b9b4ebc59e54366b"}},"sourceBranch":"main","suggestedTargetBranches":[],"targetPullRequestStates":[{"branch":"main","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/188836","number":188836,"mergeCommit":{"message":"[APM]
Fix `labels.served_from_cache` type (#188836)\n\nfixes
[#176425](https://github.com/elastic/kibana/issues/176425)\r\nfixes
[#176544](https://github.com/elastic/kibana/issues/176544)\r\n\r\n##
Summary\r\n\r\nThe problem happens intermittently, I've managed to
reproduce it a few\r\ntimes though. I suspect that since the es_archive
`8.0.0` does not have\r\na type definition for the
`label.served_from_cache` field, it leaves ES\r\nto determine what's the
appropriate type, therefore causing ingestion to\r\nsometimes
fail.\r\n\r\nThe `apm_8.0.0` has a type definition for this field for
`transaction`\r\nindex\r\nhttps://github.com/elastic/kibana/blob/main/x-pack/test/apm_api_integration/common/fixtures/es_archiver/apm_8.0.0/mappings.json#L18824,\r\nso
I followed the same configuration.\r\n\r\nCo-authored-by: Elastic
Machine
<[email protected]>","sha":"850aa6964898adbe4659a748b9b4ebc59e54366b"}}]}]
BACKPORT-->

Co-authored-by: Carlos Crespo <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
failed-test A test failure on a tracked branch, potentially flaky-test Team:obs-ux-infra_services Observability Infrastructure & Services User Experience Team
Projects
None yet
6 participants