-
Notifications
You must be signed in to change notification settings - Fork 14.2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[AIRFLOW-6091] Add flushing in execute method for BigQueryCursor #6683
Conversation
ace59d0
to
fcf0c63
Compare
@nuclearpinguin any hints what should I do to fix failing tests? I ran out of ideas :/ |
I restarted the failing job. It's a flaky "run_on_kill" test that @nuclearpinguin is solving with the upcoming #6472 change. |
Codecov Report
@@ Coverage Diff @@
## master #6683 +/- ##
==========================================
- Coverage 84.45% 84.31% -0.14%
==========================================
Files 676 676
Lines 38310 38314 +4
==========================================
- Hits 32354 32304 -50
- Misses 5956 6010 +54
Continue to review full report at Codecov.
|
Can you add tests to prevent regression? |
I will need some hints on how to do it. |
The best way to learn this is to read the source test codes for operators and hooks for other GCP services. All services except BigQuery have one style and the way they are tested. All services except BigQuery have one style and the way they are tested. It is also useful to know unittest.mock. |
I agree with @mik-laj . Analysing some other tests is the best way to learn, preferably play with then locally, break thing and what happened. I am during improving BigQuery hooks test, fell free to check them out here: #6777. |
If you execute multiple queries results of old ones will be flushed allowing to read results of recent execute without any issues.
fcf0c63
to
44bdd67
Compare
If you execute multiple queries results of old ones will be flushed allowing to read results of recent execute without any issues. (cherry picked from commit 0cf9598)
If you execute multiple queries results of old ones will be flushed allowing to read results of recent execute without any issues. (cherry picked from commit 0cf9598)
If you execute multiple queries results of old ones will be flushed allowing to read results of recent execute without any issues. (cherry picked from commit 0cf9598)
…che#6683) If you execute multiple queries results of old ones will be flushed allowing to read results of recent execute without any issues.
If you execute multiple queries results of old ones will be
flushed allowing to read results of recent execute without
any issues.
Make sure you have checked all steps below.
Jira
Description
My PR addresses issue with results buffer and reading flags not being cleaned in some cases what ends up in returning no data or even data from previous job in case of executing many queries with one cursor.
Tests
My PR does not add any functionality to test
Commits
Documentation