Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[YSQL] [Unit Test] Test testUpdateTableIndexWrites is failing with assertion error #18729

Open
1 task done
rjalan-yb opened this issue Aug 17, 2023 · 0 comments
Open
1 task done

[YSQL] [Unit Test] Test testUpdateTableIndexWrites is failing with assertion error #18729

rjalan-yb opened this issue Aug 17, 2023 · 0 comments
Assignees
@ttyusupov
Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature kind/failing-test Tests and testing infra priority/medium Medium priority issue

Comments

@rjalan-yb
Copy link
Contributor

rjalan-yb commented Aug 17, 2023 •
edited by jira bot
Loading

Jira Link: DB-7625

Description

Unit Test org.yb.pgsql.TestPgIndexSelectiveUpdate.testUpdateTableIndexWrites has been failing very frequently after this commit caf5ba7

Error:

00:54:41.426 (Time-limited test) [INFO - org.yb.pgsql.ConnectionBuilder.connect(ConnectionBuilder.java:219)] Postgres is still starting up, waiting for 500 ms. Got message: Connection to 127.228.70.235:23625 refused. Check that the hostname and port are correct and that the postmaster is accepting TCP/IP connections.
00:54:41.426 (Time-limited test) [INFO - org.yb.pgsql.ConnectionBuilder.connect(ConnectionBuilder.java:234)] Waiting for 500ms. 
00:54:46.399 (yb-nio-0) [DEBUG - org.yb.client.AsyncYBClient.removeClientFromCache(AsyncYBClient.java:2979)] Removed from IP cache: {127.194.150.236:18185} -> {TabletClient@1404905734(chan=[id: 0xc915cf5b, L:/127.0.0.1:48628 ! R:/127.194.150.236:18185], uuid=YB Master - 127.194.150.236:18185, #pending_rpcs=0, #rpcs_inflight=0)}
00:55:18.901 (Time-limited test) [ERROR - org.yb.AssertionWrappers.logError(AssertionWrappers.java:40)] Assertion error:
java.lang.AssertionError: expected:<3> but was:<2>

Complete Logs: https://jenkins.dev.yugabyte.com/job/github-yugabyte-db-alma8-master-clang16-debug-aarch64/583/testReport/junit/org.yb.pgsql/TestPgIndexSelectiveUpdate/testUpdateTableIndexWrites/

Warning: Please confirm that this issue does not contain any sensitive information

  • I confirm this issue does not contain any sensitive information.
@rjalan-yb rjalan-yb added area/ysql Yugabyte SQL (YSQL) area/unittest tracking platform unit test related issues status/awaiting-triage Issue awaiting triage labels Aug 17, 2023
@yugabyte-ci yugabyte-ci added kind/enhancement This is an enhancement of an existing feature priority/medium Medium priority issue labels Aug 17, 2023
@yugabyte-ci yugabyte-ci added kind/failing-test Tests and testing infra and removed area/unittest tracking platform unit test related issues status/awaiting-triage Issue awaiting triage labels Aug 24, 2023
karthik-ramanathan-3006 added a commit that referenced this issue Sep 23, 2024
@karthik-ramanathan-3006
[#18152, #18729] Docdb: Fix test TestPgIndexSelectiveUpdate
6fa33e6 
Summary:
D36588 introduced an optimization to convert updates to included columns of indexes from requiring a DELETE + INSERT RPC to just a single UPDATE RPC.
Furthermore, this optimization allows these index updates to be performed over a single flush to DocDB rather than requiring the previous two.

The reduction in flushes has an impact on the DocDB metric: `intentsdb_rocksdb_write_self` for the index table.
An update to an included column of an index now requires only 2 write batches to be written out to `intentsdb`:
1. The intent to update the column's value. Internal to DocDB/RocksDB, this includes the intents to delete the old value and insert the new value.
2. The intent to delete the intents mentioned in (1) once the operation has been committed to `regulardb`.

Previously, this operation would have needed 3 writes batches:
1. The intent to delete the old value of the column.
2. The intent to insert the new value of the column.
3. The intent to delete the intents mentioned in (1) and (2) once the operation has been committed to `regulardb`.

(1) and (2) were previously distinct because they happened over two separate flushes to DocDB which implies that they were part of two different write batches.

Note - The cases where the metric was previously > 3, was due to the fact that more flushes (> 2) were needed to execute the query. These queries now just need 1 flush.

This revision updates the `org.yb.pgsql.TestPgIndexSelectiveUpdate` with the new expectation for the metric `intentsdb_rocksdb_write_self`.
Jira: DB-7192, DB-7625

Test Plan:
```
./yb_build.sh --java-test 'org.yb.pgsql.TestPgIndexSelectiveUpdate'
```

Reviewers: tnayak

Reviewed By: tnayak

Subscribers: smishra, yql

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D38234
foucher pushed a commit that referenced this issue Sep 24, 2024
@yugabyte-ci @foucher
[BACKPORT pg15-cherrypicks] all: Bulk port from master - 107
218324e 
Summary:
 5d3e83e [PLAT-15199] Change TP API URLs according to latest refactoring
 a50a730 [doc][yba] YBDB compatibility (#23984)
 0c84dbe [#24029] Update the callhome diagnostics  not to send gflags details.
 b53ed3a [PLAT-15379][Fix PLAT-12510] Option to use UTC when dealing with cron exp. in backup schedule
 f0eab8f [PLAT-15278]: Fix DB Scoped XCluster replication restart
 344bc76 Revert "[PLAT-15379][Fix PLAT-12510] Option to use UTC when dealing with cron exp. in backup schedule"
 3628ba7 [PLAT-14459] Swagger fix
 bb93ebe [#24021] YSQL: Add --TEST_check_catalog_version_overflow
 9ab7806 [#23927] docdb: Add gflag for minimum thread stack size
 Excluded: 8c8adc0 [#18822] YSQL: Gate update optimizations behind preview flag
 5e86515 [#23768] YSQL: Fix table rewrite DDL before slot creation
 123d496 [PLAT-14682] Universe task should only unlock itself and make unlock aware of the lock config
 de9d4ad [doc][yba] CIS hardened OS support (#23789)
 e131b20 [#23998] DocDB: Update usearch and other header-only third-party dependencies
 1665662 Automatic commit by thirdparty_tool: update usearch to commit 240fe9c298100f9e37a2d7377b1595be6ba1f412.
 3adbdae Automatic commit by thirdparty_tool: update fp16 to commit 98b0a46bce017382a6351a19577ec43a715b6835.
 9a819f7 Automatic commit by thirdparty_tool: update hnswlib to commit 2142dc6f4dd08e64ab727a7bbd93be7f732e80b0.
 2dc58f4 Automatic commit by thirdparty_tool: update simsimd to tag v5.1.0.
 9a03432 [doc][ybm] Azure private link host (#24086)
 039c9a2 [#17378] YSQL: Testing for histogram_bounds in pg_stats
 09f7a0f [#24085] DocDB: Refactor HNSW wrappers
 555af7d [#24000] DocDB: Shutting down shared exchange could cause TServer to hang
 5743a03 [PLAT-15317]Alert emails are not in the correct format.
 8642555 [PLAT-15379][Fix PLAT-12510] Option to use UTC when dealing with cron exp. in backup schedule
 253ab07 [PLAT-15400][PLAT-15401][PLAT-13051] - Connection pooling ui issues and other ui issues
 57576ae [#16487] YSQL: Fix flakey TestPostgresPid test
 bc8ae45 Update ports for CIS hardened (#24098)
 6fa33e6 [#18152, #18729] Docdb: Fix test TestPgIndexSelectiveUpdate
 cc6d2d1 [docs] added and updated cves (#24046)
 Excluded: ed153dc [#24055] YSQL: fix pg_hint_plan regression with executing prepared statement

Test Plan: Jenkins: rebase: pg15-cherrypicks

Reviewers: jason, jenkins-bot

Differential Revision: https://phorge.dev.yugabyte.com/D38322
karthik-ramanathan-3006 added a commit that referenced this issue Sep 30, 2024
@karthik-ramanathan-3006
[BACKPORT 2024.2][#18152, #18729] Docdb: Fix test TestPgIndexSelectiv…
076ea48 
…eUpdate

Summary:
D36588 introduced an optimization to convert updates to included columns of indexes from requiring a DELETE + INSERT RPC to just a single UPDATE RPC.
Furthermore, this optimization allows these index updates to be performed over a single flush to DocDB rather than requiring the previous two.

The reduction in flushes has an impact on the DocDB metric: `intentsdb_rocksdb_write_self` for the index table.
An update to an included column of an index now requires only 2 write batches to be written out to `intentsdb`:
1. The intent to update the column's value. Internal to DocDB/RocksDB, this includes the intents to delete the old value and insert the new value.
2. The intent to delete the intents mentioned in (1) once the operation has been committed to `regulardb`.

Previously, this operation would have needed 3 writes batches:
1. The intent to delete the old value of the column.
2. The intent to insert the new value of the column.
3. The intent to delete the intents mentioned in (1) and (2) once the operation has been committed to `regulardb`.

(1) and (2) were previously distinct because they happened over two separate flushes to DocDB which implies that they were part of two different write batches.

Note - The cases where the metric was previously > 3, was due to the fact that more flushes (> 2) were needed to execute the query. These queries now just need 1 flush.

This revision updates the `org.yb.pgsql.TestPgIndexSelectiveUpdate` with the new expectation for the metric `intentsdb_rocksdb_write_self`.
Jira: DB-7192, DB-7625

Original commit: 6fa33e6 / D38234

Test Plan:
```
./yb_build.sh --java-test 'org.yb.pgsql.TestPgIndexSelectiveUpdate'
```

Reviewers: tnayak

Reviewed By: tnayak

Subscribers: yql, smishra

Tags: #jenkins-ready

Differential Revision: https://phorge.dev.yugabyte.com/D38507
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@ttyusupov ttyusupov

Labels
area/ysql Yugabyte SQL (YSQL) kind/enhancement This is an enhancement of an existing feature kind/failing-test Tests and testing infra priority/medium Medium priority issue
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@ttyusupov @yugabyte-ci @rjalan-yb