[main] Copy v21.0.0-RC2 release notes (#17048)

Signed-off-by: Rohit Nayak <[email protected]>
Co-authored-by: Rohit Nayak <[email protected]>

changelog/21.0/21.0.0/changelog.md

@@ -6,7 +6,8 @@
 #### Backup and Restore
  * fixing issue with xtrabackup and long gtids [#16304](https://github.com/vitessio/vitess/pull/16304)
  * Fix `xtrabackup`/`builtin` context when doing `AddFiles` [#16806](https://github.com/vitessio/vitess/pull/16806)
- * Fail fast when builtinbackup fails to restore a single file [#16856](https://github.com/vitessio/vitess/pull/16856) 
+ * Fail fast when builtinbackup fails to restore a single file [#16856](https://github.com/vitessio/vitess/pull/16856)
+ * [release-21.0] fix releasing the global read lock when mysqlshell backup fails (#17000) [#17012](https://github.com/vitessio/vitess/pull/17012) 
 #### Build/CI
  * Fix echo command in build docker images workflow [#16350](https://github.com/vitessio/vitess/pull/16350)
  * CI: Remove build docker images CI action until we figure out if that it is affecting our rate limits [#16759](https://github.com/vitessio/vitess/pull/16759)
@@ -63,7 +64,8 @@
  * VTTablet: smartconnpool: notify all expired waiters [#16897](https://github.com/vitessio/vitess/pull/16897)
  * [release-21.0] fix issue with json unmarshalling of operators with space in them [#16933](https://github.com/vitessio/vitess/pull/16933)
  * [release-21.0] VTGate MoveTables Buffering: Fix panic when buffering is disabled (#16922) [#16936](https://github.com/vitessio/vitess/pull/16936)
- * [release-21.0] fixes bugs around expression precedence and LIKE (#16934) [#16947](https://github.com/vitessio/vitess/pull/16947) 
+ * [release-21.0] fixes bugs around expression precedence and LIKE (#16934) [#16947](https://github.com/vitessio/vitess/pull/16947)
+ * [release-21.0] [Direct PR] bugfix: add HAVING columns inside derived tables (#16976) [#16980](https://github.com/vitessio/vitess/pull/16980) 
 #### Schema Tracker
  * Log and ignore the error in reading udfs in schema tracker [#16630](https://github.com/vitessio/vitess/pull/16630) 
 #### Throttler
@@ -86,15 +88,18 @@
  * VTGate VStream: Ensure reasonable delivery time for reshard journal event  [#16639](https://github.com/vitessio/vitess/pull/16639)
  * Workflow Status: change logic to determine whether `MoveTables` writes are switched [#16731](https://github.com/vitessio/vitess/pull/16731)
  * Migrate Workflow: Scope vindex names correctly when target and source keyspace have different names [#16769](https://github.com/vitessio/vitess/pull/16769)
- * VReplication: Support both vindex col def formats when initing target sequences [#16862](https://github.com/vitessio/vitess/pull/16862) 
+ * VReplication: Support both vindex col def formats when initing target sequences [#16862](https://github.com/vitessio/vitess/pull/16862)
+ * [release-21.0] fix: Infinite logs in case of non-existent stream logs (#17004) [#17014](https://github.com/vitessio/vitess/pull/17014)
+ * [release-21.0] VReplication: Support reversing read-only traffic in vtctldclient (#16920) [#17015](https://github.com/vitessio/vitess/pull/17015) 
 #### VTAdmin
  * VTAdmin: Upgrade websockets js package [#16504](https://github.com/vitessio/vitess/pull/16504)
  * VTAdmin: Fix serve-handler's path-to-regexp dep and add default schema refresh [#16778](https://github.com/vitessio/vitess/pull/16778) 
 #### VTCombo
  * vtcombo: close query service on drop database [#16606](https://github.com/vitessio/vitess/pull/16606) 
 #### VTGate
  * Fix `RemoveTablet` during `TabletExternallyReparented` causing connection issues [#16371](https://github.com/vitessio/vitess/pull/16371)
- * Fix panic in schema tracker in presence of keyspace routing rules [#16383](https://github.com/vitessio/vitess/pull/16383) 
+ * Fix panic in schema tracker in presence of keyspace routing rules [#16383](https://github.com/vitessio/vitess/pull/16383)
+ * [release-21.0] Fix deadlock between health check and topology watcher (#16995) [#17010](https://github.com/vitessio/vitess/pull/17010) 
 #### VTTablet
  * Fix race in `replicationLagModule` of `go/vt/throttle` [#16078](https://github.com/vitessio/vitess/pull/16078) 
 #### VTorc
@@ -119,7 +124,8 @@
  * CI: Use valid tag for the ossf-scorecard action [#16730](https://github.com/vitessio/vitess/pull/16730)
  * `endtoend`: better error reporting in Online DDL MySQL execution tests [#16786](https://github.com/vitessio/vitess/pull/16786)
  * Docker Images Build CI action: add updated version from release-20.0 back in prep for v21 release [#16799](https://github.com/vitessio/vitess/pull/16799)
- * Use `go-version-file: go.mod` in CI  [#16841](https://github.com/vitessio/vitess/pull/16841) 
+ * Use `go-version-file: go.mod` in CI  [#16841](https://github.com/vitessio/vitess/pull/16841)
+ * [release-21.0] Flakes: Address flakes in TestMoveTables* unit tests (#16942) [#16951](https://github.com/vitessio/vitess/pull/16951) 
 #### Docker
  * Docker: Update node vtadmin version [#16147](https://github.com/vitessio/vitess/pull/16147)
  * Fix `docker_lite_push` make target [#16662](https://github.com/vitessio/vitess/pull/16662) 
@@ -134,7 +140,8 @@
  * CI: wait-for rather than 'assume' in Online DDL flow [#16210](https://github.com/vitessio/vitess/pull/16210)
  * A couple changes in Online DDL CI [#16272](https://github.com/vitessio/vitess/pull/16272) 
 #### VReplication
- * CI: Lower resources used for TestVtctldMigrateSharded [#16875](https://github.com/vitessio/vitess/pull/16875) 
+ * CI: Lower resources used for TestVtctldMigrateSharded [#16875](https://github.com/vitessio/vitess/pull/16875)
+ * [release-21.0] VReplication: Restore previous minimal e2e test behavior (#17016) [#17017](https://github.com/vitessio/vitess/pull/17017) 
 #### VTAdmin
  * Update micromatch to 4.0.8 [#16660](https://github.com/vitessio/vitess/pull/16660) 
 #### VTGate
@@ -158,7 +165,8 @@
 #### Documentation
  * Changelog 20.0: Fix broken links [#16048](https://github.com/vitessio/vitess/pull/16048)
  * copy editing changes to summary [#16880](https://github.com/vitessio/vitess/pull/16880)
- * Add blurb about experimental 8.4 support [#16886](https://github.com/vitessio/vitess/pull/16886) 
+ * Add blurb about experimental 8.4 support [#16886](https://github.com/vitessio/vitess/pull/16886)
+ * [release-21.0] Add missing changelog for PR #16852 (#17002) [#17006](https://github.com/vitessio/vitess/pull/17006) 
 #### General
  * release notes: update dml related release notes [#16241](https://github.com/vitessio/vitess/pull/16241) 
 #### VReplication
@@ -167,13 +175,15 @@
  * clarify collations are also supported for `db_charset` [#16423](https://github.com/vitessio/vitess/pull/16423)
 ### Enhancement 
 #### Build/CI
- * Improve the queries upgrade/downgrade CI workflow by using same test code version as binary [#16494](https://github.com/vitessio/vitess/pull/16494) 
+ * Improve the queries upgrade/downgrade CI workflow by using same test code version as binary [#16494](https://github.com/vitessio/vitess/pull/16494)
+ * [release-21.0] Change upgrade test to still use the older version of tests (#16937) [#16970](https://github.com/vitessio/vitess/pull/16970) 
 #### CLI
  * Support specifying expected primary in ERS/PRS [#16852](https://github.com/vitessio/vitess/pull/16852) 
 #### Cluster management
  * Prefer replicas that have innodb buffer pool populated in PRS [#16374](https://github.com/vitessio/vitess/pull/16374)
  * Allow cross cell promotion in PRS [#16461](https://github.com/vitessio/vitess/pull/16461)
- * Fix: Errant GTID detection on the replicas when they set replication source [#16833](https://github.com/vitessio/vitess/pull/16833) 
+ * Fix: Errant GTID detection on the replicas when they set replication source [#16833](https://github.com/vitessio/vitess/pull/16833)
+ * [release-21.0] [Direct PR] Add RPC to read the reparent journal position [#16982](https://github.com/vitessio/vitess/pull/16982) 
 #### Docker
  * Remove the `bootstrap` dependency on the Docker images we ship [#16339](https://github.com/vitessio/vitess/pull/16339) 
 #### Evalengine
@@ -245,7 +255,8 @@
  * VTAdmin: Initiate workflow details tab [#16570](https://github.com/vitessio/vitess/pull/16570)
  * VTAdmin: Workflow status endpoint [#16587](https://github.com/vitessio/vitess/pull/16587)
  * VTAdmin(API): Add workflow start/stop endpoint [#16658](https://github.com/vitessio/vitess/pull/16658)
- * VTAdmin: Distributed transactions list on VTAdmin [#16793](https://github.com/vitessio/vitess/pull/16793) 
+ * VTAdmin: Distributed transactions list on VTAdmin [#16793](https://github.com/vitessio/vitess/pull/16793)
+ * [Release 21.0] [Direct PR] Conclude txn in VTAdmin and `GetUnresolvedTransactions` bug fix [#16949](https://github.com/vitessio/vitess/pull/16949) 
 #### VTCombo
  * VTCombo: Ensure VSchema exists when creating keyspace [#16094](https://github.com/vitessio/vitess/pull/16094) 
 #### VTGate
@@ -269,7 +280,7 @@
  * VTAdmin(web): Add workflow start/stop actions [#16675](https://github.com/vitessio/vitess/pull/16675)
  * VTAdmin(web): Some tweaks in the workflow details [#16706](https://github.com/vitessio/vitess/pull/16706)
  * VTAdmin(web): Initiate MoveTables workflow create screen [#16707](https://github.com/vitessio/vitess/pull/16707)
-### Feature Request 
+### Feature 
 #### Backup and Restore
  * adding new mysql shell backup engine [#16295](https://github.com/vitessio/vitess/pull/16295)
  * select backup engine in Backup() and ignore engines in RestoreFromBackup() [#16428](https://github.com/vitessio/vitess/pull/16428) 
@@ -388,7 +399,10 @@
  * fix: order by subquery planning [#16049](https://github.com/vitessio/vitess/pull/16049)
  * feat: add a LIMIT 1 on EXISTS subqueries to limit network overhead [#16153](https://github.com/vitessio/vitess/pull/16153)
  * bugfix: Allow cross-keyspace joins [#16520](https://github.com/vitessio/vitess/pull/16520)
+ * [release-21.0] [Direct PR] fix: route engine to handle column truncation for execute after lookup (#16981) [#16986](https://github.com/vitessio/vitess/pull/16986)
 ### Release 
+#### Build/CI
+ * [release-21.0] Fix the release workflow (#16964) [#17020](https://github.com/vitessio/vitess/pull/17020) 
 #### General
  * [main] Copy `v20.0.0-RC1` release notes [#16140](https://github.com/vitessio/vitess/pull/16140)
  * [main] Copy `v20.0.0-RC2` release notes [#16234](https://github.com/vitessio/vitess/pull/16234)
@@ -400,14 +414,17 @@
  * [main] Copy `v19.0.6` release notes [#16752](https://github.com/vitessio/vitess/pull/16752)
  * [main] Copy `v20.0.2` release notes [#16755](https://github.com/vitessio/vitess/pull/16755)
  * [release-21.0] Code Freeze for `v21.0.0-RC1` [#16912](https://github.com/vitessio/vitess/pull/16912)
+ * [release-21.0] Release of `v21.0.0-RC1` [#16950](https://github.com/vitessio/vitess/pull/16950)
+ * [release-21.0] Bump to `v21.0.0-SNAPSHOT` after the `v21.0.0-RC1` release [#16955](https://github.com/vitessio/vitess/pull/16955)
 ### Testing 
 #### Build/CI
  * Online DDL flow CI: Update golang version to 1.22.4 [#16066](https://github.com/vitessio/vitess/pull/16066)
  * Run the static checks workflow if the file changes [#16659](https://github.com/vitessio/vitess/pull/16659)
  * Fix error contain checks in vtgate package [#16672](https://github.com/vitessio/vitess/pull/16672)
  * Don't show skipped tests in summary action [#16859](https://github.com/vitessio/vitess/pull/16859) 
 #### Cluster management
- * Add semi-sync plugin test in main [#16372](https://github.com/vitessio/vitess/pull/16372) 
+ * Add semi-sync plugin test in main [#16372](https://github.com/vitessio/vitess/pull/16372)
+ * [release-21.0] Flaky test fixes (#16940) [#16960](https://github.com/vitessio/vitess/pull/16960) 
 #### General
  * CI Summary Addition [#16143](https://github.com/vitessio/vitess/pull/16143)
  * Add Summary in unit-race workflow [#16164](https://github.com/vitessio/vitess/pull/16164)

changelog/21.0/21.0.0/release_notes.md

@@ -24,6 +24,7 @@
     - **[Experimental MySQL 8.4 support](#experimental-mysql-84)**
     - **[Current Errant GTIDs Count Metric](#errant-gtid-metric)**
     - **[vtctldclient ChangeTabletTags](#vtctldclient-changetablettags)**
+    - **[Support for specifying expected primary in reparents](#reparents-expectedprimary)**
 
 
 ## <a id="major-changes"/>Major Changes</a>
@@ -236,10 +237,14 @@ This metric shows the current count of the errant GTIDs in the tablets.
 
 The `vtctldclient` command `ChangeTabletTags` was added to allow the tags of a tablet to be changed dynamically.
 
+### <a id="reparents-expectedprimary"/>Support specifying expected primary in reparents
+
+The `EmergencyReparentShard` and `PlannedReparentShard` commands and RPCs now support specifying a primary we expect to still be the current primary in order for a reparent operation to be processed. This allows reparents to be conditional on a specific state being true.
+
 ------------
 The entire changelog for this release can be found [here](https://github.com/vitessio/vitess/blob/main/changelog/21.0/21.0.0/changelog.md).
 
-The release includes 338 merged Pull Requests.
+The release includes 354 merged Pull Requests.
 
 Thanks to all our contributors: @GrahamCampbell, @GuptaManan100, @Utkar5hM, @anshikavashistha, @app/dependabot, @app/vitess-bot, @arthurschreiber, @beingnoble03, @brendar, @cameronmccord2, @chrism1001, @cuishuang, @dbussink, @deepthi, @demmer, @frouioui, @harshit-gangal, @harshitasao, @icyflame, @kirtanchandak, @mattlord, @mattrobenolt, @maxenglander, @mcrauwel, @notfelineit, @perminov, @rafer, @rohit-nayak-ps, @runewake2, @rvrangel, @shanth96, @shlomi-noach, @systay, @timvaillancourt, @vitess-bot
 

changelog/21.0/21.0.0/summary.md

@@ -18,7 +18,6 @@
     - **[Dynamic VReplication Configuration](#dynamic-vreplication-configuration)**
     - **[Reference Table Materialization](#reference-table-materialization)**
     - **[New VEXPLAIN Modes: TRACE and KEYS](#new-vexplain-modes)**
-    - **[Errant GTID Detection on VTTablets](#errant-gtid-vttablet)**
     - **[Automatically Replace MySQL auto_increment Clauses with Vitess Sequences](#auto-replace-mysql-autoinc-with-seq)**
     - **[Experimental MySQL 8.4 support](#experimental-mysql-84)**
     - **[Current Errant GTIDs Count Metric](#errant-gtid-metric)**
@@ -208,14 +207,6 @@ filter columns (potential candidates for indexes, primary keys, or sharding keys
 These new `VEXPLAIN` modes enhance Vitess's query analysis capabilities, allowing for more informed decisions about sharding 
 strategies and query optimization.
 
-### <a id="errant-gtid-vttablet"/>Errant GTID Detection on VTTablets</a>
-
-VTTablets now run an errant GTID detection logic before they join the replication stream. So, if a replica has an errant GTID, it will
-not start replicating from the primary. This protects us from running into situations which are very difficult to recover from.
-
-For users running with the vitess-operator on Kubernetes, this change means that replica tablets with errant GTIDs will have broken 
-replication and will report as unready. Users will need to manually replace and clean up these errant replica tablets.
-
 ### <a id="auto-replace-mysql-autoinc-with-seq"/>Automatically Replace MySQL auto_increment Clauses with Vitess Sequences</a>
 
 In https://github.com/vitessio/vitess/pull/16860 we added support for replacing MySQL `auto_increment` clauses with [Vitess Sequences](https://vitess.io/docs/reference/features/vitess-sequences/), performing all of the setup and initialization