[HUDI-4695] Fix flaky TestInlineCompaction#testCompactionRetryOnFailureBasedOnTime #6534
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
4 tasks
xushiyan
changed the title
xushiyan
reviewed
Aug 30, 2022
...di-spark-client/src/test/java/org/apache/hudi/table/action/compact/TestInlineCompaction.java
Show resolved
Hide resolved
yihua
added
the
priority:minor
|
going ahead w/ merging to unblock green CI |
TengHuo
pushed a commit
to TengHuo/hudi
that referenced
this pull request
Nov 28, 2022
…ilureBasedOnTime (apache#6534)
neverdizzy
pushed a commit
to neverdizzy/hudi
that referenced
this pull request
Dec 13, 2022
…ilureBasedOnTime (apache#6534) (cherry picked from commit d7a5240)
fengjian428
pushed a commit
to fengjian428/hudi
that referenced
this pull request
Apr 5, 2023
…ilureBasedOnTime (apache#6534)
vinishjail97
added a commit
to vinishjail97/hudi
that referenced
this pull request
Dec 15, 2023
* [HUDI-3984] Remove mandatory check of partiton path for cli command (apache#5458) * [HUDI-3634] Could read empty or partial HoodieCommitMetaData in downstream if using HDFS (apache#5048) Add the differentiated logic of creating immutable file in HDFS by first creating the file.tmp and then renaming the file * [HUDI-3953]Flink Hudi module should support low-level source and sink api (apache#5445) Co-authored-by: jerryyue <[email protected]> * [HUDI-4353] Column stats data skipping for flink (apache#6026) * [HUDI-3505] Add call procedure for UpgradeOrDowngradeCommand (apache#6012) Co-authored-by: superche <[email protected]> * [HUDI-3730] Improve meta sync class design and hierarchies (apache#5854) * [HUDI-3730] Improve meta sync class design and hierarchies (apache#5754) * Implements class design proposed in RFC-55 Co-authored-by: jian.feng <[email protected]> Co-authored-by: jian.feng <[email protected]> * [HUDI-3511] Add call procedure for MetadataCommand (apache#6018) * [HUDI-3730] Add ConfigTool#toMap UT (apache#6035) Co-authored-by: voonhou.su <[email protected]> * [MINOR] Improve variable names (apache#6039) * [HUDI-3116]Add a new HoodieDropPartitionsTool to let users drop table partitions through a standalone job. (apache#4459) Co-authored-by: yuezhang <[email protected]> * [HUDI-4360] Fix HoodieDropPartitionsTool based on refactored meta sync (apache#6043) * [HUDI-3836] Improve the way of fetching metadata partitions from table (apache#5286) Co-authored-by: xicm <[email protected]> * [HUDI-4359] Support show_fs_path_detail command on Call Produce Command (apache#6042) * [HUDI-4356] Fix the error when sync hive in CTAS (apache#6029) * [HUDI-4219] Merge Into when update expression "col=s.col+2" on precombine cause exception (apache#5828) * [HUDI-4357] Support flink 1.15.x (apache#6050) * [HUDI-4152] Flink offline compaction support compacting multi compaction plan at once (apache#5677) * [HUDI-4152] Flink offline compaction allow compact multi compaction plan at once * [HUDI-4152] Fix exception for duplicated uid when multi compaction plan are compacted * [HUDI-4152] Provider UT & IT for compact multi compaction plan * [HUDI-4152] Put multi compaction plans into one compaction plan source * [HUDI-4152] InstantCompactionPlanSelectStrategy allow multi instant by using comma * [HUDI-4152] Add IT for InstantCompactionPlanSelectStrategy * [HUDI-4309] fix spark32 repartition error (apache#6033) * [HUDI-4366] Synchronous cleaning for flink bounded source (apache#6051) * [minor] following 4152, refactor the clazz about plan selection strategy (apache#6060) * [HUDI-4367] Support copyToTable on call (apache#6054) * [HUDI-4335] Bug fixes in AWSGlueCatalogSyncClient post schema evolution. (apache#5995) * fix for updateTableParameters which is not excluding partition columns and updateTableProperties boolean check * Fix - serde parameters getting overrided on table property update * removing stale syncConfig * [HUDI-4276] Reconcile schema-inject null values for missing fields and add new fields (apache#6017) * [HUDI-4276] Reconcile schema-inject null values for missing fields and add new fields. * fix comments Co-authored-by: public (bdcee5037027) <[email protected]> * [HUDI-3500] Add call procedure for RepairsCommand (apache#6053) * [HUDI-2150] Rename/Restructure configs for better modularity (apache#6061) - Move clean related configuration to HoodieCleanConfig - Move Archival related configuration to HoodieArchivalConfig - hoodie.compaction.payload.class move this to HoodiePayloadConfig * [MINOR] Bump xalan from 2.7.1 to 2.7.2 (apache#6062) Bumps xalan from 2.7.1 to 2.7.2. --- updated-dependencies: - dependency-name: xalan:xalan dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> * [HUDI-4324] Remove use_jdbc config from hudi sync (apache#6072) * [HUDI-4324] Remove use_jdbc config from hudi sync * Users should use HIVE_SYNC_MODE instead * [HUDI-3730][RFC-55] Improve hudi-sync classes design and simplify configs (apache#5695) * [HUDI-4146] RFC for Improve Hive/Meta sync class design and hierarchies Co-authored-by: jian.feng <[email protected]> Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4323] Make database table names optional in sync tool (apache#6073) * [HUDI-4323] Make database table names optional in sync tool * Infer from these properties from the table config * [MINOR] Update RFCs status (apache#6078) * [HUDI-4298] When reading the mor table with QUERY_TYPE_SNAPSHOT,Unabl… (apache#5937) * [HUDI-4298] Add test case for reading mor table Signed-off-by: LinMingQiang <[email protected]> * [HUDI-4379] Bump Flink versions to 1.14.5 and 1.15.1 (apache#6080) * [HUDI-4391] Incremental read from archived commits for flink (apache#6096) * [RFC-51] [HUDI-3478] Hudi to support Change-Data-Capture (apache#5436) Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4393] Add marker file for target file when flink merge handle rolls over (apache#6103) * [HUDI-4399][RFC-57] Claim RFC 57 for DeltaStreamer proto support (apache#6112) * [HUDI-4397] Flink Inline Cluster and Compact plan distribute strategy changed from rebalance to hash to avoid potential multiple threads accessing the same file (apache#6106) Co-authored-by: jerryyue <[email protected]> * [MINOR] Disable TestHiveSyncGlobalCommitTool (apache#6119) * [HUDI-4403] Fix the end input metadata for bounded source (apache#6116) * [HUDI-4408] Reuse old rollover file as base file for flink merge handle (apache#6120) * [HUDI-3503] Add call procedure for CleanCommand (apache#6065) * [HUDI-3503] Add call procedure for CleanCommand Co-authored-by: simonssu <[email protected]> * [HUDI-4249] Fixing in-memory `HoodieData` implementation to operate lazily (apache#5855) * [HUDI-4170] Make user can use hoodie.datasource.read.paths to read necessary files (apache#5722) * Rebase codes * Move listFileSlices to HoodieBaseRelation * Fix review * Fix style * Fix bug * Fix file group count issue with metadata partitions (apache#5892) * [HUDI-4098] Support HMS for flink HudiCatalog (apache#6082) * [HUDI-4098]Support HMS for flink HudiCatalog * [HUDI-4409] Improve LockManager wait logic when catch exception (apache#6122) * [HUDI-4065] Add FileBasedLockProvider (apache#6071) * [HUDI-4416] Default database path for hoodie hive catalog (apache#6136) * [HUDI-4372] Enable matadata table by default for flink (apache#6066) * [HUDI-4401] Skip HBase version check (apache#6114) * Disable EmrFS file metadata caching and EMR Spark's data prefetcher feature * [HUDI-4427] Add a computed column IT test (apache#6150) * [HUDI-4146][RFC-55] Update config changes proposal (apache#6162) * [HUDI-3896] Porting Nested Schema Pruning optimization for Hudi's custom Relations (apache#5428) Currently, all Hudi Relations bear performance gap relative to Spark's HadoopFsRelation and the reason to that is SchemaPruning optimization rule (pruning nested schemas) that is unfortunately predicated on usage of HadoopFsRelation, meaning that it's not applied in cases when any other relation is used. This change is porting this rule to Hudi relations (MOR, Incremental, etc) by the virtue of leveraging HoodieSparkSessionExtensions mechanism injecting modified version of the original SchemaPruning rule that is adopted to work w/ Hudi's custom relations. - Added customOptimizerRules to HoodieAnalysis - Added NestedSchemaPrunning Spark's Optimizer rule - Handle Spark's Optimizer pruned data schema (to effectively prune nested schemas) - Enable HoodieClientTestHarness to inject HoodieSparkSessionExtensions - Injecting Spark Session extensions for TestMORDataSource, TestCOWDataSource - Disabled fallback to HadoopFsRelation * [HUDI-3764] Allow loading external configs while querying Hudi tables with Spark (apache#4915) Currently when doing Hudi queries w/ Spark, it won't load the external configurations. Say if customers enabled metadata listing in their global config file, then this would let them actually query w/o metadata feature enabled. This PR fixes this issue and allows loading global configs during the Hudi reading phase. Co-authored-by: Wenning Ding <[email protected]> * [HUDI-3993] Replacing UDF in Bulk Insert w/ RDD transformation (apache#5470) * [MINOR] Add logger for HoodieCopyOnWriteTableInputFormat (apache#6161) Co-authored-by: Wenning Ding <[email protected]> * [HUDI-4400] Fix missing bloom filters in metadata table in non-partitioned table (apache#6113) Fixes the missing bloom filters in metadata table in the non-partitioned table due to incorrect record key generation, because of wrong file names when generating the metadata payload for the bloom filter. * [HUDI-4204] Fixing NPE with row writer path and with OCC (apache#5850) * [HUDI-4247] Upgrading protocol buffers version for presto bundle (apache#5852) * [MINOR] Fix result missing information issue in commits_compare Procedure (apache#6165) Co-authored-by: superche <[email protected]> * [HUDI-4404] Fix insert into dynamic partition write misalignment (apache#6124) * [MINOR] Fallback to default for hive-style partitioning, url-encoding configs (apache#6175) - Fixes broken ITTestHoodieDemo#testParquetDemo * [MINOR] Fix CI issue with TestHiveSyncTool (apache#6110) * [HUDI-4039] Make sure all builtin `KeyGenerator`s properly implement Spark specific APIs (apache#5523) This set of changes makes sure that all builtin KeyGenerators properly implement Spark-specific APIs in a performant way (minimizing key-generators overhead) * [MINOR] Disable Flink compactor IT test (apache#6189) * Revert "[MINOR] Fix CI issue with TestHiveSyncTool (apache#6110)" (apache#6192) This reverts commit d5c904e. * [HUDI-3979] Optimize out mandatory columns when no merging is performed (apache#5430) For MOR, when no merging is performed there is no point in reading either primary-key or pre-combine-key values (unless query is referencing these). Avoiding reading these allows to potentially save substantial resources wasted for reading it out. * [HUDI-4303] Use Hive sentinel value as partition default to avoid type caste issues (apache#5954) * Revert "[HUDI-4324] Remove use_jdbc config from hudi sync (apache#6072)" (apache#6160) This reverts commit 046044c. * [HUDI-4435] Fix Avro field not found issue introduced by Avro 1.10 (apache#6155) Co-authored-by: Wenning Ding <[email protected]> * [HUDI-4437] Fix test conflicts by clearing file system cache (apache#6123) Co-authored-by: jian.feng <[email protected]> Co-authored-by: jian.feng <[email protected]> Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4436] Invalidate cached table in Spark after write (apache#6159) Co-authored-by: Ryan Pifer <[email protected]> * [MINOR] Fix Call Procedure code style (apache#6186) * Fix Call Procedure code style. Co-authored-by: superche <[email protected]> * [MINOR] Bump CI timeout to 150m (apache#6198) * [HUDI-4440] Treat boostrapped table as non-partitioned in HudiFileIndex if partition column is missing from schema (apache#6163) Co-authored-by: Ryan Pifer <[email protected]> * [HUDI-4071] Make NONE sort mode as default for bulk insert (apache#6195) * [HUDI-4420] Fixing table schema delineation on partition/data schema for Spark relations (apache#5708) * [HUDI-4448] Remove the latest commit refresh for timeline server (apache#6179) * [HUDI-4450] Revert the checkpoint abort notification (apache#6181) * [HUDI-4439] Fix Amazon CloudWatch reporter for metadata enabled tables (apache#6164) Co-authored-by: Udit Mehrotra <[email protected]> Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-4348] fix merge into sql data quality in concurrent scene (apache#6020) * [HUDI-3510] Add sync validate procedure (apache#6200) * [HUDI-3510] Add sync validate procedure Co-authored-by: simonssu <[email protected]> * [MINOR] Fix typos in Spark client related classes (apache#6204) * [HUDI-4456] Close FileSystem in SparkClientFunctionalTestHarness (apache#6201) * [MINOR] Only log stdout output for non-zero exit from commands in IT (apache#6199) * [HUDI-4458] Add a converter cache for flink ColumnStatsIndices (apache#6205) * [HUDI-4071] Match ROLLBACK_USING_MARKERS_ENABLE in sql as datasource (apache#6206) Co-authored-by: superche <[email protected]> * [HUDI-4455] Improve test classes for TestHiveSyncTool (apache#6202) Improve HiveTestService, HiveTestUtil, and related classes. * [HUDI-4456] Clean up test resources (apache#6203) * [HUDI-3884] Support archival beyond savepoint commits (apache#5837) Co-authored-by: sivabalan <[email protected]> * [HUDI-4250][HUDI-4202] Optimize performance of Column Stats Index reading in Data Skipping (apache#5746) We provide an alternative way of fetching Column Stats Index within the reading process to avoid the penalty of a more heavy-weight execution scheduled through a Spark engine. * [HUDI-4471] Relocate AWSDmsAvroPayload class to hudi-common * [HUDI-4474] Infer metasync configs (apache#6217) - infer repeated sync configs from original configs - `META_SYNC_BASE_FILE_FORMAT` - infer from `org.apache.hudi.common.table.HoodieTableConfig.BASE_FILE_FORMAT` - `META_SYNC_ASSUME_DATE_PARTITION` - infer from `org.apache.hudi.common.config.HoodieMetadataConfig.ASSUME_DATE_PARTITIONING` - `META_SYNC_DECODE_PARTITION` - infer from `org.apache.hudi.common.table.HoodieTableConfig.URL_ENCODE_PARTITIONING` - `META_SYNC_USE_FILE_LISTING_FROM_METADATA` - infer from `org.apache.hudi.common.config.HoodieMetadataConfig.ENABLE` As proposed in https://github.com/apache/hudi/blob/master/rfc/rfc-55/rfc-55.md#compatible-changes * [HUDI-4210] Create custom hbase index to solve data skew issue on hbase regions (apache#5797) * [HUDI-3730] Keep metasync configs backward compatible (apache#6221) * [HUDI-4469] Flip reuse flag to true in HoodieBackedTableMetadata to improve file listing (apache#6214) * [HUDI-4186] Support Hudi with Spark 3.3.0 (apache#5943) Co-authored-by: Shawn Chang <[email protected]> * [HUDI-4126] Disable file splits for Bootstrap real time queries (via InputFormat) (apache#6219) Co-authored-by: Udit Mehrotra <[email protected]> Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4490] Make AWSDmsAvroPayload class backwards compatible (apache#6229) Co-authored-by: Rahil Chertara <[email protected]> * [HUDI-4484] Add default lock config options for flink metadata table (apache#6222) * [HUDI-4494] keep the fields' order when data is written out of order (apache#6233) * [MINOR] Minor changes around Spark 3.3 support (apache#6231) Co-authored-by: Shawn Chang <[email protected]> * [HUDI-4081][HUDI-4472] Addressing Spark SQL vs Spark DS performance gap (apache#6213) * [HUDI-4495] Fix handling of S3 paths incompatible with java URI standards (apache#6237) * [HUDI-4499] Tweak default retry times for flink metadata table lock (apache#6238) * [HUDI-4221] Optimzing getAllPartitionPaths (apache#6234) - Levering spark par for dir processing * Moving to 0.13.0-SNAPSHOT on master branch. * [HUDI-4504] Disable metadata table by default for flink (apache#6241) * [HUDI-4505] Returns instead of throws if lock file exists for FileSystemBasedLockProvider (apache#6242) To avoid unnecessary exception throws * [HUDI-4507] Improve file name extraction logic in metadata utils (apache#6250) * [MINOR] Fix convertPathWithScheme tests (apache#6251) * [MINOR] Add license header (apache#6247) Add license header to TestConfigUtils * [HUDI-4025] Add Presto and Trino query node to validate queries (apache#5578) * Add Presto and Trino query nodes to hudi-integ-test * Add yamls for query validation * Add presto-jdbc and trino-jdbc to integ-test-bundle * [HUDI-4518] Free lock if allocated but not acquired (apache#6272) If the lock is not null but its state has not yet transitioned to ACQUIRED, retry fails because the lock is not de-allocated. See issue apache#5702 * [HUDI-4510] Repair config "hive_sync.metastore.uris" in flink sql hive schema sync is not effective (apache#6257) * [HUDI-3848] Fixing minor bug in listing based rollback request generation (apache#6244) * [HUDI-4512][HUDI-4513] Fix bundle name for spark3 profile (apache#6261) * [HUDI-4501] Throwing exception when restore is attempted with hoodie.arhive.beyond.savepoint is enabled (apache#6239) * [HUDI-4516] fix Task not serializable error when run HoodieCleaner after one failure (apache#6265) Co-authored-by: jian.feng <[email protected]> * remove test resources (apache#6147) Co-authored-by: root <[email protected]> * [HUDI-4477] Adjust partition number of flink sink task (apache#6218) Co-authored-by: lewinma <[email protected]> * [HUDI-4298] Mor table reading for base and log files lost sequence of events (apache#6286) * [HUDI-4298] Mor table reading for base and log files lost sequence of events Signed-off-by: HunterXHunter <[email protected]> * [HUDI-4525] Fixing Spark 3.3 `AvroSerializer` implementation (apache#6279) * [HUDI-4447] fix no partitioned path extractor error when sync meta (apache#6263) * [HUDI-4520] Support qualified table 'db.table' in call procedures (apache#6274) * [HUDI-4531] Wrong partition path for flink hive catalog when the partition fields are not in the last (apache#6292) * [HUDI-4487] support to create ro/rt table by spark sql (apache#6262) * [HUDI-4533] Fix RunCleanProcedure's ArrayIndexOutOfBoundsException (apache#6293) * [HUDI-4536] ClusteringOperator causes the NullPointerException when writing with BulkInsertWriterHelper in clustering (apache#6298) * [HUDI-4385] Support online compaction in the flink batch mode write (apache#6093) * [HUDI-4385] Support online compaction in the flink batch mode write Signed-off-by: HunterXHunter <[email protected]> * [HUDI-4530] fix default payloadclass in mor is different with cow (apache#6288) * [HUDI-4545] Do not modify the current record directly for OverwriteNonDefaultsWithLatestAvroPayload (apache#6306) * [HUDI-4544] support retain hour cleaning policy for flink (apache#6300) * [HUDI-4547] Fix SortOperatorGen sort indices (apache#6309) Signed-off-by: HunterXHunter <[email protected]> * [HUDI-4470] Remove spark dataPrefetch disabled prop in DefaultSource * [HUDI-4540] Cover different table types in functional tests of Spark structured streaming (apache#6317) * [HUDI-4514] optimize CTAS to adapt to saveAsTable api in different modes (apache#6295) * [HUDI-4474] Fix inferring props for meta sync (apache#6310) - HoodieConfig#setDefaults looks up declared fields, so should pass static class for reflection, otherwise, subclasses of HoodieSyncConfig won't set defaults properly. - Pass all write client configs of deltastreamer to meta sync - Make org.apache.hudi.hive.MultiPartKeysValueExtractor default for deltastreamer, to align with SQL and flink * [HUDI-4550] Fallback to listing based rollback for completed instant (apache#6313) Ideally, rollback is not triggered for completed instants. However, if it gets triggered due to some extraneous condition or forced while rollback strategy still configured to be marker-based, then fallback to listing-based rollback instead of failing. - CTOR changes in rollback plan and action executors. - Change in condition to determine whether to use marker-based rollback. - Added UT to cover the scenario. * [HUDI-4303] Adding 4 to 5 upgrade handler to check for old deprecated "default" partition value (apache#6248) - Added FourToFiveUpgradeHandler to detect hudi tables with "default" partition and throwing exception. - Added a new write config ("hoodie.skip.default.partition.validation") when enabled, will bypass the above validation. If users have a hudi table where "default" partition was created intentionally and not as sentinel, they can enable this config to get past the validation. * [HUDI-4546] Optimize catalog cast logic in HoodieSpark3Analysis (apache#6307) * [HUDI-4534] Fixing upgrade to reload Metaclient for deltastreamer writes (apache#6296) * [HUDI-4517] If no marker type file, fallback to timeline based marker (apache#6266) - If MARKERS.type file is not present, the logic assumes that the direct markers are stored, which causes the read failure in certain cases even where timeline server based marker is enabled. This PR handles the failure by falling back to timeline based marker in such cases. * [HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles… (apache#5884) - Adding request retry to RemoteHoodieTableFileSystemView. Users can enable using the new configs added. * [HUDI-4464] Clear warnings in Azure CI (apache#6210) Co-authored-by: jian.feng <[email protected]> * [MINOR] Update PR description template (apache#6323) * [HUDI-4508] Repair the exception when reading optimized query for mor in hive and presto/trino (apache#6254) In MOR table, file slice may just have log file but no base file, before the file slice is compacted. In this case, read-optimized query will match the condition !baseFileOpt.isPresent() in HoodieCopyOnWriteTableInputFormat.createFileStatusUnchecked() and throw IllegalStateException. Instead of throwing exception, it is more suitable to query nothing in the file slice. Co-authored-by: sivabalan <[email protected]> * [HUDI-4548] Unpack the column max/min to string instead of Utf8 for Mor table (apache#6311) * [HUDI-4447] fix SQL metasync when perform delete table operation (apache#6180) * [HUDI-4424] Add new compactoin trigger stratgy: NUM_COMMITS_AFTER_REQ… (apache#6144) * [MINOR] improve flink dummySink's parallelism (apache#6325) * [HUDI-4568] Shade dropwizard metrics-core in hudi-aws-bundle (apache#6327) * [HUDI-4572] Fix 'Not a valid schema field: ts' error in HoodieFlinkCompactor if precombine field is not ts (apache#6331) Co-authored-by: jian.feng <[email protected]> * [HUDI-4570] Fix hive sync path error due to reuse of storage descriptors. (apache#6329) * [HUDI-4571] Fix partition extractor infer function when partition field mismatch (apache#6333) Infer META_SYNC_PARTITION_FIELDS and META_SYNC_PARTITION_EXTRACTOR_CLASS from hoodie.table.partition.fields first. If not set, then from hoodie.datasource.write.partitionpath.field. Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4570] Add test for updating multiple partitions in hive sync (apache#6340) * [MINOR] Fix wrong key to determine sync sql cascade (apache#6339) * [HUDI-4581] Claim RFC-58 for data skipping integration with query engines (apache#6346) * [HUDI-4577] Adding test coverage for `DELETE FROM`, Spark Quickstart guide (apache#6318) * [HUDI-4556] Improve functional test coverage of column stats index (apache#6319) * [HUDI-4558] lost 'hoodie.table.keygenerator.class' in hoodie.properties (apache#6320) Co-authored-by: 吴文池 <[email protected]> * [HUDI-4543] Support natural order when table schema contains a field named 'ts' (apache#6246) * be able to disable precombine field when table schema contains a field named ts Co-authored-by: jian yonghua <[email protected]> * [HUDI-4569][RFC-58] Claim RFC-58 for adding a new feature named 'Multiple event_time Fields Latest Verification in a Single Table' for Hudi (apache#6328) Co-authored-by: XinyaoTian <[email protected]> * [HUDI-3503] Support more feature to call procedure CleanCommand (apache#6353) * [HUDI-4590] Add hudi-aws dependency to hudi-flink-bundle. (apache#6356) * [MINOR] fix potential npe in spark writer (apache#6363) Co-authored-by: zhanshaoxiong <shaoxiong0001@@gmail.com> * fix bug in cli show fsview all (apache#6314) * [HUDI-4488] Improve S3EventsHoodieIncrSource efficiency (apache#6228) * [HUDI-4611] Fix the duplicate creation of config in HoodieFlinkStreamer (apache#6369) Co-authored-by: linfey <[email protected]> * [HUDI-3189] Fallback to full table scan with incremental query when files are cleaned up or achived for MOR table (apache#6141) * Spark support MOR read archived commits for incremental query * [MINOR] fix progress field calculate logic in HoodieLogRecordReader (apache#6291) * [HUDI-4608] Fix upgrade command in Hudi CLI (apache#6374) * [HUDI-4609] Improve usability of upgrade/downgrade commands in Hudi CLI (apache#6377) * [HUDI-4574] Fixed timeline based marker thread safety issue (apache#6383) * fixed timeline based markers thread safety issue * add document for TimelineBasedMarkers thread safety issues * [HUDI-4621] Add validation that bucket index fields should be subset of primary keys (apache#6396) * check bucket index fields Co-authored-by: 吴文池 <[email protected]> * [HUDI-4354] Add --force-empty-sync flag to deltastreamer (apache#6027) * [HUDI-4601] Read error from MOR table after compaction with timestamp partitioning (apache#6365) * read error from mor after compaction Co-authored-by: 吴文池 <[email protected]> * [MINOR] Update DOAP with 0.12.0 Release (apache#6413) * [HUDI-4529] Tweak some default config options for flink (apache#6287) * [HUDI-4632] Remove the force active property for flink1.14 profile (apache#6415) * [HUDI-4551] Tweak the default parallelism of flink pipeline to execution env parallelism (apache#6312) * [MINOR] Improve code style of CLI Command classes (apache#6427) * [HUDI-3625] Claim RFC-60 for Federated Storage Layer (apache#6440) * [HUDI-4616] Adding `PulsarSource` to `DeltaStreamer` to support ingesting from Apache Pulsar (apache#6386) - Adding PulsarSource to DeltaStreamer to support ingesting from Apache Pulsar. - Current implementation of PulsarSource is relying on "pulsar-spark-connector" to ingest using Spark instead of building similar pipeline from scratch. * [HUDI-3579] Add timeline commands in hudi-cli (apache#5139) * [HUDI-4638] Rename payload clazz and preCombine field options for flink sql (apache#6434) * Revert "[HUDI-4632] Remove the force active property for flink1.14 profile (apache#6415)" (apache#6449) This reverts commit 9055b2f. * [HUDI-4643] MergeInto syntax WHEN MATCHED is optional but must be set (apache#6443) * [HUDI-4644] Change default flink profile to 1.15.x (apache#6445) * [HUDI-4678] Claim RFC-61 for Snapshot view management (apache#6461) Co-authored-by: jian.feng <[email protected]> * [HUDI-4676] infer cleaner policy when write concurrency mode is OCC (apache#6459) * [HUDI-4676] infer cleaner policy when write concurrency mode is OCC Co-authored-by: jian.feng <[email protected]> * [HUDI-4683] Use enum class value for default value in flink options (apache#6453) * [HUDI-4584] Cleaning up Spark utilities (apache#6351) Cleans up Spark utilities and removes duplication * [HUDI-4686] Flip option 'write.ignore.failed' to default false (apache#6467) Also fix the flaky test * [HUDI-4515] Fix savepoints will be cleaned in keeping latest versions policy (apache#6267) * [HUDI-4637] Release thread in RateLimiter doesn't been terminated (apache#6433) * [HUDI-4698] Rename the package 'org.apache.flink.table.data' to avoid conflicts with flink table core (apache#6481) * HUDI-4687 add show_invalid_parquet procedure (apache#6480) Co-authored-by: zhanshaoxiong <shaoxiong0001@@gmail.com> * [HUDI-4584] Fixing `SQLConf` not being propagated to executor (apache#6352) Fixes `HoodieSparkUtils.createRDD` to make sure `SQLConf` is properly propagated to the executor (required by `AvroSerializer`) * [HUDI-4441] Log4j2 configuration fixes and removal of log4j1 dependencies (apache#6170) * [HUDI-4665] Flipping default for "ignore failed batch" config in streaming sink to false (apache#6450) * [HUDI-4713] Fix flaky ITTestHoodieDataSource#testAppendWrite (apache#6490) * [HUDI-4696] Fix flaky TestHoodieCombineHiveInputFormat (apache#6494) * Revert "[HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles… (apache#5884)" (apache#6501) This reverts commit 660177b. * [Stacked on 6386] Fixing `DebeziumSource` to properly commit offsets; (apache#6416) * [HUDI-4399][RFC-57] Protobuf support in DeltaStreamer (apache#6111) * [HUDI-4703] use the historical schema to response time travel query (apache#6499) * [HUDI-4703] use the historical schema to response time travel query * [HUDI-4549] Remove avro from hudi-hive-sync-bundle and hudi-aws-bundle (apache#6472) * Remove avro shading from hudi-hive-sync-bundle and hudi-aws-bundle. Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4482] remove guava and use caffeine instead for cache (apache#6240) * [HUDI-4483] Fix checkstyle in integ-test module (apache#6523) * [HUDI-4340] fix not parsable text DateTimeParseException by addng a method parseDateFromInstantTimeSafely for parsing timestamp when output metrics (apache#6000) * [DOCS] Add docs about javax.security.auth.login.LoginException when starting Hudi Sink Connector (apache#6255) * [HUDI-4327] Fixing flaky deltastreamer test (testCleanerDeleteReplacedDataWithArchive) (apache#6533) * [HUDI-4730] Fix batch job cannot clean old commits files (apache#6515) * [HUDI-4370] Fix batch job cannot clean old commits files Co-authored-by: jian.feng <[email protected]> * [HUDI-4740] Add metadata fields for hive catalog #createTable (apache#6541) * [HUDI-4695] Fixing flaky TestInlineCompaction#testCompactionRetryOnFailureBasedOnTime (apache#6534) * [HUDI-4193] change protoc version to unblock hudi compilation on m1 mac (apache#6535) * [HUDI-4438] Fix flaky TestCopyOnWriteActionExecutor#testPartitionMetafileFormat (apache#6546) * [MINOR] Fix typo in HoodieArchivalConfig (apache#6542) * [HUDI-4582] Support batch synchronization of partition to HMS to avoid timeout (apache#6347) Co-authored-by: xxhua <[email protected]> * [HUDI-4742] Fix AWS Glue partition's location is wrong when updatePartition (apache#6545) Co-authored-by: xxhua <[email protected]> * [HUDI-4418] Add support for ProtoKafkaSource (apache#6135) - Adds PROTO to Source.SourceType enum. - Handles PROTO type in SourceFormatAdapter by converting to Avro from proto Message objects. Conversion to Row goes Proto -> Avro -> Row currently. - Added ProtoClassBasedSchemaProvider to generate schemas for a proto class that is currently on the classpath. - Added ProtoKafkaSource which parses byte[] into a class that is on the path. - Added ProtoConversionUtil which exposes methods for creating schemas and translating from Proto messages to Avro GenericRecords. - Added KafkaSource which provides a base class for the other Kafka sources to use. * [HUDI-4642] Adding support to hudi-cli to repair deprecated partition (apache#6438) * [HUDI-4751] Fix owner instants for transaction manager api callers (apache#6549) * [HUDI-4739] Wrong value returned when key's length equals 1 (apache#6539) * extracts key fields Co-authored-by: 吴文池 <[email protected]> * [HUDI-4528] Add diff tool to compare commit metadata (apache#6485) * Add diff tool to compare commit metadata * Add partition level info to commits and compaction command * Partition support for compaction archived timeline * Add diff command test * [HUDI-4648] Support rename partition through CLI (apache#6569) * [HUDI-4775] Fixing incremental source for MOR table (apache#6587) * Fixing incremental source for MOR table * Remove unused import Co-authored-by: Sagar Sumit <[email protected]> * [HUDI-4694] Print testcase running time for CI jobs (apache#6586) * [RFC] Claim RFC-62 for Diagnostic Reporter (apache#6599) Co-authored-by: yuezhang <[email protected]> * [minor] following HUDI-4739, fix the extraction for simple record keys (apache#6594) * [HUDI-4619] Add a remote request retry mechanism for 'Remotehoodietablefilesystemview'. (apache#6393) * [HUDI-4720] Fix HoodieInternalRow return wrong num of fields when source not contains meta fields (apache#6500) Co-authored-by: wangzixuan.wzxuan <[email protected]> * [HUDI-4389] Make HoodieStreamingSink idempotent (apache#6098) * Support checkpoint and idempotent writes in HoodieStreamingSink - Use batchId as the checkpoint key and add to commit metadata - Support multi-writer for checkpoint data model * Walk back previous commits until checkpoint is found * Handle delete operation and fix test * [MINOR] Remove redundant braces (apache#6604) * [HUDI-4618] Separate log word for CommitUitls class (apache#6392) * [HUDI-4776] Fix merge into use unresolved assignment (apache#6589) * [HUDI-4795] Fix KryoException when bulk insert into a not bucket index hudi table Co-authored-by: hbg <[email protected]> * [HUDI-4615] Return checkpoint as null for empty data from events queue. (apache#6387) Co-authored-by: sivabalan <[email protected]> * [HUDI-4782] Support TIMESTAMP_LTZ type for flink (apache#6607) * [HUDI-4731] Shutdown CloudWatch reporter when query completes (apache#6468) * [HUDI-4793] Fixing ScalaTest tests to properly respect Log4j2 configs (apache#6617) * [HUDI-4766] Strengthen flink clustering job (apache#6566) * Allow rollbacks if required during clustering * Allow size to be defined in Long instead of Integer * Fix bug where clustering will produce files of 120MB in the same filegroup * Added clean task * Fix scheduling config to be consistent with that with compaction * Fix filter mode getting ignored issue * Add --instant-time parameter * Prevent no execute() calls exception from being thrown (clustering & compaction) * Apply upstream changes * Fix compilation issues * Fix checkstyle Signed-off-by: LinMingQiang <[email protected]> Signed-off-by: HunterXHunter <[email protected]> Co-authored-by: miomiocat <[email protected]> Co-authored-by: RexAn <[email protected]> Co-authored-by: JerryYue-M <[email protected]> Co-authored-by: jerryyue <[email protected]> Co-authored-by: Danny Chan <[email protected]> Co-authored-by: superche <[email protected]> Co-authored-by: superche <[email protected]> Co-authored-by: Shiyan Xu <[email protected]> Co-authored-by: jian.feng <[email protected]> Co-authored-by: jian.feng <[email protected]> Co-authored-by: voonhous <[email protected]> Co-authored-by: voonhou.su <[email protected]> Co-authored-by: YueZhang <[email protected]> Co-authored-by: yuezhang <[email protected]> Co-authored-by: Y Ethan Guo <[email protected]> Co-authored-by: xi chaomin <[email protected]> Co-authored-by: xicm <[email protected]> Co-authored-by: ForwardXu <[email protected]> Co-authored-by: 董可伦 <[email protected]> Co-authored-by: shenjiayu17 <[email protected]> Co-authored-by: Lanyuanxiaoyao <[email protected]> Co-authored-by: KnightChess <[email protected]> Co-authored-by: 苏承祥 <[email protected]> Co-authored-by: Kumud Kumar Srivatsava Tirupati <[email protected]> Co-authored-by: xiarixiaoyao <[email protected]> Co-authored-by: liujinhui <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: 冯健 <[email protected]> Co-authored-by: Sagar Sumit <[email protected]> Co-authored-by: HunterXHunter <[email protected]> Co-authored-by: Luning (Lucas) Wang <[email protected]> Co-authored-by: Yann Byron <[email protected]> Co-authored-by: Tim Brown <[email protected]> Co-authored-by: simonsssu <[email protected]> Co-authored-by: Alexey Kudinkin <[email protected]> Co-authored-by: Sivabalan Narayanan <[email protected]> Co-authored-by: Bo Cui <[email protected]> Co-authored-by: Rahil Chertara <[email protected]> Co-authored-by: wenningd <[email protected]> Co-authored-by: Wenning Ding <[email protected]> Co-authored-by: Rahil C <[email protected]> Co-authored-by: Ryan Pifer <[email protected]> Co-authored-by: Udit Mehrotra <[email protected]> Co-authored-by: simonssu <[email protected]> Co-authored-by: Vander <[email protected]> Co-authored-by: Tim Brown <[email protected]> Co-authored-by: Dongwook Kwon <[email protected]> Co-authored-by: Shawn Chang <[email protected]> Co-authored-by: Shawn Chang <[email protected]> Co-authored-by: 5herhom <[email protected]> Co-authored-by: 吴祥平 <[email protected]> Co-authored-by: root <[email protected]> Co-authored-by: F7753 <[email protected]> Co-authored-by: lewinma <[email protected]> Co-authored-by: shaoxiong.zhan <[email protected]> Co-authored-by: Nicholas Jiang <[email protected]> Co-authored-by: Yonghua Jian_deepnova <[email protected]> Co-authored-by: leesf <[email protected]> Co-authored-by: 5herhom <[email protected]> Co-authored-by: RexXiong <[email protected]> Co-authored-by: BruceLin <[email protected]> Co-authored-by: Pratyaksh Sharma <[email protected]> Co-authored-by: wuwenchi <[email protected]> Co-authored-by: 吴文池 <[email protected]> Co-authored-by: jian yonghua <[email protected]> Co-authored-by: Xinyao Tian (Richard) <[email protected]> Co-authored-by: XinyaoTian <[email protected]> Co-authored-by: zhanshaoxiong <shaoxiong0001@@gmail.com> Co-authored-by: vamshigv <[email protected]> Co-authored-by: feiyang_deepnova <[email protected]> Co-authored-by: linfey <[email protected]> Co-authored-by: novisfff <[email protected]> Co-authored-by: Qi Ji <[email protected]> Co-authored-by: hehuiyuan <[email protected]> Co-authored-by: Zouxxyy <[email protected]> Co-authored-by: Teng <[email protected]> Co-authored-by: leandro-rouberte <[email protected]> Co-authored-by: Jon Vexler <[email protected]> Co-authored-by: smilecrazy <[email protected]> Co-authored-by: xxhua <[email protected]> Co-authored-by: komao <[email protected]> Co-authored-by: wangzixuan.wzxuan <[email protected]> Co-authored-by: felixYyu <[email protected]> Co-authored-by: Bingeng Huang <[email protected]> Co-authored-by: hbg <[email protected]> Co-authored-by: Vinish Reddy <[email protected]> Co-authored-by: junyuc25 <[email protected]> Co-authored-by: rmahindra123 <[email protected]>
vinishjail97
added a commit
to vinishjail97/hudi
that referenced
this pull request
Dec 15, 2023
* [HUDI-4354] Add --force-empty-sync flag to deltastreamer (apache#6027) * [HUDI-4601] Read error from MOR table after compaction with timestamp partitioning (apache#6365) * read error from mor after compaction Co-authored-by: 吴文池 <[email protected]> * [MINOR] Update DOAP with 0.12.0 Release (apache#6413) * [HUDI-4529] Tweak some default config options for flink (apache#6287) * [HUDI-4632] Remove the force active property for flink1.14 profile (apache#6415) * [HUDI-4551] Tweak the default parallelism of flink pipeline to execution env parallelism (apache#6312) * [MINOR] Improve code style of CLI Command classes (apache#6427) * [HUDI-3625] Claim RFC-60 for Federated Storage Layer (apache#6440) * [HUDI-4616] Adding `PulsarSource` to `DeltaStreamer` to support ingesting from Apache Pulsar (apache#6386) - Adding PulsarSource to DeltaStreamer to support ingesting from Apache Pulsar. - Current implementation of PulsarSource is relying on "pulsar-spark-connector" to ingest using Spark instead of building similar pipeline from scratch. * [HUDI-3579] Add timeline commands in hudi-cli (apache#5139) * [HUDI-4638] Rename payload clazz and preCombine field options for flink sql (apache#6434) * Revert "[HUDI-4632] Remove the force active property for flink1.14 profile (apache#6415)" (apache#6449) This reverts commit 9055b2f. * [HUDI-4643] MergeInto syntax WHEN MATCHED is optional but must be set (apache#6443) * [HUDI-4644] Change default flink profile to 1.15.x (apache#6445) * [HUDI-4678] Claim RFC-61 for Snapshot view management (apache#6461) Co-authored-by: jian.feng <[email protected]> * [HUDI-4676] infer cleaner policy when write concurrency mode is OCC (apache#6459) * [HUDI-4676] infer cleaner policy when write concurrency mode is OCC Co-authored-by: jian.feng <[email protected]> * [HUDI-4683] Use enum class value for default value in flink options (apache#6453) * [HUDI-4584] Cleaning up Spark utilities (apache#6351) Cleans up Spark utilities and removes duplication * [HUDI-4686] Flip option 'write.ignore.failed' to default false (apache#6467) Also fix the flaky test * [HUDI-4515] Fix savepoints will be cleaned in keeping latest versions policy (apache#6267) * [HUDI-4637] Release thread in RateLimiter doesn't been terminated (apache#6433) * [HUDI-4698] Rename the package 'org.apache.flink.table.data' to avoid conflicts with flink table core (apache#6481) * HUDI-4687 add show_invalid_parquet procedure (apache#6480) Co-authored-by: zhanshaoxiong <shaoxiong0001@@gmail.com> * [HUDI-4584] Fixing `SQLConf` not being propagated to executor (apache#6352) Fixes `HoodieSparkUtils.createRDD` to make sure `SQLConf` is properly propagated to the executor (required by `AvroSerializer`) * [HUDI-4441] Log4j2 configuration fixes and removal of log4j1 dependencies (apache#6170) * [HUDI-4665] Flipping default for "ignore failed batch" config in streaming sink to false (apache#6450) * [HUDI-4713] Fix flaky ITTestHoodieDataSource#testAppendWrite (apache#6490) * [HUDI-4696] Fix flaky TestHoodieCombineHiveInputFormat (apache#6494) * Revert "[HUDI-3669] Add a remote request retry mechanism for 'Remotehoodietablefiles… (apache#5884)" (apache#6501) This reverts commit 660177b. * [Stacked on 6386] Fixing `DebeziumSource` to properly commit offsets; (apache#6416) * [HUDI-4399][RFC-57] Protobuf support in DeltaStreamer (apache#6111) * [HUDI-4703] use the historical schema to response time travel query (apache#6499) * [HUDI-4703] use the historical schema to response time travel query * [HUDI-4549] Remove avro from hudi-hive-sync-bundle and hudi-aws-bundle (apache#6472) * Remove avro shading from hudi-hive-sync-bundle and hudi-aws-bundle. Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4482] remove guava and use caffeine instead for cache (apache#6240) * [HUDI-4483] Fix checkstyle in integ-test module (apache#6523) * [HUDI-4340] fix not parsable text DateTimeParseException by addng a method parseDateFromInstantTimeSafely for parsing timestamp when output metrics (apache#6000) * [DOCS] Add docs about javax.security.auth.login.LoginException when starting Hudi Sink Connector (apache#6255) * [HUDI-4327] Fixing flaky deltastreamer test (testCleanerDeleteReplacedDataWithArchive) (apache#6533) * [HUDI-4730] Fix batch job cannot clean old commits files (apache#6515) * [HUDI-4370] Fix batch job cannot clean old commits files Co-authored-by: jian.feng <[email protected]> * [HUDI-4740] Add metadata fields for hive catalog #createTable (apache#6541) * [HUDI-4695] Fixing flaky TestInlineCompaction#testCompactionRetryOnFailureBasedOnTime (apache#6534) * [HUDI-4193] change protoc version to unblock hudi compilation on m1 mac (apache#6535) * [HUDI-4438] Fix flaky TestCopyOnWriteActionExecutor#testPartitionMetafileFormat (apache#6546) * [MINOR] Fix typo in HoodieArchivalConfig (apache#6542) * [HUDI-4582] Support batch synchronization of partition to HMS to avoid timeout (apache#6347) Co-authored-by: xxhua <[email protected]> * [HUDI-4742] Fix AWS Glue partition's location is wrong when updatePartition (apache#6545) Co-authored-by: xxhua <[email protected]> * [HUDI-4418] Add support for ProtoKafkaSource (apache#6135) - Adds PROTO to Source.SourceType enum. - Handles PROTO type in SourceFormatAdapter by converting to Avro from proto Message objects. Conversion to Row goes Proto -> Avro -> Row currently. - Added ProtoClassBasedSchemaProvider to generate schemas for a proto class that is currently on the classpath. - Added ProtoKafkaSource which parses byte[] into a class that is on the path. - Added ProtoConversionUtil which exposes methods for creating schemas and translating from Proto messages to Avro GenericRecords. - Added KafkaSource which provides a base class for the other Kafka sources to use. * [HUDI-4642] Adding support to hudi-cli to repair deprecated partition (apache#6438) * [HUDI-4751] Fix owner instants for transaction manager api callers (apache#6549) * [HUDI-4739] Wrong value returned when key's length equals 1 (apache#6539) * extracts key fields Co-authored-by: 吴文池 <[email protected]> * [HUDI-4528] Add diff tool to compare commit metadata (apache#6485) * Add diff tool to compare commit metadata * Add partition level info to commits and compaction command * Partition support for compaction archived timeline * Add diff command test * [HUDI-4648] Support rename partition through CLI (apache#6569) * [HUDI-4775] Fixing incremental source for MOR table (apache#6587) * Fixing incremental source for MOR table * Remove unused import Co-authored-by: Sagar Sumit <[email protected]> * [HUDI-4694] Print testcase running time for CI jobs (apache#6586) * [RFC] Claim RFC-62 for Diagnostic Reporter (apache#6599) Co-authored-by: yuezhang <[email protected]> * [minor] following HUDI-4739, fix the extraction for simple record keys (apache#6594) * [HUDI-4619] Add a remote request retry mechanism for 'Remotehoodietablefilesystemview'. (apache#6393) * [HUDI-4720] Fix HoodieInternalRow return wrong num of fields when source not contains meta fields (apache#6500) Co-authored-by: wangzixuan.wzxuan <[email protected]> * [HUDI-4389] Make HoodieStreamingSink idempotent (apache#6098) * Support checkpoint and idempotent writes in HoodieStreamingSink - Use batchId as the checkpoint key and add to commit metadata - Support multi-writer for checkpoint data model * Walk back previous commits until checkpoint is found * Handle delete operation and fix test * [MINOR] Remove redundant braces (apache#6604) * [HUDI-4618] Separate log word for CommitUitls class (apache#6392) * [HUDI-4776] Fix merge into use unresolved assignment (apache#6589) * [HUDI-4795] Fix KryoException when bulk insert into a not bucket index hudi table Co-authored-by: hbg <[email protected]> * [HUDI-4615] Return checkpoint as null for empty data from events queue. (apache#6387) Co-authored-by: sivabalan <[email protected]> * [HUDI-4782] Support TIMESTAMP_LTZ type for flink (apache#6607) * [HUDI-4731] Shutdown CloudWatch reporter when query completes (apache#6468) * [HUDI-4793] Fixing ScalaTest tests to properly respect Log4j2 configs (apache#6617) * [HUDI-4766] Strengthen flink clustering job (apache#6566) * Allow rollbacks if required during clustering * Allow size to be defined in Long instead of Integer * Fix bug where clustering will produce files of 120MB in the same filegroup * Added clean task * Fix scheduling config to be consistent with that with compaction * Fix filter mode getting ignored issue * Add --instant-time parameter * Prevent no execute() calls exception from being thrown (clustering & compaction) * [HUDI-4797] fix merge into table for source table with different column order (apache#6620) Co-authored-by: zhanshaoxiong <shaoxiong0001@@gmail.com> * [MINOR] Typo fix for kryo in flink-bundle (apache#6639) * [HUDI-4811] Fix the checkstyle of hudi flink (apache#6633) * [HUDI-4465] Optimizing file-listing sequence of Metadata Table (apache#6016) Optimizes file-listing sequence of the Metadata Table to make sure it's on par or better than FS-based file-listing Change log: - Cleaned up avoidable instantiations of Hadoop's Path - Replaced new Path w/ createUnsafePath where possible - Cached TimestampFormatter, DateFormatter for timezone - Avoid loading defaults in Hadoop conf when init-ing HFile reader - Avoid re-instantiating BaseTableMetadata twice w/in BaseHoodieTableFileIndex - Avoid looking up FileSystem for every partition when listing partitioned table, instead do it just once * [HUDI-4807] Use base table instant for metadata initialization (apache#6629) * [HUDI-3453] Fix HoodieBackedTableMetadata concurrent reading issue (apache#5091) Co-authored-by: yuezhang <[email protected]> Co-authored-by: Sagar Sumit <[email protected]> * [HUDI-4518] Add unit test for reentrant lock in diff lockProvider (apache#6624) * [HUDI-4810] Fixing Hudi bundles requiring log4j2 on the classpath (apache#6631) Downgrading all of the log4j2 deps to "provided" scope, since these are not API modules (as advertised), but rather fully-fledged implementations adding dependency on other modules (like log4j2 in the case of "log4j-1.2-api") * [HUDI-4826] Update RemoteHoodieTableFileSystemView to allow base path in UTF-8 (apache#6544) * [HUDI-4763] Allow hoodie read client to choose index (apache#6506) Co-authored-by: Y Ethan Guo <[email protected]> * [DOCS] Fix Slack invite link in README.md (apache#6648) * [HUDI-3558] Consistent bucket index: bucket resizing (split&merge) & concurrent write during resizing (apache#4958) RFC-42 implementation - Implement bucket resizing for consistent hashing index. - Support concurrent write during bucket resizing. This change added tests and can be verified as follows: - The test of the consistent bucket index is enhanced to include the case of bucket resizing. - Tests of different bucket resizing cases. - Tests of concurrent resizing, and concurrent writes during resizing. * [MINOR] Add dev setup and spark 3.3 profile to readme (apache#6656) * [HUDI-4831] Fix AWSDmsAvroPayload#getInsertValue,combineAndGetUpdateValue to invoke correct api (apache#6637) Co-authored-by: Rahil Chertara <[email protected]> * [HUDI-4806] Use Avro version from the root pom for Flink bundle (apache#6628) Co-authored-by: Shawn Chang <[email protected]> * [HUDI-4833] Add Postgres Schema Name to Postgres Debezium Source (apache#6616) * [HUDI-4825] Remove redundant fields in serialized commit metadata in JSON (apache#6646) * [MINOR] Insert should call validateInsertSchema in HoodieFlinkWriteClient (apache#5919) Co-authored-by: 徐帅 <[email protected]> * [HUDI-3879] Suppress exceptions that are not fatal in HoodieMetadataTableValidator (apache#5344) Co-authored-by: yuezhang <[email protected]> Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-3998] Fix getCommitsSinceLastCleaning failed when async cleaning (apache#5478) - The last completed commit timestamp is used to calculate how many commit have been completed since the last clean. we might need to save this w/ clean plan so that next time when we trigger clean, we can start calculating from that. * [HUDI-3994] - Added support for initializing DeltaStreamer without a defined Spark Master (apache#5630) That will enable the usage of DeltaStreamer on environments such as AWS Glue or other serverless environments where the spark master is inherited and we do not have access to it. Co-authored-by: Angel Conde Manjon <[email protected]> * [HUDI-4628] Hudi-flink support GLOBAL_BLOOM，GLOBAL_SIMPLE，BUCKET index type (apache#6406) Co-authored-by: xiaoxingstack <[email protected]> * [HUDI-4814] Schedules new clustering plan based on latest clustering instant (apache#6574) * Keep a clustering running at the same time * Simplify filtering logic Co-authored-by: dongsj <[email protected]> * [HUDI-4817] Delete markers after full-record bootstrap operation (apache#6667) * [HUDI-4691] Cleaning up duplicated classes in Spark 3.3 module (apache#6550) As part of adding support for Spark 3.3 in Hudi 0.12, a lot of the logic from Spark 3.2 module has been simply copied over. This PR is rectifying that by: 1. Creating new module "hudi-spark3.2plus-common" (that is shared across Spark 3.2 and Spark 3.3) 2. Moving shared components under "hudi-spark3.2plus-common" * [HUDI-4752] Add dedup support for MOR table in cli (apache#6608) * [HUDI-4837] Stop sleeping where it is not necessary after the success (apache#6270) Co-authored-by: Volodymyr Burenin <[email protected]> Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-4843] Delete the useless timer in BaseRollbackActionExecutor (apache#6671) Co-authored-by: 吴文池 <[email protected]> * [HUDI-4780] hoodie.logfile.max.size It does not take effect, causing the log file to be too large (apache#6602) * hoodie.logfile.max.size It does not take effect, causing the log file to be too large Co-authored-by: [email protected] <loukey_7821> * [HUDI-4844] Skip partition value resolving when the field does not exists for MergeOnReadInputFormat#getReader (apache#6678) * [MINOR] Fix the Spark job status description for metadata-only bootstrap operation (apache#6666) * [HUDI-3403] Ensure keygen props are set for bootstrap (apache#6645) * [HUDI-4193] Upgrade Protobuf to 3.21.5 (apache#5784) * [HUDI-4785] Fix partition discovery in bootstrap operation (apache#6673) Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-4706] Fix InternalSchemaChangeApplier#applyAddChange error to add nest type (apache#6486) InternalSchemaChangeApplier#applyAddChange forget to remove parent name when calling ColumnAddChange#addColumns * [HUDI-4851] Fixing CSI not handling `InSet` operator properly (apache#6685) * [HUDI-4796] MetricsReporter stop bug (apache#6619) * [HUDI-3861] update tblp 'path' when rename table (apache#5320) * [HUDI-4853] Get field by name for OverwriteNonDefaultsWithLatestAvroPayload to avoid schema mismatch (apache#6689) * [HUDI-4813] Fix infer keygen not work in sparksql side issue (apache#6634) * [HUDI-4813] Fix infer keygen not work in sparksql side issue Co-authored-by: xiaoxingstack <[email protected]> * [HUDI-4856] Missing option for HoodieCatalogFactory (apache#6693) * [HUDI-4864] Fix AWSDmsAvroPayload#combineAndGetUpdateValue when using MOR snapshot query after delete operations with test (apache#6688) Co-authored-by: Rahil Chertara <[email protected]> * [HUDI-4841] Fix sort idempotency issue (apache#6669) * [HUDI-4865] Optimize HoodieAvroUtils#isMetadataField to use O(1) complexity (apache#6702) * [HUDI-4736] Fix inflight clean action preventing clean service to continue when multiple cleans are not allowed (apache#6536) * [HUDI-4842] Support compaction strategy based on delta log file num (apache#6670) Co-authored-by: 苏承祥 <[email protected]> * [HUDI-4282] Repair IOException in CHDFS when check block corrupted in HoodieLogFileReader (apache#6031) Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-4757] Create pyspark examples (apache#6672) * [HUDI-3959] Rename class name for spark rdd reader (apache#5409) Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-4828] Fix the extraction of record keys which may be cut out (apache#6650) Co-authored-by: yangshuo3 <[email protected]> Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-4873] Report number of messages to be processed via metrics (apache#6271) Co-authored-by: Volodymyr Burenin <[email protected]> Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-4870] Improve compaction config description (apache#6706) * [HUDI-3304] Support partial update payload (apache#4676) Co-authored-by: jian.feng <[email protected]> * [HUDI-4808] Fix HoodieSimpleBucketIndex not consider bucket num in lo… (apache#6630) * [HUDI-4808] Fix HoodieSimpleBucketIndex not consider bucket num in log file issue Co-authored-by: xiaoxingstack <[email protected]> * [HUDI-4485] Bump spring shell to 2.1.1 in CLI (apache#6489) Bumped spring shell to 2.1.1 and updated the default value for show fsview all `pathRegex` parameter. * [minor] following 3304, some code refactoring (apache#6713) * [HUDI-4832] Fix drop partition meta sync (apache#6662) * [HUDI-4810] Fix log4j imports to use bridge API (apache#6710) Co-authored-by: dongsj <[email protected]> * [HUDI-4877] Fix org.apache.hudi.index.bucket.TestHoodieSimpleBucketIndex#testTagLocation not work correct issue (apache#6717) Co-authored-by: xiaoxingstack <[email protected]> * [HUDI-4326] add updateTableSerDeInfo for HiveSyncTool (apache#5920) - This pull request fix [SUPPORT] Hudi spark datasource error after migrate from 0.8 to 0.11 apache#5861* - The issue is caused by after changing the table to spark data source table, the table SerDeInfo is missing. * Co-authored-by: Sagar Sumit <[email protected]> * [MINOR] fix indent to make build pass (apache#6721) * [HUDI-3478] Implement CDC Write in Spark (apache#6697) * [HUDI-4326] Fix hive sync serde properties (apache#6722) * [HUDI-4875] Fix NoSuchTableException when dropping temporary view after applied HoodieSparkSessionExtension in Spark 3.2 (apache#6709) * [DOCS] Improve the quick start guide for Kafka Connect Sink (apache#6708) * [HUDI-4729] Fix file group pending compaction cannot be queried when query _ro table (apache#6516) File group in pending compaction can not be queried when query _ro table with spark. This commit fixes that. Co-authored-by: zhanshaoxiong <shaoxiong0001@@gmail.com> Co-authored-by: Sagar Sumit <[email protected]> * [HUDI-3983] Fix ClassNotFoundException when using hudi-spark-bundle to write table with hbase index (apache#6715) * [HUDI-4758] Add validations to java spark examples (apache#6615) * [HUDI-4792] Batch clean files to delete (apache#6580) This patch makes use of batch call to get fileGroup to delete during cleaning instead of 1 call per partition. This limit the number of call to the view and should fix the trouble with metadata table in context of lot of partitions. Fixes issue apache#6373 Co-authored-by: sivabalan <[email protected]> * [HUDI-4363] Support Clustering row writer to improve performance (apache#6046) * [HUDI-3478][HUDI-4887] Use Avro as the format of persisted cdc data (apache#6734) * [HUDI-4851] Fixing handling of `UTF8String` w/in `InSet` operator (apache#6739) Co-authored-by: Raymond Xu <[email protected]> * [HUDI-3901] Correct the description of hoodie.index.type (apache#6749) * [MINOR] Add .mvn directory to gitignore (apache#6746) Co-authored-by: Rahil Chertara <[email protected]> * add support for unraveling proto schemas * fix some compile issues * [HUDI-4901] Add avro.version to Flink profiles (apache#6757) * Add avro.version to Flink profiles Co-authored-by: Shawn Chang <[email protected]> * [HUDI-4559] Support hiveSync command based on Call Produce Command (apache#6322) * [HUDI-4883] Supporting delete savepoint for MOR (apache#6744) Users could delete unnecessary savepoints and unblock archival for MOR table. * [HUDI-4897] Refactor the merge handle in CDC mode (apache#6740) * [HUDI-3523] Introduce AddColumnSchemaPostProcessor to support add columns to the end of a schema (apache#5031) * Revert "[HUDI-3523] Introduce AddColumnSchemaPostProcessor to support add columns to the end of a schema (apache#5031)" (apache#6768) This reverts commit 092375f. * [HUDI-3523] Introduce AddPrimitiveColumnSchemaPostProcessor to support add new primitive column to the end of a schema (apache#6769) * [HUDI-4903] Fix TestHoodieLogFormat`s minor typo (apache#6762) * [MINOR] Drastically reducing concurrency level (to avoid CI flakiness) (apache#6754) * Update HoodieIndex.java Fix a typo * [HUDI-4906] Fix the local tests for hudi-flink (apache#6763) * [HUDI-4899] Fixing compatibility w/ Spark 3.2.2 (apache#6755) * [HUDI-4892] Fix hudi-spark3-bundle (apache#6735) * [MINOR] Fix a few typos in HoodieIndex (apache#6784) Co-authored-by: xingjunwang <[email protected]> * [HUDI-4412] Fix multi writer INSERT_OVERWRITE NPE bug (apache#6130) There are two minor issues fixed here: 1. When the insert_overwrite operation is performed, the clusteringPlan in the requestedReplaceMetadata will be null. Calling getFileIdsFromRequestedReplaceMetadata will cause NPE. 2. When insert_overwrite operation, inflightCommitMetadata!=null, getOperationType should be obtained from getHoodieInflightReplaceMetadata, the original code will have a null pointer. * [MINOR] retain avro's namespace (apache#6783) * [MINOR] Simple logging fix in LockManager (apache#6765) Co-authored-by: 苏承祥 <[email protected]> * [HUDI-4433] hudi-cli repair deduplicate not working with non-partitioned dataset (apache#6349) When using the repair deduplicate command with hudi-cli, there is no way to run it on the unpartitioned dataset, so modify the cli parameter. Co-authored-by: Xingjun Wang <[email protected]> * [RFC-51][HUDI-3478] Update RFC: CDC support (apache#6256) * [HUDI-4915] improve avro serializer/deserializer (apache#6788) * [HUDI-3478] Implement CDC Read in Spark (apache#6727) * naming and style updates * [HUDI-4830] Fix testNoGlobalConfFileConfigured when add hudi-defaults.conf in default dir (apache#6652) * make test data random, reuse code * [HUDI-4760] Fixing repeated trigger of data file creations w/ clustering (apache#6561) - Apparently in clustering, data file creations are triggered twice since we don't cache the write status and for doing some validation, we do isEmpty on JavaRDD which ended up retriggering the action. Fixing the double de-referencing in this patch. * [HUDI-4914] Managed memory weight should be set when sort clustering is enabled (apache#6792) * [HUDI-4910] Fix unknown variable or type "Cast" (apache#6778) * [HUDI-4918] Fix bugs about when trying to show the non -existing key from env, NullPointException occurs. (apache#6794) * [HUDI-4718] Add Kerberos kinit command support. (apache#6719) * add test for 2 different recursion depths, fix schema cache key * add unsigned long support * better handle other types * rebase on 4904 * get all tests working * fix oneof expected schema, update tests after rebase * [HUDI-4902] Set default partitioner for SIMPLE BUCKET index (apache#6759) * [MINOR] Update PR template with documentation update (apache#6748) * revert scala binary change * try a different method to avoid avro version * [HUDI-4904] Add support for unraveling proto schemas in ProtoClassBasedSchemaProvider (apache#6761) If a user provides a recursive proto schema, it will fail when we write to parquet. We need to allow the user to specify how many levels of recursion they want before truncating the remaining data. Main changes to existing code: ProtoClassBasedSchemaProvider tracks number of times a message descriptor is seen within a branch of the schema traversal once the number of times that descriptor is seen exceeds the user provided limit, set the field to preset record that will contain two fields: 1) the remaining data serialized as a proto byte array, 2) the descriptors full name for context about what is in that byte array Converting from a proto to an avro now accounts for this truncation of the input * delete unused file * [HUDI-4907] Prevent single commit multi instant issue (apache#6766) Co-authored-by: TengHuo <[email protected]> Co-authored-by: yuzhao.cyz <[email protected]> * [HUDI-4923] Fix flaky TestHoodieReadClient.testReadFilterExistAfterBulkInsertPrepped (apache#6801) Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4848] Fixing repair deprecated partition tool (apache#6731) * [HUDI-4913] Fix HoodieSnapshotExporter for writing to a different S3 bucket or FS (apache#6785) * address PR feedback, update decimal precision * fix isNullable issue, check if class is Int64value * checkstyle fix * change wrapper descriptor set initialization * add in testing for unsigned long to BigInteger conversion * [HUDI-4453] Fix schema to include partition columns in bootstrap operation (apache#6676) Turn off the type inference of the partition column to be consistent with existing behavior. Add notes around partition column type inference. * [HUDI-2780] Fix the issue of Mor log skipping complete blocks when reading data (apache#4015) Co-authored-by: huangjing02 <[email protected]> Co-authored-by: sivabalan <[email protected]> * [HUDI-4924] Auto-tune dedup parallelism (apache#6802) * [HUDI-4687] Avoid setAccessible which breaks strong encapsulation (apache#6657) Use JOL GraphLayout for estimating deep size. * [MINOR] fixing validate async operations to poll completed clean instances (apache#6814) * [HUDI-4734] Deltastreamer table config change validation (apache#6753) Co-authored-by: sivabalan <[email protected]> * [HUDI-4934] Revert batch clean files (apache#6813) * Revert "[HUDI-4792] Batch clean files to delete (apache#6580)" This reverts commit cbf9b83. * [HUDI-4722] Added locking metrics for Hudi (apache#6502) * [HUDI-4936] Fix `as.of.instant` not recognized as hoodie config (apache#5616) Co-authored-by: leon <[email protected]> Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4861] Relaxing `MERGE INTO` constraints to permit limited casting operations w/in matched-on conditions (apache#6820) * [HUDI-4885] Adding org.apache.avro to hudi-hive-sync bundle (apache#6729) * [HUDI-4951] Fix incorrect use of Long.getLong() (apache#6828) * [MINOR] Use base path URI in ITTestDataStreamWrite (apache#6826) * [HUDI-4308] READ_OPTIMIZED read mode will temporary loss of data when compaction (apache#6664) Co-authored-by: Y Ethan Guo <[email protected]> * [HUDI-4237] Fixing empty partition-values being sync'd to HMS (apache#6821) Co-authored-by: dujunling <[email protected]> Co-authored-by: Raymond Xu <[email protected]> * [HUDI-4925] Should Force to use ExpressionPayload in MergeIntoTableCommand (apache#6355) Co-authored-by: jian.feng <[email protected]> * [HUDI-4850] Add incremental source from GCS to Hudi (apache#6665) Adds an incremental source from GCS based on a similar design as https://hudi.apache.org/blog/2021/08/23/s3-events-source * [HUDI-4957] Shade JOL in bundles to fix NoClassDefFoundError:GraphLayout (apache#6839) * [HUDI-4718] Add Kerberos kdestroy command support (apache#6810) * [HUDI-4916] Implement change log feed for Flink (apache#6840) * [HUDI-4769] Option read.streaming.skip_compaction skips delta commit (apache#6848) * [HUDI-4949] optimize cdc read to avoid the problem of reusing buffer underlying the Row (apache#6805) * [HUDI-4966] Add a partition extractor to handle partition values with slashes (apache#6851) * [MINOR] Fix testUpdateRejectForClustering (apache#6852) * [HUDI-4962] Move cloud dependencies to cloud modules (apache#6846) * [HOTFIX] Fix source release validate script (apache#6865) * [HUDI-4980] Calculate avg record size using commit only (apache#6864) Calculate average record size for Spark upsert partitioner based on commit instants only. Previously it's based on commit and replacecommit, of which the latter may be created by clustering which has inaccurately smaller average record sizes, which could result in OOM due to size underestimation. * shade protobuf dependency * Revert "[HUDI-4915] improve avro serializer/deserializer (apache#6788)" (apache#6809) This reverts commit 79b3e2b. * [HUDI-4970] Update kafka-connect readme and refactor HoodieConfig#create (apache#6857) * Enhancing README for multi-writer tests (apache#6870) * [MINOR] Fix deploy script for flink 1.15 (apache#6872) * [HUDI-4992] Fixing invalid min/max record key stats in Parquet metadata (apache#6883) * Revert "shade protobuf dependency" This reverts commit f03f961. * [HUDI-4972] Fixes to make unit tests work on m1 mac (apache#6751) * [HUDI-2786] Docker demo on mac aarch64 (apache#6859) * [HUDI-4971] Fix shading kryo-shaded with reusing configs (apache#6873) * [HUDI-3900] [UBER] Support log compaction action for MOR tables (apache#5958) - Adding log compaction support to MOR table. subsequent log blocks can now be compacted into larger log blocks without needing to go for full compaction (by merging w/ base file). - New timeline action is introduced for the purpose. Co-authored-by: sivabalan <[email protected]> * Relocate apache http package (apache#6874) * [HUDI-4975] Fix datahub bundle dependency (apache#6896) * [HUDI-4999] Refactor FlinkOptions#allOptions and CatalogOptions#allOptions (apache#6901) * [MINOR] Update GitHub setting for merge button (apache#6922) Only allow squash and merge. Disable merge and rebase * [HUDI-4993] Make DataPlatform name and Dataset env configurable in DatahubSyncTool (apache#6885) * [MINOR] Fix name spelling for RunBootstrapProcedure * [HUDI-4754] Add compliance check in github actions (apache#6575) * [HUDI-4963] Extend InProcessLockProvider to support multiple table ingestion (apache#6847) Co-authored-by: rmahindra123 <[email protected]> * [HUDI-4994] Fix bug that prevents re-ingestion of soft-deleted Datahub entities (apache#6886) * Implement Create/Drop/Show/Refresh Secondary Index (apache#5933) * remove oss pr compliance * different approach for shutdown all metrics instances * remove flink testing, update metrics shutdown Co-authored-by: Qi Ji <[email protected]> Co-authored-by: wuwenchi <[email protected]> Co-authored-by: 吴文池 <[email protected]> Co-authored-by: Sagar Sumit <[email protected]> Co-authored-by: Danny Chan <[email protected]> Co-authored-by: Nicholas Jiang <[email protected]> Co-authored-by: Y Ethan Guo <[email protected]> Co-authored-by: Alexey Kudinkin <[email protected]> Co-authored-by: 董可伦 <[email protected]> Co-authored-by: 冯健 <[email protected]> Co-authored-by: jian.feng <[email protected]> Co-authored-by: hehuiyuan <[email protected]> Co-authored-by: Zouxxyy <[email protected]> Co-authored-by: Manu <[email protected]> Co-authored-by: shaoxiong.zhan <[email protected]> Co-authored-by: zhanshaoxiong <shaoxiong0001@@gmail.com> Co-authored-by: Sivabalan Narayanan <[email protected]> Co-authored-by: Shiyan Xu <[email protected]> Co-authored-by: Yann Byron <[email protected]> Co-authored-by: KnightChess <[email protected]> Co-authored-by: Teng <[email protected]> Co-authored-by: leandro-rouberte <[email protected]> Co-authored-by: Jon Vexler <[email protected]> Co-authored-by: smilecrazy <[email protected]> Co-authored-by: xxhua <[email protected]> Co-authored-by: YueZhang <[email protected]> Co-authored-by: yuezhang <[email protected]> Co-authored-by: HunterXHunter <[email protected]> Co-authored-by: komao <[email protected]> Co-authored-by: wangzixuan.wzxuan <[email protected]> Co-authored-by: felixYyu <[email protected]> Co-authored-by: Bingeng Huang <[email protected]> Co-authored-by: hbg <[email protected]> Co-authored-by: Vinish Reddy <[email protected]> Co-authored-by: junyuc25 <[email protected]> Co-authored-by: voonhous <[email protected]> Co-authored-by: Xingcan Cui <[email protected]> Co-authored-by: Yuwei XIAO <[email protected]> Co-authored-by: wangp-nhlab <[email protected]> Co-authored-by: Nicolas Paris <[email protected]> Co-authored-by: Rahil C <[email protected]> Co-authored-by: Rahil Chertara <[email protected]> Co-authored-by: Shawn Chang <[email protected]> Co-authored-by: Shawn Chang <[email protected]> Co-authored-by: Abhishek Modi <[email protected]> Co-authored-by: shuai.xu <[email protected]> Co-authored-by: 徐帅 <[email protected]> Co-authored-by: Angel Conde <[email protected]> Co-authored-by: Angel Conde Manjon <[email protected]> Co-authored-by: FocusComputing <[email protected]> Co-authored-by: xiaoxingstack <[email protected]> Co-authored-by: eric9204 <[email protected]> Co-authored-by: dongsj <[email protected]> Co-authored-by: Volodymyr Burenin <[email protected]> Co-authored-by: Volodymyr Burenin <[email protected]> Co-authored-by: luokey <[email protected]> Co-authored-by: Sylwester Lachiewicz <[email protected]> Co-authored-by: 苏承祥 <[email protected]> Co-authored-by: 苏承祥 <[email protected]> Co-authored-by: 5herhom <[email protected]> Co-authored-by: Jon Vexler <[email protected]> Co-authored-by: simonsssu <[email protected]> Co-authored-by: y0908105023 <[email protected]> Co-authored-by: yangshuo3 <[email protected]> Co-authored-by: Paul Zhang <[email protected]> Co-authored-by: Kyle Zhike Chen <[email protected]> Co-authored-by: dohongdayi <[email protected]> Co-authored-by: RexAn <[email protected]> Co-authored-by: ForwardXu <[email protected]> Co-authored-by: wangxianghu <[email protected]> Co-authored-by: wulei <[email protected]> Co-authored-by: Xingjun Wang <[email protected]> Co-authored-by: Prasanna Rajaperumal <[email protected]> Co-authored-by: xingjunwang <[email protected]> Co-authored-by: liujinhui <[email protected]> Co-authored-by: ChanKyeong Won <[email protected]> Co-authored-by: Forus <[email protected]> Co-authored-by: hj2016 <[email protected]> Co-authored-by: huangjing02 <[email protected]> Co-authored-by: jsbali <[email protected]> Co-authored-by: Leon Tsao <[email protected]> Co-authored-by: leon <[email protected]> Co-authored-by: 申胜利 <[email protected]> Co-authored-by: aiden.dong <[email protected]> Co-authored-by: dujunling <[email protected]> Co-authored-by: Pramod Biligiri <[email protected]> Co-authored-by: Zouxxyy <[email protected]> Co-authored-by: Alexey Kudinkin <[email protected]> Co-authored-by: Surya Prasanna <[email protected]> Co-authored-by: Rajesh Mahindra <[email protected]> Co-authored-by: rmahindra123 <[email protected]> Co-authored-by: huberylee <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Change Logs
Fixed flaky InlineCompactionTest. Test had some dependency on timer. Have bumped up the timer so that its more deterministic.
Impact
Improves CI stability
Risk level: low
Contributor's checklist