-
Notifications
You must be signed in to change notification settings - Fork 1
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
master pull #5
master pull #5
Commits on Mar 14, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 1ba8220 - Browse repository at this point
Copy the full SHA 1ba8220View commit details -
[HUDI-3600] Tweak the default cleaning strategy to be more streaming …
…friendly for flink (#5010)
Configuration menu - View commit details
-
Copy full SHA for 465d553 - Browse repository at this point
Copy the full SHA 465d553View commit details -
fix NPE when run schdule using spark-sql if the commits time < hoodie…
….compact.inline.max.delta.commits (#4976) * Update CompactionHoodiePathCommand.scala fix NPE when run schdule using spark-sql if the commits time < hoodie.compact.inline.max.delta.commits * Update CompactionHoodiePathCommand.scala fix IndexOutOfBoundsException when there`s no schedule for compaction * Update CompactionHoodiePathCommand.scala fix CI issue
Configuration menu - View commit details
-
Copy full SHA for 4b75cb6 - Browse repository at this point
Copy the full SHA 4b75cb6View commit details -
Configuration menu - View commit details
-
Copy full SHA for 003c6ee - Browse repository at this point
Copy the full SHA 003c6eeView commit details -
Configuration menu - View commit details
-
Copy full SHA for 22c3ce7 - Browse repository at this point
Copy the full SHA 22c3ce7View commit details -
Configuration menu - View commit details
-
Copy full SHA for 30cf393 - Browse repository at this point
Copy the full SHA 30cf393View commit details -
Configuration menu - View commit details
-
Copy full SHA for d40adfa - Browse repository at this point
Copy the full SHA d40adfaView commit details
Commits on Mar 15, 2022
-
[HUDI-3547] Introduce MaxwellSourcePostProcessor to extract data from…
Configuration menu - View commit details
-
Copy full SHA for 3b59b76 - Browse repository at this point
Copy the full SHA 3b59b76View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6ed7106 - Browse repository at this point
Copy the full SHA 6ed7106View commit details -
[HUDI-3619] Fix HoodieOperation fromValue using wrong constant value (#…
…5033) Co-authored-by: root <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 9bdda2a - Browse repository at this point
Copy the full SHA 9bdda2aView commit details -
[HUDI-3514] Rebase Data Skipping flow to rely on MT Column Stats index (
Alexey Kudinkin authoredMar 15, 2022 Configuration menu - View commit details
-
Copy full SHA for 5e8ff8d - Browse repository at this point
Copy the full SHA 5e8ff8dView commit details -
[HUDI-3633] Allow non-string values to be set in TypedProperties (#5045)
* [HUDI-3633] Allow non-string values to be set in TypedProperties * Override getProperty to ignore instanceof string check
Configuration menu - View commit details
-
Copy full SHA for d514570 - Browse repository at this point
Copy the full SHA d514570View commit details -
[HUDI-3589] flink sync hive metadata supports table properties and se…
…rde properties (#4995)
Configuration menu - View commit details
-
Copy full SHA for 55dca96 - Browse repository at this point
Copy the full SHA 55dca96View commit details
Commits on Mar 16, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 296a0e6 - Browse repository at this point
Copy the full SHA 296a0e6View commit details -
[HUDI-3607] Support backend switch in HoodieFlinkStreamer (#5032)
Configuration menu - View commit details
-
Copy full SHA for 91849c3 - Browse repository at this point
Copy the full SHA 91849c3View commit details
Commits on Mar 17, 2022
-
[Hudi-3376] Add an option to skip under deletion files for HoodieMeta…
…dataTableValidator (#4994) Co-authored-by: yuezhang <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 8ca9a54 - Browse repository at this point
Copy the full SHA 8ca9a54View commit details -
[HUDI-3404] Automatically adjust write configs based on metadata tabl…
…e and write concurrency mode (#4975)
Configuration menu - View commit details
-
Copy full SHA for 95e6e53 - Browse repository at this point
Copy the full SHA 95e6e53View commit details -
Configuration menu - View commit details
-
Copy full SHA for 5ba2d9a - Browse repository at this point
Copy the full SHA 5ba2d9aView commit details -
[HUDI-3645] Fix NPE caused by multiple threads accessing non-thread-s…
…afe HashMap (#5028) - Change HashMap in HoodieROTablePathFilter to ConcurrentHashMap
Configuration menu - View commit details
-
Copy full SHA for bf191f8 - Browse repository at this point
Copy the full SHA bf191f8View commit details -
[HUDI-2439] Replace RDD with HoodieData in HoodieSparkTable and commi…
…t executors (#4856) - Adopt HoodieData in Spark action commit executors - Make Spark independent DeleteHelper, WriteHelper, MergeHelper in hudi-client-common - Make HoodieTable in WriteClient APIs have raw type to decouple with Client's generic types
Configuration menu - View commit details
-
Copy full SHA for 7446ff9 - Browse repository at this point
Copy the full SHA 7446ff9View commit details -
[MINOR] HoodieFileScanRDD could print null path (#5056)
Co-authored-by: Rex An <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 9ece775 - Browse repository at this point
Copy the full SHA 9ece775View commit details
Commits on Mar 18, 2022
-
[HUDI-3598] Row Data to Hoodie Record Operator parallelism needs to a…
…lways be consistent with input operator (#5049) for chaining purpose Co-authored-by: jerryyue <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 6fe4d6e - Browse repository at this point
Copy the full SHA 6fe4d6eView commit details -
[HUDI-3656] Adding medium sized dataset for clustering and minor fixe…
…s to integ tests (#5063)
Configuration menu - View commit details
-
Copy full SHA for 2551c26 - Browse repository at this point
Copy the full SHA 2551c26View commit details -
Configuration menu - View commit details
-
Copy full SHA for 316e38c - Browse repository at this point
Copy the full SHA 316e38cView commit details
Commits on Mar 19, 2022
-
[HUDI-3457] Refactored Spark DataSource Relations to avoid code dupli…
…cation (#4877) Refactoring Spark DataSource Relations to avoid code duplication. Following Relations were in scope: - BaseFileOnlyViewRelation - MergeOnReadSnapshotRelaation - MergeOnReadIncrementalRelation
Alexey Kudinkin authoredMar 19, 2022 Configuration menu - View commit details
-
Copy full SHA for 099c2c0 - Browse repository at this point
Copy the full SHA 099c2c0View commit details
Commits on Mar 20, 2022
-
[HUDI-3663] Fixing Column Stats index to properly handle first Data T…
…able commit (#5070) * Fixed metadata conversion util to extract schema from `HoodieCommitMetadata` * Fixed failure to fetch columns to index in empty table * Abort indexing seq in case there are no columns to index * Fallback to index at least primary key columns, in case no writer schema could be obtained to index all columns * Fixed `getRecordFields` incorrectly ignoring default value * Make sure Hudi metadata fields are also indexed
Alexey Kudinkin authoredMar 20, 2022 Configuration menu - View commit details
-
Copy full SHA for 1b6e201 - Browse repository at this point
Copy the full SHA 1b6e201View commit details -
Configuration menu - View commit details
-
Copy full SHA for 15d1c18 - Browse repository at this point
Copy the full SHA 15d1c18View commit details
Commits on Mar 21, 2022
-
Configuration menu - View commit details
-
Copy full SHA for 799c78e - Browse repository at this point
Copy the full SHA 799c78eView commit details -
Configuration menu - View commit details
-
Copy full SHA for a118d56 - Browse repository at this point
Copy the full SHA a118d56View commit details -
[HUDI-3559] Flink bucket index with COW table throws NoSuchElementExc…
…eption Actually method FlinkWriteHelper#deduplicateRecords does not guarantee the records sequence, but there is a implicit constraint: all the records in one bucket should have the same bucket type(instant time here), the BucketStreamWriteFunction breaks the rule and fails to comply with this constraint. close #5018
Configuration menu - View commit details
-
Copy full SHA for 26e5d2e - Browse repository at this point
Copy the full SHA 26e5d2eView commit details
Commits on Mar 22, 2022
-
[HUDI-1436]: Provide an option to trigger clean every nth commit (#4385)
- Provided option to trigger clean every nth commit with default number of commits as 1 so that existing users are not affected. Co-authored-by: sivabalan <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for ca0931d - Browse repository at this point
Copy the full SHA ca0931dView commit details -
[HUDI-3640] Set SimpleKeyGenerator as default in 2to3 table upgrade f…
…or Spark engine (#5075)
Configuration menu - View commit details
-
Copy full SHA for 9b6e138 - Browse repository at this point
Copy the full SHA 9b6e138View commit details -
[HUDI-2883] Refactor hive sync tool / config to use reflection and st…
…andardize configs (#4175) - Refactor hive sync tool / config to use reflection and standardize configs Co-authored-by: sivabalan <[email protected]> Co-authored-by: Rajesh Mahindra <[email protected]> Co-authored-by: Raymond Xu <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 5f570ea - Browse repository at this point
Copy the full SHA 5f570eaView commit details