[HUDI-3236] use fields'comments persisted in catalog to fill in schema #4587
Conversation
| throw new AnalysisException(s"Can't find column `$columnName` given table data columns " + | ||
| s"${hoodieCatalogTable.dataSchema.fieldNames.mkString("[`", "`, `", "`]")}") | ||
| ) | ||
|
|
There was a problem hiding this comment.
I don't think we support rename operation now， Why remove relevant judgments？
There was a problem hiding this comment.
This changes also do not support rename operation. See the findColumnByName's implement.
There was a problem hiding this comment.
thanks . I mean, maybe we can throw a corresponding exception to tell the user that we don't support rename at present,
There was a problem hiding this comment.
I think It is better to have the same behavior as Spark.
| } | ||
| checkAnswer(s"select id, name, price, ts, ext0 from $newTableName")( | ||
| Seq(1, "a1", 10.0, 1000, null) | ||
| ) | ||
| // Alter table column type | ||
|
|
||
| // change column's data type | ||
| spark.sql(s"alter table $newTableName change column id id bigint") |
There was a problem hiding this comment.
now， hudi on spark cannot support dataType change。 hudi use spark parquetFileFormat to read parquet file，but that reader is hardly support type change。 see the origin code of spark project ParquetVectorUpdaterFactory.getUpdater
This test is actually wrong， if you add spark.sql(s"select id from $newTableName").show(false) in line 95， this test will failed。
There was a problem hiding this comment.
I know that. See the details in https://issues.apache.org/jira/browse/HUDI-3237.
YannByron
force-pushed
the
master_3236
branch
2 times, most recently
from
790f50e
to
dec3b88
Compare
YannByron
force-pushed
the
master_3236
branch
from
dec3b88
to
4ca6772
Compare
Tips
What is the purpose of the pull request
(For example: This pull request adds quick-start document.)
Brief change log
(for example:)
Verify this pull request
(Please pick either of the following options)
This pull request is a trivial rework / code cleanup without any test coverage.
(or)
This pull request is already covered by existing tests, such as (please describe tests).
(or)
This change added tests and can be verified as follows:
(example:)
Committer checklist
Has a corresponding JIRA in PR title & commit
Commit message is descriptive of the change
CI is green
Necessary doc changes done or have another open PR
For large changes, please consider breaking it into sub-tasks under an umbrella JIRA.