Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return &Arc reference to inner trait object #11103

Merged
merged 2 commits into from
Jun 25, 2024
Merged

Conversation

linhr
Copy link
Contributor

@linhr linhr commented Jun 24, 2024

Which issue does this PR close?

Closes #11100.

Rationale for this change

It's better to return &Arc reference to inner trait object for &self methods. This makes it easier to work with Rust lifetime rules when we need to cast trait object to concrete reference types. Please refer to the corresponding issue for an example.

What changes are included in this PR?

This PR updates the return type for few methods in ScalarUDF, AggregateUDF, WindowUDF, and SessionState.

Are these changes tested?

The changes are covered by existing tests.

Are there any user-facing changes?

Yes, these are breaking changes to the public API. The changes should be small, since the user can simply call .clone() on &Arc if needed.

@github-actions github-actions bot added logical-expr Logical plan and expressions core Core DataFusion crate labels Jun 24, 2024
@alamb alamb added the api change Changes the API exposed to users of the crate label Jun 24, 2024
Copy link
Contributor

@alamb alamb left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you @linhr -- the usecase described on #11100 makes sense to me

I have flagged this PR as a breaking API change as well

@alamb
Copy link
Contributor

alamb commented Jun 24, 2024

Looks like the CI is failing -- I think running cargo fmt and checking in the result is what is needed to fix: https://github.com/apache/datafusion/actions/runs/9646694377/job/26620921903?pr=11103

@linhr
Copy link
Contributor Author

linhr commented Jun 25, 2024

Thank you @alamb ! I've fixed the code formatting issue.

@@ -236,13 +236,14 @@ mod tests {
fn setup_context() -> (SessionContext, Arc<dyn SchemaProvider>) {
let mut ctx = SessionContext::new();
ctx.register_catalog_list(Arc::new(DynamicFileCatalog::new(
ctx.state().catalog_list(),
ctx.state().catalog_list().clone(),
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when I see .clone() Im now thinking what if we comment it the similar way as .unwrap() back in the day. Like say clone is cheap here because it is Arc::clone so only reference gets cloned.... thats just an idea, its too cumbersome to make it happen

Copy link
Contributor

@alamb alamb Jun 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One thing we do in InfluxDB is use this pattern to make it clear

Suggested change
ctx.state().catalog_list().clone(),
Arc::clone(ctx.state().catalog_list())

to make it explicit.

There is a clippy lint https://rust-lang.github.io/rust-clippy/v0.0.212/#clone_on_ref_ptr we could turn on in datafusion to enable this

Here is how it is done in influxdb:

https://github.com/influxdata/influxdb3_core/blob/0f5ecbd6b17f83f7ad4ba55699fc2cd3e151cf94/Cargo.toml#L117-L118

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a good idea, let's do it

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Filed #11143 to track

Copy link
Contributor

@comphead comphead left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks @linhr this pattern makes a lot of sense to me

@alamb alamb merged commit 5f02c8a into apache:main Jun 25, 2024
23 checks passed
@linhr linhr deleted the self-arc-dyn-ref branch June 27, 2024 05:05
findepi pushed a commit to findepi/datafusion that referenced this pull request Jul 16, 2024
* Return `&Arc` reference to inner trait object

* Format code
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api change Changes the API exposed to users of the crate core Core DataFusion crate logical-expr Logical plan and expressions
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Methods should return &Arc references to inner trait objects for &self
4 participants