Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG] With_new_children not implemented for sample #2528

Merged
merged 3 commits into from
Jul 17, 2024
Merged

[BUG] With_new_children not implemented for sample #2528

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
2fbf05f
fix
Jul 17, 2024
e6934af
remove catch all
Jul 17, 2024
22be551
add test
Jul 17, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion src/daft-plan/src/logical_plan.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -199,7 +199,9 @@
Self::Sink(Sink { sink_info, .. }) => Self::Sink(Sink::try_new(input.clone(), sink_info.clone()).unwrap()),
Self::MonotonicallyIncreasingId(MonotonicallyIncreasingId {column_name, .. }) => Self::MonotonicallyIncreasingId(MonotonicallyIncreasingId::new(input.clone(), Some(column_name))),
Self::Unpivot(Unpivot {ids, values, variable_name, value_name, output_schema, ..}) => Self::Unpivot(Unpivot { input: input.clone(), ids: ids.clone(), values: values.clone(), variable_name: variable_name.clone(), value_name: value_name.clone(), output_schema: output_schema.clone() }),
_ => panic!("Logical op {} has two inputs, but got one", self),
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't catch this bug when I implemented Sample because of this catch all branch, so just gonna remove it in case someone else implements a new logical op and misses it too

Self::Sample(Sample {fraction, with_replacement, seed, ..}) => Self::Sample(Sample::new(input.clone(), *fraction, *with_replacement, *seed)),
Self::Concat(_) => panic!("Concat ops should never have only one input, but got one"),
Self::Join(_) => panic!("Join ops should never have only one input, but got one"),

Check warning on line 204 in src/daft-plan/src/logical_plan.rs

View check run for this annotation

Codecov / codecov/patch

src/daft-plan/src/logical_plan.rs#L202-L204 

Added lines #L202 - L204 were not covered by tests
},
[input1, input2] => match self {
Self::Source(_) => panic!("Source nodes don't have children, with_new_children() should never be called for Source ops"),
Expand Down
16 changes: 16 additions & 0 deletions tests/dataframe/test_sample.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -81,3 +81,19 @@ def test_sample_with_replacement(make_df, valid_data: list[dict[str, float]]) ->
assert df.column_names == list(valid_data[0].keys())
# Check that the two rows are the same, which should be for this seed.
assert all(col[0] == col[1] for col in df.to_pydict().values())


def test_sample_with_concat(make_df, valid_data: list[dict[str, float]]) -> None:
df1 = make_df(valid_data)
df2 = make_df(valid_data)

df1 = df1.sample(fraction=0.5, seed=42)
df2 = df2.sample(fraction=0.5, seed=42)

df = df1.concat(df2)
df.collect()

assert len(df) == 4
assert df.column_names == list(valid_data[0].keys())
# Check that the two rows are the same, which should be for this seed.
assert all(col[:2] == col[2:] for col in df.to_pydict().values())
Loading