[data] Change fixture from shutdown_only to `ray_start_regular_shar…

…ed` for `test_csv_read_filter_non_csv_file` (ray-project#47513)

## Why are these changes needed?
Seems that ray-project#47467 ended up
breaking some niche setup for this test, by changing the fixture from
`shutdown_only` to `ray_start_regular_shared` we are able to get the
test passing again.

## Related issue number

## Checks

- [x] I've signed off every commit(by using the -s flag, i.e., `git
commit -s`) in this PR.
- [x] I've run `scripts/format.sh` to lint the changes in this PR.
- [ ] I've included any doc changes needed for
https://docs.ray.io/en/master/.
- [ ] I've added any new APIs to the API Reference. For example, if I
added a
method in Tune, I've added it in `doc/source/tune/api/` under the
           corresponding `.rst` file.
- [x] I've made sure the tests are passing. Note that there might be a
few flaky tests, see the recent failures at https://flakey-tests.ray.io/
- Testing Strategy
   - [x] Unit tests
   - [ ] Release tests
   - [ ] This PR is not tested :(

Signed-off-by: Matthew Owen <[email protected]>
Signed-off-by: ujjawal-khare <[email protected]>

python/ray/data/tests/test_csv.py

@@ -758,9 +758,7 @@ def test_csv_read_filter_non_csv_file(ray_start_regular_shared, tmp_path):
 
     # Directory of CSV files.
     ds = ray.data.read_csv(tmp_path)
-    actual_data = sorted(ds.to_pandas().itertuples(index=False))
-    expected_data = sorted(pd.concat([df, df]).itertuples(index=False))
-    assert actual_data == expected_data, (actual_data, expected_data)
+    assert ds.to_pandas().equals(pd.concat([df, df], ignore_index=True))
 
     # Non-CSV file in Parquet format.
     table = pa.Table.from_pandas(df)