[docs] Document using a different separator for read_csv #27850
Conversation
pcmoritz
requested review from
ericl,
scv119,
clarkzinzow,
jjyao and
jianoaix
as code owners
pcmoritz
force-pushed
the
fix-tsv-reading
branch
2 times, most recently
from
2b4353a
to
cce97d4
Compare
| >>> # because by default read_csv only reads .csv files. | ||
| >>> from pyarrow import csv | ||
| >>> parse_options = csv.ParseOptions(delimiter="\\t") | ||
| >>> ray.data.read_csv( # doctest: +SKIP |
There was a problem hiding this comment.
This needs to be skipped at the moment which is very very sad -- tracked in #27853
Signed-off-by: Philipp Moritz <[email protected]>
Signed-off-by: Philipp Moritz <[email protected]>
Signed-off-by: Philipp Moritz <[email protected]>
Signed-off-by: Philipp Moritz <[email protected]>
pcmoritz
force-pushed
the
fix-tsv-reading
branch
from
3e7f01c
to
1945998
Compare
Co-authored-by: matthewdeng <[email protected]>
python/ray/data/read_api.py
Outdated
| >>> # Read files that use a different delimiter. The partition_filter=None is needed here | ||
| >>> # because by default read_csv only reads .csv files. | ||
| >>> from pyarrow import csv | ||
| >>> parse_options = csv.ParseOptions(delimiter="\\t") |
There was a problem hiding this comment.
it should be delimiter="\t" right? I tested it out in #27738 (comment) .
There was a problem hiding this comment.
I tried that before and then it didn't render correctly in the docs, let me try again :)
There was a problem hiding this comment.
Ok, so the right solution that makes this work in all cases (both raw docstring and readthedocs) seems to be to put an r before the doc string and not escape this, which I did now :)
| @@ -0,0 +1,150 @@ | |||
| 5.1 3.5 1.4 0.2 setosa | |||
There was a problem hiding this comment.
We also need to define the schema at the first line:
sepal.length sepal.width petal.length petal.width variety
There was a problem hiding this comment.
Thanks, I fixed that now :)
| 6.3 2.5 5.0 1.9 virginica | ||
| 6.5 3.0 5.2 2.0 virginica | ||
| 6.2 3.4 5.4 2.3 virginica | ||
| 5.9 3.0 5.1 1.8 virginica |
|
Thanks for all the feedback, this should be ready to merge now, @c21 can you approve when you get a chance? |
…#27850) See discussion in ray-project#27738 Co-authored-by: matthewdeng <[email protected]> Signed-off-by: Zhi Lin <[email protected]>
…#27850) See discussion in ray-project#27738 Co-authored-by: matthewdeng <[email protected]> Signed-off-by: ilee300a <[email protected]>
Why are these changes needed?
See discussion in #27738
Related issue number
Checks
git commit -s) in this PR.scripts/format.shto lint the changes in this PR.