Add splitlines

[mpastell]

Adds splitlines as suggested in #19759

[KristofferC]

Could you add a doctest here showing how it is used?

[StefanKarpinski]

LGTM, aside from a doctest example being nice (especially since folks have been working so hard at adding doctests to so many things).

[stevengj]

Or maybe just implement readlines(s::AbstractString) and eachline(s::AbstractString) methods?

[StefanKarpinski]

julia> readlines("/usr/share/dict/words")
235886-element Array{String,1}:
 "A"
 "a"
 "aa"
 "aal"
 "aalii"
 "aam"
 "Aani"
 "aardvark"
 "aardwolf"
 ⋮
 "zymurgy"
 "Zyrenian"
 "Zyrian"
 "Zyryan"
 "zythem"
 "Zythia"
 "zythum"
 "Zyzomys"
 "Zyzzogeton"

[stevengj]

Oh, right, I forgot that you can pass the filename to readlines and eachline.

[stevengj]

I'm not sure calling readlines on an IOBuffer is the right implementation here, though. The split function returns an array of substrings, so shouldn't splitlines do the same?

[stevengj]

The split function takes a duck-typed splitter argument that just has to work with search. Maybe the right thing to do would be to define a new splitter type, something like:

immutable LinebreakSplitter; chomp::Bool; end
function search(s::AbstractString, splitter::LinebreakSplitter, i::Integer)
       newline = search(s, '\n', i)
       newline < i && return newline:newline
       splitter.chomp || return nextind(s, newline):newline
       prev = prevind(s, newline)
       return prev ≥ i && s[prev] == '\r' ? (prev:newline) : (newline:newline)
end
splitlines(s::AbstractString; chomp::Bool=true) = split(s, LinebreakSplitter(chomp))

Not only does this re-use the split logic, but also this way the LinebreakSplitter could potentially be used with other functions (e.g. replace).

[mpastell]

Do we actually need the chomp option?

splitlines(str; limit=0, keep=true) = split(str, r"\n|\r\n", limit=limit, keep=keep)

would be a simple implementation that is compatible with split.

[StefanKarpinski]

I agree that LinebreakSplitter seems overly complicated. @mpastell's definition of splitlines or something equivalent to it seems sufficient to me.

[stevengj]

I'm not sure how important it is in practice, but Python's version has the equivalent of the chomp=false option. I guess this would matter if you wanted to write the data back out while preserving the line endings.

[stevengj]

The LinebreakSplitter is only about 2x faster than the regex version in a simple test on join([randstring(50)*(rand(Bool)?"\n":"\r\n") for i = 1:10^4]), so I would only use it if we decide the chomp option is important (or if we decide a 2x speed boost is important here).

[brenhinkeller]

So it looks like as of Julia 1.8.3 this is now provided by split:

julia> s = "this\nstring has several\nlines"
"this\nstring has several\nlines"

julia> split(s, '\n')
3-element Vector{SubString{String}}:
 "this"
 "string has several"
 "lines"

so I think we can close this, but if anyone disagrees (or wants to add an alias splitlines(s) = split(s, '\n') feel free to reopen or make a new PR!

[mlhetland]

Does this mean split now handles errors with \r (which was the point of this issue and PR)? If not, maybe at least the issue should be kept open?

[brenhinkeller]

Ah I missed the point

[vtjnash]

Quite conflicted for such a small change. The implementation here is already mentioned now in the docstring for eachline though and seems trivial enough not to need this PR to add it

help?> eachline

  eachline(io::IO=stdin; keep::Bool=false)
  eachline(filename::AbstractString; keep::Bool=false)

...

  To iterate over each line of a String, eachline(IOBuffer(str)) can be used.