Document/export copy-free string allocation?

[stevengj]

There are now (as of #19449) undocumented functions Base._string_n(n) (to allocate an n-byte string) and Base.StringVector(n) (to allocate an n-byte array that can be converted to a string without copying). Should some version of these be documented and exported? They seem useful for e.g. string processing and calling C APIs expecting pre-allocated string buffers.

Note also that the String(v::Vector{UInt8}) documentation, which says that it takes "ownership" of the array, seems to be wrong now (unless v was allocated with StringVector).

cc @JeffBezanson, @nalimilan

[stevengj]

At some point, @JeffBezanson said in #19945 that something like StringVector was the default for all byte arrays, but this seems to be no longer the case?

[nalimilan]

BTW, am I right that String(take!(b)) now makes a copy? This is annoying since we just deprecated takebuf_string...

[stevengj]

@nalimilan, no, that does not typically make a copy, because IOBuffer by default uses StringVector to allocate its buffer.

[JeffBezanson]

StringVector is used by IOBuffer, but is otherwise not the default for all byte arrays.

[stevengj]

Would it be bad to make this the default for all byte arrays?

[StefanKarpinski]

To the extent possible, if we're going to export some of these interfaces, I'd like to put some thought into making sure that they're future proof in the sense that the same API will continue to be usable even if we change the underlying string implementation. The codeunit change is a good example of this. If we're not sure about that, I'd rather not export, so that even if people do use these interfaces in performance-critical string code, they're fairly easy to find since they have to reach into Base to get at the interfaces in question.

[simonbyrne]

Any more thoughts on this? It is a very convenient API, and is currently used by HDF5.jl

[stevengj]

It's used in CSTParser.jl, CSV.jl, DelimitedFiles.jl, ExactConvolution.jl, Format.jl, GraphQLParser.jl, HDF5.jl, HTTP.jl, JSON2.jl, JSON3.jl, JuliaSyntax.jl, LazyJSON.jl, MySQL.jl, Parsers.jl, Pidfile.jl, ShortStrings.jl, SourceWalk.jl, StrBase.jl, StringViews.jl, and ZMQ.jl …