update "left" and "right" for @join, eliminate "overlap"

[bansp]

This is just a note for now about what essentially qualifies as an upgrade in the standard that is not yet reflected upstream, in the Guidelines: in Brussels in 2023, we determined that "left" and "right" don't make much sense universally, given the directionality algorithms. They should be "prev" and "next". And the definition of "overlap" just doesn't hold.

So we probably need a temporary fix in the standard while the matter is submitted for the Council's consideration. We can count on it getting done in 4.8.0 at the earliest, but the ticket should be submitted more or less ASAP.

[ebeshero]

@bansp I understand the logic of revising "left" and "right" for "prev" and "next", but I have two questions:

1. Could the class of att.linguistic apply to the texts of ergodic poems or anagrams where the "prev" and "next" is ambiguous and text is intended to be readable in multiple directions? Or is this not an issue? (Perhaps not an issue becuase we can present encodings of such documents by unpacking them as linear strings: One poem might produce two encodings--one with the tokens joined moving left to right, and the other right to left.)

2. What's wrong with the definition of "overlap"? Is superposition/overlap of tokens basically not possible because--such a combination would form a new token anyway? To be sure, it would be helpful to see an example handling overlap and describing the "other devices" required "to more precisely locate this token in the character stream".

Sorry if the questions are themselves misguided! We may need to clarify that non-linguists and encoders of weird anagrammatic poetry should steer clear of att.linguistic and try something else.

[bansp]

Thanks for the feedback, @ebeshero . Re: 1 -- great question, and you've even answered it yourself! ;-) Yes, it's about how the text may be streamed for processing, rather than a description of a state. So ergodic poems would need preprocessing for the word stream to get linearised, even if that happens along more than one axis. Still, this is something that the documentation should make obvious.

Re: 2 -- "overlap" was originally supposed to be usable in sequential arrangements of segments, and in such cases, it doesn't provide enough information, and the standard currently suggests resorting to standOff for cases of overlap. Still, you're right to ask about that, because, in theory, why not do "overlap,right" (whoopsie, "overlap,next"), although I keep thinking that in order for this information to be meaningful, signalling overlap in the @join attribute is insufficient. But maybe I just fail to imagine the use cases. Will bring that to Laurent's attention.

Thanks a lot!