input: Change titles to objects

[bdarcus]

Description

As an alternative to title parsing to support independent formatting of main and subtitles, this converts title strings to objects, with the following properties:

• full
• main
• sub (an array, to support multiple)
• sub-sub
• alternate
• short

I added embedded examples to the schema, drawn directly from style guides (MLA and Chicago), and here's example docs generated from the schema.

addresses #310.

Alternatives

Alternatives I considered but rejected (though I don't feel strongly against them) using arrays for parts.

--- 
title-alt-1: 
  # change main parts to an array (Denis suggested this possibility)
  content: 
    - Main
    - Subtitle
    - "Second Subtitle"
  short: Main
title-alt-2: 
  # also change alt to an array
  content: 
    - Main
    - Subtitle
    - "Second Subtitle"
  short: Main

But, after subsequent discussion below, sub is now an array.

See Also

https://discourse.citationstyles.org/t/design-principles-for-csl-json/1671

Type of change

• Variable addition (adds a new variable or type string)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)
• This change requires a documentation update

[bdarcus]

If we went with an object structure, I see a few issues here. A title object needs to always have a "full" property. That is needed to indicate the original punctuation for styles that don't normalize the punctuation (this is most of them).

Added.

I don't see a need for "alternate" in the example you give. That should just be understood in our data model as "sub".

If we're going to go this route, we might as well do it right, and an alternate title is not a subtitle.

Practically speaking, a) they're formatted differently., and b) what happens if you have a title and subtitles and an alternate title? It's certainly feasible.

[denismaier]

Also, c) what if alternate has one or two subtitles?

[bdarcus]

Also, c) what if alternate has one or two subtitles?

Yeah, I wondered about that :-)

I made the decision, perhaps unwise (IDK), that this would be unlikely.

WDYT?

[denismaier]

I made the decision, perhaps unwise (IDK), that this would be unlikely.

I don't really know. May be unlikely enough, but if we decided that these things are not subtitles or second subtitles, but fall rather in a third category then this category should be parallel to normal titles and similar in structure.

title-ver1:
  main: First main title
  sub: First subtitle
  alternate-main: First alternate title
  alternate-sub: First alternate subtitle
title-ver2:
  main: First main title
  sub: First subtitle
  alternate: 
    main: First alternate title
    sub: First alternate subtitle
title-ver3:
  - main: First main title
    sub: First subtitle
  - main: First alternate title
    sub: First alternate subtitle

Or even:

title:
  main: First main title
  sub: First subtitle
alternate-title:
  main: First alternate title
  sub: First alternate subtitle

None of these are really pretty I have to admit... Other options?

[bwiernik]

The "full" is there to support unparsed titles, but would mean the @Form='sub' and such would not work (except for "short").

In my reading of it, once this hits the processor for citation formatting, the "full" isn't really there to support unparsed titles. It's there for rendering in styles that don't normalize the punctuation. With this model, unparsed titles are handled at the preprocessor stage and parsed into their components.

In APA, for example, the processor would:

1. Render the "full" title.
2. Use "main" and "sub" to determine capitalization (e.g., matching the "sub" text in "full" and replacing it with the first character capitalizing).

Another option would be to add main-delimiter and sub-delimiter elements to hold the original punctuation. So:

--- 
title:
  main: "Finis Coronat Opus"
  main-delimiter: ":"
  sub: "A Curious Reciprocity"
  sub-delimiter: ";"
  sub-sub: "Shelley’s “When the Lamp Is Shattered”"
...

Practically speaking, a) they're formatted differently., and b) what happens if you have a title and subtitles and an alternate title? It's certainly feasible.
c) what if alternate has one or two subtitles?

I'm just really not sure (a) is the case. Looking at MLA and Chicago, such "alternate" titles are formatted just like subtitles. The only difference is the odd delimiter ; or,. Looking at examples, every one I've seen has had an "alternate" title or a "sub" title(s), but not both. How would a separate "alternate" element work with syles? Would need need a form="alternate", etc. for what is admittedly a very unusual case. Would we specify that form="sub" should render both sub and alternate elements?

(b) and (c) can be addressed by allowing "sub" to be an object, rather than having a "sub-sub" element:

--- 
title:
  full: "Finis Coronat Opus: A Curious Reciprocity; Shelley’s “When the Lamp Is Shattered”"
  main: "Finis Coronat Opus"
  sub: 
    - "A Curious Reciprocity"
    - "Shelley’s “When the Lamp Is Shattered”"
...

or:

--- 
title:
  main: "Finis Coronat Opus"
  main-delimiter: ":"
  sub: 
    - "A Curious Reciprocity"
    - "Shelley’s “When the Lamp Is Shattered”"
  sub-delimiter: ";"
...

[denismaier]

---

title:
main: "Finis Coronat Opus"
main-delimiter: ":"
sub:
- "A Curious Reciprocity"
- "Shelley’s “When the Lamp Is Shattered”"
sub-delimiter: ";"
...

Yet another option with specified punctuation;

title:
  main:
    content: "Finis Coronat Opus"
    delimiter: ": "
  sub:
    - content: "A Curious Reciprocity"
      delimiter: "; "
    - content: "Shelley’s “When the Lamp Is Shattered”"

Or, as said before, just include in main and sub:

title:
  main: "Finis Coronat Opus: "
  sub: 
    - "A Curious Reciprocity; "
    - "Shelley’s “When the Lamp Is Shattered”"

[bwiernik]

If we did the punctuation at the end of the titles, then we could specify that form="main" renders the main title without the delimiter (final punctuation mark). That would work.

[denismaier]

If we did the punctuation at the end of the titles, then we could specify that form="main" renders the main title without the delimiter (final punctuation mark). That would work.

But, of course, that would again add complexity for citeprocs as they would have to extract the punctuation.

[bwiernik]

That said, it seems like a favorable balance between data model complexity and processor complexity. Processors need to be able to substitute delimiters anyway, and this is simpler the needing to compare full to main/sub for capitalization.

[bdarcus]

I just posted a note on the main issue asking us to clarify the requirements, which might help us settle this.

If we did the punctuation at the end of the titles, then we could specify that form="main" renders the main title without the delimiter (final punctuation mark). That would work.

Exclamation and question marks would be exceptions though. In this approach, processors would still need to deal with trailing punctuation.

[bwiernik]

It’s pretty similar to existing double-punctuation requirements (e.g., to not add a period at the end of a title).

If we specified default delimiters and made normalize-title-delimiters a Boolean attribute, that makes a common set of requirements of that line up with the existing double punctuation step:

Eg,

1. If normalize-title-delimiters is true, remove : or . from the end of the title parts (this is notably similar to existing strip-periods)
2. Add a title-delimiter between main and sub
3. Follow the normal double punctuation prevention rules

[bdarcus]

So am I correct you guys are thinking we want to remove sub-sub and convert sub to an array?

[bwiernik]

So am I correct you guys are thinking we want to remove sub-sub and convert sub to an array?

A list/object, not an array, but yes.

[bdarcus]

We're talking json schema, which doesn't have lists. Array; right?

[bwiernik]

I mean {"First subtitle;", "Second subtitle"}

Not ["First subtitle;", "Second subtitle"]

[bdarcus]

I mean {"First subtitle;", "Second subtitle"}

Not ["First subtitle;", "Second subtitle"]

That's not valid JSON ;-)

The second is.

An object is an unordered list of key values.

An array is an ordered list.

The YAML examples that Denis posted are arrays.

[bdarcus]

I don't really know. May be unlikely enough, but if we decided that these things are not subtitles or second subtitles, but fall rather in a third category then this category should be parallel to normal titles and similar in structure.

On this, my thinking now is we should have the alternate property as now, but if we need it, we can later add an alternate-sub property to parallel the main sub property.

My suspicion is it won't be needed though.

[bdarcus]

Given where we are ATM, what are we doing with the full property?

Keeping it? Modifying the description?

[bwiernik]

To answer that, I think we need to answer the last question here: #324

How do we declare that a title variable is fully parsed vs needs preprocessing still?

[bwiernik]

On this, my thinking now is we should have the alternate property as now,

In a style, would alternate then be rendered with title form="long" and title form="sub"?

In the unlikely event that both a sub and alternate title are present, what then?

Finally, I am concerned about the label "alternate" and that it is likely to be confused by users (e.g., with translated-title). How about "second" instead?

[bdarcus]

On this, my thinking now is we should have the alternate property as now,

In a style, would alternate then be rendered with title form="long" and title form="sub"?

I was thinking we'd add an "alternate" title form:

<text variable="title" form="alternate"/>

In the unlikely event that both a sub and alternate title are present, what then?

<text variable="title" form="sub"/>
<text variable="title" form="alternate" prefix=", or "/>

Finally, I am concerned about the label "alternate" and that it is likely to be confused by users (e.g., with translated-title). How about "second" instead?

Here's how I ended up there.

Both MLA and Chicago talk in terms of "double titles," but the MLA website description begins "For an alternative or double title ..."

And the convention to preface the second one with the "or" in English clearly signals that it's an "alternate."

It's "second" only in the context of output formatting.

What does APA say on this?

Do you have any view on this question @denismaier?

[denismaier]

I was thinking we'd add an "alternate" title form:

<text variable="title" form="alternate"/>

I am not sure this is ideal. That would mean you will have to have three forms in a style.

<text variable="title" form="main"/>
<text variable="title" form="sub"/>
<text variable="title" form="alternate" prefix=", or "/>

And text variable="title" will have to concatenate "main", "sub" and "alternate".

[denismaier]

Both MLA and Chicago talk in terms of "double titles,"

That's why I suggested we could just treat titles as array:

title:
  - main: First main title
    sub: First subtitle
  - main: First alternate title
    sub: First alternate subtitle

Question is though if we really need the full semantic representation.

[bwiernik]

I really don’t think we should add a form="alternate". That is way too big a burden on style authoring. The most typical way for these to be cited is “Dr. Strangelove, or How I learned to stop worrying and love the bomb.” form=“long” should render that.

I’m going to reiterate my point here that it is only a few styles (Chicago, MLA) making any point of this at all. There, this second title is functionally equivalent to a subtitle with a special delimter—“or” with some punctuation. These titles are capitalized and otherwise formatted like a subtitle. APA doesn’t really treat this case specially, and the capitalization of “or” is inconsistent. Most examples have “or” after a colon, so it gets capitalized.

In modern applications of this convention, such as journal articles using the “or how I learned to stop worrying and love the ...”, it’s just treated as a subtitle. https://scholar.google.fr/scholar?hl=en&as_sdt=0%2C10&q=or+how+I+learned+to+stop+worrying+and+&btnG=

I suggest we just not cover this at all. Drop “alternate” and let the “double” title get parsed into main or sub as follows the general punctuation rules. In the vast majority of cases, this will work out fine. If a user needs to, they can manually indicate a subtitle split to control capitalization/etc. Very few styles seem to normalize away from a colon, so I don’t think we need to worry about the case of , or:

[denismaier]

I'm with @bwiernik here regarding "alternate". I think adding "alternate" would not have any practical benefits, on the contrary.

Basically, I'd say we're just dealing with a list of "title-parts", of which we will want to render all or only a subset.

title: 
  parts:
    - content: Main
      delimiter: ": "
    - content: Subtitle
      delimiter: "; or: "
    - content: "Some Alt Title"
  short: Main

If we want to align the input schema with the styles schema then let's use this:

title:
  main:
    content: "Finis Coronat Opus"
    delimiter: ": "
  sub:
    - content: "A Curious Reciprocity"
      delimiter: "; "
    - content: "Shelley’s “When the Lamp Is Shattered”"
  short: "Opus"

(Not sure regarding the punctuation yet..; we could just include it in the text fields to reduce the complexity of the schema, but ending the field with a space looks a bit odd.)

[denismaier]

Just a couple of questions/remarks to make sure we're on the same page:

It also supports localization, as the bit that connects a second/alternate title ("or" in English) could be localized.

I don't think that's an issue.

The PR also supports the non-normalized option with Denis' suggestion to allow delimiters as trailing punctuation, as would already be required to support another common case: titles ending in question or exclamation marks (the difference would simply be in styles that normalize punctuation, these two characters would not be suppressed or modified).

Just to make that clear: My suggestion was not to allow delimiters as trailing punctuation, but for this to work they must be accessible somehow.

If that's our decision, I do NOT want to include any kind of hack in the spec that treats those parts differently than any other subtitles.

What do you mean with that? What kind of hacks? I don't think this was suggested somewhere...

I also do not want us to say they are just a another subtitle, because they're not. For all intents and purposes, then, we would support a single main title, and multiple subtitles, and that's it. So either we support alternate titles now, or we don't.

So, say we don't add support for alternate this time: How will that look like in practical terms?

title:
  main: "Dr. Strangelove, or "
  sub: "How I learned to stop worrying and love the bomb"

Or just this:

title:
  main: "Dr. Strangelove, or How I learned to stop worrying and love the bomb"
  short: "Dr. Strangelove"

?

[denismaier]

Very few styles seem to normalize away from a colon, so I don’t think we need to worry about the case of , or:

Maybe English language styles don't do that, but as soon as you switch to a German style that's a different story. What speaks against just treating those "or"-constructs as a special character just like "?" and "!" ?

[bwiernik]

How will that look like in practical terms?

Any parsing by a preprocessor would happen based on the punctuation.

For German styles, would it really be writing for the colon to be replaced by a period in the Dr. Strangelove? If so, in that unusual situation, the user can ensure that the whole title is entered into main.

include any kind of hack in the spec that treats those parts differently than any other subtitles.

To be clear, Bruce, I was never suggesting these be treated any differently than any other subtitle. I was suggesting we treat them exactly like any other subtitle.

[denismaier]

For German styles, would it really be writing for the colon to be replaced by a period in the Dr. Strangelove? If so, in that unusual situation, the user can ensure that the whole title is entered into main.

If splitting happens on a ":", and ", or:" is not treated as a single token then this may lead to strange results because the colon may then possibly be normalized to a period. If we added ", or : " as a whole to the list of splitting characters that should not be normalized (just like question marks and exclamation marks),

To be clear, Bruce, I was never suggesting these be treated any differently than any other subtitle. I was suggesting we treat them exactly like any other subtitle.

I thought so too.

[bwiernik]

strange results because the colon may then possibly be normalized to a period

That’s my question. Is that strange?

Dr. Strangelove, or. How I Learned to Stop Worrying.

Is that different from
Age and Environmental Sustainability: A Meta-Analysis

They seem similar to me, both have sentence fragments following a period

[bwiernik]

And the simplest way for a user to avoid a colon becoming a period would be to enter the title in the Chicago-preferred format with commas instead. That wouldn't get normalized in any case.

[denismaier]

Having a period after "or" looks definitively strange to me, and much more than in the other case.

[bdarcus]

The practical import of what I was saying about not explicitly supporting alternate is that we would not include any "or" examples in documentation, but people can figure out on their own it worked for that purpose.

It also supports localization, as the bit that connects a second/alternate title ("or" in English) could be localized.

I don't think that's an issue.

Treating it as subtitle means, by definition, "or" is treated as a string, or at least as a string delimiter. You couldn't localize that without special, localized and brittle, parsing rules.

Treating it a a delimiter that links subtitle and alternate means you can localize.

Or do you think it's not a practical issue; that people won't care?

I'll try to catch up on the punctuation issue.

[denismaier]

Are do you think it's not a practical issue; that people won't care?

Exactly. I don't think you would ever want to localize that delimiter:
"Dr. Strangelove, oder: How I Learned to Stop Worrying." looks utterly wrong to me. That's part of the title and should not be localized.

[bdarcus]

On punctuation, I just want to point out that you guys are getting tripped up on a feature you said we're not formally supporting.

It's straightforward to say in the spec something like:

Unless X, trailing punctuation on title parts will be dropped, except for ?, !, and ___ .

But it seems another thing to look beyond punctuation to actual words, which may be in different languages.

Do we have a consensus of where we've ended up on this, that one of you could summarize? Or do you want to think about this a bit more?

I'd like to figure this out this week, but don't see a huge hurry. Better to get it right.

And, of course, we will have a comment period to hopefully get feedback.

[bwiernik]

I think these cases are rare enough that we can just not worry about it.

If a user is really concerned, they can structure their data to ensure no punctuation changes or incorrect capitalization:
A title, || {nocase: or:} How I learned

Given the rarity of these cases where this would be necessary, I think avoiding the complexity is a bigger benefit than trying to accommodate it. If we accommodated it across styles versus as a configurable feature, I can also imagine user confusion.

[denismaier]

I think these cases are rare enough that we can just not worry about it.

If a user is really concerned, they can structure their data to ensure no punctuation changes or incorrect capitalization:
A title, || {nocase: or:} How I learned

LGTM.

The only problem with this is that we have no control over how calling applications will pass their data to a citeproc. Will || etc work as an overriding mechanism? Will they have distinct fields for different parts of a title?

So to summarize this using the examples from the Chicago Manual: Titles like those will just be parsed like so, right?

a) ", or "

title:
  main: "A title, or How I learned to stop worrying and love ..."

b) "; or, "

title:
  main: "A title; "
  sub: "or, How I learned to stop worrying and love ..."

c) ", or: "

title:
  main: "A title, or:"
  sub: "How I learned to stop worrying and love ..."

a) is entirely unproblematic.
b) will require some user intervention if semicolons are normalized to colons or periods.
c) will require user intervention if colons are normalized to periods.

Is that all correct?

[bdarcus]

The only problem with this is that we have no control over how calling applications will pass their data to a citeproc. Will || etc work as an overriding mechanism? Will they have distinct fields for different parts of a title?

From the standpoint of CSL, || wouldn't exist.

So to summarize this using the examples from the Chicago Manual: Titles like those will just be parsed like so, right?

a) ", or "

title:
  main: "A title, or How I learned to stop worrying and love ..."

Yes, but in the end, it would be a user choice.

The only wrinkle with this choice is if you need to print a short variant, you probably want to drop the alternate.

But again: if we're not covering this case, I don't think we should use these examples to see how well it works.

It seems you're trying to slip the feature back in, convinced it will work, but I think you're mistaken.

Why I introduced alternate to begin with, though, which would be:

title:
  main: "A title, "
  alternate: "How I learned to stop worrying and love ..."

This would (assuming rules for dropping of trailing punctuation) require no user tweaking you noted below, and does not have the problem I note with the above.

Why is this a bad idea again?

...

a) is entirely unproblematic.

Not entirely, per my point above.

b) will require some user intervention if semicolons are normalized to colons or periods.
c) will require user intervention if colons are normalized to periods.

Is that all correct?

Mostly.

[bwiernik]

These cases are rare enough that I don’t think we need to trouble ourselves over them.

a) is entirely unproblematic.

If the user is concerned here would enter the “or How I learned” part in sub (using whatever mechanism the application provides to do that—fields, syntax parsing, etc.; that’s not our concern). The comma wouldn’t be normalized.

b) will require some user intervention if semicolons are normalized to colons or periods.

I don’t think this is a problem. I’ve definitely seen “Title. Or other title” in German journals. The “or” part can be case-protected with Markup if needed.

c) will require user intervention if colons are normalized to periods.

The user would place the “or” into the

The user intervention is to place “or” in the main

Why is this a bad idea again?

I have a few issues.

1. A minority of styles treat these any differently. APA would just capitalize the “or” after a colon of semicolon. So, supporting it would re-introduce the need to have style-configurable delimiters. Not supporting it means that we can avoid that.
2. Chicago et al. say to use the original punctuation around “or” unless none is given, so eliminating “or” from the item data and adding it back in as in your example wouldn’t actually meet the style guide anyway.
3. form="long" needs to include the “Or how I learned” part. It’s not the full title without it. Then what happens in the rare event that both sub and alternate are present? Which comes first? This is creating a bunch of processor complexity for a minor case.
4. I don’t think the label “alternate” is a good one. “Alternate” evokes the sense that this is a fungible title, but no one would ever refer to “The Whale” instead of “Moby Dick” or “How I learned to stop worrying” instead of whatever comes first. It’s not the same as the alternate element of a name, where they really are equivalent or the CSLm “alternative” structure that holds, e.g., transliterations of fields.

Given the inconsistent treatment of these across styles and their rarity, I don’t think the complexity is worth it.

[denismaier]

The only problem with this is that we have no control over how calling applications will pass their data to a citeproc. Will || etc work as an overriding mechanism? Will they have distinct fields for different parts of a title?

From the standpoint of CSL, || wouldn't exist.

Exactly. That was what I was trying to say. This is beyond CSL. But we need to assume that it will be possible somehow.

The only wrinkle with this choice is if you need to print a short variant, you probably want to drop the alternate.
Yep, you'll have to explicitly add a "short" form.

It seems you're trying to slip the feature back in, convinced it will work, but I think you're mistaken.

Well, I'm just trying to figure out how these titles will be processed. My understanding is those titles will arrive either only in "main" (as in this case "A title, or How I learned to stop worrying and love ...") or the "alternate" part appear in "sub". (I assume some automatic pre-processing will happen somewhere along the way.)

Why is this a bad idea again?

I tend to agree with the points mentioned by @bwiernik.

[bwiernik]

Automatic preprocessing will happen somewhere in the change If a user finds that a problem, they can resort to manually splitting the fields, either in separate application fields, with a splitting syntax, etc. That’s all beyond the scope of this PR. The upshot is that a user can accommodate the rare case if needed.

[bdarcus]

I just pushed a commit that removes alternate.

Final issue to discuss:

How do we see this interacting with things like translated-title? Should that be moved here?

[denismaier]

How do we see this interacting with things like translated-title? Should that be moved here?

That would actually be a good fit for alternate ;-)

[bdarcus]

This is RTM then.

I'll give @cormacrelf a bit of time if he wants to weigh in.

[bdarcus]

Do we still need full?

[denismaier]

How do we see this interacting with things like translated-title? Should that be moved here?

That would actually be a good fit for alternate ;-)

That was actually only partly a joke...

I imagine we could do something like this:

title:
  main: "Война и миръ"
alternate:
  - main: "War and peace"
    type:  "translated"
  - main: "Vojna i mir"
    type: "transliteration"

[denismaier]

Where have settled on regarding punctuation? Shouldn't we include the colon in "main"?
"main": "Finis Coronat Opus: "
(Of course only if the colon appears in the original...)

[bdarcus]

Keeping with the "let's keep simple things simple" approach, I think there should be a convention, for styles that require "original punctuation", to add a default delimiter if one doesn't exist, so that adding colons and such (by FAR the most common convention) shouldn't be necessary.

I think this should still work elegantly if someone wants to maintain their bibliographic database in CSL YAML.

[denismaier]

Ok. That might mean we will want to add an option normalize-delimiters="false". Then you could still add title-delimiter=": " to a style, which will then add a colon if no punctuation is present, but it will not override an existing period. (Exclamation marks and question marks will be protected anyway.)

[bdarcus]

Should title delimiters also be defined in locales?

Like, in English, would be a colon.

[denismaier]

Better not do this. If I use Chicago, I'll always want to have colons as title-subtitle-delimiter, even when writing in German. But a institutional style from a German university will perhaps use periods.

[denismaier]

We can allow it in a locale, but in one locale only so that there really is one standard delimiter (bit one per locale). Styles that require a different delimiter can still override this.

[bdarcus]

In that case, we can just define colon as the default delimeter, and semi-colon as the default sub delimiter.

[denismaier]

@georgd What do you think about this? I remember you've had doubts regarding localizing punctuation...

[bdarcus]

It seems his concern was about user experience; unexpected behavior.

I'd agree that should be an important consideration, but that this (default delimiters) should be a part of it.

[georgd]

@georgd What do you think about this? I remember you've had doubts regarding localizing punctuation...

My impression is that the delimiter is a locale-independent property of the style – at least I’ve never come across anything else. A style that requires colon in English will be used with colon in a German document as well, as @denismaier said. Localisation for special typographical conventions (like the space before colon in French) is not restricted to this case, likewise script specific localisation (e.g. ano teleia instead of colon in Greek script). That’s why I thought that a simple global property would be enough.

But what do you expect to do with non-latin titles? I suppose they’re currently not normalised?

[bdarcus]

I imagine we could do something like this:

title:
  main: "Война и миръ"
alternate:
  - main: "War and peace"
    type:  "translated"
  - main: "Vojna i mir"
    type: "transliteration"

Seems better and more consistent to just do this?

title:
  main:  "whatever"
  translated:
    main: "War and peace"
  transliterated: 
    main: "Vojna i mir"

Might need a language property though?

Regardless of the details, should I include this too, or just keep them separate?

I don't care; whatever seems best/most consistent.

Or we can just not worry about it now.

Edit: let's just keep as is. We can revisit later, if necessary.

[denismaier]

Lgtm.
I'll open a new issue to discuss extension possibilities...

[bdarcus]

The thread I posted over here is based on an idea that maybe we don't have to choose between the two options we considered for this feature.