Nested volumes and explicit <url> tags

[mjpost]

Started consolidating URLs per #156. The goal is to:

• Convert all href attributes to <url> tags
• Convert all <href> tags to <url> tags
• Use relative IDs for Anthology identifies (e.g., P18-1014 instead of https://www.aclweb.org/anthology/P18-1014)

Adding explicit <url> tags is easier with nested volumes (#317), so I am doing that, too:

• Convert schemas to support nesting
• Update all the XML
• Add explicit full-volume <url> tags for volumes that are present on the server
• Update the site generation code to work with the nested volumes

Additionally I think we should:

• remove <bibtype> and <bibkey> tags, which are just clutter
• Reformat all the XML to be properly indented (2 spaces? 2 spaces)

This will help with mirroring (#295 #28 #22).

Finally, I'd like to update the code not to generate URLs for papers without a URL tag, so by cross-checking against #264, we can solve a lot of problems (#226 #181 #180).

[mjpost]

Another issue has come up. We now need to generate explicit volume <url> tags. This will be easier once #317 is done, so I will move to that next.

[akoehn]

Can you do a squash-merge to keep the changes more local? I am a big fan of rewriting history (in git at least ...) so that it is easier to see where a feature was introduced.

That is just my personal opinion of course.

[mjpost]

Yes, will do a squash merge. This is a good idea.

@acl-org/anthology, this is ready for review. Here's a summary of changes:

• Introduces a nested format (closes Nested volumes in XML #317)
• Only displays URLs if they are found in the XML
• URLs are stored using a relative format for internal links (closes URLs in authoritative XML #156)
• Thereby facilitates mirroring (Anthology mirrors #295)
• Volume PDFs are displayed only if present in the block.
• Enables front matter for journals that have URLs (closes Front matter for journals #181).
• Added missing entries and removed ones whose PDFs were missing, including LREC 2014 (closes LREC 2014 proceedings are 404 #31)
• Add links to journal frontmatter where they exist (addresses Broken & missing links on the server #264)
• I inserted URLs via a scripted check for volume PDFs
• Punts on C69 formatting (closes Paper labeling in C69.xml #147)

Enabled, but yet to do:

• Creating PDF volumes from individual papers (Front matter from ACL conferences are missing and full proceedings links broken #226)

[mbollmann]

I want to build this locally and have a closer look at it later today.

[akoehn]

@mbollmann: To have a look at the results (rather than the source), build master, mv build build_master, build this branch and compare with meld (or another diff tool). Tried this for other PRs and that workflow works quite well.

[akoehn]

There is now a benoit_sagot and benoit_sagot1:

title: "Beno\u0131\u0302t Sagot"
title: "Beno\xEEt Sagot"

Maybe it was there before and now fixed in master but not yet in this PR?

[mjpost]

@akoehn this kind of diffing is what I did. I did bulk diffs on .md and .bib files, and found only differences of this sort (which I think will be resolved when we do the final merge).

I have merged in master a few times but I think there may be one that's missing.

[davidweichiang]

Good catch. I think this was fixed in master, though. The issue is that this uses a dotless i and Unicode normalization doesn't know how to deal with that. I don't think there's anywhere in the code that fixes this automatically, so I'll create a new issue for this.

[mbollmann]

This is pretty great overall, but also quite hard to diff :)

I focused on the code & templates now rather than the XML, and have some (minor?) comments.

[akoehn]

Congratulations on this huge merge!