Merge tN Verse Files to Chunks for tS API

[jag3773]

For the tS v2 API endpoint we want the tN to be presented as chunks, not as individual verses.

The tN will be converted to be verse by verse files in DCS shortly. After that happens we'll need a processor to combine those to be chunk level for the tS v2 API endpoint.

Take note that the processor will need to:

• Not overwrite the tw_cat.json (see Freeze tw_cat in tS v2 catalog #42 )
• Condense duplicate notes to 1 per chunk. For example, if chunk 1 contains verses 1-3 and verses 1 and 2 have the exact same note, only include that note once in chunk 1.

The output for the tS v2 API JSON files should so very few (if any) differences to what was published for v11.

[da1nerd]

@jag3773 fyi before we make this content change I'll need to disable overwriting tw_cat.json.
Technically this entire issue should be completed before then, but we at least want to disable overwriting the tw_cat.json.

[jag3773]

Example new structure is https://git.door43.org/destatez/en_tn .

[da1nerd]

I think I have this all done. I'm testing rebuilding the ts api to ensure it's backwards compatible with the legacy formatted tN (it works in the unit tests just making sure).

[da1nerd]

@jag3773 We are ready for a live test. Can you please submit the updated English Titus tN to the catalog?
All of my tests pass, but just to be safe I've also backed up the notes and tw_cat for Titus in the tS api.

[jag3773]

@neutrinog It's not going to be until January at the earliest that we want to push a new copy out into the API. Until then do we have a test/dev pipeline that we could it with? If not we can probably wait if that works for you.

[da1nerd]

we can wait. I've written a bunch of unit tests regarding this so I feel pretty confident.

[jag3773]

I want to keep this open a placeholder for us (at least me) to remember that we haven't tested this yet.