sort_and_finalize_staged_data is not properly tested

[willdealtry]

Describe the bug

This feature is in the v2 API, and it has happy-path tests that ensure that it fundamentally works, however it is not sufficiently tested to really recommend its use. It should have simple hypothesis tests that generate dataframes with overlapping columns and new columns with overlapping dates, and then checks that they correspond to the output that pandas would produce

Issues

• Allow unordered indexes in staged area when sort_and_finalize_staged_data is used #1738
• Staged segment of only empty DataFrames throws when sort_and_finalize_staged_data is called #1735
• Calling sort_and_finalize_staged_data with staged segment of DataFrame with 0 rows fails #1736
• Calling sort_and_finalize_staged_data throws non informative exception #1737
• Using sort_and_finalize_staged_data with append can create unordered index #1734
• Sort and finalize staged data fails with pre-epoch dates #1781
• compact_incomplete with append=True should fail if dynamic schema is false and incomplete segment fields do not exactly match the existing fields #1466
• Sort and finalize can reorder the fields in a segment and crash with static schema #1795
• Sort and finalize and regular write/append handle NaT index values differently #1797
• Sort and finalize throws when staged columns are of different but promotable types (dynamic schema) #1807

Note: All issues related to adding empty df, 0-row df, etc... are probably broken with the old API as well. They should at least be tested.

Steps/Code to Reproduce

run sort_and_finalize_staged_data

Expected Results

tests

OS, Python Version and ArcticDB Version

All versions

Backend storage used

No response

Additional Context

No response