Zip Archive Abstraction plus Minor tweaks for factor file generator #2416
Conversation
mchandschuh
requested review from
jaredbroad,
StefanoRaggi and
Martin-Molinero
Adds FirstTicker to MapFile Adds MostRecentFactorChange to FactorFile Minor clean up in StreamReaderEnumerable Make Dividend.ReferencePrice settable
Splits factors are recorded in the factor file as the previous trading day and not the actual day of the split.
ArchiveCache allows for lockless concurrent reads of zip files ArchiveImplementation provides for easy swapping of underlying implementation in case of issues running on one system or another.
mchandschuh
force-pushed
the
feature-2378-generator-factors
branch
from
1ad94f2
to
e308ce7
Compare
It was 'possible' for a race to exist between disposal of a cache entry and returning an instance to that cache entry. By removing first, the race on the returned entry will simply create a new cache entry and the old entry will be disposed.
The thinking is that this has the possibility to lead to memory leaks.
|
There are core issues in the application of mono zip libraries which cause corrupt zips and thread explosions which aren't fully understood; and rather than understanding them fully we've pulled in three separate zip libraries and applied them in various ways. I think to merge this is building layers on top of that poor initial decision and cleaning up the compression libraries should be done in a more thoughtful way disconnected from the Live-Corporate events issue. So as tempting as it is to merge it to continue the march of progress I know we'll just be installing bad patterns for the future that we'll need to undo. And rather than debating the zip implementation here; I'll close this for now as too much code unrelated to the Live-Corporate events issue and open issue (#2435) to continue archive clean up later. |
Description
Here's the zip archive abstraction I spoke about. The main entry is through
Archive.OpenReadandArchive.OpenWriteand you can pass it an enum value picking which underlying implementation will be used -- figured it may be handy -- factor file generator set up to pull from command line or config if value present. A super awesome thing in here is theArchiveCachewhich allows you to 'checkout' a zip instance and 'return' it -- like it's a freakin' library book! This support lockless multi-threaded read scenarios.Related Issue
See #2378
Motivation and Context
These changes are in support of the new factor file generation/supplementation/projection application.
Requires Documentation Change
No.
How Has This Been Tested?
Probably not well enough, really need to see days tick by w/ the ticker plant properly emitting non-finalized events.
Types of changes
Checklist:
bug-<issue#>-<descriptionorfeature-<issue#>-<description>