Merge changes from TfWM V2 #66
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
… columns not being explicitly set - and some fields assumed to be present in route table (route_desc) are not present in BODS data
…quencies' and 'feed_info' when reading / writing BODS data. Removed assumptions about table names, all additional tables encountered in zip are read and written. also performance improvement when writing.
…ay anything else that it wasn't expecting. OK for the GTFS tables generated by this tool from CIF files, but caused data loss when using BODS data. modified to keep all encountered columns and tables - passes through if there isn't any specific code for the table in question.
renamed ZZ to reflect it's a generic obfuscated freight, not a specific operator
…lendar_date exists with no calendar (rare but valid data configuration)
…g about merge() failing
…GTFS into oo-tfwm-performance
merge up to latest leeds master
1) make load and merge agnostic about expected tables and fields, so that optional tables and fields are at least passed through rather than discarded. (allows GTFS generated by this tool and BODS to be merged)
2) tighten up data type expectations for fields in BODS data
3) fix bugs when calendar dates have no calendar (this is a legitimate but range arrangement of data)
4) performance improvement while formatting files before writing to file system (3 minute improvement when writing 400mb zipped GTFS file (all GB BODS extract)
There is one line in merge that needs resolution - complex line that I couldn't understand so commented it out.
calendar_dates <- calendar_dates[sapply(calendar_dates, function(x){ifelse(is.null(nrow(x)),0,nrow(x))}) > 0]
no idea what it's supposed to do and no explanatory comment
…o the caller being a different shaped structure being passed back
…tries in LO table
change to use data.table (likely a breaking change) performance improvements add setting of route_long_name "Train from..." to change based on mode
…wrong, overlays not being applied correctly
…bles, change default options to highest performing write combination
…as also been extensively profiled and reasonable level of optimisation applied
…sts for revised overlay functionality
…s from too much scrolling
…of date+time (this means if github can't be reached should only nag once per day to update instead of on every single package load) 2) add timeout to update check on package load and make it short (but make default when explicitly called long)
(atoc data doesn't seem to have many non-public moves in it to start with)
…ily vectorise process_activity
…e loaded into OTP
… more permissive licence than ATCO data.
… they are NA instead
…t completely follow the schema. Also add 'metro' to long name for underground & Tyne&Wear metro services
…e name instead of NA
…ass time 2) when looking at non-public services include passes marked as 'no pickup/dropoff available' from a GTFS perspective
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
No description provided.