Update output file location

[nreinicke]

Takes a pass at #32 by adding a new optional argument output_file_override to the from_config_file method on the python compass application. This would allow the user to load the application as:

app = CompassApp.from_config_file("my_config.toml:, output_file_override="my_output.json")

I also played around with changing the output file after the application has been loaded (see the rust CompassApp.with_output_directory() method) but we can roll that back if seems too weird.

Going through this it still feels a bit awkward but I had a hard time finding an elegant way to give a custom config parameter outside of the context of the config file. I'm open to alternatives if you have any ideas.

[robfitzgerald]

I'm open to alternatives if you have any ideas

this does seem to go pretty deep to deliver this feature. a shallower solution could work mostly in the TOML.

if the user sets the output_file_override argument, it would first deserialize the TOML from file in python, modify it's outfile location directly, and then pass the modified TOML blob (maybe as a string) to CompassApp.

to use that on the rust-side, we would need yet another CompassApp.try_from method, but this time, something like a TryFrom<&str> that expects it needs to parse the string into a Config. it would follow a similar logic to CompassApp's TryFrom<&Path> but add a step to deserialize the Config. but then we don't need to make any other changes on the rust code.

[nreinicke]

if the user sets the output_file_override argument, it would first deserialize the TOML from file in python, modify it's outfile location directly, and then pass the modified TOML blob (maybe as a string) to CompassApp.

to use that on the rust-side, we would need yet another CompassApp.try_from method, but this time, something like a TryFrom<&str> that expects it needs to parse the string into a Config. it would follow a similar logic to CompassApp's TryFrom<&Path> but add a step to deserialize the Config. but then we don't need to make any other changes on the rust code.

Yeah that is a much cleaner and less intrusive solution, let me run with that, thanks for the quick suggestion!

[nreinicke]

Okay, I've updated to come at this from the python side and just do a manipulation on the config and pass it to rust as a string. It still doesn't feel perfect but it's much less intrusive an I think it gets us what we want in terms of functionality

[robfitzgerald]

i tried to test this using the notebook example:

$ git checkout ndr/update-output-dir
$ git status
On branch ndr/update-output-dir
Your branch is up to date with 'origin/ndr/update-output-dir'.
$ conda activate routee-rust
$ pip install -e .

then, in the notebook, running in routee-rust, i added the output file argument:

app = CompassApp.from_config_file("golden_co/osm_default_energy.toml", output_file="foo.json")  

that should have created an output file, but, even after calling app.run there is no such file.

two thoughts

• i think it fails because there is no out file output plugin in the TOML. maybe the logic should inject (append to the end of the list) the output plugin if it's missing?
• or maybe i'm doing something wrong, can you confirm/let me know if this does work for you?

[nreinicke]

Yeah this is definitely due to the fact that there is no to_disk plugin specified in the osm_default_energy.toml file. I had initially put a warning in but I think the expected behavior would be to add that plugin into the config if a user passes in that optional argument. Thanks for testing and catching this.

[nreinicke]

@robfitzgerald - this has been updated to inject a to_disk plugin if the output directory is specified and one does not already exist

[robfitzgerald]

works for me!