Progress indicator/logger for exports

[jkikstra]

For larger databases, using export_timeseries_data() on a ixmp.Platform object can take multiple hours.
With this in mind, it would be very beneficial to log some feedback or show a progress bar, such that users can at least tell what is going on and whether anything is going wrong or not while waiting.

[khaeru]

This would indeed be a good feature.

click.progressbar (already a dependency) and tqdm (would be a new dependency) are two options for providing progress bars.

With the current default/only JDBCBackend, however, this kind of file I/O operation is a single Java function call:

ixmp/ixmp/backend/jdbc.py

Lines 510 to 528 in 5ad2ad9

	elif path.suffix == '.csv' and item_type is ItemType.TS:
	models = set(filters.pop('model'))
	scenarios = set(filters.pop('scenario'))
	variables = filters.pop('variable')
	units = filters.pop('unit')
	regions = filters.pop('region')
	default = filters.pop('default')
	scen_list = self.jobj.getScenarioList(default, None, None)
	# TODO: replace with passing list of models/scenarios
	# to the method above
	run_ids = [s['run_id'] for s in scen_list
	if (len(scenarios) == 0 or s['scenario'] in scenarios)
	and (len(models) == 0 or s['model'] in models)]
	self.jobj.exportTimeseriesData(to_jlist(run_ids),
	to_jlist(variables),
	to_jlist(units),
	to_jlist(regions),
	str(path))

To display a progress indicator, the Python side / CLI code would require some kind of state variable(s) to be synchronized & checked, while that Java function continued running, i.e. this would need to be multi-threaded, something we currently don't do anywhere. Not impossible, but would require more advanced code than we currently have.