Model does not run properly for years prior to 2016 (included)

[amilovanoff]

Hi,
First, thank you for the great work!
I have been able to run the version 2.9 of the OpenIO-Canada model for the years 2020, 2019, 2018 and 2017. However, the model does not run properly for the years 2016 and prior. I tried for 2016, 2015 and 2014. Note that I was able to run the version 2.7.1 of the OpenIO-Canada model for the year 2016 but for for 2015 and 2014. I confirm that I have the exiobase and statsCAN SUT data for 2014, 2015 and 2016.

For the year 2016, it seems that the error came from the row "self.add_plastic_emissions()". The full log of the error is pasted below. Thank you for the help!
Alex

Log
Traceback (most recent call last):
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\base.py", line 3805, in get_loc
return self._engine.get_loc(casted_key)
File "index.pyx", line 167, in pandas._libs.index.IndexEngine.get_loc
File "index.pyx", line 173, in pandas._libs.index.IndexEngine.get_loc
TypeError: '['Motor vehicle plastic parts', 'Motor vehicle interior trim, seats and seat parts', 'Tires', 'Motor vehicle electrical and electronic equipment', 'Other miscellaneous motor vehicle parts']' is an invalid key

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\multi.py", line 3499, in get_locs
lvl_indexer = self._get_level_indexer(k, level=i, indexer=indexer)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\multi.py", line 3391, in _get_level_indexer
idx = self._get_loc_single_level_index(level_index, key)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\multi.py", line 2980, in _get_loc_single_level_index
return level_index.get_loc(key)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\base.py", line 3817, in get_loc
self._check_indexing_error(key)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\base.py", line 6059, in _check_indexing_error
raise InvalidIndexError(key)
pandas.errors.InvalidIndexError: ['Motor vehicle plastic parts', 'Motor vehicle interior trim, seats and seat parts', 'Tires', 'Motor vehicle electrical and electronic equipment', 'Other miscellaneous motor vehicle parts']

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\base.py", line 3805, in get_loc
return self._engine.get_loc(casted_key)
File "index.pyx", line 167, in pandas._libs.index.IndexEngine.get_loc
File "index.pyx", line 196, in pandas._libs.index.IndexEngine.get_loc
File "pandas\_libs\hashtable_class_helper.pxi", line 7081, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas\_libs\hashtable_class_helper.pxi", line 7089, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'Motor vehicle electrical and electronic equipment'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "", line 1, in
File "C:\Users\MilovanA\Run_OIOC.py", line 10, in
tables = openIO.IOTables(folder_path=folder_path,exiobase_folder=exiobase_folder,endogenizing_capitals=False)
File "C:\Users\MilovanA\OpenIO-Canada-2.9\openIO.py", line 244, in init
self.add_plastic_emissions()
File "C:\Users\MilovanA\OpenIO-Canada-2.9\openIO.py", line 2558, in add_plastic_emissions
if where_is_plastic.loc(axis=0)['CA-' + province, map_plastic_data_to_io[category]].sum() != 0:
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexing.py", line 1184, in getitem
return self._getitem_tuple(key)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexing.py", line 1368, in _getitem_tuple
return self._getitem_lowerdim(tup)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexing.py", line 1037, in _getitem_lowerdim
return self._getitem_axis(tup, axis=axis)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexing.py", line 1424, in _getitem_axis
locs = labels.get_locs(key)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\multi.py", line 3513, in get_locs
item_indexer = self._get_level_indexer(
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\multi.py", line 3391, in _get_level_indexer
idx = self._get_loc_single_level_index(level_index, key)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\multi.py", line 2980, in _get_loc_single_level_index
return level_index.get_loc(key)
File "C:\Users\MilovanA\AppData\Roaming\Python\Python39\site-packages\pandas\core\indexes\base.py", line 3812, in get_loc
raise KeyError(key) from err
KeyError: 'Motor vehicle electrical and electronic equipment'

[MaximeAgez]

Hi. Indeed the mapping for plastic emissions makes it that earlier years don't work (because the names of some sectors changed between supply and use accounts years). For example, "Motor vehicle electrical and electronic equipment" which is the name I used to map plastic flow accounts to openIO-Canada was called "Motor vehicle electrical and electronic equipment and instruments" in earlier versions.
So right now it is bugged for early years with the v2.9.
The way to correct this bug would be to modify the plastic_mapping_exio_openio.json file to operate with classification codes (e.g., MPG336320) and not names ("Motor vehicle electrical and electronic equipment"), but that also requires some code adaptation in the plastic function of openIO-Canada.
I will eventually correct this, but right now it is not in my priorities :/