2.4 release

README.md

@@ -1,29 +1,35 @@
-**OpenIO-Canada v2.3**
+## _OpenIO-Canada v2.4_
 
-Class that creates Multi-Regional symmetric Environmentally Extended Input-Output (EEIO) tables for Canada. OpenIO-Canada 
-operates at the provincial level (13 provinces). It can thus be used to compare the environmental impacts of value chains from 
-Quebec and Ontario for example.
+Python class creating Multi-Regional symmetric Environmentally Extended Input-Output (MREEIO) tables for Canada. OpenIO-Canada 
+operates at the provincial level (13 provinces). It can thus be used to compare the environmental impacts of value chains
+or consumption by households from any specific province.
 
-Through the "Detail level" of economic data from Statistics Canada, openIO-Canada covers 492 commodities and 240 industries.
+OpenIO-Canada covers 492 commodities, 310 pollutants including 3 greenhouse gases (CO2, CH4 and N2O) in 3 compartments 
+(air, water and soil), water use and energy use.
 
-Covers 310 pollutants including 3 greenhouse gases (CO2, CH4 and N2O) in 3 compartments (air, water and soil). OpenIO-Canada 
-also covers the use of water and energy.
+OpenIO-Canada is connected to the Exiobase global MRIO database to model value chains happening outside Canada.
 
-The IMPACT World+ life cycle impact assessment methodology is used to characterize the impacts of emissions on the 
-environment.
+###Getting started
 
-Tables are available both in _ixi_ (industry) and _pxp_ (product) formats.
-The fixed industry sales structure assumption was used to generate the _ixi_ format and the industry technology 
-assumption was used for the _pxp_ format. More transformation models might be added in the future.
+Clone the repository (or download it) and install the different libraries required to run this Python class (requirements.txt).
+Go to the doc folder and take a look at the demo.ipynb file to see how to generate the IO tables.
 
-Data used:
+###Data used
 - economic data: https://www150.statcan.gc.ca/n1/en/catalogue/15-602-X
+- international merchandise trade data: https://open.canada.ca/data/en/dataset/b1126a07-fd85-4d56-8395-143aba1747a4
 - ghg emissions: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=3810009701
 - water use:https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=3810025001
 - energy use: https://www150.statcan.gc.ca/t1/tbl1/en/tv.action?pid=3810009601
 - other pollutants: https://open.canada.ca/data/en/dataset/1fb7d8d4-7713-4ec6-b957-4a882a84fed3
+- exiobase3: https://doi.org/10.5281/zenodo.5589597
+- The IMPACT World+ impact assessment methodology is used
 
+
+###Miscellaneous information
 This project was in part funded by Shared Services Canada (SSC) but they are not responsible for any data/results 
 obtained from open IO Canada.
 
-An article describing the methodology of openIO-Canada is being written.
+Support for the industry version (_ixi_) is currently discontinued as it requires too much work. Support for lower detail 
+IOIC classifications (i.e., Summary, Link-1961 and Link-1997) is also discontinued.
+
+A scientific article describing the methodology and results of openIO-Canada is being written.

openIO.py

@@ -723,36 +723,27 @@ def load_merchandise_international_trade_database(self):
         conc = pd.read_excel(pkg_resources.resource_stream(__name__, '/Data/HS-IOIC.xlsx'))
 
         # load database
-        merchandise_database = pd.read_csv(pkg_resources.resource_stream(__name__, '/Data/Imports_' + str(self.year) +
-                                                                         '_HS06.csv'))
-
-        # drop useless columns
-        merchandise_database = merchandise_database.drop(['YearMonth/AnnéeMois', 'Province', 'State/État',
-                                                          'Quantity/Quantité','Unit of Measure/Unité de Mesure'],
-                                                         axis=1)
-
-        # drop international imports coming from Canada
-        merchandise_database = merchandise_database[merchandise_database['Country/Pays'] != 'CA']
-
-        # also drop nan countries for obvious reasons
-        merchandise_database = merchandise_database.dropna(subset=['Country/Pays'])
+        merchandise_database = pd.read_excel(pkg_resources.resource_stream(__name__, '/Data/Imports_' + str(self.year) +
+                                                                         '_HS06_treated.xlsx'))
+        merchandise_database = merchandise_database.ffill()
+        merchandise_database.columns = ['Country', 'HS6', 'Value']
 
         # apply concordance
         merchandise_database = merchandise_database.merge(conc, on='HS6', how='left')
 
         # only keep useful information
-        merchandise_database = merchandise_database.loc[:, ['IOIC', 'Country/Pays', 'Value/Valeur', ]]
+        merchandise_database = merchandise_database.loc[:, ['IOIC', 'Country', 'Value']]
 
-        # drop sectors that are not matched to IOIC (None)
+        # remove HS sectors that cant be matched to IOIC (identified with "None")
         merchandise_database = merchandise_database.drop(
-            merchandise_database[merchandise_database['IOIC'] == 'None'].index)
+            [i for i in merchandise_database.index if merchandise_database.loc[i, 'IOIC'] == 'None'])
 
         # change IOIC codes to sector names
         code_to_name = {j[0]: j[1] for j in self.commodities}
         merchandise_database.IOIC = [code_to_name[i] for i in merchandise_database.IOIC]
 
         # set MultiIndex with country and classification
-        merchandise_database = merchandise_database.set_index(['Country/Pays', 'IOIC'])
+        merchandise_database = merchandise_database.set_index(['Country', 'IOIC'])
 
         # regroup purchases together (on country + IOIC sector)
         merchandise_database = merchandise_database.groupby(merchandise_database.index).sum()
@@ -2364,6 +2355,36 @@ def todf(data):
     return out
 
 
+def treatment_import_data(original_file_path):
+    """Function used to treat the merchandise imports trade database file. FIle is way too big to be provided to
+    users through Github, so we treat the data to only keep what is relevant."""
+
+    # load database
+    merchandise_database = pd.read_csv(original_file_path)
+    # drop useless columns
+
+    merchandise_database = merchandise_database.drop(['YearMonth/AnnéeMois', 'Province', 'State/État',
+                                                      'Quantity/Quantité', 'Unit of Measure/Unité de Mesure'],
+                                                     axis=1)
+
+    # drop international imports coming from Canada
+    merchandise_database = merchandise_database[merchandise_database['Country/Pays'] != 'CA']
+
+    # also drop nan countries for obvious reasons
+    merchandise_database = merchandise_database.dropna(subset=['Country/Pays'])
+
+    # set the index as country/code multi-index
+    merchandise_database = merchandise_database.set_index(['Country/Pays', 'HS6'])
+
+    # regroup data from several months into a single yearly data
+    merchandise_database = merchandise_database.groupby(merchandise_database.index).sum()
+
+    # multi-index is cleaner
+    merchandise_database.index = pd.MultiIndex.from_tuples(merchandise_database.index)
+
+    return merchandise_database
+
+
 # pyomo optimization functions
 def reconcile_one_product_market(uy, u0, imp, penalty_multiplicator):
     opt = SolverFactory('ipopt')

requirements.txt

@@ -0,0 +1,5 @@
+pandas==1.3.4
+numpy==1.20.3
+regex==2.2.1
+pymrio==0.4.6
+country_converter==0.7.4