Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

chore(data): AGB 3.2 + EI 3.10 #727

Merged
merged 9 commits into from
Oct 1, 2024
Merged

chore(data): AGB 3.2 + EI 3.10 #727

merged 9 commits into from
Oct 1, 2024

Conversation

ccomb
Copy link
Collaborator

@ccomb ccomb commented Sep 5, 2024

Imports of Agribalyse 3.2 and Ecoinvent 3.10 so that they are available for process selection and in the brightway explorer

@ccomb ccomb requested a review from paulboosz September 5, 2024 09:38
@ccomb ccomb changed the title AGB 3.2 + EI 3.10 chore(data): AGB 3.2 + EI 3.10 Sep 5, 2024
@ccomb ccomb force-pushed the agb32_ei310 branch 3 times, most recently from ef2ec5f to d5cfcf7 Compare September 10, 2024 06:42
Copy link
Collaborator

@paulboosz paulboosz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great !
Could you add the new agb3.2 file to the drive ?

It would also be a lot cleaner if we could put all the datafiles in a folder
And maybe rename import_agribalyse.py to -> import_food.py as we import data from several sources (WFLDB...)

@ccomb
Copy link
Collaborator Author

ccomb commented Sep 19, 2024

Thanks for the suggestion, I've gathered the zip files in a dbfiles folder, adapted the scripts and README. Also disabled the npm cache cleaning which should be unnecessary.
Could you please retry?

@paulboosz
Copy link
Collaborator

I got this bug on the vm

ecobalyse@ecobalyse:~/ecobalyse/data$ make
docker build -t ecobalyse docker
[+] Building 1.2s (25/25) FINISHED                                   docker:default
 => [internal] load build definition from Dockerfile                           0.0s
 => => transferring dockerfile: 1.99kB                                         0.0s
 => [internal] load metadata for docker.io/jupyter/minimal-notebook:notebook-  0.7s
 => [internal] load .dockerignore                                              0.0s
 => => transferring context: 2B                                                0.0s
 => [ 1/14] FROM docker.io/jupyter/minimal-notebook:notebook-7.0.6@sha256:1c4  0.0s
 => [internal] load build context                                              0.0s
 => => transferring context: 486B                                              0.0s
 => [ 8/14] ADD https://api.github.com/repos/brightway-lca/bw_projects/git/re  0.2s
 => [ 7/14] ADD https://api.github.com/repos/brightway-lca/brightway2-analyze  0.2s
 => [ 6/14] ADD https://api.github.com/repos/brightway-lca/brightway2-calc/gi  0.2s
 => [ 5/14] ADD https://api.github.com/repos/brightway-lca/brightway2-data/gi  0.3s
 => [ 4/14] ADD https://api.github.com/repos/brightway-lca/brightway2-paramet  0.2s
 => [ 3/14] ADD https://api.github.com/repos/ccomb/brightway2-io/git/refs/tag  0.2s
 => CACHED [ 2/14] RUN mkdir -p /home/jovyan/data     && mkdir -p /home/jovya  0.0s
 => CACHED [ 3/14] ADD https://api.github.com/repos/ccomb/brightway2-io/git/r  0.0s
 => CACHED [ 4/14] ADD https://api.github.com/repos/brightway-lca/brightway2-  0.0s
 => CACHED [ 5/14] ADD https://api.github.com/repos/brightway-lca/brightway2-  0.0s
 => CACHED [ 6/14] ADD https://api.github.com/repos/brightway-lca/brightway2-  0.0s
 => CACHED [ 7/14] ADD https://api.github.com/repos/brightway-lca/brightway2-  0.0s
 => CACHED [ 8/14] ADD https://api.github.com/repos/brightway-lca/bw_projects  0.0s
 => CACHED [ 9/14] COPY requirements.txt .                                     0.0s
 => CACHED [10/14] RUN apt update     && apt install -y gosu vim     && pip i  0.0s
 => CACHED [11/14] RUN curl -L -o elm.gz https://github.com/elm/compiler/rele  0.0s
 => CACHED [12/14] COPY simapro-biosphere.json /opt/conda/lib/python3.11/site  0.0s
 => CACHED [13/14] COPY entrypoint.sh /                                        0.0s
 => CACHED [14/14] WORKDIR /home/jovyan/notebooks                              0.0s
 => exporting to image                                                         0.0s
 => => exporting layers                                                        0.0s
 => => writing image sha256:659d93a5133f9bafb524833f973dbe5cc2ea0b9e690960e23  0.0s
 => => naming to docker.io/library/ecobalyse                                   0.0s

 3 warnings found (use --debug to expand):
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 3)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 4)
 - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy "ENV key value" format (line 5)
Error response from daemon: No such container: ecobalyse
ECOBALYSE_DATA_DIR=/home/ecobalyse/ecobalyse-private
ECOBALYSE_DATA_DIR=/home/ecobalyse/ecobalyse-private
(Creating a new container)
Using environment variable BRIGHTWAY2_DIR for data directory:
/home/jovyan/data
Biosphere database already present!!! No setup is needed
Agribalyse 3.1.1 already imported
### Importing ../../dbfiles/AGB32beta_08082024.CSV.zip...
Traceback (most recent call last):
  File "/home/jovyan/ecobalyse/data/import_food.py", line 237, in <module>
    import_simapro_csv(
  File "/home/jovyan/ecobalyse/data/common/import_.py", line 188, in import_simapro_csv
    with ZipFile(datapath) as zf:
         ^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/zipfile.py", line 1284, in __init__
    self.fp = io.open(file, filemode)
              ^^^^^^^^^^^^^^^^^^^^^^^
FileNotFoundError: [Errno 2] No such file or directory: '../../dbfiles/AGB32beta_08082024.CSV.zip'
make: *** [Makefile:26: import_food] Error 1

@paulboosz
Copy link
Collaborator

paulboosz commented Sep 30, 2024

How to test

  • make clean && make
  • verify db is imported

@ccomb ccomb merged commit 43708be into master Oct 1, 2024
6 checks passed
@ccomb ccomb deleted the agb32_ei310 branch October 1, 2024 13:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants