24.08.08 (#343)

* BCO scoring changes * Add New Production Deployment md Changes to be committed: deleted: docs/bco_scores.json modified: docs/deployment/productionDeployment.md new file: docs/newProductionInstance.md * Update config.md * Add API endpoint for basic comaprison Changes to be committed: modified: biocompute/apis.py modified: biocompute/urls.py modified: requirements.txt * Fix Django CI issue (#340) biocompute-objects/portal_userdb#380 Changes to be committed: modified: .github/workflows/django.yml modified: biocompute/apis.py modified: biocompute/models.py modified: config/settings.py modified: config/urls.py * Update to BcoValidator.load_schema (#341) Updated the function to check locally for the most used schemas. Improves testing and speed in BCO validation. Changes to be committed: modified: biocompute/services.py renamed: config/IEEE/2791object.json -> config/schemas/2791object.json new file: config/schemas/dataset_extension.json renamed: config/IEEE/description_domain.json -> config/schemas/description_domain.json renamed: config/IEEE/error_domain.json -> config/schemas/error_domain.json renamed: config/IEEE/execution_domain.json -> config/schemas/execution_domain.json new file: config/schemas/fhir_extension.json new file: config/schemas/galaxy_extension.json renamed: config/IEEE/io_domain.json -> config/schemas/io_domain.json new file: config/schemas/license_extension.json renamed: config/IEEE/parametric_domain.json -> config/schemas/parametric_domain.json renamed: config/IEEE/provenance_domain.json -> config/schemas/provenance_domain.json new file: config/schemas/scm_extension.json renamed: config/IEEE/usability_domain.json -> config/schemas/usability_domain.json * Add additional schema files and reorganize Changes to be committed: modified: biocompute/services.py new file: config/schemas/1.1.0/dataset_extension.json new file: config/schemas/1.1.0/fhir_extension.json new file: config/schemas/1.1.0/galaxy_extension.json new file: config/schemas/1.1.0/license_extension.json new file: config/schemas/1.1.0/scm_extension.json renamed: config/schemas/dataset_extension.json -> config/schemas/1.2.0/dataset_extension.json renamed: config/schemas/fhir_extension.json -> config/schemas/1.2.0/fhir_extension.json renamed: config/schemas/galaxy_extension.json -> config/schemas/1.2.0/galaxy_extension.json renamed: config/schemas/license_extension.json -> config/schemas/1.2.0/license_extension.json renamed: config/schemas/scm_extension.json -> config/schemas/1.2.0/scm_extension.json renamed: config/schemas/2791object.json -> config/schemas/2791/2791object.json renamed: config/schemas/description_domain.json -> config/schemas/2791/description_domain.json renamed: config/schemas/error_domain.json -> config/schemas/2791/error_domain.json renamed: config/schemas/execution_domain.json -> config/schemas/2791/execution_domain.json renamed: config/schemas/io_domain.json -> config/schemas/2791/io_domain.json renamed: config/schemas/parametric_domain.json -> config/schemas/2791/parametric_domain.json renamed: config/schemas/provenance_domain.json -> config/schemas/2791/provenance_domain.json renamed: config/schemas/usability_domain.json -> config/schemas/2791/usability_domain.json modified: config/settings.py modified: requirements.txt * Add instructions for Docker deployment Changes to be committed: modified: Dockerfile modified: docker-compose.yml modified: docs/deployment/dockerDeployment.md new file: entrypoint.sh --------- Co-authored-by: Kiran Sen <[email protected]> Co-authored-by: Kiran Sen <[email protected]>
biocompute-objects · Aug 9, 2024 · c39a741 · c39a741
1 parent 4efc685
commit c39a741
Show file tree

Hide file tree

Showing 28 changed files with 728 additions and 63 deletions.
diff --git a/.github/workflows/django.yml b/.github/workflows/django.yml
@@ -24,9 +24,7 @@ jobs:
       run: |
         python -m pip install --upgrade pip
         pip install -r requirements.txt
-    - name: Create .secret file
-      run: |
-        echo "${{ secrets.PORTAL_USERDB_SECRET }}" > server/.secret
+
     - name: Run Tests
       run: |
         python3.9 manage.py test
diff --git a/Dockerfile b/Dockerfile
@@ -6,29 +6,22 @@ LABEL org.opencontainers.image.source=https://github.com/octocat/my-repo
 LABEL org.opencontainers.image.description="My container image"
 LABEL org.opencontainers.image.licenses=MIT
 
-# Note that this is just for debug / test purposes; should not be set via the setup for production
-# ENV DJANGO_SUPERUSER_PASSWORD="BioCompute123"
-# ENV DJANGO_SUPERUSER_USERNAME="BioComputeSuperUser"
-# ENV DJANGO_SUPERUSER_EMAIL="[email protected]"
 
 RUN apt-get -qq update && apt-get install -y python3.9 python3-dev python3-pip
 
 RUN python3 -m pip install --upgrade pip
 
 WORKDIR /biocompute_api
 
-COPY requirements.txt .
-
+COPY requirements.txt /biocompute_api/
 RUN python3 -m pip install -r requirements.txt
 
-COPY . ./
-
-WORKDIR /biocompute_api/
+COPY . /biocompute_api/
 
-# RUN python3 manage.py migrate
-# RUN python3 manage.py createsuperuser --no-input
+WORKDIR /biocompute_api
 
 EXPOSE 8000
-#CMD ["bash"]
-ENTRYPOINT ["python3", "manage.py", "runserver"]
-CMD ["0.0.0.0:8000"]
+
+ENTRYPOINT ["./entrypoint.sh"]
+
+CMD ["python3", "manage.py", "runserver", "0.0.0.0:8000"]
diff --git a/biocompute/apis.py b/biocompute/apis.py
@@ -244,7 +244,6 @@ def post(self, request) -> Response:
                 data=bco_results
             )
 
-        import pdb; pdb.set_trace()
         return Response(status=status.HTTP_200_OK, data=response_data)
 
         # for index, object in enumerate(data):
@@ -307,7 +306,6 @@ def post(self, request) -> Response:
         #         continue
 
         #     bco_results = validator.parse_and_validate(bco_instance.contents)
-        #     import pdb; pdb.set_trace()
         #     identifier, results = bco_results.popitem()
 
         #     if results["number_of_errors"] > 0:
@@ -499,7 +497,7 @@ def post(self, request) -> Response:
                 message=message,
                 data=results
             ))
-
+        
         status_code = response_status(accepted_requests, rejected_requests)
         return Response(status=status_code, data=response_data)
 

diff --git a/biocompute/models.py b/biocompute/models.py
@@ -19,7 +19,7 @@
 ]
 
 class Bco(models.Model):
-    """BioComput Object Model.
+    """BioCompute Object Model.
 
     Attributes:
     -----------

diff --git a/biocompute/services.py b/biocompute/services.py
@@ -50,12 +50,25 @@ def load_schema(schema_uri):
         Returns:
         - dict: The loaded JSON schema.
         """
+        schema_mapping = {
+            "https://w3id.org/ieee/ieee-2791-schema/2791object.json": f"{BASE_DIR}/config/schemas/2791/2791object.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.1.0/dataset/dataset_extension.json": f"{BASE_DIR}/config/schemas/1.1.0/dataset_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.1.0/fhir/fhir_extension.json": f"{BASE_DIR}/config/schemas/1.1.0/fhir_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.1.0/galaxy/galaxy_extension.json": f"{BASE_DIR}/config/schemas/1.1.0/galaxy_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.1.0/license/license_extension.json": f"{BASE_DIR}/config/schemas/1.1.0/license_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.1.0/scm/scm_extension.json": f"{BASE_DIR}/config/schemas/1.1.0/scm_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.2.0/dataset/dataset_extension.json": f"{BASE_DIR}/config/schemas/1.2.0/dataset_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.2.0/fhir/fhir_extension.json": f"{BASE_DIR}/config/schemas/1.2.0/fhir_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.2.0/galaxy/galaxy_extension.json": f"{BASE_DIR}/config/schemas/1.2.0/galaxy_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.2.0/license/license_extension.json": f"{BASE_DIR}/config/schemas/1.2.0/license_extension.json",
+            "https://raw.githubusercontent.com/biocompute-objects/extension_domain/1.2.0/scm/scm_extension.json": f"{BASE_DIR}/config/schemas/1.2.0/scm_extension.json"
+        }
 
-        if schema_uri == \
-          "https://w3id.org/ieee/ieee-2791-schema/2791object.json":
-            return jsonref.load_uri(
-                f"file://{BASE_DIR}/config/IEEE/2791object.json"
-            )
+        if schema_uri in schema_mapping:
+            print(schema_uri in schema_mapping, schema_uri)
+            return jsonref.load_uri(f"file://{schema_mapping[schema_uri]}")
+        else:
+            print(schema_uri in schema_mapping, schema_uri)
         try:
             return jsonref.load_uri(schema_uri)
         except (JSONDecodeError, TypeError, RequestsConnectionError) as e:

diff --git a/config/schemas/1.1.0/dataset_extension.json b/config/schemas/1.1.0/dataset_extension.json
@@ -0,0 +1,86 @@
+{
+    "$schema": "http://json-schema.org/draft-07/schema#",
+    "$id": "http://www.w3id.org/biocompute/extension_domain/1.1.0/dataset/dataset_extension.json",
+    "title": "dataset_extension",
+    "type": "object",
+    "description": "The external references extension to list additional licenses and dataset catagories for a dataset BCO (dsBCO).",
+    "required": ["dataset_extension", "extension_schema"],
+    "additionalProperties": false,
+    "properties": {
+        "dataset_extension": {
+            "type": "object",
+            "additionalProperties": false,
+            "required": ["dataset_categories"],
+            "properties": {
+                "additional_license": {
+                    "type": "object",
+                    "description": "The additional license property contains the details about the licenses applied to the dataset and the script or tool/software used to process the given dataset.",
+                    "additionalProperties": false,
+                    "properties": {
+                        "data_license": {
+                            "title": "data_license",
+                            "type": "string",
+                            "description": "The license applied to the data or the dataset by the author",
+                            "examples": [
+                                "https://creativecommons.org/licenses/by/4.0/"
+                            ]
+                        },
+                        "script_license": {
+                            "title": "script_license",
+                            "type": "string",
+                            "description": "The license applied to the computational script or the tool/software developed to process (parse, QC, align) the input dataset for a final output dataset.",
+                            "examples": [
+                                "https://www.gnu.org/licenses/gpl-3.0.en.html"
+                            ]
+                        }
+                    }
+                },
+                "dataset_categories": {
+                    "title": "dataset_categories",
+                    "type": "array",
+                    "description": "Dataset categories describe and provide more information about the dataset which can be used to classify, group, sort and filter datasets.",
+                    "items": {
+                        "required": [
+                            "category_value",
+                            "category_name"
+                        ],
+                        "additionalProperties": false,
+                        "properties": {
+                            "category_value": {
+                                "type": "string",
+                                "title": "category_value",
+                                "description": "An explanation about the purpose of this instance.",
+                                "examples": [
+                                    "Homo sapiens"
+                                ]
+                            },
+                            "category_name": {
+                                "type": "string",
+                                "title": "category_name",
+                                "description": "An explanation about the purpose of this instance.",
+                                "enum": [
+                                    "species",
+                                    "molecule",
+                                    "tag",
+                                    "file_type",
+                                    "status",
+                                    "scope"
+                                ]
+                            }
+                        }
+                    }
+                }
+            }
+        },
+		"extension_schema": {
+            "title": "extension_schema",
+            "type": "string",
+			"format": "uri",
+            "description": "The schema applied to the extension object",
+            "examples": [
+                "http://www.w3id.org/biocompute/extension_domain/example.json"
+            ]
+
+		}
+    }
+}
diff --git a/config/schemas/1.1.0/fhir_extension.json b/config/schemas/1.1.0/fhir_extension.json
@@ -0,0 +1,67 @@
+{
+    "$schema": "http://json-schema.org/draft-07/schema#",
+    "$id": "https://w3id.org/biocompute/extension_domain/1.1.0/fhir/fhir_extension.json",
+    "title": "fhir_extension",
+    "type": "object",
+    "description": "The external references extension to FHIR resource",
+    "required": ["fhir_extension", "extension_schema"],
+    "additionalProperties": false,
+    "properties":{
+        "extension_schema": {
+            "title": "extension_schema",
+            "type": "string",
+            "format": "uri",
+            "description": "The schema applied to the extension object",
+            "examples": [
+                "http://www.w3id.org/biocompute/extension_domain/example.json"
+            ]
+
+        },
+        "fhir_extension":{
+            "title": "",
+            "type": "array",
+            "decription": "",
+            "items": {
+                "required": [
+                    "fhir_endpoint",
+                    "fhir_version",
+                    "fhir_resources"
+                ],
+                "properties": {
+                    "fhir_endpoint": {
+                        "type": "string",
+                        "description": "Base URI of FHIR server where the resources are stored",
+                        "examples": [
+                            "http://fhirtest.uhn.ca/baseDstu3"
+                        ],
+                        "format": "uri"
+                    },
+                    "fhir_version": {
+                        "type": "string",
+                        "description": "FHIR version of the server endpoint"
+                    },
+                    "fhir_resources": {
+                        "type": "array",
+                        "items": {
+                            "type": "object",
+                            "required": [
+                                "fhir_resource",
+                                "fhir_id"
+                            ],
+                            "properties": {
+                                "fhir_resource": {
+                                    "type": "string",
+                                    "description": "Type of FHIR resource used"
+                                },
+                                "fhir_id": {
+                                    "type": "string",
+                                    "description": "Server-specific identifier string"
+                                }
+                            }
+                        }
+                    }
+                }
+            }
+        }
+    }
+}
diff --git a/config/schemas/1.1.0/galaxy_extension.json b/config/schemas/1.1.0/galaxy_extension.json
@@ -0,0 +1,44 @@
+{
+    "$schema": "http://json-schema.org/draft-07/schema#",
+    "$id": "http://www.w3id.org/biocompute/extension_domain/1.1.0/galaxy/galaxy_extension.json",
+    "title": "galaxy_extension",
+    "type": "object",
+    "description": "The external references **example** extension for a Galaxy BCO.",
+    "required": ["galaxy_extension", "extension_schema"],
+    "additionalProperties": false,
+    "properties": {
+        "extension_schema": {
+            "type": "string",
+            "format": "uri",
+            "description": "The schema applied to the extension object",
+            "examples": [
+                "http://www.w3id.org/biocompute/extension_domain/example.json"
+            ]
+
+        },
+        "galaxy_extension": {
+            "type": "object",
+            "required": [
+                "galaxy_url",
+                "galaxy_version"
+            ],
+            "properties": {
+                "galaxy_url": {
+                    "type": "string",
+                    "description": "The base url for the galaxy instance used to create the BioCompute Object.",
+                    "examples": [
+                        "https://galaxy.aws.biochemistry.gwu.edu/"
+                    ],
+                    "format": "uri"
+                },
+                "galaxy_version": {
+                    "type": "string",
+                    "description": "The specific version of the Galaxy software used to generate the BioCompute Object.",
+                    "examples": [
+                        "20.01"
+                    ]
+                }
+            }
+        }
+    }
+}
diff --git a/config/schemas/1.1.0/license_extension.json b/config/schemas/1.1.0/license_extension.json
@@ -0,0 +1,41 @@
+{
+    "$schema": "http://json-schema.org/draft-07/schema#",
+    "$id": "http://www.w3id.org/biocompute/extension_domain/1.1.0/license/license_extension.json",
+    "type": "object",
+    "description": "The extension to add additional licenses.",
+    "required": ["licence_extension", "extension_schema"],
+    "additionalProperties": false,
+    "properties":{
+        "extension_schema": {
+            "title": "extension_schema",
+            "type": "string",
+            "format": "uri",
+            "description": "The schema applied to the extension object",
+            "examples": [
+                "http://www.w3id.org/biocompute/extension_domain/example.json"
+            ]
+        },
+        "licence_extension":{
+            "required": [
+                "data_license",
+                "scripts_license"
+            ],
+            "properties": {
+                "data_license": {
+                    "type": "string",
+                    "examples": [
+                        "https://github.com/example/repo1"
+                    ],
+                    "format": "uri"
+                },
+                "scripts_license": {
+                    "type": "string",
+                    "examples": [
+                        "https://github.com/example/repo1"
+                    ],
+                    "format": "uri"
+                }
+            }
+        }
+    }
+}