Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Improve query performance and templating #8046

Merged
merged 2 commits into from
Feb 16, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CocoComboGUIDMap.json

Large diffs are not rendered by default.

2 changes: 2 additions & 0 deletions bom/build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,7 @@ ext {
// TODO: Held as data engine breaks
mockitoVersion = '4.11.0'
plexusVersion = '4.0.0'
postgresVersion = '42.7.1'
prometheusVersion = '1.12.0'
nettyVersion = '4.1.106.Final'
quartzVersion = '2.3.2'
Expand Down Expand Up @@ -201,6 +202,7 @@ dependencies {
api("org.janusgraph:janusgraph-inmemory:${janusVersion}")
api("org.janusgraph:janusgraph-driver:${janusVersion}")
api("org.javassist:${javassistVersion}")
api("org.postgresql:postgresql:${postgresVersion}")
api("org.quartz-scheduler:quartz:${quartzVersion}")
api("org.reflections:reflections:${reflectionsVersion}")
api("org.slf4j:jcl-over-slf4j:${slf4jVersion}")
Expand Down
2 changes: 1 addition & 1 deletion content-packs/CocoBusinessSystemsArchive.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/CocoClinicalTrialsTemplatesArchive.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/CocoComboArchive.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/CocoComboArchive.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/CocoGovernanceEngineDefinitionsArchive.json

Large diffs are not rendered by default.

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/CocoGovernanceProgramArchive.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/CocoOrganizationArchive.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/CocoSustainabilityArchive.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/CocoTypesArchive.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/OpenMetadataTypes.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/OpenMetadataTypes.omarchive

Large diffs are not rendered by default.

71 changes: 71 additions & 0 deletions content-packs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
<!-- SPDX-License-Identifier: CC-BY-4.0 -->
<!-- Copyright Contributors to the ODPi Egeria project 2020. -->

# Egeria Content Packs

The Egeria content packs provide a means to distribute standard metadata to different organizations.
They are formatted as [Open Metadata Archives](https://egeria-project.org/concepts/open-metadata-archive)
and can be loaded using the Administration Services
[at server start up](https://egeria-project.org/guides/admin/servers/configuring-the-startup-archives)
or [while the server is running](https://egeria-project.org/guides/operations/adding-archive-to-running-server).

They are as follows:

* **CloudInformationModel.json** - glossary from the **Cloud Information Model**'s
[JSONLD formatted model](https://raw.githubusercontent.com/cloudinformationmodel/cloudinformationmodel/master/dist/model.jsonld).
The content covers basic commerce concepts such as Party, Product, Invoice and Shipping.
The cloud information project has been archived. However, this is a useful starter set glossary,

The next four archives are built from the [coco-metadata-archives](https://github.com/odpi/egeria/tree/main/open-metadata-resources/open-metadata-samples/sample-metadata/coco-metadata-archives) module.
They supply metadata to support the [Coco Pharmaceuticals scenarios](https://egeria-project.org/practices/coco-pharmaceuticals/).

* **CocoBusinessSystemsArchive** provides a catalog of the business systems and the lineage between
them and the load of their data into the data lake. This archive simulates the type of metadata expected from
an ETL tool suite. It is intended for **cocoMDS5** in the open metadata labs but can be used in any server.

* **Coco Pharmaceuticals Organizations** - provides the profiles, user identifies and team of the featured
personas of Coco Pharmaceuticals.

* **CocoClinicalTrialsTemplatesArchive** - provides the template assets used for onboarding weekly patient measurements during a clinical trial.

* **Governance Program Definitions** - provides the metadata to describe Coco Pharmaceuticals governance program.

* **CocoGovernanceEngineDefinitionsArchive** - provides the metadata to describe Coco Pharmaceuticals three governance engines:
`AssetGovernance`, `AssetDiscovery` and `AssetQuality`.

* **Coco Pharmaceuticals Sustainability Initiative** provides the base definitions for Coco Pharmaceutical's
sustainability initiative.

* **CocoTypesArchive** - provides additional types for Coco Pharmaceuticals. These are `BiopsyScope` Enum, `BiopsyReport` Entity, `BiopsySupportingEvidence` Relationship and
`ReviewedByClinicalTrials` Classification.

These next archives contain connector type and connector category definitions.

* **DataStoreConnectorTypes** - A subset of the *OpenConnectorsArchive* retained in case anyone is using it.
It should be considered deprecated and the OpenConnectorsArchive used instead.

* **OpenConnectorsArchive** - Connector type metadata for connecting to data sources and other third party technology.

This archive is a summary of all the open metadata type definitions provided by Egeria. It is supplied for
external utilities since each OMAG server capable of being a cohort member will load these types on start up.

* **OpenMetadataTypes** - Archive of the open metadata types defined by Egeria.

This final four archives are for a demo showing 4 metadata access servers connected together in a single cohort.
The archives are each loaded into a different server.
It is then possible to show how the cohort integrates metadata from different catalogs.
These archives are used in the *Development labs* which are part of the [Open Metadata Labs](https://egeria-project.org/education/open-metadata-labs/overview/).

* **SimpleAPICatalog** - API metadata typically found in an API catalog.

* **SimpleDataCatalog** - Data Source metadata typically found in an Data catalog.

* **SimpleEventCatalog** - Event metadata typically found in an API catalog.

* **SimpleGovernanceCatalog** - A glossary term linked to metadata elements in the API, Event, Data catalogs.

There is also **SimpleCatalog** that rolls up the content of the four catalogs into a single content pack for use in other scenarios.

----
License: [CC BY 4.0](https://creativecommons.org/licenses/by/4.0/),
Copyright Contributors to the ODPi Egeria project.
2 changes: 1 addition & 1 deletion content-packs/SimpleAPICatalog.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleAPICatalog.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleCatalog.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleCatalog.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleDataCatalog.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleDataCatalog.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleEventCatalog.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleEventCatalog.omarchive

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleGovernanceCatalog.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion content-packs/SimpleGovernanceCatalog.omarchive

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,6 @@ Provides a basic implementation of the interface javax.sql.DataSource interface
jdbcConnector.asDataSource()
```

See [JDBC Integration Connector](../jdbc-integration-connector/README.MD) for actual usage example.
Its Jar file includes the PostgreSQL client driver.

See [JDBC Resource Connector](https://egeria-project.org/connectors/resource/jdbc-resource-connector/) for documentation.
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
description = 'JDBC Resource Connector for Egeria'

dependencies {
runtimeOnly 'org.postgresql:postgresql'
compileOnly 'org.slf4j:slf4j-api'
compileOnly 'com.fasterxml.jackson.core:jackson-annotations'
compileOnly project(':open-metadata-implementation:frameworks:audit-log-framework')
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2833,6 +2833,8 @@ public List<B> assetZoneScan(String userId,
resultTypeGUID,
resultTypeName,
OpenMetadataProperty.QUALIFIED_NAME.name,
null,
null,
forLineage,
forDuplicateProcessing,
startFrom,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2386,6 +2386,8 @@ public List<B> getEngineActions(String userId,
OpenMetadataType.ENGINE_ACTION_TYPE_GUID,
OpenMetadataType.ENGINE_ACTION_TYPE_NAME,
OpenMetadataProperty.QUALIFIED_NAME.name,
null,
null,
false,
false,
0,
Expand Down Expand Up @@ -2453,6 +2455,8 @@ public List<B> getActiveEngineActions(String userId,
OpenMetadataType.ENGINE_ACTION_TYPE_GUID,
OpenMetadataType.ENGINE_ACTION_TYPE_NAME,
OpenMetadataProperty.QUALIFIED_NAME.name,
null,
null,
false,
false,
0,
Expand Down Expand Up @@ -2542,6 +2546,8 @@ public List<B> getActiveClaimedEngineActions(String userId,
properties,
MatchCriteria.ANY,
OpenMetadataProperty.QUALIFIED_NAME.name,
null,
null,
false,
false,
0,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -632,6 +632,8 @@ private EntityDetail getExternalIdEntity(String userId,
true,
false,
null,
null,
null,
forLineage,
forDuplicateProcessing,
0,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -624,6 +624,8 @@ public List<B> findGlossaryCategories(String userId,
false,
false,
null,
null,
null,
forLineage,
forDuplicateProcessing,
0,
Expand Down Expand Up @@ -862,6 +864,8 @@ public List<B> getGlossaryCategoriesByName(String userId,
true,
false,
null,
null,
null,
forLineage,
forDuplicateProcessing,
0,
Expand Down
Loading
Loading