Skip to content

Latest commit

 

History

History
115 lines (88 loc) · 7.65 KB

README.md

File metadata and controls

115 lines (88 loc) · 7.65 KB
Raw

The Objectives of Egeria

ODPi Egeria implements a set of open APIs, metadata types, frameworks, connectors and interchange protocols to allow all types of tools and metadata repositories to share and exchange metadata.

It is essentially a comprehensive toolkit for integrating and distributing metadata between different tools and technologies. In addition, it has a multi-tenant platform that supports horizontal scale-out in Kubernetes and yet is light enough to run as an edge server on a Raspberry Pi. This platform is used to host the integration and metadata exchange capabilities along with open governance, discovery and access services to automate the collection, management and use of metadata across an enterprise.

The result is an enterprise catalog of data and IT resources that are transparently assessed, governed and consumed through many types of tools and technologies in order to deliver maximum value to the enterprise.

Why open source?

Delivering this capability as open source is a critical part of the project since multiple vendors must buy into this ecosystem. They are not going to do this if one organization dominates the technology base. Thus the open metadata and governance technology must be freely available with an open source governance model that allows a community of organizations and practitioners to develop and evolve the base and then use it in their offerings and deployments.

Project Objectives

Figure 1 summarizes the objectives of Egeria: Automation, Business Value and Connectivity.

Figure 1: The ABC of open metadata and governance

Figure 1: The ABC of open metadata and governance

  • Automation

    It is expensive (and error prone) to expect individuals to maintain a metadata catalog by hand. Egeria helps to automate metadata management through its specialist governance servers that extract and synchronize metadata with different technologies. Figure 1 shows two of these servers, the Stewardship Server and Discovery Server managing metadata and storing it in an open metadata repository.

  • Business Value

    Once metadata is integrated, new insight is possible into the data and services owned by the enterprise. Egeria provides access services and open lineage services that use federated metadata queries to create this enterprise view.

  • Connectivity

    Finally, the automation and and business value would not be possible without the ability to integrate and distribute metadata. Connectivity enables a peer-to-peer Metadata Highway offering open metadata exchange, linking and federation between heterogeneous metadata repositories.

Further reading

Technical Components

These are the principle technical components of Egeria

  • Open Metadata Repository Services (OMRS) - Open metadata repository APIs and notifications to enable metadata repositories to exchange metadata in a peer-to-peer metadata repository cohort. This capability is located in each metadata repository and collectively they enable what is referred to as the "metadata highway".
  • Open Metadata Access Services (OMAS) - Consumer-centric APIs and notifications for specific classes of tools and applications. The OMAS services call the OMRS to access metadata from any open metadata repository.
  • New frameworks:
    • Audit Log Framework (ALF) - provides factories for connectors with access APIs for data resources and metadata together. The OMRS is also built as a set of metadata repository connectors and the OMAS services use the OCF to connect to the appropriate OMRS connector.
    • Open Connector Framework (OCF) - provides factories for connectors with access APIs for data resources and metadata together. The OMRS is also built as a set of metadata repository connectors and the OMAS services use the OCF to connect to the appropriate OMRS connector.
    • Open Discovery Framework (ODF) - provides management for automated processes and analytics to analyze the content of data resources and update the metadata about them.
    • Governance Action Framework (GAF) - provides audit logging and governance enforcement services for implementing enforcement points in data engines, security managers such as Apache Ranger, and APIs. It also adds stewardship services for analyzing audit logs and resolving issues identified in exceptions raised by the enforcement services.
  • Open Metadata Graph Repository - A set of stores linked together with a graph database. These stores provide linkage between business, technical and operational metadata along with logs for auditing, operational lineage, metering and exception management.
  • Governance Servers for automating the management, governance and use of metadata with different tools, platforms and engines.
  • View Servers for REST APIs that support user interfaces.
  • Open Lineage Services - Services for collecting and querying lineage information across multiple heterogeneous metadata repositories.
  • Administration Services for configuring Egeria
  • Platform Services for querying the status of the servers running on the platform.
  • Common Services for common services such as security and first failure data capture (FFDC).
  • Adapters connectors and plugins for different types of technologies.
  • Server Chassis the base component of the OMAG Server Platform.

Further Help

Additional contributions by the ODPi Community

License: CC BY 4.0, Copyright Contributors to the ODPi Egeria project.