Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Don't use database as fallback when no schema parsed. #2023

Merged
merged 1 commit into from
Aug 7, 2023

Conversation

JDarDagran
Copy link
Contributor

Problem

With cross-database queries to information schema there shouldn't be used database as fallback for no schema parsed.

Solution

Set schema to None in TablesHierarchy. This will lead to no filter on schema level in information schema query.

Checklist

  • You've signed-off your work
  • Your pull request title follows our guidelines
  • Your changes are accompanied by tests (if relevant)
  • Your change contains a small diff and is self-contained
  • You've updated any relevant documentation (if relevant)
  • Your comment includes a one-liner for the changelog about the specific purpose of the change (if necessary)
  • You've versioned the core OpenLineage model or facets according to SchemaVer (if relevant)
  • You've added a header to source files (if relevant)

SPDX-License-Identifier: Apache-2.0
Copyright 2018-2023 contributors to the OpenLineage project

@codecov-commenter
Copy link

codecov-commenter commented Jul 31, 2023

Codecov Report

Merging #2023 (93e71e0) into main (6b95181) will increase coverage by 1.36%.
The diff coverage is n/a.

❗ Your organization is not using the GitHub App Integration. As a result you may experience degraded service beginning May 15th. Please install the Github App Integration for your organization. Read more.

@@             Coverage Diff              @@
##               main    #2023      +/-   ##
============================================
+ Coverage     81.14%   82.51%   +1.36%     
============================================
  Files            88       54      -34     
  Lines          3724     3140     -584     
  Branches         29        0      -29     
============================================
- Hits           3022     2591     -431     
+ Misses          666      549     -117     
+ Partials         36        0      -36     
Files Changed Coverage Δ
...ow/openlineage/airflow/extractors/sql_extractor.py 90.37% <ø> (ø)

... and 34 files with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more

@mobuchowski mobuchowski merged commit c3a5211 into main Aug 7, 2023
2 checks passed
@mobuchowski mobuchowski deleted the airflow/no-db-fallback-on-no-schema-parsed branch August 7, 2023 17:21
Sheeri pushed a commit to Sheeri/OpenLineage that referenced this pull request Nov 22, 2023
harels added a commit that referenced this pull request Nov 22, 2023
* add proposal for OpenLineage registry

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix headers

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* improve clarity

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added a requirement
Trying out a commit in a new branch...Hopefully I did it correctly :D

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added an entry for "core" to propose changes to the spec with a "known" starting point.

This entry:
---
root_doc_URL: "https://openlineage.io/spec/facets/"
produced_facets: [
  "ol:core:1-0-0/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-1/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-0/DataQualityAssertionsDatasetFacet.json"
]
---

indicates that the documentation for the produced facets are at:
https://openlineage.io/spec/facets/1-0-0/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-0/DataQualityAssertionsDatasetFacet.json

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Proposing a new formatting of producer/consumer objects instead of an array of strings.

I have modified the "core" example, to show the translation between old and new format.
Added an "egeria" example which is both a producer and consumer.

- Producer and consumer doc root URLs may differ, so they are set inside the producer/consumer object
- It is assumed that the documentation link only applies to the facets owned by the same entity
  - e.g. egeria owns NewCustomFacet so the docs are at the Egeria doc URL
  - egeria does not own ColumnLineageDatasetFacet so there are no docs at the Egeria doc URL (unless it's extended by egeria?)
- "sample_URL" is where the examples/tests can be found (recommended but not required)
- "owner" is added for clarification. egeria produces their own custom facet plus one facet from core in this example
- spec_versions array for compatibility
- use_cases to better create the documentation page
- since there are "consumer" and "producer" objects, "facets" replaces "produced_facets" and "consumed_facets"

Signed-off-by: Sheeri K. Cabral <[email protected]>

* adding another proposed requirement

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added note about accuracy of registry entries

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added Acceptance guidelines

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added a note about reserving names for the future.

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update registry.md

Removing open questions that we don't have an answer to

Signed-off-by: Sheeri K. Cabral <[email protected]>

* bump 0.30.0 release date (#2002)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: remove java macos arm parser build (#2003)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix changelog (#2005)

* fix changelog

Signed-off-by: Michael Robinson <[email protected]>

* add missing change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove $ref facets from core spec. (#1997)

Move from boon to jv.

Add test facets.

Add pre-commit usage guide.

Change facets versions bump to REVISION level.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: convert lineage from legacy file definition (#2006)

* airflow: convert lineage from legacy file definition

Signed-off-by: Maciej Obuchowski <[email protected]>

* Update integration/airflow/openlineage/airflow/extractors/converters.py

Co-authored-by: JDarDagran <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Co-authored-by: JDarDagran <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix error message to avoid confusion (#2001)

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: add file transport documentation (#2008)

Signed-off-by: Alexandre Bergere <[email protected]>
Co-authored-by: Alexandre Bergere <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: make sure we cannot fail in thread despite direct execution (#2010)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Change log level to DEBUG when extractor isn't found (#2012)

There isn't much a user can do when an extractor is not even available for them to use. So changing this to DEBUG makes more sense IM

Signed-off-by: Kaxil Naik <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.3 to 1.5.4 in /integration/spark (#2021)

Bumps org.gradle.test-retry from 1.5.3 to 1.5.4.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit.jupiter:junit-jupiter in /client/java (#2022)

Bumps [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/flink (#2015)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/flink (#2016)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http, snowflake: stop using reusable session by default, do not send full event on snowflake complete (#2025)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add missing changes to changelog for 1.0.0 release (#2027)

* add missing changes to changelog for 1.0.0 release

Signed-off-by: Michael Robinson <[email protected]>

* add missing changes to changelog for 1.0.0 release continued

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.0.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.1.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] filter unwanted events (#1987)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/spark (#2032)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] merge into delta integration test (#2026)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] read configuration from flink conf (#2033)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.6 to 2.9.7 in /integration/airflow (#2031)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.6 to 2.9.7.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.6...2.9.7)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Don't use database as fallback when no schema parsed. (#2023)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add javadoc to the java client (#2004)

* add javadoc to the java client

Signed-off-by: Julien Le Dem <[email protected]>

* change for compiler version compat

Signed-off-by: Julien Le Dem <[email protected]>

* fix javadoc

Signed-off-by: Julien Le Dem <[email protected]>

---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: fix wrong naming of JDBC datasets (#2035)

* spark: jdbc namespaces should not have database in them

Signed-off-by: Maciej Obuchowski <[email protected]>

* tests tests

Signed-off-by: Maciej Obuchowski <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: bump Go image version used in root CI job (#2047)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] append output dataset name to job name (#2036)

* [SPARK] append output dataset name to job name

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] use dot separator within job name parts

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add codespell pre-commit hook. (#2011)

* Add codespell pre-commit hook.

Fix mispells.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Shorten allow list.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] upgrade latest supported version to 3.4.1 (#2057)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/spark (#2018)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] replace dbfs init scripts (#2055)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] fix a bug when getting schema for KafkaSink (#2042)

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

---------

Signed-off-by: pentium3 <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bug/fix ignored event adaptive spark plan databricks (#2061)

* removed/adaptive_spark_plan from excludedNodes of DatabricksEventFilter. see :https://github.com/OpenLineage/OpenLineage/issues/2058

Signed-off-by: Abdallah Terrab <[email protected]>

* added Databricks integration tests testNarrowTransformation and testWideTransformation related to
https://github.com/OpenLineage/OpenLineage/issues/2058
Signed-off-by: Abdallah Terrab <[email protected]>

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew :app:spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply
w/ java version "1.8.0_381"

Signed-off-by: Abdallah Terrab <[email protected]>

---------

Signed-off-by: Abdallah Terrab <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.1.0 (#2062)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.1.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.2.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2068)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.18.1 to 1.19.0 in /integration/spark (#2064)

Bumps `testcontainersVersion` from 1.18.1 to 1.19.0.

Updates `org.testcontainers:junit-jupiter` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:postgresql` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:mockserver` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:kafka` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2065)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.0 to 3.42.0.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.0...3.42.0.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.codehaus.groovy:groovy-all in /integration/flink (#2070)

Bumps [org.codehaus.groovy:groovy-all](https://github.com/apache/groovy) from 3.0.18 to 3.0.19.
- [Commits](https://github.com/apache/groovy/commits)

---
updated-dependencies:
- dependency-name: org.codehaus.groovy:groovy-all
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2069)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update jackson (#2071)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.0 in /client/java (#2066)

Bumps org.openapi.generator from 6.6.0 to 7.0.0.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Ci/fix pre commit step (#2072)

* Change python to python3.8.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Bump cache version due to change in cimg image.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix RDD missing inputs (#2039)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.20.0 to 6.21.0 in /integration/flink (#2080)

Bumps com.diffplug.spotless from 6.20.0 to 6.21.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2077)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.1 to 3.43.0.0.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.1...3.43.0.0)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.tomakehurst:wiremock in /integration/flink (#2079)

Bumps [com.github.tomakehurst:wiremock](https://github.com/wiremock/wiremock) from 2.27.2 to 3.0.1.
- [Release notes](https://github.com/wiremock/wiremock/releases)
- [Commits](https://github.com/wiremock/wiremock/compare/2.27.2...3.0.1)

---
updated-dependencies:
- dependency-name: com.github.tomakehurst:wiremock
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] don't send RUNNING events after COMPLETE (#2075)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http: use non-deprecated apiKey if loading it from env variables (#2029)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.0 to 7.4.1 in /integration/airflow (#2081)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.0 to 7.4.1.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.0...7.4.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: serde filtering (#2044)

* fix: serde filtering

Signed-off-by: Xiang Li <[email protected]>

* Rewrite lambda function.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Xiang Li <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
Co-authored-by: Xiang Li <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `sqlparser` main dependency in ifaces. (#2090)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: publish ProcessingEngineRunFacet (#2089)

* spark: publish ProcessingEngineRunFacet

Previously, the Spark integration published a custom facet named
'SparkVersion'. As it is custom, it isn't defined in the OpenLineage
spec. However, the OpenLineage spec defines a ProcessingEngineRunFacet
that is meant to capture details about the things that runs a job.

This change introduces this in the form of the
'SparkProcessingEngineRunFacetBuilder' and
'SparkProcessingEngineRunFacetBuilderDelegate'.

As the names suggest, these classes are meant to create and populate
ProcessingEngineRunFacet.

The reason for the existence of the delegate is because there are two
code paths that interact with the run facets, namely the code path
within the RddExecutionContext and the other in the
SparkSqlExecutionContext, albeit via a very roundabout way.

The delegate is the object that actually constructs the facet, whilst
the builder provides an adapter that uses the CustomFacetBuilder
interface.

Yes, it's a hacky way of doing it and may need to be changed in the
future. For now though, its good enough.

Closes: https://github.com/OpenLineage/OpenLineage/issues/2086
Signed-off-by: Damien Hawes <[email protected]>

* spark: Deprecated the SparkVersionFacet, alongside the version-facet.json

Signed-off-by: Damien Hawes <[email protected]>

* spark: Used a mocked SparkContext instead inside SparkProcessingEngineFacetBuilderTest

Signed-off-by: Damien Hawes <[email protected]>

* changelog: Updated the changelog indicating that the SparkVersionFacet will be removed in 1.4.0

Signed-off-by: Damien Hawes <[email protected]>

---------

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK][FLINK] Unify dataset naming from URI objects. (#2083)

* [SPARK][FLINK] Unify dataset naming from URI objects.

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] move DatasetIdentifier to openlineage-java

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix ol proxy chart (#2091)

Signed-off-by: Harel Shein <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.slf4j:slf4j-simple from 2.0.7 to 2.0.9 in /client/java (#2096)

Bumps org.slf4j:slf4j-simple from 2.0.7 to 2.0.9.

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-simple
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] verify dataset naming on databricks and limit amount of events sent (#2076)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix circle CI caches (#2101)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Capture clusterAllTags variable from databricks (#2099)

* capture clusterAllTags var

Signed-off-by: anirudh.shrinivason <[email protected]>

* Update changelod

Signed-off-by: anirudh.shrinivason <[email protected]>

* Changelog nit

Signed-off-by: anirudh.shrinivason <[email protected]>

---------

Signed-off-by: anirudh.shrinivason <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* python: fix custom http transport TokenProvider (#2100)

* (init)

Signed-off-by: John Lukenoff <[email protected]>

* unlint

Signed-off-by: John Lukenoff <[email protected]>

---------

Signed-off-by: John Lukenoff <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix failing S3 test on main (#2102)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: Support parsing dbt dbt_project.yml without target-path (#2106)

As of dbt v1.5, usage of target-path in the dbt_project.yml file has been deprecated, now preferring a CLI flag or env var. It will be removed in a future version. See dbt-labs/dbt-core#6882

Docs: https://docs.getdbt.com/reference/project-configs/target-path

This change allows users to run DbtLocalArtifactProcessor in dbt projects that don't declare target-path

Fix: #2093

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: Add openlineage-integration-common PyPI links (#2108)

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.0 (#2111)

* update changelog for 1.2.0

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog (#2112)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix checksum for release sql java (#2114)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.1 (#2119)

* update changelog for 1.2.1

Signed-off-by: Michael Robinson <[email protected]>

* revert changelog change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert bump: org.openapi.generator required JDK 11 (#2113)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.2 (#2120)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.2

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the openlineage-java client's documentation (#2123)

- Added missing transports documentation to the README.md
- Streamlined descriptions for clarity and consistency across all transport types.
- Organized configuration details and examples for better readability.
- Highlighted key notes and behaviors for each transport.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/spark (#2125)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/flink (#2128)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.1 in /client/java (#2126)

Bumps org.openapi.generator from 6.6.0 to 7.0.1.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok from 1.18.28 to 1.18.30 in /client/java (#2127)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] modify CircleCi resource class for macos (#2133)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.4 to 1.5.5 in /integration/spark (#2116)

Bumps org.gradle.test-retry from 1.5.4 to 1.5.5.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add timers to ol emit calls (#1845)

Add tests for stats.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.1 to 7.4.2 in /integration/airflow (#2094)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.1 to 7.4.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.1...7.4.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dbt: Add SQLSERVER to supported dbt profile types (#2136)

* Add SQLSERVER to supported dbt profile types

Signed-off-by: Erik Alfthan <[email protected]>

* Signed commit

Signed-off-by: Erik Alfthan <[email protected]>

* Update integration/common/openlineage/common/provider/dbt/processor.py

Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined) (#2138)

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined)

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched test

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched function

Signed-off-by: Erik Alfthan <[email protected]>

* Apply ruff sort

Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updated the README.md of the sql parser (#2140)

This just surfaces the list of supported dialects to make it more accessible to readers.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add more graceful logging when no OL provider installed. (#2141)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix find-links path in tox. (#2139)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.7 to 2.9.8 in /integration/airflow (#2146)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.7 to 2.9.8.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.7...2.9.8)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Spark: Fixed scheme preservation bug in PathUtils (#2142)

Summary:

Fixed a bug in the PathUtils' prepareDatasetIdentifierFromDefaultTablePath(CatalogTable) method, ensuring the correct scheme preservation from the CatalogTable's location.

Details:

Previously, when generating a DatasetIdentifier from a CatalogTable's default path, the scheme (like "hdfs") could be incorrectly set to "file". This fix addresses the issue, ensuring that the proper scheme from the CatalogTable's location is always preserved.

Impact:

This fix ensures the accuracy and correctness of the DatasetIdentifier's namespace.

Testing:

A unit test was added in PathUtilsTest#testFromCatalogTableShouldReturnADatasetIdentifierWithTheActualScheme

Issue: https://github.com/OpenLineage/OpenLineage/issues/2132

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.21.0 to 6.22.0 in /integration/flink (#2145)

Bumps com.diffplug.spotless from 6.21.0 to 6.22.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.avro:avro from 1.11.2 to 1.11.3 in /integration/flink (#2144)

Bumps org.apache.avro:avro from 1.11.2 to 1.11.3.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support for Spark 3.5 (#2118)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the changelog (#2148)

* Updates the changelog.

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Reverts org.openapi.generator version to enable Java client release. (#2152)

* Reverts org.openapi.generator version to enable java client release.

Signed-off-by: Michael Robinson <[email protected]>

* Configures dependabot to ignore org.openapi.generator.

Signed-off-by: Michael Robinson <[email protected]>

* Updates changelog for 1.3.1.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] expand iceberg source types (#2149)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.19.0 to 1.19.1 in /integration/spark (#2154)

Bumps `testcontainersVersion` from 1.19.0 to 1.19.1.

Updates `org.testcontainers:junit-jupiter` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:postgresql` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:mockserver` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:kafka` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.scala-lang.modules:scala-collection-compat_2.12 (#2157)

Bumps [org.scala-lang.modules:scala-collection-compat_2.12](https://github.com/scala/scala-collection-compat) from 2.1.2 to 2.11.0.
- [Release notes](https://github.com/scala/scala-collection-compat/releases)
- [Commits](https://github.com/scala/scala-collection-compat/compare/v2.1.2...v2.11.0)

---
updated-dependencies:
- dependency-name: org.scala-lang.modules:scala-collection-compat_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Allow setting client's endpoint via environment variable (#2151)

* Allow setting client's endpoint via environment variable

Signed-off-by: Mars Lan <[email protected]>

* Fix lint errors

Signed-off-by: Mars Lan <[email protected]>

---------

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.5 to 1.5.6 in /integration/spark (#2156)

Bumps org.gradle.test-retry from 1.5.5 to 1.5.6.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.8 to 2.9.9 in /integration/airflow (#2155)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.8 to 2.9.9.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.8...2.9.9)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] add debug facet to help resolving Spark integration issues (#2147)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0 in /client/java (#2172)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2174)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients in /integration/spark (#2171)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Enable nessie rest catalog (#2165)

Add a new case to the if-else statement in the getDatasetIdentifier() method to handle Nessie catalogs.
#2084

Signed-off-by: WINKJUL <[email protected]>
Co-authored-by: WINKJUL <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2173)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.0. (#2178)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `deref()` from openlineage-sql impl. (#2179)

Fix mispell in changelog.

Fix `unwrap_or_else`.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.1. (#2180)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] migrate RddExecutionContext to PlanUtils (#2181)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Skip redaction on `ColumnLineageDatasetFacetFieldsAdditionalInputFields`. (#2177)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.amazonaws:amazon-kinesis-producer in /client/java (#2196)

Bumps [com.amazonaws:amazon-kinesis-producer](https://github.com/awslabs/amazon-kinesis-producer) from 0.15.7 to 0.15.8.
- [Release notes](https://github.com/awslabs/amazon-kinesis-producer/releases)
- [Changelog](https://github.com/awslabs/amazon-kinesis-producer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/amazon-kinesis-producer/compare/v0.15.7...v0.15.8)

---
updated-dependencies:
- dependency-name: com.amazonaws:amazon-kinesis-producer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin:gradle-avro-plugin (#2193)

Bumps [com.github.davidmc24.gradle.plugin:gradle-avro-plugin](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin:gradle-avro-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin.avro in /integration/flink (#2195)

Bumps [com.github.davidmc24.gradle.plugin.avro](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin.avro
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix duplicate COMPLETE events (#2103)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] support Flink cassandra lineage (#2175)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* athena: change dataset name to its location (#2167)

* fix ci

athena: change dataset name to its location

Signed-off-by: dkt-sophie-ly <[email protected]>

* add s3 location in symlink facets

Signed-off-by: dkt-sophie-ly <[email protected]>

* add test for athena extractor

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix ci

Signed-off-by: dkt-sophie-ly <[email protected]>

* constraint airflow version

Signed-off-by: dkt-sophie-ly <[email protected]>

* small fix

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix typo in version

Signed-off-by: dkt-sophie-ly <[email protected]>

---------

Signed-off-by: dkt-sophie-ly <[email protected]>
Co-authored-by: dkt-sophie-ly <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* minor changes to spark/README.md file to avoid (#2202)

hick ups in first time setup of spark integration

Signed-off-by: savan navalgi <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2205)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.20.0 to 2.21.0.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.typesafe:config from 1.4.2 to 1.4.3 in /integration/flink (#2206)

Bumps [com.typesafe:config](https://github.com/lightbend/config) from 1.4.2 to 1.4.3.
- [Release notes](https://github.com/lightbend/config/releases)
- [Changelog](https://github.com/lightbend/config/blob/main/NEWS.md)
- [Commits](https://github.com/lightbend/config/compare/v1.4.2...v1.4.3)

---
updated-dependencies:
- dependency-name: com.typesafe:config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2207)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.0.0 to 3.43.2.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.0.0...3.43.2.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] write scala integration test (#2188)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support databricks 13.3. (#2185)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update fluentd proxy to validate against 2.0 spec. (#2213)

Add unit tests to CI.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add always step. (#2182)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Loosen attrs and requests versions. (#2107)

Remove unnecessary dependency in openlineage-airflow.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix bitnami image hash (#2216)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2217)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.21.0 to 2.21.1.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add script to dev for generating release docs for the website (#2219)

* Adds script for generating release doc.

Signed-off-by: Michael Robinson <[email protected]>

* Adds docstring about purpose of script.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Render yaml configs lazily. (#2221)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.5.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.6.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates the changelog for 1.5.0. (#2224)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2231)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.2.1 to 3.43.2.2.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.2.1...3.43.2.2)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* upgrade gradle and jackson (#2233)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Pin yq version. (#2235)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert #2216 as is no longer required (#2234)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] add option for flink job listener to read from flink conf (#2229)

* allow flink job listener to read config values from stream execution environment

Signed-off-by: ensctom <[email protected]>

* add changelog and update flink readme

Signed-off-by: ensctom <[email protected]>

---------

Signed-off-by: ensctom <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spec: add clarity to snowflake naming docs (#2223)

Signed-off-by: David Goss <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add Kafka to naming schema (#2226)

Add missing Kafka to Naming.md

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update README.md (#2236)

Add latest spark version supported
---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Run always workflow really always. (#2238)

Fix spelling.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dagster: support dagster 1.5.x (#2220)

* fix(dagster): fix missing imports in conftest.py

fix missing imports in conftest.py
---
Signed-off-by: George T. C., Lai <[email protected]>
Closes #2043
Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing argument and AttributeError

EventRecordsFilter accepts an event_type as mandatory argument so that
we have to get event records for each event_type.

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for utils.py

correct tests for utils.py

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for sensor evaluation

correct tests for sensor evaluation

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing arguments for sensor

sensor factory method now accepts additional event_type with default set
PIPELINE_EVENTS and STEP_EVENTS for filtering event
records

---

Signed-off-by: George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* docs(dagster): correct requirements for dagster version

correct requirements for Dagster version to 0.15.0+

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): support for Dagster version >=1.0.0

support for Dagster version >=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): create DagsterRun for Dagster version>=1.0.0

handle DagsterRun instance creation for Dagster
version>=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): need not to handle change to EventLogEntry

we don't need to handle removed message field from EventLogEntry since
0.14.3 (minimum version is 1.0.0 now)

---

Signed-off-by: George T. C.
Lai <tsungchih.hd@gma…
julienledem added a commit that referenced this pull request May 24, 2024
* add proposal for OpenLineage registry

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix headers

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* improve clarity

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added a requirement
Trying out a commit in a new branch...Hopefully I did it correctly :D

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added an entry for "core" to propose changes to the spec with a "known" starting point.

This entry:
---
root_doc_URL: "https://openlineage.io/spec/facets/"
produced_facets: [
  "ol:core:1-0-0/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-1/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-0/DataQualityAssertionsDatasetFacet.json"
]
---

indicates that the documentation for the produced facets are at:
https://openlineage.io/spec/facets/1-0-0/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-0/DataQualityAssertionsDatasetFacet.json

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Proposing a new formatting of producer/consumer objects instead of an array of strings.

I have modified the "core" example, to show the translation between old and new format.
Added an "egeria" example which is both a producer and consumer.

- Producer and consumer doc root URLs may differ, so they are set inside the producer/consumer object
- It is assumed that the documentation link only applies to the facets owned by the same entity
  - e.g. egeria owns NewCustomFacet so the docs are at the Egeria doc URL
  - egeria does not own ColumnLineageDatasetFacet so there are no docs at the Egeria doc URL (unless it's extended by egeria?)
- "sample_URL" is where the examples/tests can be found (recommended but not required)
- "owner" is added for clarification. egeria produces their own custom facet plus one facet from core in this example
- spec_versions array for compatibility
- use_cases to better create the documentation page
- since there are "consumer" and "producer" objects, "facets" replaces "produced_facets" and "consumed_facets"

Signed-off-by: Sheeri K. Cabral <[email protected]>

* adding another proposed requirement

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added note about accuracy of registry entries

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added Acceptance guidelines

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added a note about reserving names for the future.

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update registry.md

Removing open questions that we don't have an answer to

Signed-off-by: Sheeri K. Cabral <[email protected]>

* bump 0.30.0 release date (#2002)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: remove java macos arm parser build (#2003)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix changelog (#2005)

* fix changelog

Signed-off-by: Michael Robinson <[email protected]>

* add missing change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove $ref facets from core spec. (#1997)

Move from boon to jv.

Add test facets.

Add pre-commit usage guide.

Change facets versions bump to REVISION level.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: convert lineage from legacy file definition (#2006)

* airflow: convert lineage from legacy file definition

Signed-off-by: Maciej Obuchowski <[email protected]>

* Update integration/airflow/openlineage/airflow/extractors/converters.py

Co-authored-by: JDarDagran <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Co-authored-by: JDarDagran <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix error message to avoid confusion (#2001)

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: add file transport documentation (#2008)

Signed-off-by: Alexandre Bergere <[email protected]>
Co-authored-by: Alexandre Bergere <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: make sure we cannot fail in thread despite direct execution (#2010)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Change log level to DEBUG when extractor isn't found (#2012)

There isn't much a user can do when an extractor is not even available for them to use. So changing this to DEBUG makes more sense IM

Signed-off-by: Kaxil Naik <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.3 to 1.5.4 in /integration/spark (#2021)

Bumps org.gradle.test-retry from 1.5.3 to 1.5.4.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit.jupiter:junit-jupiter in /client/java (#2022)

Bumps [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/flink (#2015)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/flink (#2016)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http, snowflake: stop using reusable session by default, do not send full event on snowflake complete (#2025)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add missing changes to changelog for 1.0.0 release (#2027)

* add missing changes to changelog for 1.0.0 release

Signed-off-by: Michael Robinson <[email protected]>

* add missing changes to changelog for 1.0.0 release continued

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.0.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.1.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] filter unwanted events (#1987)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/spark (#2032)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] merge into delta integration test (#2026)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] read configuration from flink conf (#2033)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.6 to 2.9.7 in /integration/airflow (#2031)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.6 to 2.9.7.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.6...2.9.7)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Don't use database as fallback when no schema parsed. (#2023)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add javadoc to the java client (#2004)

* add javadoc to the java client

Signed-off-by: Julien Le Dem <[email protected]>

* change for compiler version compat

Signed-off-by: Julien Le Dem <[email protected]>

* fix javadoc

Signed-off-by: Julien Le Dem <[email protected]>

---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: fix wrong naming of JDBC datasets (#2035)

* spark: jdbc namespaces should not have database in them

Signed-off-by: Maciej Obuchowski <[email protected]>

* tests tests

Signed-off-by: Maciej Obuchowski <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: bump Go image version used in root CI job (#2047)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] append output dataset name to job name (#2036)

* [SPARK] append output dataset name to job name

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] use dot separator within job name parts

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add codespell pre-commit hook. (#2011)

* Add codespell pre-commit hook.

Fix mispells.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Shorten allow list.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] upgrade latest supported version to 3.4.1 (#2057)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/spark (#2018)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] replace dbfs init scripts (#2055)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] fix a bug when getting schema for KafkaSink (#2042)

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

---------

Signed-off-by: pentium3 <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bug/fix ignored event adaptive spark plan databricks (#2061)

* removed/adaptive_spark_plan from excludedNodes of DatabricksEventFilter. see :https://github.com/OpenLineage/OpenLineage/issues/2058

Signed-off-by: Abdallah Terrab <[email protected]>

* added Databricks integration tests testNarrowTransformation and testWideTransformation related to
https://github.com/OpenLineage/OpenLineage/issues/2058
Signed-off-by: Abdallah Terrab <[email protected]>

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew :app:spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply
w/ java version "1.8.0_381"

Signed-off-by: Abdallah Terrab <[email protected]>

---------

Signed-off-by: Abdallah Terrab <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.1.0 (#2062)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.1.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.2.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2068)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.18.1 to 1.19.0 in /integration/spark (#2064)

Bumps `testcontainersVersion` from 1.18.1 to 1.19.0.

Updates `org.testcontainers:junit-jupiter` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:postgresql` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:mockserver` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:kafka` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2065)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.0 to 3.42.0.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.0...3.42.0.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.codehaus.groovy:groovy-all in /integration/flink (#2070)

Bumps [org.codehaus.groovy:groovy-all](https://github.com/apache/groovy) from 3.0.18 to 3.0.19.
- [Commits](https://github.com/apache/groovy/commits)

---
updated-dependencies:
- dependency-name: org.codehaus.groovy:groovy-all
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2069)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update jackson (#2071)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.0 in /client/java (#2066)

Bumps org.openapi.generator from 6.6.0 to 7.0.0.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Ci/fix pre commit step (#2072)

* Change python to python3.8.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Bump cache version due to change in cimg image.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix RDD missing inputs (#2039)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.20.0 to 6.21.0 in /integration/flink (#2080)

Bumps com.diffplug.spotless from 6.20.0 to 6.21.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2077)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.1 to 3.43.0.0.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.1...3.43.0.0)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.tomakehurst:wiremock in /integration/flink (#2079)

Bumps [com.github.tomakehurst:wiremock](https://github.com/wiremock/wiremock) from 2.27.2 to 3.0.1.
- [Release notes](https://github.com/wiremock/wiremock/releases)
- [Commits](https://github.com/wiremock/wiremock/compare/2.27.2...3.0.1)

---
updated-dependencies:
- dependency-name: com.github.tomakehurst:wiremock
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] don't send RUNNING events after COMPLETE (#2075)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http: use non-deprecated apiKey if loading it from env variables (#2029)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.0 to 7.4.1 in /integration/airflow (#2081)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.0 to 7.4.1.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.0...7.4.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: serde filtering (#2044)

* fix: serde filtering

Signed-off-by: Xiang Li <[email protected]>

* Rewrite lambda function.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Xiang Li <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
Co-authored-by: Xiang Li <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `sqlparser` main dependency in ifaces. (#2090)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: publish ProcessingEngineRunFacet (#2089)

* spark: publish ProcessingEngineRunFacet

Previously, the Spark integration published a custom facet named
'SparkVersion'. As it is custom, it isn't defined in the OpenLineage
spec. However, the OpenLineage spec defines a ProcessingEngineRunFacet
that is meant to capture details about the things that runs a job.

This change introduces this in the form of the
'SparkProcessingEngineRunFacetBuilder' and
'SparkProcessingEngineRunFacetBuilderDelegate'.

As the names suggest, these classes are meant to create and populate
ProcessingEngineRunFacet.

The reason for the existence of the delegate is because there are two
code paths that interact with the run facets, namely the code path
within the RddExecutionContext and the other in the
SparkSqlExecutionContext, albeit via a very roundabout way.

The delegate is the object that actually constructs the facet, whilst
the builder provides an adapter that uses the CustomFacetBuilder
interface.

Yes, it's a hacky way of doing it and may need to be changed in the
future. For now though, its good enough.

Closes: https://github.com/OpenLineage/OpenLineage/issues/2086
Signed-off-by: Damien Hawes <[email protected]>

* spark: Deprecated the SparkVersionFacet, alongside the version-facet.json

Signed-off-by: Damien Hawes <[email protected]>

* spark: Used a mocked SparkContext instead inside SparkProcessingEngineFacetBuilderTest

Signed-off-by: Damien Hawes <[email protected]>

* changelog: Updated the changelog indicating that the SparkVersionFacet will be removed in 1.4.0

Signed-off-by: Damien Hawes <[email protected]>

---------

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK][FLINK] Unify dataset naming from URI objects. (#2083)

* [SPARK][FLINK] Unify dataset naming from URI objects.

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] move DatasetIdentifier to openlineage-java

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix ol proxy chart (#2091)

Signed-off-by: Harel Shein <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.slf4j:slf4j-simple from 2.0.7 to 2.0.9 in /client/java (#2096)

Bumps org.slf4j:slf4j-simple from 2.0.7 to 2.0.9.

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-simple
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] verify dataset naming on databricks and limit amount of events sent (#2076)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix circle CI caches (#2101)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Capture clusterAllTags variable from databricks (#2099)

* capture clusterAllTags var

Signed-off-by: anirudh.shrinivason <[email protected]>

* Update changelod

Signed-off-by: anirudh.shrinivason <[email protected]>

* Changelog nit

Signed-off-by: anirudh.shrinivason <[email protected]>

---------

Signed-off-by: anirudh.shrinivason <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* python: fix custom http transport TokenProvider (#2100)

* (init)

Signed-off-by: John Lukenoff <[email protected]>

* unlint

Signed-off-by: John Lukenoff <[email protected]>

---------

Signed-off-by: John Lukenoff <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix failing S3 test on main (#2102)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: Support parsing dbt dbt_project.yml without target-path (#2106)

As of dbt v1.5, usage of target-path in the dbt_project.yml file has been deprecated, now preferring a CLI flag or env var. It will be removed in a future version. See dbt-labs/dbt-core#6882

Docs: https://docs.getdbt.com/reference/project-configs/target-path

This change allows users to run DbtLocalArtifactProcessor in dbt projects that don't declare target-path

Fix: #2093

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: Add openlineage-integration-common PyPI links (#2108)

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.0 (#2111)

* update changelog for 1.2.0

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog (#2112)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix checksum for release sql java (#2114)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.1 (#2119)

* update changelog for 1.2.1

Signed-off-by: Michael Robinson <[email protected]>

* revert changelog change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert bump: org.openapi.generator required JDK 11 (#2113)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.2 (#2120)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.2

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the openlineage-java client's documentation (#2123)

- Added missing transports documentation to the README.md
- Streamlined descriptions for clarity and consistency across all transport types.
- Organized configuration details and examples for better readability.
- Highlighted key notes and behaviors for each transport.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/spark (#2125)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/flink (#2128)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.1 in /client/java (#2126)

Bumps org.openapi.generator from 6.6.0 to 7.0.1.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok from 1.18.28 to 1.18.30 in /client/java (#2127)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] modify CircleCi resource class for macos (#2133)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.4 to 1.5.5 in /integration/spark (#2116)

Bumps org.gradle.test-retry from 1.5.4 to 1.5.5.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add timers to ol emit calls (#1845)

Add tests for stats.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.1 to 7.4.2 in /integration/airflow (#2094)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.1 to 7.4.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.1...7.4.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dbt: Add SQLSERVER to supported dbt profile types (#2136)

* Add SQLSERVER to supported dbt profile types

Signed-off-by: Erik Alfthan <[email protected]>

* Signed commit

Signed-off-by: Erik Alfthan <[email protected]>

* Update integration/common/openlineage/common/provider/dbt/processor.py

Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined) (#2138)

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined)

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched test

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched function

Signed-off-by: Erik Alfthan <[email protected]>

* Apply ruff sort

Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updated the README.md of the sql parser (#2140)

This just surfaces the list of supported dialects to make it more accessible to readers.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add more graceful logging when no OL provider installed. (#2141)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix find-links path in tox. (#2139)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.7 to 2.9.8 in /integration/airflow (#2146)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.7 to 2.9.8.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.7...2.9.8)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Spark: Fixed scheme preservation bug in PathUtils (#2142)

Summary:

Fixed a bug in the PathUtils' prepareDatasetIdentifierFromDefaultTablePath(CatalogTable) method, ensuring the correct scheme preservation from the CatalogTable's location.

Details:

Previously, when generating a DatasetIdentifier from a CatalogTable's default path, the scheme (like "hdfs") could be incorrectly set to "file". This fix addresses the issue, ensuring that the proper scheme from the CatalogTable's location is always preserved.

Impact:

This fix ensures the accuracy and correctness of the DatasetIdentifier's namespace.

Testing:

A unit test was added in PathUtilsTest#testFromCatalogTableShouldReturnADatasetIdentifierWithTheActualScheme

Issue: https://github.com/OpenLineage/OpenLineage/issues/2132

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.21.0 to 6.22.0 in /integration/flink (#2145)

Bumps com.diffplug.spotless from 6.21.0 to 6.22.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.avro:avro from 1.11.2 to 1.11.3 in /integration/flink (#2144)

Bumps org.apache.avro:avro from 1.11.2 to 1.11.3.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support for Spark 3.5 (#2118)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the changelog (#2148)

* Updates the changelog.

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Reverts org.openapi.generator version to enable Java client release. (#2152)

* Reverts org.openapi.generator version to enable java client release.

Signed-off-by: Michael Robinson <[email protected]>

* Configures dependabot to ignore org.openapi.generator.

Signed-off-by: Michael Robinson <[email protected]>

* Updates changelog for 1.3.1.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] expand iceberg source types (#2149)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.19.0 to 1.19.1 in /integration/spark (#2154)

Bumps `testcontainersVersion` from 1.19.0 to 1.19.1.

Updates `org.testcontainers:junit-jupiter` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:postgresql` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:mockserver` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:kafka` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.scala-lang.modules:scala-collection-compat_2.12 (#2157)

Bumps [org.scala-lang.modules:scala-collection-compat_2.12](https://github.com/scala/scala-collection-compat) from 2.1.2 to 2.11.0.
- [Release notes](https://github.com/scala/scala-collection-compat/releases)
- [Commits](https://github.com/scala/scala-collection-compat/compare/v2.1.2...v2.11.0)

---
updated-dependencies:
- dependency-name: org.scala-lang.modules:scala-collection-compat_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Allow setting client's endpoint via environment variable (#2151)

* Allow setting client's endpoint via environment variable

Signed-off-by: Mars Lan <[email protected]>

* Fix lint errors

Signed-off-by: Mars Lan <[email protected]>

---------

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.5 to 1.5.6 in /integration/spark (#2156)

Bumps org.gradle.test-retry from 1.5.5 to 1.5.6.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.8 to 2.9.9 in /integration/airflow (#2155)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.8 to 2.9.9.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.8...2.9.9)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] add debug facet to help resolving Spark integration issues (#2147)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0 in /client/java (#2172)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2174)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients in /integration/spark (#2171)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Enable nessie rest catalog (#2165)

Add a new case to the if-else statement in the getDatasetIdentifier() method to handle Nessie catalogs.
#2084

Signed-off-by: WINKJUL <[email protected]>
Co-authored-by: WINKJUL <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2173)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.0. (#2178)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `deref()` from openlineage-sql impl. (#2179)

Fix mispell in changelog.

Fix `unwrap_or_else`.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.1. (#2180)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] migrate RddExecutionContext to PlanUtils (#2181)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Skip redaction on `ColumnLineageDatasetFacetFieldsAdditionalInputFields`. (#2177)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.amazonaws:amazon-kinesis-producer in /client/java (#2196)

Bumps [com.amazonaws:amazon-kinesis-producer](https://github.com/awslabs/amazon-kinesis-producer) from 0.15.7 to 0.15.8.
- [Release notes](https://github.com/awslabs/amazon-kinesis-producer/releases)
- [Changelog](https://github.com/awslabs/amazon-kinesis-producer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/amazon-kinesis-producer/compare/v0.15.7...v0.15.8)

---
updated-dependencies:
- dependency-name: com.amazonaws:amazon-kinesis-producer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin:gradle-avro-plugin (#2193)

Bumps [com.github.davidmc24.gradle.plugin:gradle-avro-plugin](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin:gradle-avro-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin.avro in /integration/flink (#2195)

Bumps [com.github.davidmc24.gradle.plugin.avro](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin.avro
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix duplicate COMPLETE events (#2103)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] support Flink cassandra lineage (#2175)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* athena: change dataset name to its location (#2167)

* fix ci

athena: change dataset name to its location

Signed-off-by: dkt-sophie-ly <[email protected]>

* add s3 location in symlink facets

Signed-off-by: dkt-sophie-ly <[email protected]>

* add test for athena extractor

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix ci

Signed-off-by: dkt-sophie-ly <[email protected]>

* constraint airflow version

Signed-off-by: dkt-sophie-ly <[email protected]>

* small fix

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix typo in version

Signed-off-by: dkt-sophie-ly <[email protected]>

---------

Signed-off-by: dkt-sophie-ly <[email protected]>
Co-authored-by: dkt-sophie-ly <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* minor changes to spark/README.md file to avoid (#2202)

hick ups in first time setup of spark integration

Signed-off-by: savan navalgi <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2205)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.20.0 to 2.21.0.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.typesafe:config from 1.4.2 to 1.4.3 in /integration/flink (#2206)

Bumps [com.typesafe:config](https://github.com/lightbend/config) from 1.4.2 to 1.4.3.
- [Release notes](https://github.com/lightbend/config/releases)
- [Changelog](https://github.com/lightbend/config/blob/main/NEWS.md)
- [Commits](https://github.com/lightbend/config/compare/v1.4.2...v1.4.3)

---
updated-dependencies:
- dependency-name: com.typesafe:config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2207)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.0.0 to 3.43.2.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.0.0...3.43.2.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] write scala integration test (#2188)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support databricks 13.3. (#2185)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update fluentd proxy to validate against 2.0 spec. (#2213)

Add unit tests to CI.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add always step. (#2182)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Loosen attrs and requests versions. (#2107)

Remove unnecessary dependency in openlineage-airflow.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix bitnami image hash (#2216)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2217)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.21.0 to 2.21.1.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add script to dev for generating release docs for the website (#2219)

* Adds script for generating release doc.

Signed-off-by: Michael Robinson <[email protected]>

* Adds docstring about purpose of script.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Render yaml configs lazily. (#2221)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.5.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.6.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates the changelog for 1.5.0. (#2224)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2231)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.2.1 to 3.43.2.2.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.2.1...3.43.2.2)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* upgrade gradle and jackson (#2233)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Pin yq version. (#2235)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert #2216 as is no longer required (#2234)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] add option for flink job listener to read from flink conf (#2229)

* allow flink job listener to read config values from stream execution environment

Signed-off-by: ensctom <[email protected]>

* add changelog and update flink readme

Signed-off-by: ensctom <[email protected]>

---------

Signed-off-by: ensctom <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spec: add clarity to snowflake naming docs (#2223)

Signed-off-by: David Goss <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add Kafka to naming schema (#2226)

Add missing Kafka to Naming.md

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update README.md (#2236)

Add latest spark version supported
---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Run always workflow really always. (#2238)

Fix spelling.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dagster: support dagster 1.5.x (#2220)

* fix(dagster): fix missing imports in conftest.py

fix missing imports in conftest.py
---
Signed-off-by: George T. C., Lai <[email protected]>
Closes #2043
Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing argument and AttributeError

EventRecordsFilter accepts an event_type as mandatory argument so that
we have to get event records for each event_type.

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for utils.py

correct tests for utils.py

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for sensor evaluation

correct tests for sensor evaluation

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing arguments for sensor

sensor factory method now accepts additional event_type with default set
PIPELINE_EVENTS and STEP_EVENTS for filtering event
records

---

Signed-off-by: George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* docs(dagster): correct requirements for dagster version

correct requirements for Dagster version to 0.15.0+

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): support for Dagster version >=1.0.0

support for Dagster version >=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): create DagsterRun for Dagster version>=1.0.0

handle DagsterRun instance creation for Dagster
version>=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): need not to handle change to EventLogEntry

we don't need to handle removed message field from EventLogEntry since
0.14.3 (minimum version is 1.0.0 now)

---

Signed-off-by: George T. C.
Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* docs(dagster): bump Dagster version requirement to 1.0.0+

bump Dagster version requirement to 1.0.0+

---

Signed-off-by: George T.
C. Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* style(dagster): fix ruff check errors

fix ruff check errors

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* refactor(dagster): use hasattr to handle attr change in DagsterRun

use  to handle attribute change in DagsterRun instead of

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* chore(dagster): correct factory function signature

remove `Optional` for argument `concerned_event_type`

---

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): support for Dagster version >=1.0.0

support for Dagster version >=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): create DagsterRun for Dagster version>=1.0.0

handle DagsterRun instance creation for Dagster
version>=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* style(dagster): fix ruff check errors

fix ruff check errors

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* refactor(dagster): use hasattr to handle attr change in DagsterRun

use  to handle attribute change in DagsterRun instead of

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

---------

Signed-off-by: George T…
julienledem added a commit that referenced this pull request May 24, 2024
* add proposal for OpenLineage registry

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix headers

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* improve clarity

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added a requirement
Trying out a commit in a new branch...Hopefully I did it correctly :D

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added an entry for "core" to propose changes to the spec with a "known" starting point.

This entry:
---
root_doc_URL: "https://openlineage.io/spec/facets/"
produced_facets: [
  "ol:core:1-0-0/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-1/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-0/DataQualityAssertionsDatasetFacet.json"
]
---

indicates that the documentation for the produced facets are at:
https://openlineage.io/spec/facets/1-0-0/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-0/DataQualityAssertionsDatasetFacet.json

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Proposing a new formatting of producer/consumer objects instead of an array of strings.

I have modified the "core" example, to show the translation between old and new format.
Added an "egeria" example which is both a producer and consumer.

- Producer and consumer doc root URLs may differ, so they are set inside the producer/consumer object
- It is assumed that the documentation link only applies to the facets owned by the same entity
  - e.g. egeria owns NewCustomFacet so the docs are at the Egeria doc URL
  - egeria does not own ColumnLineageDatasetFacet so there are no docs at the Egeria doc URL (unless it's extended by egeria?)
- "sample_URL" is where the examples/tests can be found (recommended but not required)
- "owner" is added for clarification. egeria produces their own custom facet plus one facet from core in this example
- spec_versions array for compatibility
- use_cases to better create the documentation page
- since there are "consumer" and "producer" objects, "facets" replaces "produced_facets" and "consumed_facets"

Signed-off-by: Sheeri K. Cabral <[email protected]>

* adding another proposed requirement

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added note about accuracy of registry entries

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added Acceptance guidelines

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added a note about reserving names for the future.

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update registry.md

Removing open questions that we don't have an answer to

Signed-off-by: Sheeri K. Cabral <[email protected]>

* bump 0.30.0 release date (#2002)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: remove java macos arm parser build (#2003)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix changelog (#2005)

* fix changelog

Signed-off-by: Michael Robinson <[email protected]>

* add missing change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove $ref facets from core spec. (#1997)

Move from boon to jv.

Add test facets.

Add pre-commit usage guide.

Change facets versions bump to REVISION level.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: convert lineage from legacy file definition (#2006)

* airflow: convert lineage from legacy file definition

Signed-off-by: Maciej Obuchowski <[email protected]>

* Update integration/airflow/openlineage/airflow/extractors/converters.py

Co-authored-by: JDarDagran <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Co-authored-by: JDarDagran <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix error message to avoid confusion (#2001)

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: add file transport documentation (#2008)

Signed-off-by: Alexandre Bergere <[email protected]>
Co-authored-by: Alexandre Bergere <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: make sure we cannot fail in thread despite direct execution (#2010)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Change log level to DEBUG when extractor isn't found (#2012)

There isn't much a user can do when an extractor is not even available for them to use. So changing this to DEBUG makes more sense IM

Signed-off-by: Kaxil Naik <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.3 to 1.5.4 in /integration/spark (#2021)

Bumps org.gradle.test-retry from 1.5.3 to 1.5.4.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit.jupiter:junit-jupiter in /client/java (#2022)

Bumps [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/flink (#2015)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/flink (#2016)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http, snowflake: stop using reusable session by default, do not send full event on snowflake complete (#2025)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add missing changes to changelog for 1.0.0 release (#2027)

* add missing changes to changelog for 1.0.0 release

Signed-off-by: Michael Robinson <[email protected]>

* add missing changes to changelog for 1.0.0 release continued

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.0.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.1.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] filter unwanted events (#1987)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/spark (#2032)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] merge into delta integration test (#2026)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] read configuration from flink conf (#2033)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.6 to 2.9.7 in /integration/airflow (#2031)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.6 to 2.9.7.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.6...2.9.7)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Don't use database as fallback when no schema parsed. (#2023)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add javadoc to the java client (#2004)

* add javadoc to the java client

Signed-off-by: Julien Le Dem <[email protected]>

* change for compiler version compat

Signed-off-by: Julien Le Dem <[email protected]>

* fix javadoc

Signed-off-by: Julien Le Dem <[email protected]>

---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: fix wrong naming of JDBC datasets (#2035)

* spark: jdbc namespaces should not have database in them

Signed-off-by: Maciej Obuchowski <[email protected]>

* tests tests

Signed-off-by: Maciej Obuchowski <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: bump Go image version used in root CI job (#2047)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] append output dataset name to job name (#2036)

* [SPARK] append output dataset name to job name

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] use dot separator within job name parts

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add codespell pre-commit hook. (#2011)

* Add codespell pre-commit hook.

Fix mispells.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Shorten allow list.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] upgrade latest supported version to 3.4.1 (#2057)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/spark (#2018)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] replace dbfs init scripts (#2055)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] fix a bug when getting schema for KafkaSink (#2042)

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

---------

Signed-off-by: pentium3 <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bug/fix ignored event adaptive spark plan databricks (#2061)

* removed/adaptive_spark_plan from excludedNodes of DatabricksEventFilter. see :https://github.com/OpenLineage/OpenLineage/issues/2058

Signed-off-by: Abdallah Terrab <[email protected]>

* added Databricks integration tests testNarrowTransformation and testWideTransformation related to
https://github.com/OpenLineage/OpenLineage/issues/2058
Signed-off-by: Abdallah Terrab <[email protected]>

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew :app:spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply
w/ java version "1.8.0_381"

Signed-off-by: Abdallah Terrab <[email protected]>

---------

Signed-off-by: Abdallah Terrab <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.1.0 (#2062)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.1.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.2.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2068)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.18.1 to 1.19.0 in /integration/spark (#2064)

Bumps `testcontainersVersion` from 1.18.1 to 1.19.0.

Updates `org.testcontainers:junit-jupiter` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:postgresql` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:mockserver` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:kafka` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2065)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.0 to 3.42.0.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.0...3.42.0.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.codehaus.groovy:groovy-all in /integration/flink (#2070)

Bumps [org.codehaus.groovy:groovy-all](https://github.com/apache/groovy) from 3.0.18 to 3.0.19.
- [Commits](https://github.com/apache/groovy/commits)

---
updated-dependencies:
- dependency-name: org.codehaus.groovy:groovy-all
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2069)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update jackson (#2071)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.0 in /client/java (#2066)

Bumps org.openapi.generator from 6.6.0 to 7.0.0.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Ci/fix pre commit step (#2072)

* Change python to python3.8.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Bump cache version due to change in cimg image.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix RDD missing inputs (#2039)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.20.0 to 6.21.0 in /integration/flink (#2080)

Bumps com.diffplug.spotless from 6.20.0 to 6.21.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2077)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.1 to 3.43.0.0.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.1...3.43.0.0)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.tomakehurst:wiremock in /integration/flink (#2079)

Bumps [com.github.tomakehurst:wiremock](https://github.com/wiremock/wiremock) from 2.27.2 to 3.0.1.
- [Release notes](https://github.com/wiremock/wiremock/releases)
- [Commits](https://github.com/wiremock/wiremock/compare/2.27.2...3.0.1)

---
updated-dependencies:
- dependency-name: com.github.tomakehurst:wiremock
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] don't send RUNNING events after COMPLETE (#2075)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http: use non-deprecated apiKey if loading it from env variables (#2029)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.0 to 7.4.1 in /integration/airflow (#2081)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.0 to 7.4.1.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.0...7.4.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: serde filtering (#2044)

* fix: serde filtering

Signed-off-by: Xiang Li <[email protected]>

* Rewrite lambda function.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Xiang Li <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
Co-authored-by: Xiang Li <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `sqlparser` main dependency in ifaces. (#2090)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: publish ProcessingEngineRunFacet (#2089)

* spark: publish ProcessingEngineRunFacet

Previously, the Spark integration published a custom facet named
'SparkVersion'. As it is custom, it isn't defined in the OpenLineage
spec. However, the OpenLineage spec defines a ProcessingEngineRunFacet
that is meant to capture details about the things that runs a job.

This change introduces this in the form of the
'SparkProcessingEngineRunFacetBuilder' and
'SparkProcessingEngineRunFacetBuilderDelegate'.

As the names suggest, these classes are meant to create and populate
ProcessingEngineRunFacet.

The reason for the existence of the delegate is because there are two
code paths that interact with the run facets, namely the code path
within the RddExecutionContext and the other in the
SparkSqlExecutionContext, albeit via a very roundabout way.

The delegate is the object that actually constructs the facet, whilst
the builder provides an adapter that uses the CustomFacetBuilder
interface.

Yes, it's a hacky way of doing it and may need to be changed in the
future. For now though, its good enough.

Closes: https://github.com/OpenLineage/OpenLineage/issues/2086
Signed-off-by: Damien Hawes <[email protected]>

* spark: Deprecated the SparkVersionFacet, alongside the version-facet.json

Signed-off-by: Damien Hawes <[email protected]>

* spark: Used a mocked SparkContext instead inside SparkProcessingEngineFacetBuilderTest

Signed-off-by: Damien Hawes <[email protected]>

* changelog: Updated the changelog indicating that the SparkVersionFacet will be removed in 1.4.0

Signed-off-by: Damien Hawes <[email protected]>

---------

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK][FLINK] Unify dataset naming from URI objects. (#2083)

* [SPARK][FLINK] Unify dataset naming from URI objects.

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] move DatasetIdentifier to openlineage-java

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix ol proxy chart (#2091)

Signed-off-by: Harel Shein <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.slf4j:slf4j-simple from 2.0.7 to 2.0.9 in /client/java (#2096)

Bumps org.slf4j:slf4j-simple from 2.0.7 to 2.0.9.

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-simple
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] verify dataset naming on databricks and limit amount of events sent (#2076)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix circle CI caches (#2101)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Capture clusterAllTags variable from databricks (#2099)

* capture clusterAllTags var

Signed-off-by: anirudh.shrinivason <[email protected]>

* Update changelod

Signed-off-by: anirudh.shrinivason <[email protected]>

* Changelog nit

Signed-off-by: anirudh.shrinivason <[email protected]>

---------

Signed-off-by: anirudh.shrinivason <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* python: fix custom http transport TokenProvider (#2100)

* (init)

Signed-off-by: John Lukenoff <[email protected]>

* unlint

Signed-off-by: John Lukenoff <[email protected]>

---------

Signed-off-by: John Lukenoff <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix failing S3 test on main (#2102)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: Support parsing dbt dbt_project.yml without target-path (#2106)

As of dbt v1.5, usage of target-path in the dbt_project.yml file has been deprecated, now preferring a CLI flag or env var. It will be removed in a future version. See dbt-labs/dbt-core#6882

Docs: https://docs.getdbt.com/reference/project-configs/target-path

This change allows users to run DbtLocalArtifactProcessor in dbt projects that don't declare target-path

Fix: #2093

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: Add openlineage-integration-common PyPI links (#2108)

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.0 (#2111)

* update changelog for 1.2.0

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog (#2112)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix checksum for release sql java (#2114)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.1 (#2119)

* update changelog for 1.2.1

Signed-off-by: Michael Robinson <[email protected]>

* revert changelog change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert bump: org.openapi.generator required JDK 11 (#2113)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.2 (#2120)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.2

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the openlineage-java client's documentation (#2123)

- Added missing transports documentation to the README.md
- Streamlined descriptions for clarity and consistency across all transport types.
- Organized configuration details and examples for better readability.
- Highlighted key notes and behaviors for each transport.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/spark (#2125)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/flink (#2128)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.1 in /client/java (#2126)

Bumps org.openapi.generator from 6.6.0 to 7.0.1.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok from 1.18.28 to 1.18.30 in /client/java (#2127)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] modify CircleCi resource class for macos (#2133)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.4 to 1.5.5 in /integration/spark (#2116)

Bumps org.gradle.test-retry from 1.5.4 to 1.5.5.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add timers to ol emit calls (#1845)

Add tests for stats.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.1 to 7.4.2 in /integration/airflow (#2094)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.1 to 7.4.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.1...7.4.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dbt: Add SQLSERVER to supported dbt profile types (#2136)

* Add SQLSERVER to supported dbt profile types

Signed-off-by: Erik Alfthan <[email protected]>

* Signed commit

Signed-off-by: Erik Alfthan <[email protected]>

* Update integration/common/openlineage/common/provider/dbt/processor.py

Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined) (#2138)

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined)

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched test

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched function

Signed-off-by: Erik Alfthan <[email protected]>

* Apply ruff sort

Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updated the README.md of the sql parser (#2140)

This just surfaces the list of supported dialects to make it more accessible to readers.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add more graceful logging when no OL provider installed. (#2141)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix find-links path in tox. (#2139)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.7 to 2.9.8 in /integration/airflow (#2146)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.7 to 2.9.8.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.7...2.9.8)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Spark: Fixed scheme preservation bug in PathUtils (#2142)

Summary:

Fixed a bug in the PathUtils' prepareDatasetIdentifierFromDefaultTablePath(CatalogTable) method, ensuring the correct scheme preservation from the CatalogTable's location.

Details:

Previously, when generating a DatasetIdentifier from a CatalogTable's default path, the scheme (like "hdfs") could be incorrectly set to "file". This fix addresses the issue, ensuring that the proper scheme from the CatalogTable's location is always preserved.

Impact:

This fix ensures the accuracy and correctness of the DatasetIdentifier's namespace.

Testing:

A unit test was added in PathUtilsTest#testFromCatalogTableShouldReturnADatasetIdentifierWithTheActualScheme

Issue: https://github.com/OpenLineage/OpenLineage/issues/2132

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.21.0 to 6.22.0 in /integration/flink (#2145)

Bumps com.diffplug.spotless from 6.21.0 to 6.22.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.avro:avro from 1.11.2 to 1.11.3 in /integration/flink (#2144)

Bumps org.apache.avro:avro from 1.11.2 to 1.11.3.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support for Spark 3.5 (#2118)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the changelog (#2148)

* Updates the changelog.

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Reverts org.openapi.generator version to enable Java client release. (#2152)

* Reverts org.openapi.generator version to enable java client release.

Signed-off-by: Michael Robinson <[email protected]>

* Configures dependabot to ignore org.openapi.generator.

Signed-off-by: Michael Robinson <[email protected]>

* Updates changelog for 1.3.1.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] expand iceberg source types (#2149)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.19.0 to 1.19.1 in /integration/spark (#2154)

Bumps `testcontainersVersion` from 1.19.0 to 1.19.1.

Updates `org.testcontainers:junit-jupiter` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:postgresql` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:mockserver` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:kafka` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.scala-lang.modules:scala-collection-compat_2.12 (#2157)

Bumps [org.scala-lang.modules:scala-collection-compat_2.12](https://github.com/scala/scala-collection-compat) from 2.1.2 to 2.11.0.
- [Release notes](https://github.com/scala/scala-collection-compat/releases)
- [Commits](https://github.com/scala/scala-collection-compat/compare/v2.1.2...v2.11.0)

---
updated-dependencies:
- dependency-name: org.scala-lang.modules:scala-collection-compat_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Allow setting client's endpoint via environment variable (#2151)

* Allow setting client's endpoint via environment variable

Signed-off-by: Mars Lan <[email protected]>

* Fix lint errors

Signed-off-by: Mars Lan <[email protected]>

---------

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.5 to 1.5.6 in /integration/spark (#2156)

Bumps org.gradle.test-retry from 1.5.5 to 1.5.6.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.8 to 2.9.9 in /integration/airflow (#2155)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.8 to 2.9.9.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.8...2.9.9)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] add debug facet to help resolving Spark integration issues (#2147)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0 in /client/java (#2172)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2174)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients in /integration/spark (#2171)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Enable nessie rest catalog (#2165)

Add a new case to the if-else statement in the getDatasetIdentifier() method to handle Nessie catalogs.
#2084

Signed-off-by: WINKJUL <[email protected]>
Co-authored-by: WINKJUL <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2173)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.0. (#2178)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `deref()` from openlineage-sql impl. (#2179)

Fix mispell in changelog.

Fix `unwrap_or_else`.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.1. (#2180)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] migrate RddExecutionContext to PlanUtils (#2181)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Skip redaction on `ColumnLineageDatasetFacetFieldsAdditionalInputFields`. (#2177)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.amazonaws:amazon-kinesis-producer in /client/java (#2196)

Bumps [com.amazonaws:amazon-kinesis-producer](https://github.com/awslabs/amazon-kinesis-producer) from 0.15.7 to 0.15.8.
- [Release notes](https://github.com/awslabs/amazon-kinesis-producer/releases)
- [Changelog](https://github.com/awslabs/amazon-kinesis-producer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/amazon-kinesis-producer/compare/v0.15.7...v0.15.8)

---
updated-dependencies:
- dependency-name: com.amazonaws:amazon-kinesis-producer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin:gradle-avro-plugin (#2193)

Bumps [com.github.davidmc24.gradle.plugin:gradle-avro-plugin](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin:gradle-avro-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin.avro in /integration/flink (#2195)

Bumps [com.github.davidmc24.gradle.plugin.avro](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin.avro
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix duplicate COMPLETE events (#2103)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] support Flink cassandra lineage (#2175)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* athena: change dataset name to its location (#2167)

* fix ci

athena: change dataset name to its location

Signed-off-by: dkt-sophie-ly <[email protected]>

* add s3 location in symlink facets

Signed-off-by: dkt-sophie-ly <[email protected]>

* add test for athena extractor

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix ci

Signed-off-by: dkt-sophie-ly <[email protected]>

* constraint airflow version

Signed-off-by: dkt-sophie-ly <[email protected]>

* small fix

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix typo in version

Signed-off-by: dkt-sophie-ly <[email protected]>

---------

Signed-off-by: dkt-sophie-ly <[email protected]>
Co-authored-by: dkt-sophie-ly <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* minor changes to spark/README.md file to avoid (#2202)

hick ups in first time setup of spark integration

Signed-off-by: savan navalgi <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2205)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.20.0 to 2.21.0.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.typesafe:config from 1.4.2 to 1.4.3 in /integration/flink (#2206)

Bumps [com.typesafe:config](https://github.com/lightbend/config) from 1.4.2 to 1.4.3.
- [Release notes](https://github.com/lightbend/config/releases)
- [Changelog](https://github.com/lightbend/config/blob/main/NEWS.md)
- [Commits](https://github.com/lightbend/config/compare/v1.4.2...v1.4.3)

---
updated-dependencies:
- dependency-name: com.typesafe:config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2207)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.0.0 to 3.43.2.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.0.0...3.43.2.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] write scala integration test (#2188)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support databricks 13.3. (#2185)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update fluentd proxy to validate against 2.0 spec. (#2213)

Add unit tests to CI.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add always step. (#2182)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Loosen attrs and requests versions. (#2107)

Remove unnecessary dependency in openlineage-airflow.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix bitnami image hash (#2216)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2217)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.21.0 to 2.21.1.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add script to dev for generating release docs for the website (#2219)

* Adds script for generating release doc.

Signed-off-by: Michael Robinson <[email protected]>

* Adds docstring about purpose of script.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Render yaml configs lazily. (#2221)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.5.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.6.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates the changelog for 1.5.0. (#2224)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2231)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.2.1 to 3.43.2.2.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.2.1...3.43.2.2)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* upgrade gradle and jackson (#2233)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Pin yq version. (#2235)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert #2216 as is no longer required (#2234)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] add option for flink job listener to read from flink conf (#2229)

* allow flink job listener to read config values from stream execution environment

Signed-off-by: ensctom <[email protected]>

* add changelog and update flink readme

Signed-off-by: ensctom <[email protected]>

---------

Signed-off-by: ensctom <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spec: add clarity to snowflake naming docs (#2223)

Signed-off-by: David Goss <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add Kafka to naming schema (#2226)

Add missing Kafka to Naming.md

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update README.md (#2236)

Add latest spark version supported
---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Run always workflow really always. (#2238)

Fix spelling.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dagster: support dagster 1.5.x (#2220)

* fix(dagster): fix missing imports in conftest.py

fix missing imports in conftest.py
---
Signed-off-by: George T. C., Lai <[email protected]>
Closes #2043
Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing argument and AttributeError

EventRecordsFilter accepts an event_type as mandatory argument so that
we have to get event records for each event_type.

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for utils.py

correct tests for utils.py

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for sensor evaluation

correct tests for sensor evaluation

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing arguments for sensor

sensor factory method now accepts additional event_type with default set
PIPELINE_EVENTS and STEP_EVENTS for filtering event
records

---

Signed-off-by: George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* docs(dagster): correct requirements for dagster version

correct requirements for Dagster version to 0.15.0+

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): support for Dagster version >=1.0.0

support for Dagster version >=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): create DagsterRun for Dagster version>=1.0.0

handle DagsterRun instance creation for Dagster
version>=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): need not to handle change to EventLogEntry

we don't need to handle removed message field from EventLogEntry since
0.14.3 (minimum version is 1.0.0 now)

---

Signed-off-by: George T. C.
Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* docs(dagster): bump Dagster version requirement to 1.0.0+

bump Dagster version requirement to 1.0.0+

---

Signed-off-by: George T.
C. Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* style(dagster): fix ruff check errors

fix ruff check errors

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* refactor(dagster): use hasattr to handle attr change in DagsterRun

use  to handle attribute change in DagsterRun instead of

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* chore(dagster): correct factory function signature

remove `Optional` for argument `concerned_event_type`

---

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): support for Dagster version >=1.0.0

support for Dagster version >=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): create DagsterRun for Dagster version>=1.0.0

handle DagsterRun instance creation for Dagster
version>=1.0.0

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* style(dagster): fix ruff check errors

fix ruff check errors

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* refactor(dagster): use hasattr to handle attr change in DagsterRun

use  to handle attribute change in DagsterRun instead of

---

Signed-off-by: George T. C. Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

---------

Signed-off-by: George T…
harels added a commit that referenced this pull request May 24, 2024
…eage (#2228)

* add proposal for OpenLineage registry

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Julien Le Dem <julien.ledem@nlb-int-svc-proxy-06a01ef5ff9ffd47.elb.us-gov-west-1.amazonaws.com>

* fix headers

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Julien Le Dem <julien.ledem@nlb-int-svc-proxy-06a01ef5ff9ffd47.elb.us-gov-west-1.amazonaws.com>

* improve clarity

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Julien Le Dem <julien.ledem@nlb-int-svc-proxy-06a01ef5ff9ffd47.elb.us-gov-west-1.amazonaws.com>

* Proposed requirements and expanded spec for facet registrations (#2237)

* add proposal for OpenLineage registry

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix headers

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* improve clarity

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added a requirement
Trying out a commit in a new branch...Hopefully I did it correctly :D

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added an entry for "core" to propose changes to the spec with a "known" starting point.

This entry:
---
root_doc_URL: "https://openlineage.io/spec/facets/"
produced_facets: [
  "ol:core:1-0-0/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-1/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-0/DataQualityAssertionsDatasetFacet.json"
]
---

indicates that the documentation for the produced facets are at:
https://openlineage.io/spec/facets/1-0-0/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-0/DataQualityAssertionsDatasetFacet.json

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Proposing a new formatting of producer/consumer objects instead of an array of strings.

I have modified the "core" example, to show the translation between old and new format.
Added an "egeria" example which is both a producer and consumer.

- Producer and consumer doc root URLs may differ, so they are set inside the producer/consumer object
- It is assumed that the documentation link only applies to the facets owned by the same entity
  - e.g. egeria owns NewCustomFacet so the docs are at the Egeria doc URL
  - egeria does not own ColumnLineageDatasetFacet so there are no docs at the Egeria doc URL (unless it's extended by egeria?)
- "sample_URL" is where the examples/tests can be found (recommended but not required)
- "owner" is added for clarification. egeria produces their own custom facet plus one facet from core in this example
- spec_versions array for compatibility
- use_cases to better create the documentation page
- since there are "consumer" and "producer" objects, "facets" replaces "produced_facets" and "consumed_facets"

Signed-off-by: Sheeri K. Cabral <[email protected]>

* adding another proposed requirement

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added note about accuracy of registry entries

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added Acceptance guidelines

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added a note about reserving names for the future.

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update registry.md

Removing open questions that we don't have an answer to

Signed-off-by: Sheeri K. Cabral <[email protected]>

* bump 0.30.0 release date (#2002)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: remove java macos arm parser build (#2003)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix changelog (#2005)

* fix changelog

Signed-off-by: Michael Robinson <[email protected]>

* add missing change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove $ref facets from core spec. (#1997)

Move from boon to jv.

Add test facets.

Add pre-commit usage guide.

Change facets versions bump to REVISION level.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: convert lineage from legacy file definition (#2006)

* airflow: convert lineage from legacy file definition

Signed-off-by: Maciej Obuchowski <[email protected]>

* Update integration/airflow/openlineage/airflow/extractors/converters.py

Co-authored-by: JDarDagran <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Co-authored-by: JDarDagran <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix error message to avoid confusion (#2001)

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: add file transport documentation (#2008)

Signed-off-by: Alexandre Bergere <[email protected]>
Co-authored-by: Alexandre Bergere <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: make sure we cannot fail in thread despite direct execution (#2010)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Change log level to DEBUG when extractor isn't found (#2012)

There isn't much a user can do when an extractor is not even available for them to use. So changing this to DEBUG makes more sense IM

Signed-off-by: Kaxil Naik <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.3 to 1.5.4 in /integration/spark (#2021)

Bumps org.gradle.test-retry from 1.5.3 to 1.5.4.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit.jupiter:junit-jupiter in /client/java (#2022)

Bumps [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/flink (#2015)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/flink (#2016)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http, snowflake: stop using reusable session by default, do not send full event on snowflake complete (#2025)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add missing changes to changelog for 1.0.0 release (#2027)

* add missing changes to changelog for 1.0.0 release

Signed-off-by: Michael Robinson <[email protected]>

* add missing changes to changelog for 1.0.0 release continued

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.0.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.1.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] filter unwanted events (#1987)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/spark (#2032)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] merge into delta integration test (#2026)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] read configuration from flink conf (#2033)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.6 to 2.9.7 in /integration/airflow (#2031)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.6 to 2.9.7.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.6...2.9.7)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Don't use database as fallback when no schema parsed. (#2023)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add javadoc to the java client (#2004)

* add javadoc to the java client

Signed-off-by: Julien Le Dem <[email protected]>

* change for compiler version compat

Signed-off-by: Julien Le Dem <[email protected]>

* fix javadoc

Signed-off-by: Julien Le Dem <[email protected]>

---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: fix wrong naming of JDBC datasets (#2035)

* spark: jdbc namespaces should not have database in them

Signed-off-by: Maciej Obuchowski <[email protected]>

* tests tests

Signed-off-by: Maciej Obuchowski <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: bump Go image version used in root CI job (#2047)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] append output dataset name to job name (#2036)

* [SPARK] append output dataset name to job name

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] use dot separator within job name parts

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add codespell pre-commit hook. (#2011)

* Add codespell pre-commit hook.

Fix mispells.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Shorten allow list.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] upgrade latest supported version to 3.4.1 (#2057)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/spark (#2018)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] replace dbfs init scripts (#2055)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] fix a bug when getting schema for KafkaSink (#2042)

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

---------

Signed-off-by: pentium3 <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bug/fix ignored event adaptive spark plan databricks (#2061)

* removed/adaptive_spark_plan from excludedNodes of DatabricksEventFilter. see :https://github.com/OpenLineage/OpenLineage/issues/2058

Signed-off-by: Abdallah Terrab <[email protected]>

* added Databricks integration tests testNarrowTransformation and testWideTransformation related to
https://github.com/OpenLineage/OpenLineage/issues/2058
Signed-off-by: Abdallah Terrab <[email protected]>

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew :app:spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply
w/ java version "1.8.0_381"

Signed-off-by: Abdallah Terrab <[email protected]>

---------

Signed-off-by: Abdallah Terrab <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.1.0 (#2062)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.1.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.2.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2068)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.18.1 to 1.19.0 in /integration/spark (#2064)

Bumps `testcontainersVersion` from 1.18.1 to 1.19.0.

Updates `org.testcontainers:junit-jupiter` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:postgresql` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:mockserver` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:kafka` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2065)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.0 to 3.42.0.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.0...3.42.0.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.codehaus.groovy:groovy-all in /integration/flink (#2070)

Bumps [org.codehaus.groovy:groovy-all](https://github.com/apache/groovy) from 3.0.18 to 3.0.19.
- [Commits](https://github.com/apache/groovy/commits)

---
updated-dependencies:
- dependency-name: org.codehaus.groovy:groovy-all
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2069)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update jackson (#2071)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.0 in /client/java (#2066)

Bumps org.openapi.generator from 6.6.0 to 7.0.0.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Ci/fix pre commit step (#2072)

* Change python to python3.8.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Bump cache version due to change in cimg image.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix RDD missing inputs (#2039)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.20.0 to 6.21.0 in /integration/flink (#2080)

Bumps com.diffplug.spotless from 6.20.0 to 6.21.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2077)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.1 to 3.43.0.0.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.1...3.43.0.0)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.tomakehurst:wiremock in /integration/flink (#2079)

Bumps [com.github.tomakehurst:wiremock](https://github.com/wiremock/wiremock) from 2.27.2 to 3.0.1.
- [Release notes](https://github.com/wiremock/wiremock/releases)
- [Commits](https://github.com/wiremock/wiremock/compare/2.27.2...3.0.1)

---
updated-dependencies:
- dependency-name: com.github.tomakehurst:wiremock
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] don't send RUNNING events after COMPLETE (#2075)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http: use non-deprecated apiKey if loading it from env variables (#2029)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.0 to 7.4.1 in /integration/airflow (#2081)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.0 to 7.4.1.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.0...7.4.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: serde filtering (#2044)

* fix: serde filtering

Signed-off-by: Xiang Li <[email protected]>

* Rewrite lambda function.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Xiang Li <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
Co-authored-by: Xiang Li <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `sqlparser` main dependency in ifaces. (#2090)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: publish ProcessingEngineRunFacet (#2089)

* spark: publish ProcessingEngineRunFacet

Previously, the Spark integration published a custom facet named
'SparkVersion'. As it is custom, it isn't defined in the OpenLineage
spec. However, the OpenLineage spec defines a ProcessingEngineRunFacet
that is meant to capture details about the things that runs a job.

This change introduces this in the form of the
'SparkProcessingEngineRunFacetBuilder' and
'SparkProcessingEngineRunFacetBuilderDelegate'.

As the names suggest, these classes are meant to create and populate
ProcessingEngineRunFacet.

The reason for the existence of the delegate is because there are two
code paths that interact with the run facets, namely the code path
within the RddExecutionContext and the other in the
SparkSqlExecutionContext, albeit via a very roundabout way.

The delegate is the object that actually constructs the facet, whilst
the builder provides an adapter that uses the CustomFacetBuilder
interface.

Yes, it's a hacky way of doing it and may need to be changed in the
future. For now though, its good enough.

Closes: https://github.com/OpenLineage/OpenLineage/issues/2086
Signed-off-by: Damien Hawes <[email protected]>

* spark: Deprecated the SparkVersionFacet, alongside the version-facet.json

Signed-off-by: Damien Hawes <[email protected]>

* spark: Used a mocked SparkContext instead inside SparkProcessingEngineFacetBuilderTest

Signed-off-by: Damien Hawes <[email protected]>

* changelog: Updated the changelog indicating that the SparkVersionFacet will be removed in 1.4.0

Signed-off-by: Damien Hawes <[email protected]>

---------

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK][FLINK] Unify dataset naming from URI objects. (#2083)

* [SPARK][FLINK] Unify dataset naming from URI objects.

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] move DatasetIdentifier to openlineage-java

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix ol proxy chart (#2091)

Signed-off-by: Harel Shein <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.slf4j:slf4j-simple from 2.0.7 to 2.0.9 in /client/java (#2096)

Bumps org.slf4j:slf4j-simple from 2.0.7 to 2.0.9.

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-simple
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] verify dataset naming on databricks and limit amount of events sent (#2076)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix circle CI caches (#2101)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Capture clusterAllTags variable from databricks (#2099)

* capture clusterAllTags var

Signed-off-by: anirudh.shrinivason <[email protected]>

* Update changelod

Signed-off-by: anirudh.shrinivason <[email protected]>

* Changelog nit

Signed-off-by: anirudh.shrinivason <[email protected]>

---------

Signed-off-by: anirudh.shrinivason <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* python: fix custom http transport TokenProvider (#2100)

* (init)

Signed-off-by: John Lukenoff <[email protected]>

* unlint

Signed-off-by: John Lukenoff <[email protected]>

---------

Signed-off-by: John Lukenoff <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix failing S3 test on main (#2102)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: Support parsing dbt dbt_project.yml without target-path (#2106)

As of dbt v1.5, usage of target-path in the dbt_project.yml file has been deprecated, now preferring a CLI flag or env var. It will be removed in a future version. See dbt-labs/dbt-core#6882

Docs: https://docs.getdbt.com/reference/project-configs/target-path

This change allows users to run DbtLocalArtifactProcessor in dbt projects that don't declare target-path

Fix: #2093

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: Add openlineage-integration-common PyPI links (#2108)

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.0 (#2111)

* update changelog for 1.2.0

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog (#2112)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix checksum for release sql java (#2114)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.1 (#2119)

* update changelog for 1.2.1

Signed-off-by: Michael Robinson <[email protected]>

* revert changelog change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert bump: org.openapi.generator required JDK 11 (#2113)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.2 (#2120)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.2

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the openlineage-java client's documentation (#2123)

- Added missing transports documentation to the README.md
- Streamlined descriptions for clarity and consistency across all transport types.
- Organized configuration details and examples for better readability.
- Highlighted key notes and behaviors for each transport.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/spark (#2125)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/flink (#2128)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.1 in /client/java (#2126)

Bumps org.openapi.generator from 6.6.0 to 7.0.1.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok from 1.18.28 to 1.18.30 in /client/java (#2127)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] modify CircleCi resource class for macos (#2133)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.4 to 1.5.5 in /integration/spark (#2116)

Bumps org.gradle.test-retry from 1.5.4 to 1.5.5.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add timers to ol emit calls (#1845)

Add tests for stats.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.1 to 7.4.2 in /integration/airflow (#2094)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.1 to 7.4.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.1...7.4.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dbt: Add SQLSERVER to supported dbt profile types (#2136)

* Add SQLSERVER to supported dbt profile types

Signed-off-by: Erik Alfthan <[email protected]>

* Signed commit

Signed-off-by: Erik Alfthan <[email protected]>

* Update integration/common/openlineage/common/provider/dbt/processor.py

Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined) (#2138)

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined)

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched test

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched function

Signed-off-by: Erik Alfthan <[email protected]>

* Apply ruff sort

Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updated the README.md of the sql parser (#2140)

This just surfaces the list of supported dialects to make it more accessible to readers.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add more graceful logging when no OL provider installed. (#2141)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix find-links path in tox. (#2139)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.7 to 2.9.8 in /integration/airflow (#2146)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.7 to 2.9.8.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.7...2.9.8)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Spark: Fixed scheme preservation bug in PathUtils (#2142)

Summary:

Fixed a bug in the PathUtils' prepareDatasetIdentifierFromDefaultTablePath(CatalogTable) method, ensuring the correct scheme preservation from the CatalogTable's location.

Details:

Previously, when generating a DatasetIdentifier from a CatalogTable's default path, the scheme (like "hdfs") could be incorrectly set to "file". This fix addresses the issue, ensuring that the proper scheme from the CatalogTable's location is always preserved.

Impact:

This fix ensures the accuracy and correctness of the DatasetIdentifier's namespace.

Testing:

A unit test was added in PathUtilsTest#testFromCatalogTableShouldReturnADatasetIdentifierWithTheActualScheme

Issue: https://github.com/OpenLineage/OpenLineage/issues/2132

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.21.0 to 6.22.0 in /integration/flink (#2145)

Bumps com.diffplug.spotless from 6.21.0 to 6.22.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.avro:avro from 1.11.2 to 1.11.3 in /integration/flink (#2144)

Bumps org.apache.avro:avro from 1.11.2 to 1.11.3.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support for Spark 3.5 (#2118)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the changelog (#2148)

* Updates the changelog.

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Reverts org.openapi.generator version to enable Java client release. (#2152)

* Reverts org.openapi.generator version to enable java client release.

Signed-off-by: Michael Robinson <[email protected]>

* Configures dependabot to ignore org.openapi.generator.

Signed-off-by: Michael Robinson <[email protected]>

* Updates changelog for 1.3.1.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] expand iceberg source types (#2149)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.19.0 to 1.19.1 in /integration/spark (#2154)

Bumps `testcontainersVersion` from 1.19.0 to 1.19.1.

Updates `org.testcontainers:junit-jupiter` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:postgresql` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:mockserver` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:kafka` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.scala-lang.modules:scala-collection-compat_2.12 (#2157)

Bumps [org.scala-lang.modules:scala-collection-compat_2.12](https://github.com/scala/scala-collection-compat) from 2.1.2 to 2.11.0.
- [Release notes](https://github.com/scala/scala-collection-compat/releases)
- [Commits](https://github.com/scala/scala-collection-compat/compare/v2.1.2...v2.11.0)

---
updated-dependencies:
- dependency-name: org.scala-lang.modules:scala-collection-compat_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Allow setting client's endpoint via environment variable (#2151)

* Allow setting client's endpoint via environment variable

Signed-off-by: Mars Lan <[email protected]>

* Fix lint errors

Signed-off-by: Mars Lan <[email protected]>

---------

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.5 to 1.5.6 in /integration/spark (#2156)

Bumps org.gradle.test-retry from 1.5.5 to 1.5.6.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.8 to 2.9.9 in /integration/airflow (#2155)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.8 to 2.9.9.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.8...2.9.9)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] add debug facet to help resolving Spark integration issues (#2147)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0 in /client/java (#2172)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2174)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients in /integration/spark (#2171)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Enable nessie rest catalog (#2165)

Add a new case to the if-else statement in the getDatasetIdentifier() method to handle Nessie catalogs.
#2084

Signed-off-by: WINKJUL <[email protected]>
Co-authored-by: WINKJUL <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2173)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.0. (#2178)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `deref()` from openlineage-sql impl. (#2179)

Fix mispell in changelog.

Fix `unwrap_or_else`.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.1. (#2180)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] migrate RddExecutionContext to PlanUtils (#2181)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Skip redaction on `ColumnLineageDatasetFacetFieldsAdditionalInputFields`. (#2177)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.amazonaws:amazon-kinesis-producer in /client/java (#2196)

Bumps [com.amazonaws:amazon-kinesis-producer](https://github.com/awslabs/amazon-kinesis-producer) from 0.15.7 to 0.15.8.
- [Release notes](https://github.com/awslabs/amazon-kinesis-producer/releases)
- [Changelog](https://github.com/awslabs/amazon-kinesis-producer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/amazon-kinesis-producer/compare/v0.15.7...v0.15.8)

---
updated-dependencies:
- dependency-name: com.amazonaws:amazon-kinesis-producer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin:gradle-avro-plugin (#2193)

Bumps [com.github.davidmc24.gradle.plugin:gradle-avro-plugin](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin:gradle-avro-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin.avro in /integration/flink (#2195)

Bumps [com.github.davidmc24.gradle.plugin.avro](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin.avro
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix duplicate COMPLETE events (#2103)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] support Flink cassandra lineage (#2175)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* athena: change dataset name to its location (#2167)

* fix ci

athena: change dataset name to its location

Signed-off-by: dkt-sophie-ly <[email protected]>

* add s3 location in symlink facets

Signed-off-by: dkt-sophie-ly <[email protected]>

* add test for athena extractor

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix ci

Signed-off-by: dkt-sophie-ly <[email protected]>

* constraint airflow version

Signed-off-by: dkt-sophie-ly <[email protected]>

* small fix

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix typo in version

Signed-off-by: dkt-sophie-ly <[email protected]>

---------

Signed-off-by: dkt-sophie-ly <[email protected]>
Co-authored-by: dkt-sophie-ly <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* minor changes to spark/README.md file to avoid (#2202)

hick ups in first time setup of spark integration

Signed-off-by: savan navalgi <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2205)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.20.0 to 2.21.0.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.typesafe:config from 1.4.2 to 1.4.3 in /integration/flink (#2206)

Bumps [com.typesafe:config](https://github.com/lightbend/config) from 1.4.2 to 1.4.3.
- [Release notes](https://github.com/lightbend/config/releases)
- [Changelog](https://github.com/lightbend/config/blob/main/NEWS.md)
- [Commits](https://github.com/lightbend/config/compare/v1.4.2...v1.4.3)

---
updated-dependencies:
- dependency-name: com.typesafe:config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2207)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.0.0 to 3.43.2.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.0.0...3.43.2.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] write scala integration test (#2188)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support databricks 13.3. (#2185)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update fluentd proxy to validate against 2.0 spec. (#2213)

Add unit tests to CI.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add always step. (#2182)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Loosen attrs and requests versions. (#2107)

Remove unnecessary dependency in openlineage-airflow.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix bitnami image hash (#2216)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2217)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.21.0 to 2.21.1.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add script to dev for generating release docs for the website (#2219)

* Adds script for generating release doc.

Signed-off-by: Michael Robinson <[email protected]>

* Adds docstring about purpose of script.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Render yaml configs lazily. (#2221)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.5.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.6.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates the changelog for 1.5.0. (#2224)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2231)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.2.1 to 3.43.2.2.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.2.1...3.43.2.2)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* upgrade gradle and jackson (#2233)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Pin yq version. (#2235)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert #2216 as is no longer required (#2234)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] add option for flink job listener to read from flink conf (#2229)

* allow flink job listener to read config values from stream execution environment

Signed-off-by: ensctom <[email protected]>

* add changelog and update flink readme

Signed-off-by: ensctom <[email protected]>

---------

Signed-off-by: ensctom <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spec: add clarity to snowflake naming docs (#2223)

Signed-off-by: David Goss <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add Kafka to naming schema (#2226)

Add missing Kafka to Naming.md

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update README.md (#2236)

Add latest spark version supported
---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Run always workflow really always. (#2238)

Fix spelling.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dagster: support dagster 1.5.x (#2220)

* fix(dagster): fix missing imports in conftest.py

fix missing imports in conftest.py
---
Signed-off-by: George T. C., Lai <[email protected]>
Closes #2043
Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing argument and AttributeError

EventRecordsFilter accepts an event_type as mandatory argument so that
we have to get event records for each event_type.

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for utils.py

correct tests for utils.py

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for sensor evaluation

correct tests for sensor evaluation

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing arguments for sensor

sensor factory method now accepts additional event_type with default set
PIPELINE_EVENTS and STEP_EVENTS for filtering event
records

---

Signed-off-by: George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* docs(dagster): correct requirements for dagster version

correct requirements for Dagster version to 0.15.0+

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): support for Dagster version >=1.0.0

support for Dagster version >=1.…
fafnirZ pushed a commit to fafnirZ/OpenLineage that referenced this pull request Jul 3, 2024
… in OpenLineage (OpenLineage#2228)

* add proposal for OpenLineage registry

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Julien Le Dem <julien.ledem@nlb-int-svc-proxy-06a01ef5ff9ffd47.elb.us-gov-west-1.amazonaws.com>

* fix headers

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Julien Le Dem <julien.ledem@nlb-int-svc-proxy-06a01ef5ff9ffd47.elb.us-gov-west-1.amazonaws.com>

* improve clarity

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Julien Le Dem <julien.ledem@nlb-int-svc-proxy-06a01ef5ff9ffd47.elb.us-gov-west-1.amazonaws.com>

* Proposed requirements and expanded spec for facet registrations (#2237)

* add proposal for OpenLineage registry

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix headers

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* improve clarity

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added a requirement
Trying out a commit in a new branch...Hopefully I did it correctly :D

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Added an entry for "core" to propose changes to the spec with a "known" starting point.

This entry:
---
root_doc_URL: "https://openlineage.io/spec/facets/"
produced_facets: [
  "ol:core:1-0-0/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-1/ColumnLineageDatasetFacet.json",
  "ol:core:1-0-0/DataQualityAssertionsDatasetFacet.json"
]
---

indicates that the documentation for the produced facets are at:
https://openlineage.io/spec/facets/1-0-0/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-1/ColumnLineageDatasetFacet.json
https://openlineage.io/spec/facets/1-0-0/DataQualityAssertionsDatasetFacet.json

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Proposing a new formatting of producer/consumer objects instead of an array of strings.

I have modified the "core" example, to show the translation between old and new format.
Added an "egeria" example which is both a producer and consumer.

- Producer and consumer doc root URLs may differ, so they are set inside the producer/consumer object
- It is assumed that the documentation link only applies to the facets owned by the same entity
  - e.g. egeria owns NewCustomFacet so the docs are at the Egeria doc URL
  - egeria does not own ColumnLineageDatasetFacet so there are no docs at the Egeria doc URL (unless it's extended by egeria?)
- "sample_URL" is where the examples/tests can be found (recommended but not required)
- "owner" is added for clarification. egeria produces their own custom facet plus one facet from core in this example
- spec_versions array for compatibility
- use_cases to better create the documentation page
- since there are "consumer" and "producer" objects, "facets" replaces "produced_facets" and "consumed_facets"

Signed-off-by: Sheeri K. Cabral <[email protected]>

* adding another proposed requirement

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added note about accuracy of registry entries

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added Acceptance guidelines

Signed-off-by: Sheeri K. Cabral <[email protected]>

* added a note about reserving names for the future.

Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update registry.md

Removing open questions that we don't have an answer to

Signed-off-by: Sheeri K. Cabral <[email protected]>

* bump 0.30.0 release date (#2002)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: remove java macos arm parser build (#2003)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 0.30.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.0.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix changelog (#2005)

* fix changelog

Signed-off-by: Michael Robinson <[email protected]>

* add missing change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove $ref facets from core spec. (#1997)

Move from boon to jv.

Add test facets.

Add pre-commit usage guide.

Change facets versions bump to REVISION level.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: convert lineage from legacy file definition (#2006)

* airflow: convert lineage from legacy file definition

Signed-off-by: Maciej Obuchowski <[email protected]>

* Update integration/airflow/openlineage/airflow/extractors/converters.py

Co-authored-by: JDarDagran <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Co-authored-by: JDarDagran <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix error message to avoid confusion (#2001)

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: add file transport documentation (#2008)

Signed-off-by: Alexandre Bergere <[email protected]>
Co-authored-by: Alexandre Bergere <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* airflow: make sure we cannot fail in thread despite direct execution (#2010)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Change log level to DEBUG when extractor isn't found (#2012)

There isn't much a user can do when an extractor is not even available for them to use. So changing this to DEBUG makes more sense IM

Signed-off-by: Kaxil Naik <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.3 to 1.5.4 in /integration/spark (#2021)

Bumps org.gradle.test-retry from 1.5.3 to 1.5.4.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit.jupiter:junit-jupiter in /client/java (#2022)

Bumps [org.junit.jupiter:junit-jupiter](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/flink (#2015)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/flink (#2016)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http, snowflake: stop using reusable session by default, do not send full event on snowflake complete (#2025)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add missing changes to changelog for 1.0.0 release (#2027)

* add missing changes to changelog for 1.0.0 release

Signed-off-by: Michael Robinson <[email protected]>

* add missing changes to changelog for 1.0.0 release continued

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.0.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.1.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] filter unwanted events (#1987)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.junit:junit-bom from 5.9.3 to 5.10.0 in /integration/spark (#2032)

Bumps [org.junit:junit-bom](https://github.com/junit-team/junit5) from 5.9.3 to 5.10.0.
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit:junit-bom
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] merge into delta integration test (#2026)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] read configuration from flink conf (#2033)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.6 to 2.9.7 in /integration/airflow (#2031)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.6 to 2.9.7.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.6...2.9.7)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Don't use database as fallback when no schema parsed. (#2023)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add javadoc to the java client (#2004)

* add javadoc to the java client

Signed-off-by: Julien Le Dem <[email protected]>

* change for compiler version compat

Signed-off-by: Julien Le Dem <[email protected]>

* fix javadoc

Signed-off-by: Julien Le Dem <[email protected]>

---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: fix wrong naming of JDBC datasets (#2035)

* spark: jdbc namespaces should not have database in them

Signed-off-by: Maciej Obuchowski <[email protected]>

* tests tests

Signed-off-by: Maciej Obuchowski <[email protected]>

---------

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* ci: bump Go image version used in root CI job (#2047)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] append output dataset name to job name (#2036)

* [SPARK] append output dataset name to job name

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] use dot separator within job name parts

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add codespell pre-commit hook. (#2011)

* Add codespell pre-commit hook.

Fix mispells.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Shorten allow list.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] upgrade latest supported version to 3.4.1 (#2057)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump junit5Version from 5.9.3 to 5.10.0 in /integration/spark (#2018)

Bumps `junit5Version` from 5.9.3 to 5.10.0.

Updates `org.junit.jupiter:junit-jupiter` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

Updates `org.junit.jupiter:junit-jupiter-params` from 5.9.3 to 5.10.0
- [Release notes](https://github.com/junit-team/junit5/releases)
- [Commits](https://github.com/junit-team/junit5/compare/r5.9.3...r5.10.0)

---
updated-dependencies:
- dependency-name: org.junit.jupiter:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.junit.jupiter:junit-jupiter-params
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] replace dbfs init scripts (#2055)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] fix a bug when getting schema for KafkaSink (#2042)

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

* fix a bug when getting schema for KafkaSink

Signed-off-by: pentium3 <[email protected]>

---------

Signed-off-by: pentium3 <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bug/fix ignored event adaptive spark plan databricks (#2061)

* removed/adaptive_spark_plan from excludedNodes of DatabricksEventFilter. see :https://github.com/OpenLineage/OpenLineage/issues/2058

Signed-off-by: Abdallah Terrab <[email protected]>

* added Databricks integration tests testNarrowTransformation and testWideTransformation related to
https://github.com/OpenLineage/OpenLineage/issues/2058
Signed-off-by: Abdallah Terrab <[email protected]>

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew :app:spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply

Signed-off-by: Abdallah Terrab <[email protected]>

* gradlew spotlessApply
w/ java version "1.8.0_381"

Signed-off-by: Abdallah Terrab <[email protected]>

---------

Signed-off-by: Abdallah Terrab <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.1.0 (#2062)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.1.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.2.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2068)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.18.1 to 1.19.0 in /integration/spark (#2064)

Bumps `testcontainersVersion` from 1.18.1 to 1.19.0.

Updates `org.testcontainers:junit-jupiter` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:postgresql` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:mockserver` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

Updates `org.testcontainers:kafka` from 1.18.1 to 1.19.0
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.18.1...1.19.0)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-minor
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2065)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.0 to 3.42.0.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.0...3.42.0.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.codehaus.groovy:groovy-all in /integration/flink (#2070)

Bumps [org.codehaus.groovy:groovy-all](https://github.com/apache/groovy) from 3.0.18 to 3.0.19.
- [Commits](https://github.com/apache/groovy/commits)

---
updated-dependencies:
- dependency-name: org.codehaus.groovy:groovy-all
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2069)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.4.1 to 7.5.0.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.4.1...v7.5.0)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update jackson (#2071)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.0 in /client/java (#2066)

Bumps org.openapi.generator from 6.6.0 to 7.0.0.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Ci/fix pre commit step (#2072)

* Change python to python3.8.

Signed-off-by: Jakub Dardzinski <[email protected]>

* Bump cache version due to change in cimg image.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix RDD missing inputs (#2039)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.20.0 to 6.21.0 in /integration/flink (#2080)

Bumps com.diffplug.spotless from 6.20.0 to 6.21.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2077)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.42.0.1 to 3.43.0.0.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.42.0.1...3.43.0.0)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.tomakehurst:wiremock in /integration/flink (#2079)

Bumps [com.github.tomakehurst:wiremock](https://github.com/wiremock/wiremock) from 2.27.2 to 3.0.1.
- [Release notes](https://github.com/wiremock/wiremock/releases)
- [Commits](https://github.com/wiremock/wiremock/compare/2.27.2...3.0.1)

---
updated-dependencies:
- dependency-name: com.github.tomakehurst:wiremock
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] don't send RUNNING events after COMPLETE (#2075)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* http: use non-deprecated apiKey if loading it from env variables (#2029)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.0 to 7.4.1 in /integration/airflow (#2081)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.0 to 7.4.1.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.0...7.4.1)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: serde filtering (#2044)

* fix: serde filtering

Signed-off-by: Xiang Li <[email protected]>

* Rewrite lambda function.

Signed-off-by: Jakub Dardzinski <[email protected]>

---------

Signed-off-by: Xiang Li <[email protected]>
Signed-off-by: Jakub Dardzinski <[email protected]>
Co-authored-by: Xiang Li <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `sqlparser` main dependency in ifaces. (#2090)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spark: publish ProcessingEngineRunFacet (#2089)

* spark: publish ProcessingEngineRunFacet

Previously, the Spark integration published a custom facet named
'SparkVersion'. As it is custom, it isn't defined in the OpenLineage
spec. However, the OpenLineage spec defines a ProcessingEngineRunFacet
that is meant to capture details about the things that runs a job.

This change introduces this in the form of the
'SparkProcessingEngineRunFacetBuilder' and
'SparkProcessingEngineRunFacetBuilderDelegate'.

As the names suggest, these classes are meant to create and populate
ProcessingEngineRunFacet.

The reason for the existence of the delegate is because there are two
code paths that interact with the run facets, namely the code path
within the RddExecutionContext and the other in the
SparkSqlExecutionContext, albeit via a very roundabout way.

The delegate is the object that actually constructs the facet, whilst
the builder provides an adapter that uses the CustomFacetBuilder
interface.

Yes, it's a hacky way of doing it and may need to be changed in the
future. For now though, its good enough.

Closes: https://github.com/OpenLineage/OpenLineage/issues/2086
Signed-off-by: Damien Hawes <[email protected]>

* spark: Deprecated the SparkVersionFacet, alongside the version-facet.json

Signed-off-by: Damien Hawes <[email protected]>

* spark: Used a mocked SparkContext instead inside SparkProcessingEngineFacetBuilderTest

Signed-off-by: Damien Hawes <[email protected]>

* changelog: Updated the changelog indicating that the SparkVersionFacet will be removed in 1.4.0

Signed-off-by: Damien Hawes <[email protected]>

---------

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK][FLINK] Unify dataset naming from URI objects. (#2083)

* [SPARK][FLINK] Unify dataset naming from URI objects.

Signed-off-by: Pawel Leszczynski <[email protected]>

* [SPARK] move DatasetIdentifier to openlineage-java

Signed-off-by: Pawel Leszczynski <[email protected]>

---------

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix ol proxy chart (#2091)

Signed-off-by: Harel Shein <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.slf4j:slf4j-simple from 2.0.7 to 2.0.9 in /client/java (#2096)

Bumps org.slf4j:slf4j-simple from 2.0.7 to 2.0.9.

---
updated-dependencies:
- dependency-name: org.slf4j:slf4j-simple
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] verify dataset naming on databricks and limit amount of events sent (#2076)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix circle CI caches (#2101)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Capture clusterAllTags variable from databricks (#2099)

* capture clusterAllTags var

Signed-off-by: anirudh.shrinivason <[email protected]>

* Update changelod

Signed-off-by: anirudh.shrinivason <[email protected]>

* Changelog nit

Signed-off-by: anirudh.shrinivason <[email protected]>

---------

Signed-off-by: anirudh.shrinivason <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* python: fix custom http transport TokenProvider (#2100)

* (init)

Signed-off-by: John Lukenoff <[email protected]>

* unlint

Signed-off-by: John Lukenoff <[email protected]>

---------

Signed-off-by: John Lukenoff <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix failing S3 test on main (#2102)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* fix: Support parsing dbt dbt_project.yml without target-path (#2106)

As of dbt v1.5, usage of target-path in the dbt_project.yml file has been deprecated, now preferring a CLI flag or env var. It will be removed in a future version. See dbt-labs/dbt-core#6882

Docs: https://docs.getdbt.com/reference/project-configs/target-path

This change allows users to run DbtLocalArtifactProcessor in dbt projects that don't declare target-path

Fix: #2093

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* docs: Add openlineage-integration-common PyPI links (#2108)

Signed-off-by: Tatiana Al-Chueyr <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.0 (#2111)

* update changelog for 1.2.0

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog (#2112)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] fix checksum for release sql java (#2114)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.1 (#2119)

* update changelog for 1.2.1

Signed-off-by: Michael Robinson <[email protected]>

* revert changelog change

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert bump: org.openapi.generator required JDK 11 (#2113)

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* update changelog for 1.2.2 (#2120)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.2.2

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.3.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the openlineage-java client's documentation (#2123)

- Added missing transports documentation to the README.md
- Streamlined descriptions for clarity and consistency across all transport types.
- Organized configuration details and examples for better readability.
- Highlighted key notes and behaviors for each transport.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/spark (#2125)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok in /integration/flink (#2128)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.openapi.generator from 6.6.0 to 7.0.1 in /client/java (#2126)

Bumps org.openapi.generator from 6.6.0 to 7.0.1.

---
updated-dependencies:
- dependency-name: org.openapi.generator
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.projectlombok:lombok from 1.18.28 to 1.18.30 in /client/java (#2127)

Bumps [org.projectlombok:lombok](https://github.com/projectlombok/lombok) from 1.18.28 to 1.18.30.
- [Changelog](https://github.com/projectlombok/lombok/blob/master/doc/changelog.markdown)
- [Commits](https://github.com/projectlombok/lombok/compare/v1.18.28...v1.18.30)

---
updated-dependencies:
- dependency-name: org.projectlombok:lombok
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [CI] modify CircleCi resource class for macos (#2133)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.4 to 1.5.5 in /integration/spark (#2116)

Bumps org.gradle.test-retry from 1.5.4 to 1.5.5.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add timers to ol emit calls (#1845)

Add tests for stats.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump pytest from 7.4.1 to 7.4.2 in /integration/airflow (#2094)

Bumps [pytest](https://github.com/pytest-dev/pytest) from 7.4.1 to 7.4.2.
- [Release notes](https://github.com/pytest-dev/pytest/releases)
- [Changelog](https://github.com/pytest-dev/pytest/blob/main/CHANGELOG.rst)
- [Commits](https://github.com/pytest-dev/pytest/compare/7.4.1...7.4.2)

---
updated-dependencies:
- dependency-name: pytest
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dbt: Add SQLSERVER to supported dbt profile types (#2136)

* Add SQLSERVER to supported dbt profile types

Signed-off-by: Erik Alfthan <[email protected]>

* Signed commit

Signed-off-by: Erik Alfthan <[email protected]>

* Update integration/common/openlineage/common/provider/dbt/processor.py

Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Co-authored-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined) (#2138)

* #2130 Add columns as schema facet for airflow.lineage.Table (if defined)

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched test

Signed-off-by: Erik Alfthan <[email protected]>

* #2130 Format import - revert black formatting on untouched function

Signed-off-by: Erik Alfthan <[email protected]>

* Apply ruff sort

Signed-off-by: Erik Alfthan <[email protected]>

---------

Signed-off-by: Erik Alfthan <[email protected]>
Co-authored-by: Erik Alfthan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updated the README.md of the sql parser (#2140)

This just surfaces the list of supported dialects to make it more accessible to readers.

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add more graceful logging when no OL provider installed. (#2141)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Fix find-links path in tox. (#2139)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.7 to 2.9.8 in /integration/airflow (#2146)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.7 to 2.9.8.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.7...2.9.8)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Spark: Fixed scheme preservation bug in PathUtils (#2142)

Summary:

Fixed a bug in the PathUtils' prepareDatasetIdentifierFromDefaultTablePath(CatalogTable) method, ensuring the correct scheme preservation from the CatalogTable's location.

Details:

Previously, when generating a DatasetIdentifier from a CatalogTable's default path, the scheme (like "hdfs") could be incorrectly set to "file". This fix addresses the issue, ensuring that the proper scheme from the CatalogTable's location is always preserved.

Impact:

This fix ensures the accuracy and correctness of the DatasetIdentifier's namespace.

Testing:

A unit test was added in PathUtilsTest#testFromCatalogTableShouldReturnADatasetIdentifierWithTheActualScheme

Issue: https://github.com/OpenLineage/OpenLineage/issues/2132

Signed-off-by: Damien Hawes <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.diffplug.spotless from 6.21.0 to 6.22.0 in /integration/flink (#2145)

Bumps com.diffplug.spotless from 6.21.0 to 6.22.0.

---
updated-dependencies:
- dependency-name: com.diffplug.spotless
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.avro:avro from 1.11.2 to 1.11.3 in /integration/flink (#2144)

Bumps org.apache.avro:avro from 1.11.2 to 1.11.3.

---
updated-dependencies:
- dependency-name: org.apache.avro:avro
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support for Spark 3.5 (#2118)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update the changelog (#2148)

* Updates the changelog.

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Reverts org.openapi.generator version to enable Java client release. (#2152)

* Reverts org.openapi.generator version to enable java client release.

Signed-off-by: Michael Robinson <[email protected]>

* Configures dependabot to ignore org.openapi.generator.

Signed-off-by: Michael Robinson <[email protected]>

* Updates changelog for 1.3.1.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.3.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.4.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] expand iceberg source types (#2149)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump testcontainersVersion from 1.19.0 to 1.19.1 in /integration/spark (#2154)

Bumps `testcontainersVersion` from 1.19.0 to 1.19.1.

Updates `org.testcontainers:junit-jupiter` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:postgresql` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:mockserver` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

Updates `org.testcontainers:kafka` from 1.19.0 to 1.19.1
- [Release notes](https://github.com/testcontainers/testcontainers-java/releases)
- [Changelog](https://github.com/testcontainers/testcontainers-java/blob/main/CHANGELOG.md)
- [Commits](https://github.com/testcontainers/testcontainers-java/compare/1.19.0...1.19.1)

---
updated-dependencies:
- dependency-name: org.testcontainers:junit-jupiter
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:postgresql
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:mockserver
  dependency-type: direct:production
  update-type: version-update:semver-patch
- dependency-name: org.testcontainers:kafka
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.scala-lang.modules:scala-collection-compat_2.12 (#2157)

Bumps [org.scala-lang.modules:scala-collection-compat_2.12](https://github.com/scala/scala-collection-compat) from 2.1.2 to 2.11.0.
- [Release notes](https://github.com/scala/scala-collection-compat/releases)
- [Commits](https://github.com/scala/scala-collection-compat/compare/v2.1.2...v2.11.0)

---
updated-dependencies:
- dependency-name: org.scala-lang.modules:scala-collection-compat_2.12
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Allow setting client's endpoint via environment variable (#2151)

* Allow setting client's endpoint via environment variable

Signed-off-by: Mars Lan <[email protected]>

* Fix lint errors

Signed-off-by: Mars Lan <[email protected]>

---------

Signed-off-by: Mars Lan <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.gradle.test-retry from 1.5.5 to 1.5.6 in /integration/spark (#2156)

Bumps org.gradle.test-retry from 1.5.5 to 1.5.6.

---
updated-dependencies:
- dependency-name: org.gradle.test-retry
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump psycopg2-binary from 2.9.8 to 2.9.9 in /integration/airflow (#2155)

Bumps [psycopg2-binary](https://github.com/psycopg/psycopg2) from 2.9.8 to 2.9.9.
- [Changelog](https://github.com/psycopg/psycopg2/blob/master/NEWS)
- [Commits](https://github.com/psycopg/psycopg2/compare/2.9.8...2.9.9)

---
updated-dependencies:
- dependency-name: psycopg2-binary
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] add debug facet to help resolving Spark integration issues (#2147)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0 in /client/java (#2172)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-avro-serializer in /integration/flink (#2174)

Bumps [io.confluent:kafka-avro-serializer](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-avro-serializer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.kafka:kafka-clients in /integration/spark (#2171)

Bumps org.apache.kafka:kafka-clients from 3.5.1 to 3.6.0.

---
updated-dependencies:
- dependency-name: org.apache.kafka:kafka-clients
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Enable nessie rest catalog (#2165)

Add a new case to the if-else statement in the getDatasetIdentifier() method to handle Nessie catalogs.
#2084

Signed-off-by: WINKJUL <[email protected]>
Co-authored-by: WINKJUL <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump io.confluent:kafka-schema-registry-client in /integration/flink (#2173)

Bumps [io.confluent:kafka-schema-registry-client](https://github.com/confluentinc/schema-registry) from 7.5.0 to 7.5.1.
- [Commits](https://github.com/confluentinc/schema-registry/compare/v7.5.0...v7.5.1)

---
updated-dependencies:
- dependency-name: io.confluent:kafka-schema-registry-client
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.0. (#2178)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Remove `deref()` from openlineage-sql impl. (#2179)

Fix mispell in changelog.

Fix `unwrap_or_else`.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates changelog for 1.4.1. (#2180)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.4.1

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.5.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] migrate RddExecutionContext to PlanUtils (#2181)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Skip redaction on `ColumnLineageDatasetFacetFieldsAdditionalInputFields`. (#2177)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.amazonaws:amazon-kinesis-producer in /client/java (#2196)

Bumps [com.amazonaws:amazon-kinesis-producer](https://github.com/awslabs/amazon-kinesis-producer) from 0.15.7 to 0.15.8.
- [Release notes](https://github.com/awslabs/amazon-kinesis-producer/releases)
- [Changelog](https://github.com/awslabs/amazon-kinesis-producer/blob/master/CHANGELOG.md)
- [Commits](https://github.com/awslabs/amazon-kinesis-producer/compare/v0.15.7...v0.15.8)

---
updated-dependencies:
- dependency-name: com.amazonaws:amazon-kinesis-producer
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin:gradle-avro-plugin (#2193)

Bumps [com.github.davidmc24.gradle.plugin:gradle-avro-plugin](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin:gradle-avro-plugin
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.github.davidmc24.gradle.plugin.avro in /integration/flink (#2195)

Bumps [com.github.davidmc24.gradle.plugin.avro](https://github.com/davidmc24/gradle-avro-plugin) from 1.8.0 to 1.9.1.
- [Release notes](https://github.com/davidmc24/gradle-avro-plugin/releases)
- [Changelog](https://github.com/davidmc24/gradle-avro-plugin/blob/master/CHANGES.md)
- [Commits](https://github.com/davidmc24/gradle-avro-plugin/compare/1.8.0...1.9.1)

---
updated-dependencies:
- dependency-name: com.github.davidmc24.gradle.plugin.avro
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix duplicate COMPLETE events (#2103)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [Flink] support Flink cassandra lineage (#2175)

Signed-off-by: Zhenqiu Huang <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* athena: change dataset name to its location (#2167)

* fix ci

athena: change dataset name to its location

Signed-off-by: dkt-sophie-ly <[email protected]>

* add s3 location in symlink facets

Signed-off-by: dkt-sophie-ly <[email protected]>

* add test for athena extractor

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix ci

Signed-off-by: dkt-sophie-ly <[email protected]>

* constraint airflow version

Signed-off-by: dkt-sophie-ly <[email protected]>

* small fix

Signed-off-by: dkt-sophie-ly <[email protected]>

* fix typo in version

Signed-off-by: dkt-sophie-ly <[email protected]>

---------

Signed-off-by: dkt-sophie-ly <[email protected]>
Co-authored-by: dkt-sophie-ly <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* minor changes to spark/README.md file to avoid (#2202)

hick ups in first time setup of spark integration

Signed-off-by: savan navalgi <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2205)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.20.0 to 2.21.0.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump com.typesafe:config from 1.4.2 to 1.4.3 in /integration/flink (#2206)

Bumps [com.typesafe:config](https://github.com/lightbend/config) from 1.4.2 to 1.4.3.
- [Release notes](https://github.com/lightbend/config/releases)
- [Changelog](https://github.com/lightbend/config/blob/main/NEWS.md)
- [Commits](https://github.com/lightbend/config/compare/v1.4.2...v1.4.3)

---
updated-dependencies:
- dependency-name: com.typesafe:config
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2207)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.0.0 to 3.43.2.1.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.0.0...3.43.2.1)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] write scala integration test (#2188)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] support databricks 13.3. (#2185)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update fluentd proxy to validate against 2.0 spec. (#2213)

Add unit tests to CI.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add always step. (#2182)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Loosen attrs and requests versions. (#2107)

Remove unnecessary dependency in openlineage-airflow.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [SPARK] fix bitnami image hash (#2216)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.apache.logging.log4j:log4j-slf4j-impl in /integration/flink (#2217)

Bumps org.apache.logging.log4j:log4j-slf4j-impl from 2.21.0 to 2.21.1.

---
updated-dependencies:
- dependency-name: org.apache.logging.log4j:log4j-slf4j-impl
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Add script to dev for generating release docs for the website (#2219)

* Adds script for generating release doc.

Signed-off-by: Michael Robinson <[email protected]>

* Adds docstring about purpose of script.

Signed-off-by: Michael Robinson <[email protected]>

---------

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Render yaml configs lazily. (#2221)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare for release 1.5.0

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Prepare next development version 1.6.0-SNAPSHOT

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Updates the changelog for 1.5.0. (#2224)

Signed-off-by: Michael Robinson <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Bump org.xerial:sqlite-jdbc in /integration/spark (#2231)

Bumps [org.xerial:sqlite-jdbc](https://github.com/xerial/sqlite-jdbc) from 3.43.2.1 to 3.43.2.2.
- [Release notes](https://github.com/xerial/sqlite-jdbc/releases)
- [Changelog](https://github.com/xerial/sqlite-jdbc/blob/master/CHANGELOG)
- [Commits](https://github.com/xerial/sqlite-jdbc/compare/3.43.2.1...3.43.2.2)

---
updated-dependencies:
- dependency-name: org.xerial:sqlite-jdbc
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* upgrade gradle and jackson (#2233)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Pin yq version. (#2235)

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* revert #2216 as is no longer required (#2234)

Signed-off-by: Pawel Leszczynski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* [FLINK] add option for flink job listener to read from flink conf (#2229)

* allow flink job listener to read config values from stream execution environment

Signed-off-by: ensctom <[email protected]>

* add changelog and update flink readme

Signed-off-by: ensctom <[email protected]>

---------

Signed-off-by: ensctom <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* spec: add clarity to snowflake naming docs (#2223)

Signed-off-by: David Goss <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* add Kafka to naming schema (#2226)

Add missing Kafka to Naming.md

Signed-off-by: Maciej Obuchowski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Update README.md (#2236)

Add latest spark version supported
---------

Signed-off-by: Julien Le Dem <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* Run always workflow really always. (#2238)

Fix spelling.

Signed-off-by: Jakub Dardzinski <[email protected]>
Signed-off-by: Sheeri K. Cabral <[email protected]>

* dagster: support dagster 1.5.x (#2220)

* fix(dagster): fix missing imports in conftest.py

fix missing imports in conftest.py
---
Signed-off-by: George T. C., Lai <[email protected]>
Closes #2043
Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing argument and AttributeError

EventRecordsFilter accepts an event_type as mandatory argument so that
we have to get event records for each event_type.

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for utils.py

correct tests for utils.py

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* test(dagster): correct tests for sensor evaluation

correct tests for sensor evaluation

---

Signed-off-by: George T. C., Lai
<[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): fix missing arguments for sensor

sensor factory method now accepts additional event_type with default set
PIPELINE_EVENTS and STEP_EVENTS for filtering event
records

---

Signed-off-by: George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* docs(dagster): correct requirements for dagster version

correct requirements for Dagster version to 0.15.0+

---

Signed-off-by:
George T. C., Lai <[email protected]>

Closes #2043

Signed-off-by: George T. C. Lai <[email protected]>

* fix(dagster): support for Dagster version >=1.0.0

support for Dagster version >=1.…
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area:integration/airflow openlineage-airflow
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants