-
Notifications
You must be signed in to change notification settings - Fork 261
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] Incomplete and unpackable distribution file with gradle build #7310
Comments
I have no issues unpacking the archive ie:
But note this archive is a .gz, not a tar. 'modern' versions of tar can handle gz files natively, but the invocation above works regardless (old hat unix...). That being said, I checked and even /usr/bin/tar on MacOS 13 works just fine ie:
So it's likely a permissions issue locally. I would check where you are unpacking to. Great to see more use of the gradle build so we can flush out these issues (hopefully will be minor, as I use it all the time) before we flip over to use as default |
One thing to consider - should the sample data & content packs be in this archive? We have discussed creating container images with more of our contents and sample content, perhaps less so what we'd do for anyone just consuming directly (perhaps the answer is to only document) -- since these content packs and samples probably shouldn't be in the core. That being said, in the content of a build migration, consistency as a first step does make sense, so unless we wish to prereq that change, then we should add them |
The sample-data and content-packs need to be in the install image because they are used by the labs. Not everyone will run with K8 so we need to insure the install image is complete. |
In that case I think the right longer term solution would be to create an assembly outside the core egeria repository, which can aggregate across repos. Just as we will for an aggregate container image. (maybe it's the same repo doing this aggregation). Then we can have samples, content in the best place, including additional connectors etc, whilst also ensuring easy to use across k8s and non-k8s environments. But, as above, still makes sense to keep these 2 aspects separate, and add in the content here for now to maintain consistency with the maven build. |
See: #7224 |
I have done some more investigation: If I use The failure to directly unpack the .gz file is because it is readonly (correctly) and so the tar command is not able to over-write the original tar file. Here is the command sequence I used ...
|
The directory structure is created by the distribution plugin. There's no obvious, simple way to customize. When originally coded I felt it appropriate to stick with default conventions setup by our build tools -- even though in this case gradle differs from maven. However a similar issue is reported in stack overflow which adds some customization - but it may have other implications. Additionally the steps that unpack & run FVTs will need minor changes if the distribution archive is changed. If no immediate change is obvious, I suggest we split these observations into individually manageble chunks so
|
ok - I will do the doc update - was making changes to original instructions when discovered this. |
I have:
These changes means that the resources needed by the developer dojo can be sourced from the gradle build (and I can continue converting it from using maven to gradle). |
Sounds good. |
Existing/related issue?
No response
Current Behavior
The gradle build creates a distribution file that is supposed to be the same as the maven distribution file. Trying the 3.14 release there are two problems:
This is a list of the contents:
% tar -tf * egeria-3.14-distribution.tar.gz/ egeria-3.14-distribution.tar.gz/server/ egeria-3.14-distribution.tar.gz/server/server-chassis-spring-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/ egeria-3.14-distribution.tar.gz/server/lib/audit-log-console-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/audit-log-event-topic-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/audit-log-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/audit-log-slf4j-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/cohort-registry-file-store-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/open-metadata-archive-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/open-metadata-archive-directory-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/inmemory-repository-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/graph-repository-connector-jar-with-dependencies-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/omrs-rest-repository-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/avro-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/basic-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/csv-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/data-folder-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/configuration-encrypted-file-store-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/configuration-file-store-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/discovery-service-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/dynamic-archiver-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/governance-action-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/files-integration-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/kafka-integration-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/openapi-integration-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/openlineage-integration-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/elasticsearch-integration-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/inmemory-open-metadata-topic-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/kafka-open-metadata-topic-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/open-lineage-janus-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/open-metadata-security-samples-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/spring-rest-client-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/governance-services-sample-3.14.jar egeria-3.14-distribution.tar.gz/utilities/ egeria-3.14-distribution.tar.gz/utilities/glossary-canonical-model-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/utilities/open-metadata-types-utility-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/ egeria-3.14-distribution.tar.gz/samples/asset-create-avro-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/asset-create-csv-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/asset-reader-avro-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/asset-reader-csv-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/governance-leadership-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/governance-subject-area-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/governance-zone-create-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/subject-area-definition-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/admin-services-config-metadata-server-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/user-interface/ egeria-3.14-distribution.tar.gz/user-interface/ui-chassis-spring-3.14-plain.jar egeria-3.14-distribution.tar.gz/conformance-suite/ egeria-3.14-distribution.tar.gz/conformance-suite/open-metadata-conformance-suite-client-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/LICENSE egeria-3.14-distribution.tar.gz/truststore.p12 egeria-3.14-distribution.tar.gz/NOTICE egeria-3.14-distribution.tar.gz/keystore.p12 egeria-3.14-distribution.tar.gz/content-packs/ egeria-3.14-distribution.tar.gz/sample-data/
These are the errors when it is unpacked:
% tar -xf egeria-3.14-distribution.tar.gz egeria-3.14-distribution.tar.gz/: Refusing to overwrite archive egeria-3.14-distribution.tar.gz/server/: Could not stat egeria-3.14-distribution.tar.gz/server egeria-3.14-distribution.tar.gz/server/server-chassis-spring-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/server-chassis-spring-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/: Could not stat egeria-3.14-distribution.tar.gz/server/lib egeria-3.14-distribution.tar.gz/server/lib/audit-log-console-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/audit-log-console-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/audit-log-event-topic-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/audit-log-event-topic-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/audit-log-file-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/audit-log-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/audit-log-slf4j-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/audit-log-slf4j-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/cohort-registry-file-store-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/cohort-registry-file-store-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/open-metadata-archive-file-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/open-metadata-archive-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/open-metadata-archive-directory-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/open-metadata-archive-directory-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/inmemory-repository-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/inmemory-repository-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/graph-repository-connector-jar-with-dependencies-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/graph-repository-connector-jar-with-dependencies-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/omrs-rest-repository-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/omrs-rest-repository-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/avro-file-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/avro-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/basic-file-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/basic-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/csv-file-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/csv-file-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/data-folder-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/data-folder-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/configuration-encrypted-file-store-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/configuration-encrypted-file-store-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/configuration-file-store-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/configuration-file-store-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/discovery-service-connectors-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/discovery-service-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/dynamic-archiver-connectors-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/dynamic-archiver-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/governance-action-connectors-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/governance-action-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/files-integration-connectors-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/files-integration-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/kafka-integration-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/kafka-integration-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/openapi-integration-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/openapi-integration-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/openlineage-integration-connectors-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/openlineage-integration-connectors-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/elasticsearch-integration-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/elasticsearch-integration-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/inmemory-open-metadata-topic-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/inmemory-open-metadata-topic-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/kafka-open-metadata-topic-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/kafka-open-metadata-topic-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/open-lineage-janus-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/open-lineage-janus-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/open-metadata-security-samples-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/open-metadata-security-samples-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/spring-rest-client-connector-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/spring-rest-client-connector-3.14.jar egeria-3.14-distribution.tar.gz/server/lib/governance-services-sample-3.14.jar: Could not stat egeria-3.14-distribution.tar.gz/server/lib/governance-services-sample-3.14.jar egeria-3.14-distribution.tar.gz/utilities/: Could not stat egeria-3.14-distribution.tar.gz/utilities egeria-3.14-distribution.tar.gz/utilities/glossary-canonical-model-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/utilities/glossary-canonical-model-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/utilities/open-metadata-types-utility-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/utilities/open-metadata-types-utility-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/: Could not stat egeria-3.14-distribution.tar.gz/samples egeria-3.14-distribution.tar.gz/samples/asset-create-avro-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/asset-create-avro-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/asset-create-csv-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/asset-create-csv-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/asset-reader-avro-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/asset-reader-avro-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/asset-reader-csv-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/asset-reader-csv-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/governance-leadership-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/governance-leadership-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/governance-subject-area-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/governance-subject-area-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/governance-zone-create-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/governance-zone-create-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/subject-area-definition-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/subject-area-definition-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/samples/admin-services-config-metadata-server-sample-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/samples/admin-services-config-metadata-server-sample-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/user-interface/: Could not stat egeria-3.14-distribution.tar.gz/user-interface egeria-3.14-distribution.tar.gz/user-interface/ui-chassis-spring-3.14-plain.jar: Could not stat egeria-3.14-distribution.tar.gz/user-interface/ui-chassis-spring-3.14-plain.jar egeria-3.14-distribution.tar.gz/conformance-suite/: Could not stat egeria-3.14-distribution.tar.gz/conformance-suite egeria-3.14-distribution.tar.gz/conformance-suite/open-metadata-conformance-suite-client-3.14-jar-with-dependencies.jar: Could not stat egeria-3.14-distribution.tar.gz/conformance-suite/open-metadata-conformance-suite-client-3.14-jar-with-dependencies.jar egeria-3.14-distribution.tar.gz/LICENSE: Could not stat egeria-3.14-distribution.tar.gz/LICENSE egeria-3.14-distribution.tar.gz/truststore.p12: Could not stat egeria-3.14-distribution.tar.gz/truststore.p12 egeria-3.14-distribution.tar.gz/NOTICE: Could not stat egeria-3.14-distribution.tar.gz/NOTICE egeria-3.14-distribution.tar.gz/keystore.p12: Could not stat egeria-3.14-distribution.tar.gz/keystore.p12 egeria-3.14-distribution.tar.gz/content-packs/: Could not stat egeria-3.14-distribution.tar.gz/content-packs egeria-3.14-distribution.tar.gz/sample-data/: Could not stat egeria-3.14-distribution.tar.gz/sample-data tar: Error exit delayed from previous errors.
Expected Behavior
The following command should unpack the distribution file and create a directory called
egeria-3.14
with the full distribution contents.Steps To Reproduce
Build release 3.14 using gradle then:
cd open-metadata-distribution/open-metadata-assemblies/build/distributions
List the files:
The distribution tar file is
{{release}}-distribution.tar.gz
oregeria-3.14-distribution.tar.gz
in this example.Create a directory for the install and copy the tar file into it. The two commands shown below create an install directory at the same level in the file system as the egeria build library and then copies the egeria distribution file into it.
mkdir ~/egeria-install
These next commands change to the new directory and lists its contents.
It is now possible to unpack the tar file.
tar -xf egeria*-distribution.tar.gz
Environment
Any Further Information?
No response
The text was updated successfully, but these errors were encountered: