Skip to content

Commit

Permalink
Merged all remaining work from the 1.3.x to master (#1127)
Browse files Browse the repository at this point in the history
* updated the release workflow to hopefully work on the 1.3.x branch

* Cherry-pick tests fixes to 1.3.x (#952)

* force junit 5.7.0 (#4)

* ui tests - fix selenium tests when run in k8s testsuite (#936)

* Increase retries on all tests and increase timeouts in some UI tests (#938)

* Added some documentation about configuring the registry UI (#946)

* Added some documentation about configuring the registry UI

* renamed user interface to web console

* address feedback

* added some documentation about artifact meta-data and custom properties (#950)

* added some documentation about artifact meta-data and custom properties

* address review feedback

* Added avro and json schema serde documentation (#957)

* Added avro and json schema serde documentation, mostly to have a place to document the new Avro encoding option

* updated docs based on review feedback

* Update antora.yml

* Add registry client documentation (#955)

* Add registry client documentation

* Improve custom header docs

* Update con-registry-client.adoc

* Update ref-registry-client.adoc

* Update proc-writing-registry-client.adoc

* Improve rest client docs

* Address comments

Co-authored-by: Eric Wittmann <[email protected]>

* Various maven related documentation changes (#964)

* re-organized the maven plugin docs a bit and added some info about "test-update" and file extensions

* incorporate feedback from review

* documented how to override the topic names for kafka storage (#968)

* documented how to override the topic names for kafka storage

* changes based on review feedback

* Added docs for configuring default global rules (#963)

* Added docs for configuring default global rules

* changes based on review feedback

* added feedback from review

* Document artifact states 965 (#967)

* Add artifact state docs section

* Improve state docs

* Address comments

* Address comments

* Add Datum and Protobuf docs. (#971)

* Add Datum and Protobuf docs.

* Update con-registry-serdes-types.adoc

Co-authored-by: Eric Wittmann <[email protected]>

* Update antora.yml

* Automated update to Release Version:: 1.3.2.Final

* Automated update to next Snapshot Version: 1.3.3-SNAPSHOT

* Update antora.yml

* clean up registry user docs and restructure (#1004)

* add revew feedback and tidy up (#1013)

* more doc clean up, fomatting, modularization- no technical changes (#1016)

* Automated update to Release Version:: 1.3.2.Final

* Automated update to next Snapshot Version

* Fix serializer to use the right (Registry's) schema. (#1021)

* Add check-period to globalId strategy. (#1025)

* Add check-period to globalId strategy.

* Add check period test and docs.

* Doc release clean up (#1050)

* fix upstream branding and tidy up xref attributes

* restructure and tidy up schema look up strategies

* minor tidy up

* clean up install steps based on verification testing with openshift 4.5/4.6 (#1082)

* Push docker images using "1.3.x-snapshot" tag

* Update verify.yaml

* Update release.yaml

* Fixed some merge issues (bad choices when resolving conflicts or missed merge effects).

* Retry the schema fetch in the Kafka serializer to account for async storages

* Fix for the check period test to support async storages

Co-authored-by: Fabian Martinez <[email protected]>
Co-authored-by: Carles Arnal <[email protected]>
Co-authored-by: Aleš Justin <[email protected]>
Co-authored-by: apicurio-ci <[email protected]>
Co-authored-by: Stephen McCarthy <[email protected]>
  • Loading branch information
6 people authored Jan 15, 2021
1 parent f48f196 commit 4bbcfac
Show file tree
Hide file tree
Showing 71 changed files with 1,298 additions and 383 deletions.
8 changes: 4 additions & 4 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -69,8 +69,8 @@ jobs:
git config --global user.email "[email protected]"
git remote add origin "https://apicurio-ci:${{ secrets.ACCESS_TOKEN }}@github.com/Apicurio/apicurio-registry-examples.git"
git fetch
git checkout master
git branch --set-upstream-to=origin/master
git checkout ${{steps.metadata.outputs.branch}}
git branch --set-upstream-to=origin/${{steps.metadata.outputs.branch}}
git pull
- name: Apicurio Website Checkout
Expand All @@ -82,8 +82,8 @@ jobs:
git config --global user.email "[email protected]"
git remote add origin "https://apicurio-ci:${{ secrets.ACCESS_TOKEN }}@github.com/Apicurio/apicurio.github.io.git"
git fetch
git checkout master
git branch --set-upstream-to=origin/master
git checkout ${{steps.metadata.outputs.branch}}
git branch --set-upstream-to=origin/${{steps.metadata.outputs.branch}}
git pull
- name: Apicurio Playbook Checkout
Expand Down
29 changes: 29 additions & 0 deletions app/src/test/java/io/apicurio/registry/RegistrySerdeTest.java
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@
import io.apicurio.registry.support.TestCmmn;
import io.apicurio.registry.support.Tester;
import io.apicurio.registry.types.ArtifactType;
import io.apicurio.registry.utils.IoUtil;
import io.apicurio.registry.utils.serde.AvroEncoding;
import io.apicurio.registry.utils.serde.AvroKafkaDeserializer;
import io.apicurio.registry.utils.serde.AvroKafkaSerializer;
Expand Down Expand Up @@ -126,6 +127,34 @@ public void testCachedSchema() throws Exception {
Assertions.assertEquals(id, idStrategy.findId(restClient, artifactId, ArtifactType.AVRO, schema));
}

@Test
public void testCheckPeriod() throws Exception {
Schema schema = new Schema.Parser().parse("{\"type\":\"record\",\"name\":\"myrecord5x\",\"fields\":[{\"name\":\"bar\",\"type\":\"string\"}]}");
String artifactId = generateArtifactId();
byte[] schemaContent = IoUtil.toBytes(schema.toString());
ArtifactMetaData amd = restClient.createArtifact(artifactId, ArtifactType.AVRO, new ByteArrayInputStream(schemaContent));
this.waitForGlobalId(amd.getGlobalId());

long pc = 5000L; // 5seconds check period ...

Map<String, Object> config = new HashMap<>();
config.put(SerdeConfig.CHECK_PERIOD_MS, String.valueOf(pc));
GlobalIdStrategy<Schema> idStrategy = new FindLatestIdStrategy<>();
idStrategy.configure(config, false);

long id1 = idStrategy.findId(restClient, artifactId, ArtifactType.AVRO, schema);
long id2 = idStrategy.findId(restClient, artifactId, ArtifactType.AVRO, schema);
Assertions.assertEquals(id1, id2); // should be less than 5seconds ...
retry(() -> restClient.getArtifactMetaDataByGlobalId(id2));

ArtifactMetaData amd_v2 = restClient.updateArtifact(artifactId, ArtifactType.AVRO, new ByteArrayInputStream(schemaContent));
this.waitForGlobalId(amd_v2.getGlobalId());
Thread.sleep(pc + 1);
retry(() -> Assertions.assertNotEquals(id2, restClient.getArtifactMetaData(artifactId).getGlobalId()));

Assertions.assertNotEquals(id2, idStrategy.findId(restClient, artifactId, ArtifactType.AVRO, schema));
}

@SuppressWarnings("unchecked")
@Test
public void testConfiguration() throws Exception {
Expand Down
4 changes: 2 additions & 2 deletions docs/getting-started/master.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ include::attributes.adoc[]
include::assemblies/getting-started/assembly-intro-to-the-registry.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-intro-to-registry-rules.adoc[leveloffset=+1]
ifdef::apicurio-registry[]
include::assemblies/getting-started/assembly-installing-the-registry-docker.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-installing-registry-docker.adoc[leveloffset=+1]
endif::[]
include::assemblies/getting-started/assembly-installing-the-registry-openshift.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-installing-registry-openshift.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-configuring-the-registry.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-managing-registry-artifacts-ui.adoc[leveloffset=+1]
include::assemblies/getting-started/assembly-managing-registry-artifacts-api.adoc[leveloffset=+1]
Expand Down
17 changes: 12 additions & 5 deletions docs/local-test-playbook.yml
Original file line number Diff line number Diff line change
Expand Up @@ -14,16 +14,23 @@ ui:
url: https://raw.githubusercontent.com/Apicurio/apicurio-docs-ui/master/dist/ui-bundle.zip
snapshot: true

runtime:
cache_dir: ./target/antora-cache

output:
dir: ./target/dist

asciidoc:
attributes:
plantuml-server-url: 'http://www.plantuml.com/plantuml'
plantuml-fetch-diagram: true
mod-loc: partial$
registry-overview: link:assembly-intro-to-the-registry.html[Introduction to Apicurio Registry]
registry-rules: link:assembly-intro-to-registry-rules.html[Introduction to Apicurio Registry rules]
registry-artifact-types: link:assembly-registry-reference.html[Apicurio Registry reference]
registry-rule-types: link:assembly-registry-reference.html[Apicurio Registry reference]
registry-rule-maturity-matrix: link:assembly-registry-reference.html[Apicurio Registry reference]
registry-reference: link:assembly-registry-reference.html[Apicurio Registry artifact reference]
installing-the-registry-docker: link:assembly-installing-registry-docker.html[Installing Apicurio Registry using Docker]
installing-the-registry-openshift: link:assembly-installing-registry-openshift.html[Installing Apicurio Registry on OpenShift]
installing-the-registry-storage-openshift: link:assembly-installing-registry-storage-openshift.html[Installing Apicurio Registry storage on OpenShift]
managing-registry-artifacts-ui: link:assembly-managing-registry-artifacts-ui.html[Managing Apicurio Registry content using the web console]
installing-the-registry-docker: link:assembly-installing-the-registry-docker.html[Installing Apicurio Registry using Docker]
installing-the-registry-openshift: link:assembly-installing-the-registry-openshift.html[Installing Apicurio Registry on OpenShift]
managing-registry-artifacts-api: link:assembly-managing-registry-artifacts-api.html[Managing Apicurio Registry content using the REST API]
kafka-client-serdes: link:assembly-using-kafka-client-serdes.html[Validating schemas using Kafka client serializers/deserializers]
13 changes: 8 additions & 5 deletions docs/modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -1,11 +1,14 @@
* xref:getting-started/assembly-intro-to-the-registry.adoc[]
* xref:getting-started/assembly-intro-to-registry-rules.adoc[]
ifdef:getting-started/:apicurio-registry[]
* xref:getting-started/assembly-installing-the-registry-docker.adoc[]
endif:getting-started/:[]
* xref:getting-started/assembly-installing-the-registry-openshift.adoc[]
ifdef::apicurio-registry[]
* xref:getting-started/assembly-installing-registry-docker.adoc[]
endif::[]
* xref:getting-started/assembly-installing-registry-openshift.adoc[]
* xref:getting-started/assembly-installing-registry-storage-openshift.adoc[]
* xref:getting-started/assembly-configuring-the-registry.adoc[]
* xref:getting-started/assembly-managing-registry-artifacts-ui.adoc[]
* xref:getting-started/assembly-using-kafka-client-serdes.adoc[]
* xref:getting-started/assembly-managing-registry-artifacts-api.adoc[]
* xref:getting-started/assembly-managing-registry-artifacts-maven.adoc[]
* xref:getting-started/assembly-using-the-registry-client.adoc[]
* xref:getting-started/assembly-using-kafka-client-serdes.adoc[]
* xref:getting-started/assembly-registry-reference.adoc[]
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@
include::{mod-loc}shared/all-attributes.adoc[]

[id="configuring-the-registry"]
= Configuring {registry} on OpenShift
= Configuring {registry} deployment on OpenShift

This chapter explains how to configure optional settings for {registry} health checks on OpenShift:

Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="installing-the-registry-docker"]
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,24 @@
// Metadata created by nebel
include::{mod-loc}shared/all-attributes.adoc[]

[id="installing-registry-ocp"]
= Installing {registry} on OpenShift

This chapter explains how to install {registry}:

* xref:installing-registry-operatorhub[]
//* xref:installing-registry-kafka-streams-template-storage[]

.Prerequisites
* {registry-overview}

NOTE: You can install more than one instance of {registry} depending on your environment. The number of instances depends on the number and type of artifacts stored in {registry} and on your chosen storage option.

ifdef::apicurio-registry[]
.Additional resources
* For details on building from source, see https://github.com/Apicurio/apicurio-registry.
endif::[]

//INCLUDES
//include::{mod-loc}getting-started/proc_installing-registry-kafka-streams-template-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-installing-registry-operatorhub.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,60 +1,49 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="installing-the-registry"]
= Installing {registry} on OpenShift

This chapter explains how to first install {registry} and then how to set up your chosen registry storage option: {kafka-streams}, embedded Infinispan, or PostgreSQL database.

.Prerequisites
* {registry-overview}
[id="installing-registry-storage"]
= Installing {registry} storage on OpenShift

.{registry} installation
* xref:installing-registry-operatorhub[]
//* xref:installing-registry-kafka-streams-template-storage[]
This chapter explains how to install and configure your chosen registry storage option: {kafka-streams}, embedded Infinispan, or PostgreSQL database.

.{kafka-streams} storage
* xref:installing-kafka-streams-operatorhub[]
* xref:installing-kafka-streams-operatorhub[]
* xref:setting-up-kafka-streams-storage[]
* xref:registry-kafka-topic-names[]

ifdef::apicurio-registry[]
.Embedded Infinispan storage
* xref:setting-up-infinispan-storage[]
.Embedded Infinispan storage
* xref:setting-up-infinispan-storage[]

.PostgreSQL database storage
* xref:installing-postgresql-operatorhub[]
* xref:setting-up-postgresql-storage[]
* xref:setting-up-postgresql-storage[]
endif::[]

ifdef::rh-service-registry[]
.Embedded Infinispan storage (Technology Preview)
* xref:setting-up-infinispan-storage[]
* xref:setting-up-infinispan-storage[]

.PostgreSQL database storage (Technology Preview)
* xref:installing-postgresql-operatorhub[]
* xref:setting-up-postgresql-storage[]
* xref:setting-up-postgresql-storage[]

[IMPORTANT]
====
{registry} storage in Infinispan or PostgreSQL is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production.
{registry} storage in Infinispan or PostgreSQL is a Technology Preview feature only. Technology Preview features are not supported with Red Hat production service level agreements (SLAs) and might not be functionally complete. Red Hat does not recommend using them in production.
These features provide early access to upcoming product features, enabling customers to test functionality and provide feedback during the development process. For more information about the support scope of Red Hat Technology Preview features, see https://access.redhat.com/support/offerings/techpreview.
====
endif::[]

NOTE: You can install more than one instance of {registry} depending on your environment. The number of instances depends on your storage option, for example, your Kafka, Infinispan, or database cluster configuration, and on the number and type of artifacts stored in {registry}.

ifdef::apicurio-registry[]
.Additional resources
* For details on building from source, see https://github.com/Apicurio/apicurio-registry.
endif::[]
.Prerequisites
* {installing-the-registry-openshift}

//INCLUDES
//include::{mod-loc}getting-started/proc_installing-registry-kafka-streams-template-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-installing-registry-operatorhub.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-installing-kafka-streams-operatorhub.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-setting-up-kafka-streams-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-kafka-topic-names.adoc[leveloffset=+2]
include::{mod-loc}getting-started/proc-setting-up-infinispan-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-installing-postgresql-operatorhub.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-setting-up-postgresql-storage.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,17 +1,16 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="intro-to-registry-rules"]
= {registry} content rules
//If the assembly covers a task, start the title with a verb in the gerund form, such as Creating or Configuring.

This chapter introduces the optional rules used to govern registry content and provides details on the available rule types:
This chapter introduces the optional rules used to govern registry content and provides details on the available rule configuration:

* xref:registry-rules[]
* xref:registry-rules-apply[]
* xref:registry-rules-work[]

* xref:registry-rules-work[]
* xref:registry-rules-config[]

//INCLUDES
include::{mod-loc}getting-started/con-registry-rules.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="intro-to-the-registry"]
Expand All @@ -14,6 +13,7 @@ This chapter introduces {registry} concepts and features and provides details on
* xref:registry-rest-api[]
* xref:registry-security[]
* xref:registry-storage[]
* xref:registry-web-console[]
* xref:client-serde[]
* xref:kafka-connect[]
* xref:registry-demo[]
Expand All @@ -26,6 +26,7 @@ include::{mod-loc}getting-started/con-registry-web-console.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-security.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-rest-api.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-storage.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-web-console.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-serde.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-kafka-connect-converters.adoc[leveloffset=+1]
include::{mod-loc}getting-started/con-registry-demo.adoc[leveloffset=+1]
Expand Down
Original file line number Diff line number Diff line change
@@ -1,18 +1,18 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="managing-registry-artifacts-api"]
= Managing {registry} content using the REST API
= Managing {registry} content using the REST API
//If the assembly covers a task, start the title with a verb in the gerund form, such as Creating or Configuring.

This chapter explains how to manage artifacts stored in the registry using the Registry REST API. This includes using Registry REST API commands, a Maven plug-in, or a Java client application:
This chapter describes the Registry REST API and shows how to use it manage artifacts stored in the registry:

* xref:registry-rest-api[]
* xref:managing-artifacts-using-rest-api[]
* xref:managing-artifacts-using-maven-plugin[]
* xref:managing-artifacts-using-client-code[]

.Additional resources
* link:{attachmentsdir}/registry-rest-api.htm[Apicurio Registry REST API documentation]

//INCLUDES
include::{mod-loc}getting-started/con-registry-rest-api.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-managing-artifacts-using-rest-api.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-managing-artifacts-using-maven-plugin.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-managing-artifacts-using-client-code.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
// Metadata created by nebel
include::{mod-loc}shared/all-attributes.adoc[]

[id="managing-registry-artifacts-maven"]
= Managing {registry} content using the Maven plug-in

This chapter explains how to manage artifacts stored in the registry using the {registry} Maven plug-in:

* xref:managing-artifacts-using-maven-plugin[]

.Prerequisites
* See {registry-overview}
* {registry} must be installed and running in your environment
* Maven must be installed and configured in your environment

//INCLUDES
include::{mod-loc}getting-started/proc-managing-artifacts-using-maven-plugin.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="managing-registry-artifacts-ui"]
Expand All @@ -8,11 +7,13 @@ include::{mod-loc}shared/all-attributes.adoc[]

This chapter explains how to manage artifacts stored in the registry using the {registry} web console. This includes uploading and browsing registry content, and configuring optional rules:

* xref:configuring-registry-ui[]
* xref:adding-artifacts-using-console[]
* xref:browsing-artifacts-using-console[]
* xref:configuring-rules-using-console[]

//INCLUDES
include::{mod-loc}getting-started/proc-configuring-registry-ui.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-adding-artifacts-using-console.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-browsing-artifacts-using-console.adoc[leveloffset=+1]
include::{mod-loc}getting-started/proc-configuring-rules-using-console.adoc[leveloffset=+1]
Original file line number Diff line number Diff line change
@@ -1,21 +1,25 @@
// Metadata created by nebel

include::{mod-loc}shared/all-attributes.adoc[]

[id="artifact-and-rule-types"]
= {registry} reference
[id="registry-artifact-reference"]
= {registry} artifact reference
//If the assembly covers a task, start the title with a verb in the gerund form, such as Creating or Configuring.

This chapter lists the supported artifact types and content rule types that are stored in {registry}.
This chapter provides details on the supported artifact types, states, metadata, and content rules that are stored in {registry}.

* xref:registry-artifact-types[]
* xref:registry-artifact-states[]
* xref:registry-artifact-metadata[]
* xref:registry-rule-types[]
* xref:registry-rule-maturity-matrix[]


.Additional resources
* For more detailed information on artifact and rule types, see the link:{attachmentsdir}/registry-rest-api.htm[Apicurio Registry REST API documentation]
* For more detailed information, see the link:{attachmentsdir}/registry-rest-api.htm[Apicurio Registry REST API documentation]

//INCLUDES
include::{mod-loc}getting-started/ref-registry-artifact-types.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-artifact-states.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-artifact-metadata.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-rule-types.adoc[leveloffset=+1]
include::{mod-loc}getting-started/ref-registry-rule-maturity-matrix.adoc[leveloffset=+1]
Loading

0 comments on commit 4bbcfac

Please sign in to comment.