Skip to content

Commit

Permalink
[Schema Registry Avro] Rename Encoder to Serializer (Azure#20731)
Browse files Browse the repository at this point in the history
  • Loading branch information
deyaaeldeen authored and WeiJun428 committed Mar 19, 2022
1 parent 97ac513 commit a426fa2
Show file tree
Hide file tree
Showing 14 changed files with 200 additions and 193 deletions.
7 changes: 7 additions & 0 deletions sdk/schemaregistry/schema-registry-avro/CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,12 @@
# Release History

## 1.0.0-beta.7 (Unreleased)

### Breaking Changes
- `AvroEncoder` has been renamed to `AvroSerializer`
- The `encodeMessageData` method has been renamed to `serializeMessageData`
- The `decodeMessageData` method has been renamed to `deserializeMessageData`

## 1.0.0-beta.6 (2022-02-10)

### Features Added
Expand Down
42 changes: 21 additions & 21 deletions sdk/schemaregistry/schema-registry-avro/README.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
# Azure Schema Registry Avro Encoder client library for JavaScript
# Azure Schema Registry Avro Serializer client library for JavaScript

Azure Schema Registry is a schema repository service hosted by Azure Event Hubs,
providing schema storage, versioning, and management. This package provides an
Avro encoder capable of encoding and decoding payloads containing
Avro-encoded data.
Avro serializer capable of serializing and deserializing payloads containing
Avro-serialized data.

Key links:

Expand Down Expand Up @@ -31,58 +31,58 @@ npm install @azure/schema-registry-avro

## Key concepts

### AvroEncoder
### AvroSerializer

Provides API to encode to and decode from Avro Binary Encoding wrapped in a message
Provides API to serialize to and deserialize from Avro Binary Encoding wrapped in a message
with a content type field containing the schema ID. Uses
`SchemaRegistryClient` from the [@azure/schema-registry](https://www.npmjs.com/package/@azure/schema-registry) package
to get schema IDs from schema definition or vice versa. The provided API has internal cache to avoid calling the schema registry service when possible.

### Messages

By default, the encoder will create messages structured as follows:
By default, the serializer will create messages structured as follows:

- `body`: a byte array containing data in the Avro Binary Encoding. Note that it
is NOT Avro Object Container File. The latter includes the schema and creating
it defeats the purpose of using this encoder to move the schema out of the
it defeats the purpose of using this serializer to move the schema out of the
message payload and into the schema registry.

- `contentType`: a string of the following format `avro/binary+<Schema ID>` where
the `avro/binary` part signals that this message has an Avro-encoded payload
the `avro/binary` part signals that this message has an Avro-serialized payload
and the `<Schema Id>` part is the Schema ID the Schema Registry service assigned
to the schema used to encode this payload.
to the schema used to serialize this payload.

Not all messaging services are supporting the same message structure. To enable
integration with such services, the encoder can act on custom message structures
integration with such services, the serializer can act on custom message structures
by setting the `messageAdapter` option in the constructor with a corresponding
message producer and consumer. Azure messaging client libraries export default
adapters for their message types.

### Backward Compatibility

The encoder v1.0.0-beta.5 and under encodes data into binary arrays. Starting from
v1.0.0-beta.6, the encoder returns messages instead that contain the encoded payload.
For a smooth transition to using the newer versions, the encoder also supports
decoding messages with payloads that follow the old format where the schema ID
The serializer v1.0.0-beta.5 and under serializes data into binary arrays. Starting from
v1.0.0-beta.6, the serializer returns messages instead that contain the serialized payload.
For a smooth transition to using the newer versions, the serializer also supports
deserializing messages with payloads that follow the old format where the schema ID
is part of the payload.

This backward compatibility is temporary and will be removed in v1.0.0.

## Examples

### Encode and decode an `@azure/event-hubs`'s `EventData`
### Serialize and deserialize an `@azure/event-hubs`'s `EventData`

```javascript
const { DefaultAzureCredential } = require("@azure/identity");
import { createEventDataAdapter } from "@azure/event-hubs";
const { SchemaRegistryClient } = require("@azure/schema-registry");
const { AvroEncoder } = require("@azure/schema-registry-avro");
const { AvroSerializer } = require("@azure/schema-registry-avro");

const client = new SchemaRegistryClient(
"<fully qualified namespace>",
new DefaultAzureCredential()
);
const encoder = new AvroEncoder(client, {
const serializer = new AvroSerializer(client, {
groupName: "<group>",
messageAdapter: createEventDataAdapter(),
});
Expand All @@ -98,11 +98,11 @@ const schema = JSON.stringify({
// Example value that matches the Avro schema above
const value = { score: 42 };

// Encode value to a message
const message = await encoder.encodeMessageData(value, schema);
// Serialize value to a message
const message = await serializer.serializeMessageData(value, schema);

// Decode a message to value
const decodedValue = await encoder.decodeMessageData(message);
// Deserialize a message to value
const deserializedValue = await serializer.deserializeMessageData(message);
```

## Troubleshooting
Expand Down
2 changes: 1 addition & 1 deletion sdk/schemaregistry/schema-registry-avro/package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "@azure/schema-registry-avro",
"version": "1.0.0-beta.6",
"version": "1.0.0-beta.7",
"description": "Schema Registry Avro Serializer Library with typescript type definitions for node.js and browser.",
"sdk-type": "client",
"main": "dist/index.js",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,21 +7,21 @@
import { SchemaRegistry } from '@azure/schema-registry';

// @public
export class AvroEncoder<MessageT = MessageWithMetadata> {
constructor(client: SchemaRegistry, options?: AvroEncoderOptions<MessageT>);
decodeMessageData(message: MessageT, options?: DecodeMessageDataOptions): Promise<unknown>;
encodeMessageData(value: unknown, schema: string): Promise<MessageT>;
export class AvroSerializer<MessageT = MessageWithMetadata> {
constructor(client: SchemaRegistry, options?: AvroSerializerOptions<MessageT>);
deserializeMessageData(message: MessageT, options?: DeserializeMessageDataOptions): Promise<unknown>;
serializeMessageData(value: unknown, schema: string): Promise<MessageT>;
}

// @public
export interface AvroEncoderOptions<MessageT> {
export interface AvroSerializerOptions<MessageT> {
autoRegisterSchemas?: boolean;
groupName?: string;
messageAdapter?: MessageAdapter<MessageT>;
}

// @public
export interface DecodeMessageDataOptions {
export interface DeserializeMessageDataOptions {
schema?: string;
}

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
// Licensed under the MIT License.

/**
* @summary Demonstrates the use of SchemaRegistryAvroEncoder to create messages with avro-encoded payload using schema from Schema Registry.
* @summary Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry.
*/

import { DefaultAzureCredential } from "@azure/identity";
import { SchemaRegistryClient, SchemaDescription } from "@azure/schema-registry";
import { AvroEncoder } from "@azure/schema-registry-avro";
import { AvroSerializer } from "@azure/schema-registry-avro";

// Load the .env file if it exists
import * as dotenv from "dotenv";
Expand Down Expand Up @@ -61,23 +61,23 @@ export async function main() {
);

// Register the schema. This would generally have been done somewhere else.
// You can also skip this step and let `encodeMessageData` automatically register
// You can also skip this step and let `serializeMessageData` automatically register
// schemas using autoRegisterSchemas=true, but that is NOT recommended in production.
await client.registerSchema(schemaDescription);

// Create a new encoder backed by the client
const encoder = new AvroEncoder(client, { groupName });
// Create a new serializer backed by the client
const serializer = new AvroSerializer(client, { groupName });

// encode an object that matches the schema and put it in a message
// serialize an object that matches the schema and put it in a message
const value: User = { firstName: "Jane", lastName: "Doe" };
const message = await encoder.encodeMessageData(value, schema);
const message = await serializer.serializeMessageData(value, schema);
console.log("Created message:");
console.log(JSON.stringify(message));

// decode the message back to an object
const decodedObject = await encoder.decodeMessageData(message);
console.log("Decoded object:");
console.log(JSON.stringify(decodedObject as User));
// deserialize the message back to an object
const deserializedObject = await serializer.deserializeMessageData(message);
console.log("Deserialized object:");
console.log(JSON.stringify(deserializedObject as User));
}

main().catch((err) => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
// Licensed under the MIT License.

/**
* @summary Demonstrates the use of SchemaRegistryAvroEncoder to create messages with avro-encoded payload using schema from Schema Registry and send them to an Event Hub using the EventHub Buffered Producer Client.
* @summary Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry and send them to an Event Hub using the EventHub Buffered Producer Client.
*/

import { DefaultAzureCredential } from "@azure/identity";
import { SchemaRegistryClient, SchemaDescription } from "@azure/schema-registry";
import { AvroEncoder } from "@azure/schema-registry-avro";
import { AvroSerializer } from "@azure/schema-registry-avro";
import { EventHubBufferedProducerClient, createEventDataAdapter } from "@azure/event-hubs";

// Load the .env file if it exists
Expand Down Expand Up @@ -69,12 +69,12 @@ export async function main() {
);

// Register the schema. This would generally have been done somewhere else.
// You can also skip this step and let `encodeMessageData` automatically register
// You can also skip this step and let `serializeMessageData` automatically register
// schemas using autoRegisterSchemas=true, but that is NOT recommended in production.
await schemaRegistryClient.registerSchema(schemaDescription);

// Create a new encoder backed by the client
const encoder = new AvroEncoder(schemaRegistryClient, {
// Create a new serializer backed by the client
const serializer = new AvroSerializer(schemaRegistryClient, {
groupName,
messageAdapter: createEventDataAdapter(),
});
Expand All @@ -86,9 +86,9 @@ export async function main() {
}
);

// encode an object that matches the schema
// serialize an object that matches the schema
const value: User = { firstName: "Jane", lastName: "Doe" };
const message = await encoder.encodeMessageData(value, schema);
const message = await serializer.serializeMessageData(value, schema);
console.log("Created message:");
console.log(message);

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
// Licensed under the MIT License.

/**
* @summary Demonstrates the use of SchemaRegistryAvroEncoder to decode messages with avro-encoded payload received from the Event Hub Consumer Client.
* @summary Demonstrates the use of AvroSerializer to deserialize messages with avro-serialized payload received from the Event Hub Consumer Client.
*/

import { DefaultAzureCredential } from "@azure/identity";
import { SchemaRegistryClient, SchemaDescription } from "@azure/schema-registry";
import { AvroEncoder } from "@azure/schema-registry-avro";
import { AvroSerializer } from "@azure/schema-registry-avro";
import {
EventHubConsumerClient,
earliestEventPosition,
Expand Down Expand Up @@ -69,12 +69,12 @@ export async function main() {
);

// Register the schema. This would generally have been done somewhere else.
// You can also skip this step and let `encodeMessageData` automatically register
// You can also skip this step and let `serializeMessageData` automatically register
// schemas using autoRegisterSchemas=true, but that is NOT recommended in production.
await schemaRegistryClient.registerSchema(schemaDescription);

// Create a new encoder backed by the client
const encoder = new AvroEncoder(schemaRegistryClient, {
// Create a new serializer backed by the client
const serializer = new AvroSerializer(schemaRegistryClient, {
groupName,
messageAdapter: createEventDataAdapter(),
});
Expand Down Expand Up @@ -104,8 +104,8 @@ export async function main() {
if (event.contentType !== undefined && event.body) {
const contentTypeParts = event.contentType.split("+");
if (contentTypeParts[0] === "avro/binary") {
const decodedEvent = await encoder.decodeMessageData(event);
console.log(`Decoded message: '${JSON.stringify(decodedEvent)}'`);
const deserializedEvent = await serializer.deserializeMessageData(event);
console.log(`Deserialized message: '${JSON.stringify(deserializedEvent)}'`);
}
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@
// Licensed under the MIT License.

/**
* @summary Demonstrates the use of SchemaRegistryAvroEncoder to create messages with avro-encoded payload using schema from Schema Registry and send them to an Event Hub using the EventHub Producer Client.
* @summary Demonstrates the use of AvroSerializer to create messages with avro-serialized payload using schema from Schema Registry and send them to an Event Hub using the EventHub Producer Client.
*/

import { DefaultAzureCredential } from "@azure/identity";
import { SchemaRegistryClient, SchemaDescription } from "@azure/schema-registry";
import { AvroEncoder } from "@azure/schema-registry-avro";
import { AvroSerializer } from "@azure/schema-registry-avro";
import { EventHubProducerClient, createEventDataAdapter } from "@azure/event-hubs";

// Load the .env file if it exists
Expand Down Expand Up @@ -68,12 +68,12 @@ export async function main() {
);

// Register the schema. This would generally have been done somewhere else.
// You can also skip this step and let `encodeMessageData` automatically register
// You can also skip this step and let `serializeMessageData` automatically register
// schemas using autoRegisterSchemas=true, but that is NOT recommended in production.
await schemaRegistryClient.registerSchema(schemaDescription);

// Create a new encoder backed by the client
const encoder = new AvroEncoder(schemaRegistryClient, {
// Create a new serializer backed by the client
const serializer = new AvroSerializer(schemaRegistryClient, {
groupName,
messageAdapter: createEventDataAdapter(),
});
Expand All @@ -83,9 +83,9 @@ export async function main() {
eventHubName
);

// encode an object that matches the schema
// serialize an object that matches the schema
const value: User = { firstName: "Joe", lastName: "Doe" };
const message = await encoder.encodeMessageData(value, schema);
const message = await serializer.serializeMessageData(value, schema);
console.log("Created message:");
console.log(message);

Expand Down
Loading

0 comments on commit a426fa2

Please sign in to comment.