Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Schema Registry: Change to use BinaryContent and change Encoder to Serializer #27598

Closed
wants to merge 16 commits into from

Conversation

conniey
Copy link
Member

@conniey conniey commented Mar 10, 2022

Description

Based on conversations with architects, this updates:

  • Updates MessageWithMetadata to BinaryContent
  • Changes encode/decode Message to serialize/deserialize Message

PRs from other languages:

If an SDK is being regenerated based on a new swagger spec, a link to the pull request containing these swagger spec changes has been included above.

All SDK Contribution checklist:

  • The pull request does not introduce [breaking changes]
  • CHANGELOG is updated for new features, bug fixes or other significant changes.
  • I have read the contribution guidelines.

General Guidelines and Best Practices

  • Title of the pull request is clear and informative.
  • There are a small number of commits, each of which have an informative message. This means that previously merged commits do not appear in the history of the PR. For more information on cleaning up the commits in your PR, see this page.

Testing Guidelines

  • Pull request includes test coverage for the included changes.

@ghost ghost added Azure.Core azure-core Schema Registry labels Mar 10, 2022
@conniey conniey self-assigned this Mar 10, 2022
@conniey conniey changed the title Schemaregistry/binary content Schema Registry: Change to use BinaryContent and change Encoder to Serializer Mar 10, 2022
@azure-sdk
Copy link
Collaborator

API changes have been detected in com.azure:azure-core-experimental. You can review API changes here

@azure-sdk
Copy link
Collaborator

API changes have been detected in com.azure:azure-data-schemaregistry-apacheavro. You can review API changes here

@conniey conniey force-pushed the schemaregistry/binary-content branch from 9a00ddd to 6091b10 Compare March 10, 2022 23:05
@@ -10,7 +10,7 @@
* An abstraction for a message containing a content type along with its data.
*/
@Fluent
public class MessageWithMetadata {
public class BinaryContent {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We would now have BinaryData and BinaryContent as public types and another implementation type i.e. BinaryDataContent. This is likely going to confuse us in future :)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is going to be confusing. BinaryData and BinaryContent are very similar.

Copy link
Member Author

@conniey conniey Mar 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was based on architect feedback. The other languages also have moved their names to this and also have BinaryData. I can add you to the thread if you want.

https://github.com/Azure/azure-sdk-for-net/blob/main/sdk/core/Azure.Core.Experimental/api/Azure.Core.Experimental.netstandard2.0.cs#L9

Copy link
Member

@JoshLove-msft JoshLove-msft Mar 11, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The idea is that BinaryContent contains BinaryData along with the content type. We didn't think it would be confusing as they are two related and similarly named types, rather than being just two similarly named types.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/cc @tg-msft

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@pallavit, @srnagar, could you please elaborate what's confusing here and what concrete implications of this confusion you see? There are many types in all platforms that are similar, yet they are not considered confusing, e.g. Int32 and Int64 are super similar.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I set-up a quick chat on Monday. Might be easier to get all the thoughts in the same room.

Copy link
Member

@srnagar srnagar Mar 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Java already has BinaryDataContent which is wrapped by BinaryData but that's currently internal. Also, when deciding which type to use, would there be a scenario where we want to use BinaryData and not BinaryContent given that content type in BinaryContent is optional?

One option was to move content type property to BinaryData but this may not be possible as we want to make this the super type for Event Hubs and Service Bus event and message types. If this type is intended for extension in Event Hubs and Service Bus, should we make this type abstract?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was initially abstract, but we made it concrete to make it easier for use with Schema Registry that didn't involve Service Bus/Event Hubs. However, I think we can still make this scenario easy by having a private implementation that we return if the user doesn't specify a more specific BinaryContent type. This would allow us to move the type back to being abstract which I believe would address your concerns around any confusion that might arise from how this type is meant to be used.

Copy link
Member

@JoshLove-msft JoshLove-msft Mar 12, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here is what this change would look like in .NET - Azure/azure-sdk-for-net#27525

After making the class abstract, I needed to add two additional generic overloads that only took the TData instead of both the TEnvelope and TData. This was necessary in order to leave in our generic type constraint that TEnvelope have an empty public constructor. When using the new overloads we would construct the internal DefaultBinaryContent and return that to the user. The new overloads make the sample code simpler since now the generic arguments can be completely inferred based on the input data.

@@ -10,7 +10,7 @@
* An abstraction for a message containing a content type along with its data.
*/
@Fluent
public class MessageWithMetadata {
public class BinaryContent {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this is going to be confusing. BinaryData and BinaryContent are very similar.

@azure-sdk
Copy link
Collaborator

API changes have been detected in com.azure:azure-data-schemaregistry-apacheavro. You can review API changes here

@conniey conniey closed this Mar 22, 2022
@conniey conniey deleted the schemaregistry/binary-content branch March 22, 2022 22:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants