Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Removing blob storage SDK leakage into the code base #9981

Draft
wants to merge 20 commits into
base: dev
Choose a base branch
from
Draft

Conversation

agr
Copy link
Contributor

@agr agr commented May 29, 2024

Addresses: https://github.com/NuGet/Engineering/issues/5456

We have wrappers around blob storage SDK (CloudBlobWrapper, CloudBlobContainerWrapper, CloudBlobClientWrapper), unfortunately, they leak underlying SDK types as function arguments, return types and exceptions thrown, which defeats the purpose and complicates swapping the underlying implementation.

This change aims to stop that leakage. Introduced more wrappers around concepts that we use (BlobListContinuationToken, CloudBlobCopyState, CloudBlobCopyStatus, CloudBlobLocationMode, CloudBlobPropertiesWrapper, ListingDetails), propagated other wrappers that we had, but didn't use everywhere (AccessConditionWrapper, FileUriPermissions), wrapped StorageException with our own exception hierarchy following the checks existing in the code, simplifying catch expressions.

@@ -2,7 +2,7 @@
<PropertyGroup>
<NuGetClientPackageVersion>6.9.1</NuGetClientPackageVersion>
<ServerCommonPackageVersion>2.120.0</ServerCommonPackageVersion>
<NuGetJobsPackageVersion>4.3.0-main-9408418</NuGetJobsPackageVersion>
<NuGetJobsPackageVersion>4.3.0-agr-azst-9643015</NuGetJobsPackageVersion>
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would have to be updated with a separate PR as NuGet.Jobs release depends on this PR.

Copy link
Contributor

@drewgillies drewgillies May 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems circular. Should we get away from this dependency? (sometime in the future)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sometimes this circular dependency makes hard to reason with runtime issue caused by dependency injection. Not sure exactly what was the actual dll consumed.

Copy link
Contributor Author

@agr agr May 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We have jobs in Gallery repo that use base classes from NuGet.Jobs.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I figured it would be something like that--I'll create a task for moving them, although that would also involve nugetizing some additional gallery dependencies to support them, I expect. DI, for one may make this a pipe dream.

drewgillies
drewgillies previously approved these changes May 29, 2024
Copy link
Contributor

@drewgillies drewgillies left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Excellent! :)

{
return await @delegate();
}
catch (StorageException ex) when (ex.RequestInformation?.ExtendedErrorInformation?.ErrorCode == BlobErrorCodeStrings.ContainerNotFound)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

To avoid code duplication, consider creating a single method to handle the exceptions, and call this method from both WrapStorageException and WrapStorageException.

Copy link
Contributor Author

@agr agr May 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you have a change in mind? I wasn't able to come up with a solution that would work for both cases.

lyndaidaii
lyndaidaii previously approved these changes May 29, 2024
@agr agr dismissed stale reviews from lyndaidaii and drewgillies via cdf1063 May 29, 2024 17:38
@erdembayar
Copy link
Contributor

Could please add tracking issue into desc?

@drewgillies
Copy link
Contributor

drewgillies commented May 29, 2024

Could please add tracking issue into desc?

@erdembayar Done.


namespace NuGetGallery
{
public enum CloudBlobLocationMode
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It looks this one is copy of https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.storage.retrypolicies.locationmode?view=azure-dotnet-legacy.
Should we add a comment above so the next person who is reading the code knows where those enum values are coming from?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants