Skip to content

Wraps the .NET SDK for Azure Cosmos DB abstracting away the complexity, exposing a simple CRUD-based repository pattern

License

Notifications You must be signed in to change notification settings

hesperanca/azure-cosmos-dotnet-repository

Β 
Β 

Repository files navigation

build CodeQL NuGet .NET code metrics Discord

All Contributors

Azure Cosmos DB Repository .NET SDK

This package wraps the NuGet: Microsoft.Azure.Cosmos package, exposing a simple dependency-injection enabled IRepository<T> interface.

Cosmos Repository

The repository is responsible for all of the create, read, update, and delete (CRUD) operations on objects where T : Item. The Item type adds several properties, one which is a globally unique identifier defined as:

[JsonProperty("id")]
public string Id { get; set; } = Guid.NewGuid().ToString();

Additionally, a type property exists which indicates the subclass name (this is used for filtering implicitly on your behalf):

[JsonProperty("type")]
public string Type { get; set; }

Finally, a partition key property is used internally to manage partitioning on your behalf. This can optionally be overridden on an item per item basis.

πŸ“£ Azure Cosmos DB - Official Blog

Getting started

  1. Create an Azure Cosmos DB SQL resource.

  2. Obtain the resource connection string from the Keys blade, be sure to get a connection string and not the key - these are different. The connection string is a compound key and endpoint URL.

  3. Call AddCosmosRepository:

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddCosmosRepository();
    }

    The optional setupAction allows consumers to manually configure the RepositoryOptions object:

    public void ConfigureServices(IServiceCollection services)
    {
        services.AddCosmosRepository(
            options =>
            {
                options.CosmosConnectionString = "< connection string >";
                options.ContainerId = "data-store";
                options.DatabaseId = "samples";
            });
    }
  4. Define your object graph, objects must inherit Item, for example:

    using Microsoft.Azure.CosmosRepository;
    
    public class Person : Item
    {
        public string FirstName { get; set; }
        public string LastName { get; set; }
    }
  5. Ask for an instance of IRepository<TItem>, in this case the TItem is Person:

    using Microsoft.Azure.CosmosRepository;
    
    public class Consumer
    {
        readonly IRepository<Person> _repository;
    
        public Consumer(IRepository<Person> repository) =>
            _repository = repository;
    
        // Use the repo...
    }
  6. Perform any of the operations on the _repository instance, create Person records, update them, read them, or delete.

  7. Enjoy!

Configuration

When OptimizeBandwidth is true (its default value), the repository SDK reduces networking and CPU load by not sending the resource back over the network and serializing it to the client. This is specific to writes, such as create, update, and delete. For more information, see Optimizing bandwidth in the Azure Cosmos DB .NET SDK.

There is much debate with how to structure your database and corresponding containers. Many developers with relational database design experience might prefer to have a single container per item type, while others understand that Azure Cosmos DB will handle things correctly regardless. By default, the ContainerPerItemType option is false and all items are persisted into the same container. However, when it is true, each distinct subclass of Item gets its own container named by the class itself.

Well-known keys

Depending on the .NET configuration provider your app is using, there are several well-known keys that map to the repository options that configure your usage of the repository SDK. When using environment variables, such as those in Azure App Service configuration or Azure Key Vault secrets, the following keys map to the RepositoryOptions instance:

Key Data type Default value
RepositoryOptions__CosmosConnectionString string null
RepositoryOptions__AccountEndpoint string null
RepositoryOptions__DatabaseId string "database"
RepositoryOptions__ContainerId string "container"
RepositoryOptions__OptimizeBandwidth boolean true
RepositoryOptions__ContainerPerItemType boolean false
RepositoryOptions__AllowBulkExecution boolean false
RepositoryOptionsSerializationOptionsIgnoreNullValues boolean false
RepositoryOptionsSerializationOptionsIndented boolean false
RepositoryOptionsSerializationOptionsPropertyNamingPolicy CosmosPropertyNamingPolicy CosmosPropertyNamingPolicy.CamelCase

Example appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "AllowedHosts": "*",
  "RepositoryOptions": {
    "CosmosConnectionString": "<Your-CosmosDB-ConnectionString>",
    "AccountEndpoint": "<Your-CosmosDB-URI>"
    "DatabaseId": "<Your-CosmosDB-DatabaseName>",
    "ContainerId": "<Your-CosmosDB-ContainerName>",
    "OptimizeBandwidth": true,
    "ContainerPerItemType": true,
    "AllowBulkExecution": true,
    "SerializationOptions": {
      "IgnoreNullValues": true,
     "PropertyNamingPolicy": "CamelCase"
    }
  }
}

For more information, see JSON configuration provider.

Example appsettings.json with Azure Functions

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  "AllowedHosts": "*",
  "Values": {
    "RepositoryOptions:CosmosConnectionString": "<Your-CosmosDB-ConnectionString>",
    "RepositoryOptions:AccountEndpoint": "<Your-CosmosDB-URI>",
    "RepositoryOptions:DatabaseId": "<Your-CosmosDB-DatabaseName>",
    "RepositoryOptions:ContainerId": "<Your-CosmosDB-ContainerName>",
    "RepositoryOptions:OptimizeBandwidth": true,
    "RepositoryOptions:ContainerPerItemType": true,
    "RepositoryOptions:AllowBulkExecution": true,
    "RepositoryOptions:SerializationOptions:IgnoreNullValues": true,
    "RepositoryOptions:SerializationOptions:PropertyNamingPolicy": "CamelCase"
  }
}

For more information, see Customizing configuration sources.

Authenticating using an identity

The Azure Cosmos DB .NET SDK also supports authentication using identities, which are considered superior from an audit and granularity of permissions perspective. Authenticating using a connection string essentially provides full access to perform operations within the data plane of your Cosmos DB Account. More information on the Azure control plane and data plane is available here.

This libary also supports authentication using an identity. To authenticate using an identity (User, Group, Application Registration, or Managed Identity) you will need to set the AccountEndpoint and TokenCredential options that are available on the RepositoryOptions class.

In a basic scenario, there are three steps that need to be completed:

  1. If the identity that you would like to use, does not exist in Azure Active Directory, create it now.

  2. Use the Azure CLI to assign the appropriate role to your identity at the desired scope. - In most cases, using the built-in roles will be sufficient. However, there is support for creating custom role definitions using the Azure CLI, you can read more on this here.

  3. Configure your application using the AddCosmosRepository method in your Startup.cs file:

    using Azure.Identity;
    
    public void ConfigureServices(IServiceCollection services)
    {
        DefaultAzureCredential credential = new();
    
        services.AddCosmosRepository(
            options =>
            {
                options.TokenCredential = credential;
                options.AccountEndpoint = "< account endpoint URI >";
                options.ContainerId = "data-store";
                options.DatabaseId = "samples";
            });
    }

The example above is using the DefaultAzureCredential object provided by the Azure Identity NuGet package, which provides seamless integration with Azure Active Directory. More information on this package is available here.

Advanced partitioning strategy

As a consumer of Azure Cosmos DB, you can choose how to partition your data. By default, this repository SDK will partition items using their Item.Id value as the /id partition in the storage container. However, you can override this default behavior by:

  1. Declaratively specifying the partition key path with PartitionKeyPathAttribute
  2. Override the Item.GetPartitionKeyValue() method
  3. Ensure the the property value of the composite or synthetic key is serialized to match the partition key path
  4. Set RepositoryOptions__ContainerPerItemType to true, to ensure that your item with explicit partitioning is correctly maintained

As an example, considering the following:

using Microsoft.Azure.CosmosRepository;
using Microsoft.Azure.CosmosRepository.Attributes;
using Newtonsoft.Json;
using System;

namespace Example
{
    [PartitionKeyPath("/synthetic")]
    public class Person : Item
    {
        public string FirstName { get; set; } = null!;
        public string? MiddleName { get; set; }
        public string LastName { get; set; } = null!;

        [JsonProperty("synthetic")]
        public string SyntheticPartitionKey =>
            $"{FirstName}-{LastName}"; // Also known as a "composite key".

        protected override string GetPartitionKeyValue() => SyntheticPartitionKey;
    }
}

In-memory Repository

This library also includes an in-memory version of IRepository<T>. To use it swap out the normal services.AddCosmosRepository() for services.AddInMemoryCosmosRepository() and have all of your items stored in memory. This is a great tool for running integration tests using a package such as Microsoft.AspNetCore.Mvc.Testing, and not having to incur the cost of data stored in an Azure Cosmos DB resource.

Optimistic Concurrency Control with Etags

The default repository now supports etags and will pass them when IItemWithEtag is implemented correctly or the base classes EtagItem or FullItem are used. The etag check is enforced on all updates when TItem is of the correct type. It can however be bypassed by setting the ignoreEtag optional parameter in the relevant async methods. The InMemory repository also supports OCC with Etags. The OptimisticCurrencyControl sample shows these features.

Getting the new etag

When creating a new object, if storing in memory, it is important to store the result from the create call to ensure you have the correct etag for future updated.

For example your code should look something like this:

TItem currentItem = new TItem(...);
currentItem = _repository.CreateAsync(currentItem);

Sequential updates

When doing sequential updates to the same item it is important to use the result from the update method (when OptimizeBandwith is false) or refetch the updated data each time (when OptimizeBandwith is true) otherwise the etag value will not be updated. The following code shows what to do in each case:

Optimize Bandwith Off

TItem currentItem = _repository.CreateAsync(itemConfig);
currentItem = _repository.UpdateAsync(currentItem);
currentItem = _repository.UpdateAsync(currentItem);

Optimize Bandwith On

TItem currentItem = _repository.CreateAsync(itemConfig);
_repository.UpdateAsync(currentItem);
currentItem = _repository.GetAsync(currentItem.Id);
_repository.UpdateAsync(currentItem);
currentItem = _repository.GetAsync(currentItem);
currentItem = _repository.UpdateAsync(currentItem);

Catching mismatched etag errors

The following code shows how to catch the error when the etags do not match.

  try
  {
      currentBankAccount = await repository.UpdateAsync(currentBankAccount);
      Console.WriteLine($"Updated bank account: {currentBankAccount}.");
  }
  catch (CosmosException exception) when (exception.StatusCode == HttpStatusCode.PreconditionFailed)
  {
      Console.WriteLine("Failed to update balance as the etags did not match.");
  }

Ignoring the etag

The following code shows how to ignore the etag when doing an update.

await repository.UpdateAsync(currentBankAccount, ignoreEtag: true);

Passing the etag to a patch update

The following code shows how to pass the etag when doing a update to specific properties.

await repository.UpdateAsync(currentBankAccount.Id,
  builder => builder.Replace(account => account.Balance, currentBankAccount.Balance - 250), etag: currentBankAccount.Etag);

Time To Live

The time to live property can be set at both an item and container level. At a container level this can be done through the container options builder:

options.ContainerBuilder.Configure<BankAccount>(
  x => x.WithContainerDefaultTimeToLive(TimeSpan.FromHours(2)));

In the above example the container would have a default item lifespan of 2 hours. This can be overriden at the item level by using the TimeToLive property when correctly implemented. This is available through the FullItem and TimeToLiveItem base classes. The example below shows this been overriden so the item has a lifespan of 4 hours rather than the default of 2:

BankAccount currentBankAccount = await repository.CreateAsync(
  new BankAccount()
    {
        Name = "Current Account",
        Balance = 500.0,
        TimeToLive = TimeSpan.FromHours(4)
    });

If you didn't want that specific item to ever expire the following code can be used:

BankAccount currentBankAccount = await repository.CreateAsync(
  new BankAccount()
    {
        Name = "Current Account",
        Balance = 500.0,
        TimeToLive = TimeSpan.FromSeconds(-1)
    });

The demo BankAccount class can be found in the OptimisticCurrencyControl sample and its implementation looks like the following:

using Microsoft.Azure.CosmosRepository;
using Microsoft.Azure.CosmosRepository.Attributes;

namespace OptimisticConcurrencyControl;

[Container("accounts")]
[PartitionKeyPath("/id")]
public class BankAccount : FullItem
{
    public string Name { get; set; } = string.Empty;
    public double Balance { get; set; }

    public void Withdraw(double amount)
    {
        if (Balance - amount < 0.0) throw new InvalidOperationException("Cannot go overdrawn");

        Balance -= amount;
    }

    public void Deposit(double amount)
    {
        Balance += amount;
    }

    public override string ToString() =>
        $"Account (Name = {Name}, Balance = {Balance}, Etag = {Etag})";
}

This page goes into more detail about the various combinations.

Created and Last Updated

The last updated value is retrieved from the _ts property that Cosmos DB sets; as documented here. This property is deserialised and is available in the raw seconds (LastUpdatedTimeRaw) since epoch and a human readable format (LastUpdatedTimeUtc). Both the base classes FullItem and TimeStampedItem contain these properties.

The CreatedTimeUtc time property available in both the base classes FullItem and TimeStampedItem is set when CreateAsync is called on the repository. However, this property can be set prior to calling CreateAsync in which case it wont be overwritten; allowing you to set your own CreatedTimeUtc value. This does mean that when using existing date the CreatedTimeUtc property will be null.

Samples

Visit the Microsoft.Azure.CosmosRepository.Samples directory for samples on how to use the library with:

Deep-dive video

A deep dive into the Azure Cosmos DB repository pattern NET SDK

Discord

Get extra support on our dedicated Discord channel.

alt Join the conversation

Contributors ✨

Thanks goes to these wonderful people (emoji key):


David Pine

πŸ’» ⚠️ πŸ’‘ πŸ‘€

Invvard

⚠️ πŸ’»

Richard Mercer

πŸ’»

Daniel Marbach

πŸ’»

Manuel Sidler

πŸ’»

Dave Brock

πŸ“– πŸ’»

Cagdas Erman Afacan

πŸ’» πŸ’‘

dcuccia

πŸ’»

VeryCautious

πŸ’» ⚠️

Billy Mumby

πŸ’» πŸ“– πŸ€”

Michael Zhang

πŸ€” πŸ’»

Shay Rojansky

πŸ‘€

Junior Macedo

πŸ’» πŸ€”

Emre KARA

πŸ’»

Brad Westness

πŸ‘€

Matt Stannett

πŸ’» πŸ“– ⚠️

mustafarabie

πŸ’» ⚠️

Robert Bennett

⚠️ πŸ’»

Rabosa616

πŸ’» ⚠️ πŸ“–

Adam Storr

πŸ’» ⚠️

Kevin Benton

πŸ’» ⚠️

Filip Persson

πŸ’» ⚠️

This project follows the all-contributors specification. Contributions of any kind welcome!

About

Wraps the .NET SDK for Azure Cosmos DB abstracting away the complexity, exposing a simple CRUD-based repository pattern

Resources

License

Code of conduct

Stars

Watchers

Forks

Packages

No packages published

Languages

  • C# 100.0%