Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cosmos triggered Functions #237

Merged
merged 11 commits into from
May 14, 2019
2 changes: 1 addition & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Improvements

- Serverless mixins now support the Event Hubs trigger.
- Serverless mixins now support the Event Hubs and Cosmos DB triggers.

## 0.18.2 (Released May 6th, 2019)

Expand Down
3 changes: 3 additions & 0 deletions examples/cosmosdb/Pulumi.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
name: azure-cosmosdb-changefeed
runtime: nodejs
description: Azure Function subscribed to Cosmos DB Change Feed
29 changes: 29 additions & 0 deletions examples/cosmosdb/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
// Copyright 2016-2019, Pulumi Corporation. All rights reserved.

import * as azure from "@pulumi/azure";
import * as cosmosdb from "@pulumi/azure/cosmosdb";

const location = "West US 2";

const resourceGroup = new azure.core.ResourceGroup("test", {
location: location,
});

let cosmosdb = new azure.cosmosdb.Account("test", {
resourceGroupName: resourceGroup.name,
offerType: "Standard",
consistencyPolicy: {
consistencyLevel: "Session",
maxIntervalInSeconds: 5,
maxStalenessPrefix: 100,
},
});

cosmosdb.onChange("test", {
databaseName: "testdb",
collectionName: "testc",
callback: async (context, items) => {
console.log("ctx: " + JSON.stringify(context, null, 4));
console.log(`${items.length} received`);
}
});
21 changes: 21 additions & 0 deletions examples/cosmosdb/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"name": "azure-cosmosdb-change-feed",
"version": "0.1.0",
"main": "bin/index.js",
"typings": "bin/index.d.ts",
"scripts": {
"build": "tsc"
},
"dependencies": {
"@pulumi/pulumi": "^0.17.8",
"mime-types": "^2.1.19"
},
"devDependencies": {
"@types/node": "^10.0.0",
"typescript": "^3.4.5"
},
"peerDependencies": {
"@pulumi/azure": "latest"
},
"license": "Apache 2.0"
}
19 changes: 19 additions & 0 deletions examples/cosmosdb/tsconfig.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"compilerOptions": {
"outDir": "bin",
"target": "es6",
"module": "commonjs",
"moduleResolution": "node",
"sourceMap": true,
"experimentalDecorators": true,
"pretty": true,
"noFallthroughCasesInSwitch": true,
"noImplicitAny": true,
"noImplicitReturns": true,
"forceConsistentCasingInFileNames": true,
"strictNullChecks": true
},
"files": [
"index.ts"
]
}
5 changes: 5 additions & 0 deletions resources.go
Original file line number Diff line number Diff line change
Expand Up @@ -890,6 +890,11 @@ func Provider() tfbridge.ProviderInfo {
"zMixins_http.ts",
},
},
"cosmosdb": {
DestFiles: []string{
"zMixins.ts",
},
},
"eventhub": {
DestFiles: []string{
"zMixins.ts",
Expand Down
1 change: 1 addition & 0 deletions sdk/nodejs/cosmosdb/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,4 @@
// Export members:
export * from "./account";
export * from "./getAccount";
export * from "./zMixins";
203 changes: 203 additions & 0 deletions sdk/nodejs/cosmosdb/zMixins.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,203 @@
// Copyright 2016-2018, Pulumi Corporation.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

import * as pulumi from "@pulumi/pulumi";
import { Account } from "./account";

import * as appservice from "../appservice";
import * as core from "../core";
import * as util from "../util";

interface CosmosBindingDefinition extends appservice.BindingDefinition {
/**
* The name of the property in the context object to bind the actual items to.
*/
name: string;

/**
* The type of a Cosmos DB binding. Must be 'cosmosDBTrigger'.
*/
type: "cosmosDBTrigger";

/**
* The direction of the binding. We only support collection items being inputs to functions.
*/
direction: "in";

/**
* The name of the database we are subscribing to.
*/
databaseName: pulumi.Input<string>;

/**
* The name of the collection inside the database we are subscribing to.
*/
collectionName: pulumi.Input<string>;

/**
* When set, it adds a prefix to the leases created in the Lease collection for this Function, effectively allowing
* two separate Azure Functions to share the same Lease collection by using different prefixes.
*/
leaseCollectionPrefix: pulumi.Input<string>;

/**
* When set to true, the leases collection is automatically created when it doesn't already exist.
* The default value is false.
mikhailshilkov marked this conversation as resolved.
Show resolved Hide resolved
*/
createLeaseCollectionIfNotExists: boolean;

/**
* The name of an app setting that contains the Cosmos DB connection string to use for this binding.
*/
connectionStringSetting: string;

/**
* When set, it customizes the maximum amount of items received per Function call.
*/
maxItemsPerInvocation?: pulumi.Input<number>;

/**
* When set, it tells the Trigger to start reading changes from the beginning of the history of the collection instead of the current time.
* This only works the first time the Trigger starts, as in subsequent runs, the checkpoints are already stored. Setting this to true when
* there are leases already created has no effect.
*/
startFromBeginning?: pulumi.Input<boolean>;
}

/**
* Data that will be passed along in the context object to the CosmosCallback.
*/
export interface CosmosChangeFeedContext extends appservice.Context<void> {
invocationId: string;
executionContext: {
invocationId: string;
functionName: string;
functionDirectory: string;
};
bindings: { items: any[] };
bindingData: {
sys: {
methodName: string;
utcNow: string;
},
invocationId: string;
};
}

/**
* Signature of the callback that can receive Cosmos Change Feed notifications.
*/
export type CosmosChangeFeedCallback = appservice.Callback<CosmosChangeFeedContext, any[], void>;

export type CosmosChangeFeedSubscriptionArgs = util.Overwrite<appservice.CallbackFunctionAppArgs<CosmosChangeFeedContext, any[], void>, {
/**
* The name of the database we are subscribing to.
*/
databaseName: pulumi.Input<string>;

/**
* The name of the collection inside the database we are subscribing to.
*/
collectionName: pulumi.Input<string>;

/**
* When set, it customizes the maximum amount of items received per Function call.
*/
maxItemsPerInvocation?: pulumi.Input<number>;

/**
* When set, it tells the Trigger to start reading changes from the beginning of the history of the collection instead of the current time.
* This only works the first time the Trigger starts, as in subsequent runs, the checkpoints are already stored. Setting this to true when
* there are leases already created has no effect.
*/
startFromBeginning?: pulumi.Input<boolean>;

/**
* The resource group in which to create the event subscription. If not supplied, the Topic's
* resource group will be used.
*/
resourceGroup?: core.ResourceGroup;

/**
* The name of the resource group in which to create the event subscription. If not supplied,
* the Topic's resource group will be used.
*/
resourceGroupName?: pulumi.Input<string>;

/**
* Specifies the supported Azure location where the resource exists. Changing this forces a new
* resource to be created. If not supplied, the location of the Topic's ResourceGroup will be
* used.
*/
location?: pulumi.Input<string>;
}>;

declare module "./account" {
interface Account {
/**
* Creates a new subscription to events fired from Cosmos DB Change Feed to the handler provided, along
* with options to control the behavior of the subscription.
*/
onChange(
name: string, args: CosmosChangeFeedSubscriptionArgs, opts?: pulumi.ComponentResourceOptions): CosmosChangeFeedSubscription;
}
}

Account.prototype.onChange = function(this: Account, name, args, opts) {
return new CosmosChangeFeedSubscription(name, this, args, opts);
}

export class CosmosChangeFeedSubscription extends appservice.EventSubscription<CosmosChangeFeedContext, any[], void> {
readonly account: Account;

constructor(
name: string, account: Account,
args: CosmosChangeFeedSubscriptionArgs, opts: pulumi.ComponentResourceOptions = {}) {

opts = { parent: account, ...opts };

const { resourceGroupName, location } = appservice.getResourceGroupNameAndLocation(args, account.resourceGroupName);

const bindingConnectionKey = "BindingConnectionAppSettingsKey";

const bindings: CosmosBindingDefinition[] = [{
name: "items",
direction: "in",
type: "cosmosDBTrigger",
leaseCollectionPrefix: name,
createLeaseCollectionIfNotExists: true,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be user configurable?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to expose all properties as configurable? (there are more of them) Or take some defaults for now, which make sense in 95% of cases? (my call)

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we can totally not expose if we think this is reasonable. users can always request in the future. though we should likely doc why we think this is the right default.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added a comment to cover both.

connectionStringSetting: bindingConnectionKey,
databaseName: args.databaseName,
collectionName: args.collectionName,
maxItemsPerInvocation: args.maxItemsPerInvocation,
startFromBeginning: args.startFromBeginning,
}];

// Place the mapping from the well known key name to the Cosmos DB connection string in
// the 'app settings' object.

const appSettings = pulumi.all([args.appSettings, account.connectionStrings]).apply(
([appSettings, connectionStrings]) => ({ ...appSettings, [bindingConnectionKey]: connectionStrings[0] }));
super("azure:eventhub:CosmosChangeFeedSubscription", name, bindings, {
...args,
resourceGroupName,
location,
appSettings
}, opts);

this.account = account;

this.registerOutputs();
}
}
1 change: 1 addition & 0 deletions sdk/nodejs/tsconfig.json
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,7 @@
"cosmosdb/account.ts",
"cosmosdb/getAccount.ts",
"cosmosdb/index.ts",
"cosmosdb/zMixins.ts",
"databricks/index.ts",
"databricks/workspace.ts",
"datafactory/datasetMysql.ts",
Expand Down