Skip to content

Commit

Permalink
samples: add samples for cloud storage buckets (googleapis#1931)
Browse files Browse the repository at this point in the history
* chore: ignore samples/build/

* fix: convert string ports to number ports to match grpc

* feat: provide redacted topic and subscription admin clients

* tests: allow for storage bucket name gen

* samples: add sample for subscription with cloud storage bucket

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

* samples: remove gapic getters to be split to another PR

* samples: update for review changes

* 🦉 Updates from OwlBot post-processor

See https://github.com/googleapis/repo-automation-bots/blob/main/packages/owl-bot/README.md

---------

Co-authored-by: Owl Bot <gcf-owl-bot[bot]@users.noreply.github.com>
  • Loading branch information
feywind and gcf-owl-bot[bot] authored Jul 8, 2024
1 parent 34b8c03 commit ac48418
Show file tree
Hide file tree
Showing 8 changed files with 258 additions and 1 deletion.
1 change: 1 addition & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -131,6 +131,7 @@ Samples are in the [`samples/`](https://github.com/googleapis/nodejs-pubsub/tree
| Create Push Subscription | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createPushSubscription.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createPushSubscription.js,samples/README.md) |
| Create Push Subscription With No Wrapper | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createPushSubscriptionNoWrapper.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createPushSubscriptionNoWrapper.js,samples/README.md) |
| Create Subscription | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createSubscription.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createSubscription.js,samples/README.md) |
| Create a Cloud Storage subscription | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createSubscriptionWithCloudStorage.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createSubscriptionWithCloudStorage.js,samples/README.md) |
| Create Subscription With Dead Letter Policy | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createSubscriptionWithDeadLetterPolicy.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createSubscriptionWithDeadLetterPolicy.js,samples/README.md) |
| Create an exactly-once delivery subscription | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createSubscriptionWithExactlyOnceDelivery.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createSubscriptionWithExactlyOnceDelivery.js,samples/README.md) |
| Create Subscription With Filtering | [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createSubscriptionWithFiltering.js) | [![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createSubscriptionWithFiltering.js,samples/README.md) |
Expand Down
20 changes: 20 additions & 0 deletions samples/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ guides.
* [Create Push Subscription](#create-push-subscription)
* [Create Push Subscription With No Wrapper](#create-push-subscription-with-no-wrapper)
* [Create Subscription](#create-subscription)
* [Create a Cloud Storage subscription](#create-a-cloud-storage-subscription)
* [Create Subscription With Dead Letter Policy](#create-subscription-with-dead-letter-policy)
* [Create an exactly-once delivery subscription](#create-an-exactly-once-delivery-subscription)
* [Create Subscription With Filtering](#create-subscription-with-filtering)
Expand Down Expand Up @@ -253,6 +254,25 @@ __Usage:__



### Create a Cloud Storage subscription

Demonstrates how to create a subscription with Cloud Storage.

View the [source code](https://github.com/googleapis/nodejs-pubsub/blob/main/samples/createSubscriptionWithCloudStorage.js).

[![Open in Cloud Shell][shell_img]](https://console.cloud.google.com/cloudshell/open?git_repo=https://github.com/googleapis/nodejs-pubsub&page=editor&open_in_editor=samples/createSubscriptionWithCloudStorage.js,samples/README.md)

__Usage:__


`node createSubscriptionWithCloudStorage.js <topic-name> <subscription-name> <bucket> <filename-prefix> <filename-suffix> <max-duration>`


-----




### Create Subscription With Dead Letter Policy

Creates a new subscription With Dead Letter Policy.
Expand Down
99 changes: 99 additions & 0 deletions samples/createSubscriptionWithCloudStorage.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,99 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

// This is a generated sample, using the typeless sample bot. Please
// look for the source TypeScript sample (.ts) for modifications.
'use strict';

/**
* This snippet demonstrates how to perform basic operations on
* schemas with the Google Cloud Pub/Sub API.
*
* For more information, see the README.md under /pubsub and the documentation
* at https://cloud.google.com/pubsub/docs.
*/

// sample-metadata:
// title: Create a Cloud Storage subscription
// description: Demonstrates how to create a subscription with Cloud Storage.
// usage: node createSubscriptionWithCloudStorage.js <topic-name> <subscription-name> <bucket> <filename-prefix> <filename-suffix> <max-duration>

// [START pubsub_create_cloud_storage_subscription]
/**
* TODO(developer): Uncomment these variables before running the sample.
*/
// const topicName = 'YOUR_TOPIC_NAME';
// const subscriptionName = 'YOUR_SUBSCRIPTION_NAME';
// const bucket = 'YOUR_BUCKET_ID';
// const filenamePrefix = 'YOUR_FILENAME_PREFIX';
// const filenameSuffix = 'YOUR_FILENAME_SUFFIX';
// const maxDuration = 60;

// Imports the Google Cloud client library
const {PubSub} = require('@google-cloud/pubsub');

// Creates a client; cache this for further use
const pubSubClient = new PubSub();

async function createCloudStorageSubscription(
topicName,
subscriptionName,
bucket,
filenamePrefix,
filenameSuffix,
maxDuration
) {
const options = {
cloudStorageConfig: {
bucket,
filenamePrefix,
filenameSuffix,
maxDuration: {
seconds: maxDuration,
},
},
};

await pubSubClient
.topic(topicName)
.createSubscription(subscriptionName, options);

console.log(
`Created subscription ${subscriptionName} with a cloud storage configuration.`
);
}
// [END pubsub_create_cloud_storage_subscription]

function main(
topicName = 'YOUR_TOPIC_NAME',
subscriptionName = 'YOUR_SUBSCRIPTION_NAME',
bucket = 'YOUR_BUCKET_NAME',
filenamePrefix = 'YOUR_FILENAME_PREFIX',
filenameSuffix = 'YOUR_FILENAME_SUFFIX',
maxDuration = 60
) {
createCloudStorageSubscription(
topicName,
subscriptionName,
bucket,
filenamePrefix,
filenameSuffix,
maxDuration
).catch(err => {
console.error(err.message);
process.exitCode = 1;
});
}

main(...process.argv.slice(2));
1 change: 1 addition & 0 deletions samples/package.json
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@
},
"dependencies": {
"@google-cloud/pubsub": "^4.5.0",
"@google-cloud/storage": "^7.11.1",
"@opentelemetry/api": "^1.6.0",
"@opentelemetry/tracing": "^0.24.0",
"avro-js": "^1.11.3",
Expand Down
32 changes: 31 additions & 1 deletion samples/system-test/subscriptions.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
// limitations under the License.

import {BigQuery} from '@google-cloud/bigquery';
import {Bucket, Storage} from '@google-cloud/storage';
import {
CreateSubscriptionOptions,
PubSub,
Expand Down Expand Up @@ -74,7 +75,7 @@ describe('subscriptions', () => {

async function createBigQueryTable(datasetId: string, tableId: string) {
const bigquery = new BigQuery({
projectId: projectId,
projectId,
});

const datasetOptions = {
Expand Down Expand Up @@ -108,6 +109,17 @@ describe('subscriptions', () => {
await bigquery.dataset(datasetId).delete(deleteOptions);
}

async function createStorageBucket(testName: string): Promise<Bucket> {
const storage = new Storage({
projectId,
});

const name = resources.generateStorageName(testName);

const [bucket] = await storage.createBucket(name);
return bucket;
}

async function cleanSubs() {
const [subscriptions] = await pubsub.getSubscriptions();
await Promise.all(
Expand Down Expand Up @@ -428,6 +440,24 @@ describe('subscriptions', () => {
assert(subscriptionDetached === true);
});

it('should create a subscription with a cloud storage config', async () => {
const testId = 'sub_storage';
const topic = await createTopic(testId);
const subName = reserveSub(testId);
const bucket = await createStorageBucket(testId);
const bucketName = bucket.name;
try {
const output = execSync(
`${commandFor('createSubscriptionWithCloudStorage')} ${
topic.name
} projects/${projectId}/subscriptions/${subName} ${bucketName} 'prefix' 'suffix' 60`
);
assert.include(output, 'Created subscription');
} finally {
bucket.delete();
}
});

it('should create a subscription with dead letter policy.', async () => {
const testId = 'dead_letter';
const topic = await createTopic(testId),
Expand Down
10 changes: 10 additions & 0 deletions samples/system-test/testResources.ts
Original file line number Diff line number Diff line change
Expand Up @@ -101,6 +101,16 @@ export class TestResources {
);
}

/**
* Generates a unique resource name for one run of a test within
* a test suite for Cloud Storage resources.
*/
generateStorageName(testId: string): string {
return [normalizeId(this.getPrefix(testId)), this.tokenMaker.uuid()].join(
'_'
);
}

/*!
* Given a list of resource names (and a test ID), this will return
* a list of all resources that should be deleted to clean up for
Expand Down
95 changes: 95 additions & 0 deletions samples/typescript/createSubscriptionWithCloudStorage.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,95 @@
// Copyright 2024 Google LLC
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// http://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.

/**
* This snippet demonstrates how to perform basic operations on
* schemas with the Google Cloud Pub/Sub API.
*
* For more information, see the README.md under /pubsub and the documentation
* at https://cloud.google.com/pubsub/docs.
*/

// sample-metadata:
// title: Create a Cloud Storage subscription
// description: Demonstrates how to create a subscription with Cloud Storage.
// usage: node createSubscriptionWithCloudStorage.js <topic-name> <subscription-name> <bucket> <filename-prefix> <filename-suffix> <max-duration>

// [START pubsub_create_cloud_storage_subscription]
/**
* TODO(developer): Uncomment these variables before running the sample.
*/
// const topicName = 'YOUR_TOPIC_NAME';
// const subscriptionName = 'YOUR_SUBSCRIPTION_NAME';
// const bucket = 'YOUR_BUCKET_ID';
// const filenamePrefix = 'YOUR_FILENAME_PREFIX';
// const filenameSuffix = 'YOUR_FILENAME_SUFFIX';
// const maxDuration = 60;

// Imports the Google Cloud client library
import {CreateSubscriptionOptions, PubSub} from '@google-cloud/pubsub';

// Creates a client; cache this for further use
const pubSubClient = new PubSub();

async function createCloudStorageSubscription(
topicName: string,
subscriptionName: string,
bucket: string,
filenamePrefix: string,
filenameSuffix: string,
maxDuration: number
) {
const options: CreateSubscriptionOptions = {
cloudStorageConfig: {
bucket,
filenamePrefix,
filenameSuffix,
maxDuration: {
seconds: maxDuration,
},
},
};

await pubSubClient
.topic(topicName)
.createSubscription(subscriptionName, options);

console.log(
`Created subscription ${subscriptionName} with a cloud storage configuration.`
);
}
// [END pubsub_create_cloud_storage_subscription]

function main(
topicName = 'YOUR_TOPIC_NAME',
subscriptionName = 'YOUR_SUBSCRIPTION_NAME',
bucket = 'YOUR_BUCKET_NAME',
filenamePrefix = 'YOUR_FILENAME_PREFIX',
filenameSuffix = 'YOUR_FILENAME_SUFFIX',
maxDuration = 60
) {
createCloudStorageSubscription(
topicName,
subscriptionName,
bucket,
filenamePrefix,
filenameSuffix,
maxDuration
).catch(err => {
console.error(err.message);
process.exitCode = 1;
});
}

main(...process.argv.slice(2));
1 change: 1 addition & 0 deletions src/pubsub.ts
Original file line number Diff line number Diff line change
Expand Up @@ -307,6 +307,7 @@ export class PubSub {
allScopes[scope] = true;
}
}

this.options = Object.assign(
{
libName: 'gccl',
Expand Down

0 comments on commit ac48418

Please sign in to comment.