Allow configuration of multiple Uplink URLs #1283

snoopdave · 2021-12-09T13:19:05Z

This PR adds two new configuration fields to the existing ManagedGatewayConfig interface:

  uplinkEndpoints?: string[];
  uplinkMaxRetries?: number;

If uplinkEndpoints is not set and there is an existing setting for schemaDeliveryMaxRetries (deprecated) it will be honored.

Also, the existing env variable APOLLO_SCHEMA_CONFIG_DELIVERY_ENDPOINT is now treated as a comma-separated list of Uplink URLS.

uplinkMaxRetries will default to three times the number of entries in the end-points array.

If you specify more than one Uplink URL, the loadSupergraphSdlFromUplinks() logic will round-robin retry and will return the first result it receives.

…ma separated list of Uplink endpoints and introduce new method loadSupergraphSdlFromUplinks() to fetch from all end-points and accept newest result.

…est.

trevor-scheer

I went digging a bit and turned up some issues with our nock usage in loadSupergraphSdlFromStorage.test.ts. I pushed up a fix and landed via #1309, so you'll probably need to rebase.

My nock comments may be affected by the changes I landed, but happy to discuss tomorrow.

gateway-js/src/__tests__/integration/configuration.test.ts

gateway-js/src/config.ts

gateway-js/src/__tests__/integration/nockMocks.ts

gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

gateway-js/src/index.ts

gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

# Conflicts: # gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts # gateway-js/src/index.ts

Use const instead of var Co-authored-by: Trevor Scheer <[email protected]>

…opdave/federation into snoopdave/multi-cloud-uplink # Conflicts: # gateway-js/src/index.ts

snoopdave · 2021-12-15T22:01:17Z

I downloaded the Gateway package from CircleCI and tested it locally with the original (GCP powered) Uplink URL, with the new AWS URL and with both. I added some console logging and verified that both URLs work and Gateway will alternate between them.

gateway-js/src/__tests__/integration/configuration.test.ts

gateway-js/src/__tests__/integration/nockMocks.ts

gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

gateway-js/src/index.ts

gateway-js/src/loadSupergraphSdlFromStorage.ts

gateway-js/src/index.ts

Co-authored-by: Trevor Scheer <[email protected]>

…opdave/federation into snoopdave/multi-cloud-uplink # Conflicts: # gateway-js/src/index.ts # gateway-js/src/loadSupergraphSdlFromStorage.ts

snoopdave · 2021-12-16T20:55:44Z

thanks for the awesome reviews @trevor-scheer. I think I have hit addressed of your concerns and suggestions.

trevor-scheer

LGTM! Just one comment about some stray whitespace changes. This needs a changelog entry in the gateway-js/CHANGELOG.md file. It should explain how the API's been updated and the deprecation of the old option.

gateway-js/src/__tests__/integration/configuration.test.ts

snoopdave · 2021-12-16T22:55:52Z

@trevor-scheer OK, we should be good to go on this. I downloaded the latest build from CI, tested it with the Space Camp Federation demo and verified that both Uplink URLs are being used.

glasser · 2021-12-17T00:31:23Z

@snoopdave I just want to make sure that updating https://www.apollographql.com/docs/studio/data-privacy/ is on your radar. This is where we list all URLs that any of our software connects to. The source is at https://github.com/apollographql/apollo/blob/main/studio-docs/source/data-privacy.md

snoopdave · 2021-12-17T00:50:21Z

@snoopdave I just want to make sure that updating https://www.apollographql.com/docs/studio/data-privacy/ is on your radar. This is where we list all URLs that any of our software connects to. The source is at https://github.com/apollographql/apollo/blob/main/studio-docs/source/data-privacy.md

That was not on my radar but it is now. Thanks

gateway-js/src/__tests__/integration/sample-schema.ts

trevor-scheer

🎉

This commit adds two new configuration fields to the existing `ManagedGatewayConfig` interface: * `uplinkEndpoints?: string[];` * `uplinkMaxRetries?: number;` If `uplinkEndpoints` is not set and there is an existing setting for `schemaDeliveryMaxRetries` (deprecated) it will be honored. Also, the existing env variable `APOLLO_SCHEMA_CONFIG_DELIVERY_ENDPOINT` is now treated as a comma-separated list of Uplink URLS. `uplinkMaxRetries` will default to three times the number of entries in the end-points array. If you specify more than one Uplink URL, the `loadSupergraphSdlFromUplinks()` logic will round-robin retry and will return the first result it receives.

This commit adds two new configuration fields to the existing `ManagedGatewayConfig` interface: * `uplinkEndpoints?: string[];` * `uplinkMaxRetries?: number;` If `uplinkEndpoints` is not set and there is an existing setting for `schemaDeliveryMaxRetries` (deprecated) it will be honored. Also, the existing env variable `APOLLO_SCHEMA_CONFIG_DELIVERY_ENDPOINT` is now treated as a comma-separated list of Uplink URLS. `uplinkMaxRetries` will default to three times the number of entries in the end-points array. If you specify more than one Uplink URL, the `loadSupergraphSdlFromUplinks()` logic will round-robin retry and will return the first result it receives. Co-authored-by: David M. Johnson <[email protected]>

Dave Johnson added 8 commits December 9, 2021 08:13

Change process.env.APOLLO_SCHEMA_CONFIG_DELIVERY_ENDPOINT to be a com…

eeaa76c

…ma separated list of Uplink endpoints and introduce new method loadSupergraphSdlFromUplinks() to fetch from all end-points and accept newest result.

Merge branch 'main' into snoopdave/multi-cloud-uplink

4d5975f

oops

3cdec3e

Use simple round robin approach.

8119161

Make loadSupergraph max retries configurable and the beginning of a t…

39925d9

…est.

Merge branch 'main' into snoopdave/multi-cloud-uplink

ebd0afe

Fix tests to work with new multi-cloud uplink config.

2143f09

Test for failure case.

c3fcb32

snoopdave changed the title ~~WIP: support for multi-cloud Uplink~~ Allow configuration of multiple Uplink URLs Dec 13, 2021

Dave Johnson added 2 commits December 14, 2021 15:49

Merge branch 'main' into snoopdave/multi-cloud-uplink

693044f

Correct AWS Uplink URL.

dfebc75

trevor-scheer mentioned this pull request Dec 14, 2021

chore(gateway): Fix OOB testing strategy (main) #1309

Merged

trevor-scheer suggested changes Dec 15, 2021

View reviewed changes

Dave Johnson and others added 5 commits December 15, 2021 13:27

Merge branch 'main' into snoopdave/multi-cloud-uplink.

8752260

# Conflicts: # gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts # gateway-js/src/index.ts

Update gateway-js/src/index.ts

e8da909

Use const instead of var Co-authored-by: Trevor Scheer <[email protected]>

Fixes suggested by Trevor.

5738540

Merge branch 'snoopdave/multi-cloud-uplink' of https://github.com/sno…

1e53455

…opdave/federation into snoopdave/multi-cloud-uplink # Conflicts: # gateway-js/src/index.ts

Merge branch 'main' into snoopdave/multi-cloud-uplink

9f61c52

snoopdave requested a review from trevor-scheer December 15, 2021 22:08

trevor-scheer suggested changes Dec 16, 2021

View reviewed changes

trevor-scheer reviewed Dec 16, 2021

View reviewed changes

gateway-js/src/index.ts Outdated Show resolved Hide resolved

snoopdave and others added 8 commits December 16, 2021 09:20

Update gateway-js/src/loadSupergraphSdlFromStorage.ts

6f12db3

Co-authored-by: Trevor Scheer <[email protected]>

Update gateway-js/src/loadSupergraphSdlFromStorage.ts

09c3bfe

Co-authored-by: Trevor Scheer <[email protected]>

Update gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

efbd64b

Co-authored-by: Trevor Scheer <[email protected]>

Update gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

3873626

Co-authored-by: Trevor Scheer <[email protected]>

Update gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

3c23ed9

Co-authored-by: Trevor Scheer <[email protected]>

Update gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

9c1d819

Co-authored-by: Trevor Scheer <[email protected]>

Update gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

7af28ae

Co-authored-by: Trevor Scheer <[email protected]>

Update gateway-js/src/__tests__/loadSupergraphSdlFromStorage.test.ts

7ebe30e

Co-authored-by: Trevor Scheer <[email protected]>

snoopdave and others added 3 commits December 16, 2021 09:22

Update gateway-js/src/index.ts

56a0a3f

Co-authored-by: Trevor Scheer <[email protected]>

More fixes and improvements suggested by Trevor.

5ea7b42

Merge branch 'snoopdave/multi-cloud-uplink' of https://github.com/sno…

0d33ad5

…opdave/federation into snoopdave/multi-cloud-uplink # Conflicts: # gateway-js/src/index.ts # gateway-js/src/loadSupergraphSdlFromStorage.ts

snoopdave requested a review from trevor-scheer December 16, 2021 20:55

trevor-scheer approved these changes Dec 16, 2021

View reviewed changes

gateway-js/src/__tests__/integration/configuration.test.ts Outdated Show resolved Hide resolved

Dave Johnson added 3 commits December 16, 2021 16:57

unnecessary whitespace changes.

87204ba

unnecessary whitespace changes.

aa05f6c

Changelog entry for multi-cloud uplink

ad0f9cc

trevor-scheer reviewed Dec 17, 2021

View reviewed changes

gateway-js/src/__tests__/integration/sample-schema.ts Outdated Show resolved Hide resolved

Delete unnecessary file.

4c83722

trevor-scheer mentioned this pull request Dec 17, 2021

Release @apollo/[email protected] #1236

Merged

trevor-scheer approved these changes Dec 17, 2021

View reviewed changes

trevor-scheer merged commit 873b79c into apollographql:main Dec 17, 2021

trevor-scheer mentioned this pull request Dec 17, 2021

feat(gateway): Allow configuration of multiple Uplink URLs version-0.x (duped from #1283) #1327

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Allow configuration of multiple Uplink URLs #1283

Allow configuration of multiple Uplink URLs #1283

snoopdave commented Dec 9, 2021 •

edited

Loading

trevor-scheer left a comment

snoopdave commented Dec 15, 2021

snoopdave commented Dec 16, 2021

trevor-scheer left a comment

snoopdave commented Dec 16, 2021

glasser commented Dec 17, 2021

snoopdave commented Dec 17, 2021

trevor-scheer left a comment

Allow configuration of multiple Uplink URLs #1283

Allow configuration of multiple Uplink URLs #1283

Conversation

snoopdave commented Dec 9, 2021 • edited Loading

trevor-scheer left a comment

Choose a reason for hiding this comment

snoopdave commented Dec 15, 2021

snoopdave commented Dec 16, 2021

trevor-scheer left a comment

Choose a reason for hiding this comment

snoopdave commented Dec 16, 2021

glasser commented Dec 17, 2021

snoopdave commented Dec 17, 2021

trevor-scheer left a comment

Choose a reason for hiding this comment

snoopdave commented Dec 9, 2021 •

edited

Loading