Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Deployer missing files #1599

Merged
merged 14 commits into from
Jan 10, 2023
Merged

Deployer missing files #1599

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
a5ec62b
Add check for missing airnode deployment files
vponline Dec 21, 2022
cdd8d03
Enable ora warn and info logging while spinner is spinning
vponline Dec 21, 2022
9d7623c
Add changeset
vponline Dec 21, 2022
790080a
Fix missing latestDeployment children object
vponline Dec 21, 2022
c05171c
Skip fetching deployment files if config.json or secrets.env is missing
vponline Dec 22, 2022
3c1e64c
Merge branch 'master' into deployer-missing-files
vponline Jan 3, 2023
64d38cc
Refactor checking for bucket missing files, add check to deployAirnode
vponline Jan 9, 2023
3d0f967
Fix missing file check
vponline Jan 9, 2023
45c9c7e
Fix missing files object keys
vponline Jan 9, 2023
63fa520
Merge branch 'master' into deployer-missing-files
vponline Jan 9, 2023
5694345
Remove use of TF_STATE_FILENAME to fix tests
vponline Jan 9, 2023
410b315
Remove duplicate logging of invalid bucket directory
vponline Jan 10, 2023
df4d761
Resolve conflicts
vponline Jan 10, 2023
e5de4de
Wrap file fetching and JSON.parse with go, refactor names
vponline Jan 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/mighty-dryers-cheat.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
'@api3/airnode-deployer': patch
---

Add check for missing secrets.env, config.json, default.tfstate files in bucket
33 changes: 29 additions & 4 deletions packages/airnode-deployer/src/infrastructure/index.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,18 +460,43 @@ async function fetchDeployments(cloudProviderType: CloudProvider['type'], deploy
}

const latestDeployment = Object.keys(stageDirectory.children).sort().reverse()[0];
const latestDepolymentFileNames = Object.keys((stageDirectory.children[latestDeployment] as Directory).children);

const requiredFileNames = ['config.json', 'secrets.env', 'default.tfstate'];
const missingRequiredFiles = requiredFileNames.filter(
(requiredFileName) => !latestDepolymentFileNames.includes(requiredFileName)
);
vponline marked this conversation as resolved.
Show resolved Hide resolved
if (missingRequiredFiles.length) {
vponline marked this conversation as resolved.
Show resolved Hide resolved
logger.warn(
`Airnode '${airnodeAddress}' with stage '${stage}' is missing files: ${missingRequiredFiles.join(
', '
)}. Deployer commands may fail and manual removal may be necessary.`
);
vponline marked this conversation as resolved.
Show resolved Hide resolved
}

Copy link
Contributor Author

@vponline vponline Dec 21, 2022 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will detect any of the three missing files, but note that if the missing file is either secrets.env or config.json, it is not possible to derive the deploymentId so the commands will fail with an error:

No deployment with ID 'aws...' found

So would it make sense to just throw an error directly if either of these files are missing (but presumably the deployment is found). But this could exit the loop too early if there are multiple deployments in the bucket 🤔

image

const bucketLatestDeploymentPath = `${airnodeAddress}/${stage}/${latestDeployment}`;

const bucketConfigPath = `${bucketLatestDeploymentPath}/config.json`;
logger.debug(`Fetching configuration file '${bucketConfigPath}'`);
const config = JSON.parse(
await cloudProviderLib[cloudProviderType].getFileFromBucket(bucket.name, bucketConfigPath)
const goGetConfigFileFromBucket = await go(() =>
cloudProviderLib[cloudProviderType].getFileFromBucket(bucket.name, bucketConfigPath)
);
if (!goGetConfigFileFromBucket.success) {
logger.warn(`Failed to fetch configuration file. Error: ${goGetConfigFileFromBucket.error.message} Skipping.`);
continue;
}
const config = JSON.parse(goGetConfigFileFromBucket.data);
vponline marked this conversation as resolved.
Show resolved Hide resolved

logger.debug(`Fetching secrets file '${bucketConfigPath}'`);
const bucketSecretsPath = `${bucketLatestDeploymentPath}/secrets.env`;
const secrets = dotenv.parse(
await cloudProviderLib[cloudProviderType].getFileFromBucket(bucket.name, bucketSecretsPath)
const goGetSecretsFileFromBucket = await go(() =>
cloudProviderLib[cloudProviderType].getFileFromBucket(bucket.name, bucketSecretsPath)
);
if (!goGetSecretsFileFromBucket.success) {
logger.warn(`Failed to fetch secrets file. Error: ${goGetSecretsFileFromBucket.error.message} Skipping.`);
continue;
}
const secrets = dotenv.parse(goGetSecretsFileFromBucket.data);
const interpolatedConfig = unsafeParseConfigWithSecrets(config, secrets);

const cloudProvider = interpolatedConfig.nodeSettings.cloudProvider as CloudProvider;
Expand Down
12 changes: 11 additions & 1 deletion packages/airnode-deployer/src/utils/logger.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,25 @@ export function fail(text: string) {
}

export function warn(text: string) {
const currentOra = getSpinner();
if (currentOra.isSpinning) {
currentOra.clear();
currentOra.frame();
}
oraInstance().warn(text);
}

export function info(text: string) {
const currentOra = getSpinner();
if (currentOra.isSpinning) {
currentOra.clear();
currentOra.frame();
}
oraInstance().info(text);
}

export function debug(text: string) {
if (debugModeFlag) oraInstance().info(text);
if (debugModeFlag) info(text);
Comment on lines +39 to +57
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes are to allow printing a line to the console while the ora spinner is still spinning. WIthout these the spinner will overwrite any logs.

}

export function debugSpinner(text: string) {
Expand Down