Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(fsx): specify file system type version for the Lustre file system #31136

Merged
merged 22 commits into from
Oct 16, 2024

Conversation

badmintoncryer
Copy link
Contributor

Issue # (if applicable)

Closes #31130

Reason for this change

We cannot specify Lustre version for the file system.

Description of changes

Add fileSystemTypeVersion prop to LustreFileSystemProps.

There are some restrictions about fileSystemTypeVersion.

  • 2.10 is supported by the Scratch and Persistent_1 Lustre deployment types.
  • 2.12 is supported by all Lustre deployment types, except for PERSISTENT_2 with a metadata configuration mode.
  • 2.15 is supported by all Lustre deployment types and is recommended for all new file systems.

https://docs.aws.amazon.com/ja_jp/AWSCloudFormation/latest/UserGuide/aws-resource-fsx-filesystem.html#cfn-fsx-filesystem-filesystemtypeversion

I've only added validation of V2.10 is supported by the Scratch and Persistent_1 deployment types because LustreFileSystemProps does not have metadata configuration props.

  private validateFiileSystemTypeVersion(deploymentType: LustreDeploymentType, fileSystemTypeVersion?: FileSystemTypeVersion): void {
    if (fileSystemTypeVersion === undefined) {
      return;
    }

    if (fileSystemTypeVersion === FileSystemTypeVersion.V_2_10) {
      if (!deploymentType.startsWith('SCRATCH') && deploymentType !== LustreDeploymentType.PERSISTENT_1) {
        throw new Error('fileSystemTypeVersion V_2_10 is only supported for SCRATCH and PERSISTENT_1 deployment types');
      }
    }
  }

Description of how you validated changes

Add both unit and integ tests.

Checklist


By submitting this pull request, I confirm that my contribution is made under the terms of the Apache-2.0 license

@aws-cdk-automation aws-cdk-automation requested a review from a team August 18, 2024 00:59
@github-actions github-actions bot added effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2 distinguished-contributor [Pilot] contributed 50+ PRs to the CDK labels Aug 18, 2024
Copy link
Collaborator

@aws-cdk-automation aws-cdk-automation left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The pull request linter has failed. See the aws-cdk-automation comment below for failure reasons. If you believe this pull request should receive an exemption, please comment and provide a justification.

A comment requesting an exemption should contain the text Exemption Request. Additionally, if clarification is needed add Clarification Request to a comment.

@badmintoncryer badmintoncryer marked this pull request as ready for review August 18, 2024 01:32
@aws-cdk-automation aws-cdk-automation dismissed their stale review August 18, 2024 01:34

✅ Updated pull request passes all PRLinter validations. Dismissing previous PRLinter review.

Comment on lines 70 to 79
**Note**: The `fileSystemTypeVersion` has a restrictions on the values that can be set based on the `deploymentType`.

- `V_2_10` is supported by the Scratch and `PERSISTENT_1` deployment types.
- `V_2_12` is supported by all Lustre deployment types, except for `PERSISTENT_2` with a metadata configuration mode.
- `V_2_15` is supported by all Lustre deployment types and is recommended for all new file systems.

**Note**: The default value of `fileSystemTypeVersion` is `V_2_10` except for the following cases:

- Default value is `V_2_12` when `deploymentType` is set to `PERSISTENT_2` without a metadata configuration mode.
- Default value is `V_2_15` when `deploymentType` is set to `PERSISTENT_2` with a metadata configuration mode.
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I have included descriptions related to metadata configuration mode, but the current LustreFileSystem construct does not allow setting metadata. Should descriptions related to metadata be included?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it might be good to either add a note stating that Metadata Configuration cannot be set yet, or alternatively, include the metadata configuration property in this PR.
What do you think?

The MetadataConfiguration property already exists, so it seems possible to add it. However, since it's a new property, it appears that descriptions haven't been defined for it yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your opinion!

I think it is better to support MetadataConfiguration initially, I was planning to address the metadataConfiguration in this PR as well, but since all the CloudFormation documentation was marked as "not available," I decided to try again once that issue is resolved.

I have updated the README to reflect the assumption that metadataConfiguration is not configured.

@aws-cdk-automation aws-cdk-automation added the pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. label Aug 19, 2024
Copy link
Contributor

@mazyu36 mazyu36 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for your contribution. I've made some comments.

/**
* The Lustre version for the file system.
*
* @see https://docs.aws.amazon.com/ja_jp/AWSCloudFormation/latest/UserGuide/aws-resource-fsx-filesystem.html#cfn-fsx-filesystem-filesystemtypeversion
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: should remove ja_jp

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! I've removed ja_JP/

Comment on lines 70 to 79
**Note**: The `fileSystemTypeVersion` has a restrictions on the values that can be set based on the `deploymentType`.

- `V_2_10` is supported by the Scratch and `PERSISTENT_1` deployment types.
- `V_2_12` is supported by all Lustre deployment types, except for `PERSISTENT_2` with a metadata configuration mode.
- `V_2_15` is supported by all Lustre deployment types and is recommended for all new file systems.

**Note**: The default value of `fileSystemTypeVersion` is `V_2_10` except for the following cases:

- Default value is `V_2_12` when `deploymentType` is set to `PERSISTENT_2` without a metadata configuration mode.
- Default value is `V_2_15` when `deploymentType` is set to `PERSISTENT_2` with a metadata configuration mode.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I thought it might be good to either add a note stating that Metadata Configuration cannot be set yet, or alternatively, include the metadata configuration property in this PR.
What do you think?

The MetadataConfiguration property already exists, so it seems possible to add it. However, since it's a new property, it appears that descriptions haven't been defined for it yet.

@badmintoncryer
Copy link
Contributor Author

@mazyu36 Thank you for your review! I've addressed your comments.

Copy link
Contributor

@mazyu36 mazyu36 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks! LGTM

@badmintoncryer
Copy link
Contributor Author

@mazyu36 Thanks!

@badmintoncryer
Copy link
Contributor Author

badmintoncryer commented Sep 22, 2024

@pahud mazyu36 (and I) has been selected as a community reviewer, but even after approving, the 'needs-community-review' label is not being updated. Do you know the reason for this?

I'm sorry. This problem has resolved.

@aws-cdk-automation aws-cdk-automation added pr/needs-maintainer-review This PR needs a review from a Core Team Member and removed pr/needs-community-review This PR needs a review from a Trusted Community Member or Core Team Member. labels Sep 24, 2024
packages/aws-cdk-lib/aws-fsx/lib/lustre-file-system.ts Outdated Show resolved Hide resolved
if (fileSystemTypeVersion === FileSystemTypeVersion.V_2_10) {
if (!deploymentType.startsWith('SCRATCH') && deploymentType !== LustreDeploymentType.PERSISTENT_1) {
throw new Error('fileSystemTypeVersion V_2_10 is only supported for SCRATCH and PERSISTENT_1 deployment types');
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we also validate



2.12 is supported by all Lustre deployment types, except for PERSISTENT_2 with a metadata configuration mode.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As I described above, LustreFileSystemProps does not have metadata configuration prop and I think it is difficult to add validation for that.

What do you think about this?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My bad. I totally missed the note in the PR description. Since this prop doesn't exists yet, I think it's fine to skip this validation now.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But can you add a TODO comment to indicate that we should add one once we support metadata config?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've added TODO comment. Could you please confirm?

@mergify mergify bot dismissed GavinZZ’s stale review October 16, 2024 22:08

Pull request has been modified.

Copy link
Contributor

@GavinZZ GavinZZ left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!!

Copy link
Contributor

mergify bot commented Oct 16, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation aws-cdk-automation removed the pr/needs-maintainer-review This PR needs a review from a Core Team Member label Oct 16, 2024
Copy link
Contributor

mergify bot commented Oct 16, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

@aws-cdk-automation
Copy link
Collaborator

AWS CodeBuild CI Report

  • CodeBuild project: AutoBuildv2Project1C6BFA3F-wQm2hXv2jqQv
  • Commit ID: afbc949
  • Result: SUCCEEDED
  • Build Logs (available for 30 days)

Powered by github-codebuild-logs, available on the AWS Serverless Application Repository

@mergify mergify bot merged commit 252cca9 into aws:main Oct 16, 2024
9 checks passed
Copy link
Contributor

mergify bot commented Oct 16, 2024

Thank you for contributing! Your pull request will be updated from main and then merged automatically (do not update manually, and be sure to allow changes to be pushed to your fork).

Copy link

Comments on closed issues and PRs are hard for our team to see.
If you need help, please open a new issue that references this one.

@github-actions github-actions bot locked as resolved and limited conversation to collaborators Oct 16, 2024
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
distinguished-contributor [Pilot] contributed 50+ PRs to the CDK effort/small Small work item – less than a day of effort feature-request A feature should be added or improved. p2
Projects
None yet
Development

Successfully merging this pull request may close these issues.

FSx: Enable file_system_type_version property for LustreFileSystem construct
4 participants