Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(assorted): Assorted changes for the v1 read only system #86

Merged
merged 90 commits into from
Aug 18, 2023
Merged
Show file tree
Hide file tree
Changes from 33 commits
Commits
Show all changes
90 commits
Select commit Hold shift + click to select a range
08ae8f8
Checking in.... error free in vs code but lambda having an issue...
mdial89f Jul 27, 2023
7314463
Fix null RAI issue
mdial89f Jul 27, 2023
f00d6b2
correct some logic. working now
mdial89f Jul 27, 2023
d242227
added the breaks
mdial89f Jul 28, 2023
53e9aa2
add lead analyst... but its all null
mdial89f Jul 28, 2023
d10af16
update table lists
benjaminpaige Jul 28, 2023
f251e69
Merge branch 'master' into type-shark
benjaminpaige Jul 28, 2023
411c465
Add text helpers
benjaminpaige Jul 31, 2023
a6dafe4
Item details UI buildout (#67)
benjaminpaige Aug 1, 2023
4ea924e
Merge branch 'master' into type-shark
13bfrancis Aug 1, 2023
c2dc966
come up with a typesafe way to validate
13bfrancis Aug 3, 2023
dba703b
remove file
13bfrancis Aug 3, 2023
8e26186
validate against data and rewrite code
13bfrancis Aug 3, 2023
09b3004
Merge branch 'master' into type-shark
13bfrancis Aug 3, 2023
2b585fa
Merge branch 'master' into type-shark
13bfrancis Aug 3, 2023
eb63346
add rai recieved date
13bfrancis Aug 4, 2023
835faad
Merge branch 'master' into type-shark
13bfrancis Aug 4, 2023
04c9f0c
dry run validation seatool sink lambda
13bfrancis Aug 7, 2023
78dda92
change validation logic to account for null values
13bfrancis Aug 8, 2023
a1baf64
add test data so ci test passes
13bfrancis Aug 8, 2023
fb96875
add onemac schema
13bfrancis Aug 8, 2023
1def089
ship it
13bfrancis Aug 9, 2023
a64a52e
add error messages
13bfrancis Aug 9, 2023
0bbeac9
thank you Mike
13bfrancis Aug 9, 2023
40f75ff
add opensearch type
13bfrancis Aug 9, 2023
eec6089
refactor and fix seatool
13bfrancis Aug 10, 2023
934717b
add to barrel file
13bfrancis Aug 10, 2023
44857b0
add logs
13bfrancis Aug 10, 2023
da9d56c
fix type error
13bfrancis Aug 10, 2023
c2df38b
don't include certain seatool records
13bfrancis Aug 10, 2023
1e5a732
move id to after if check
13bfrancis Aug 11, 2023
4c8b7f9
attempt to get better logging
13bfrancis Aug 11, 2023
48e9c5a
add more logging
13bfrancis Aug 11, 2023
f51a21e
Handle onemac bad validation
mdial89f Aug 11, 2023
4b77eaf
fixes (yet untested) for the seatool sink errors
mdial89f Aug 11, 2023
106bd01
add received gat helper
benjaminpaige Aug 14, 2023
04e8b59
Merge from master
benjaminpaige Aug 14, 2023
da6e00b
update
benjaminpaige Aug 14, 2023
a33cb68
update
benjaminpaige Aug 14, 2023
08b099d
Update
benjaminpaige Aug 14, 2023
30283cc
update
benjaminpaige Aug 14, 2023
6f0c680
update
benjaminpaige Aug 14, 2023
5e67899
update
benjaminpaige Aug 14, 2023
bbd7d7d
update types
benjaminpaige Aug 15, 2023
408c335
refactor
benjaminpaige Aug 15, 2023
8d77745
Add componentType
benjaminpaige Aug 15, 2023
653fc49
rm new proj creation
mdial89f Aug 15, 2023
94978ea
lng
mdial89f Aug 15, 2023
6a4c234
overview match readme
mdial89f Aug 15, 2023
f8a2adc
api
mdial89f Aug 16, 2023
b2a8ab5
Use component type in lable
benjaminpaige Aug 16, 2023
279c220
remove old perms
mdial89f Aug 16, 2023
5dca967
docs
mdial89f Aug 16, 2023
6bb823d
remove old perms
mdial89f Aug 16, 2023
e19bcd1
remove comments
mdial89f Aug 16, 2023
ee1e40b
stuff
mdial89f Aug 16, 2023
5a6a0df
stuff
mdial89f Aug 16, 2023
4e71696
Add faq page
benjaminpaige Aug 16, 2023
fd95df2
docs
mdial89f Aug 16, 2023
a5997db
build a raw seatool index
mdial89f Aug 16, 2023
a92e9cc
remove the plan type id to name mapping in favor of using plan_types …
mdial89f Aug 16, 2023
207693e
add action type
mdial89f Aug 16, 2023
d49843b
Update styles
benjaminpaige Aug 17, 2023
cec9073
Check for null types in transform
benjaminpaige Aug 17, 2023
db3abb3
Fix detail nav types
benjaminpaige Aug 17, 2023
2eaf3ef
Update type based on unit tests
benjaminpaige Aug 17, 2023
94aec05
Merge branch 'master' into type-shark
13bfrancis Aug 17, 2023
13167f8
maybe this will work
13bfrancis Aug 17, 2023
e9b077c
nullify
13bfrancis Aug 17, 2023
ebe4935
add logging
13bfrancis Aug 17, 2023
44f8ead
fix error
13bfrancis Aug 17, 2023
101f4f2
Update type based on unit tests
benjaminpaige Aug 17, 2023
aa7fdb4
Remove submission origin
mdial89f Aug 17, 2023
8ba9936
disable the package actions until later
mdial89f Aug 17, 2023
384c7cc
remove reviewer details
mdial89f Aug 17, 2023
9fc35be
fixes
mdial89f Aug 17, 2023
99b2773
Merge branch 'rmorigin' into goblin
mdial89f Aug 17, 2023
be1be2f
reduce reuse recycle
mdial89f Aug 17, 2023
efd1ff5
fix action types thing
mdial89f Aug 17, 2023
08daceb
provisioned cap
mdial89f Aug 17, 2023
e087dda
user session timeout
mdial89f Aug 17, 2023
3ddb13f
Merge branch 'nado' into goblin
mdial89f Aug 17, 2023
20528dd
stop
mdial89f Aug 17, 2023
b30fc8f
set to undefined instead of null
13bfrancis Aug 18, 2023
76f83eb
remove componentType and update dashboard
benjaminpaige Aug 18, 2023
6784dab
do fun things
13bfrancis Aug 18, 2023
2fb4cf7
update details comp
benjaminpaige Aug 18, 2023
f43662a
codeclimate update
benjaminpaige Aug 18, 2023
6f4ccaa
merge in type-shark
benjaminpaige Aug 18, 2023
2dde1fc
fix tombstone type
benjaminpaige Aug 18, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion docs/docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ The 10,000ft view

## Overview

The {{ site.repo.name }} project is TBD.
The {{ site.repo.name }} project is a redesign of MACPRO Onemac. The mission to be a modern submission and review portal for select CMS data remains the same, but the architecture is different in some important ways.

## Architecture

Expand Down
3 changes: 1 addition & 2 deletions docs/docs/services/alerts.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
layout: default
title: alerts
parent: Services
nav_order: 1
---

# alerts
Expand All @@ -12,7 +11,7 @@ nav_order: 1

The alerts service deploys a Simple Notification Service (SNS) topic to REGION_A. This topic can be leveraged by any other service for sending alerts.

#### Notes
#### Detail

- To subscribe an email, phone number, or something else to the topic, find the SNS topic using the AWS Console and add the subscription manually.
- No SNS subscriptions are made by the deployment process. The topic is created, and several other services are configured to publish notifications to the topic, but the topic itself is not automatically configured to fan out any notifications. Here's why:
Expand Down
56 changes: 14 additions & 42 deletions docs/docs/services/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,57 +2,29 @@
layout: default
title: api
parent: Services
nav_order: 6
---

# API
# api
{: .no_toc }

## Table of contents
{: .no_toc .text-delta }
## Summary

- TOC
{:toc}
The api service deploys a lambda-backed API Gateway that is used by the frontend to interact with the data layer. Access to any of its endpoints is guarded at a high level by AWS Cognito, ensuring only authenticated users may reach it. The lambda functions that back each endpoint enforce further fine-grain access according to business rules.

## Overview
This stack is used to deploy a RESTful API service to AWS. The service includes a set of functions that can be used to interact with the API, and it is secured with various AWS security policies.

## Service
The service name is ${self:custom.project}-api, where ${self:custom.project} is a parameter provided by the user. This ensures that the service name is unique to the user's project.
## Detail

## Package
The package section is used to configure how the deployment package for the service is generated. The "individually: true" setting is used to generate separate packages for each function in the service. This makes it easier to deploy and update individual functions without having to deploy the entire service.
The largest component of the api service is the API Gateway itself. This is a standard deployment of a regional, REST API Gateway. We do not apply custom certificates or DNS names to the api gateway endpoint (yet); instead, our application uses the amazon generated SSL endpoint.

## Plugins
The plugins section is used to specify the plugins that will be used during the deployment of the service. The following plugins are used:
There are three endpoints on the api. Each is guarded by AWS IAM, meaning that while the API Gateway is publicly available, the API will not forward your request to the backing lambda unless you provide valid credentials obtained through AWS Cognito. This way, only users with an account that we can authenticate may successfully call endpoints. The three endpoints are:
- /search (POST): This endpoint accepts search queries from clients in the form of OpenSearch Query DSL queries. Once the query is received, the lambda adds extra query filters to ensure fine grain auth. This works by looking up the user making the call in Cognito, determining what type of user (cms or state) is making the call, determining what states that user has access to (if appropriate), and modifying the query in a way that will only return results for those states. By design, the only thing the search endpoint adds is related to authentication; the rest of the query building is left to the frontend for faster and more flexible development.
- /item (POST): The item endpoint is used to fetch details for exactly one record. While you can form a query to do this and use the search endpoint, the item endpoint is for convenience. Simply make a post call containing the ID of the desired record to the item endpoint, and the record will be returned. Note that fine grain auth is still enforced in an identical way to search, whereby you will only obtain results for that ID if you should have access to that ID.
- /getAttachmentUrl (POST): This endpoint is used to generate a presigned url for direct client downloading of S3 data, enforcing fine grain auth along the way. This is how we securely allow download of submission attachment data. From the details page, a user may click a file to download. Once clicked, their client makes a post to /getAttachmentUrl with the attachment metadata. The lambda function determines if the caller should or should not have access based on identical logic as the other endpoints (the UI would not display something they cannot download, but this guards against bad actors). If access is allowed, the lambda function generates a presigned url good for 60 seconds and returns it to the client browser, at which point files are downloaded automatically.

serverless-bundle: A plugin that optimizes the packaging and deployment process for serverless applications.
serverless-stack-termination-protection: A plugin that applies CloudFormation termination protection to the specified stages, ensuring that accidental deletions of the service do not occur.
"@stratiformdigital/serverless-iam-helper": A plugin that simplifies the creation and management of AWS IAM roles and policies.
"@stratiformdigital/serverless-s3-security-helper": A plugin that adds security best practices to S3 buckets.
All endpoints and backing functions interact with the OpenSearch data layer. As such, and because OpenSearch is deployed within a VPC, all lambda functions of the api service are VPC based. The functions share a security group that allows outbound traffic.

## Provider
The provider section is used to configure the cloud provider (AWS), and any additional settings for the provider. The following settings are used:

- name: aws
-runtime: nodejs18.x
region: ${env:REGION_A} (This setting retrieves the region value from an environment variable called "REGION_A").
stackTags: Specifies tags to be applied to the CloudFormation stack. The tags include PROJECT and SERVICE, which are set to the custom.project and service values, respectively.
iam: Specifies IAM related settings for the CloudFormation stack. The role setting specifies the path and permissions boundary for the IAM role. The statements setting specifies the permissions granted to the role. In this case, the role is granted permission to access all CloudWatch resources.
Custom:
The custom section is used to specify custom settings for the service. The following settings are used:

project: ${env:PROJECT} (This setting retrieves the project value from an environment variable called "PROJECT").
accountId: !Sub "${AWS::AccountId}" (This setting retrieves the account ID for the AWS account in which the stack is deployed).
stage: ${opt:stage, self:provider.stage} (This setting specifies the deployment stage for the service. It is retrieved from an option called "stage", and if the option is not set, it defaults to the value specified in provider.stage).
serverlessTerminationProtection: Specifies the stages to which CloudFormation termination protection will be applied.

## Endpoints
The service is defining an API with five endpoints: getPosts, getPost, createPost, deletePost, and updatePost, which will handle GET, POST, PUT and DELETE requests for /posts and /posts/{id} paths.

## Resources
The resources section is used to specify the additional AWS resources that the service requires. In this case, two gateway responses are created for 4xx and 5xx responses.

## Outputs
The output values include the name and URL of the API Gateway, as well as the AWS region in which the stack is deployed.
All function share an IAM role. This is for convenicence; we can do one role per function if we find that valuable. The permissions include:
- OpenSearch permissions to allow access to the data layer
- Cognito permissions to look up user attributes; allows for enforcement of fine grain auth.
- AssumeRole permissions for a very specific cross account role, which is required to generate the presigned urls for the legacy OneMac data.

22 changes: 22 additions & 0 deletions docs/docs/services/auth.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
---
layout: default
title: auth
parent: Services
---

# auth
{: .no_toc }

## Summary
The auth service builds the infrastructure for our authentication and authorization solution: Amazon Cognito. A user pool and identity pool is deployed, and may conditionally be pointed to IDM (external identity provider).

## Detail
The core of the api service is a cognito user pool and identity pool, which work together to provide an auth solution:
- user pool: this is the user directory, where all active users and their attributes are stored.
- This is where we specify the user attribute schema, informed by but not beholden to IDM.
- The attribute schema is difficiult to update, and often requires deleting the user pool. This is acceptable for two reasons. One, updating the attribute schema would be a rare event. Two, since in higher environments all users are federated, the user pool itself holds no unique data; as such, it is safe to delete and simply rebuild without having data loss.
- identity pool: this is associated with the user pool, and allows us to grant certain AWS permissions to authenticated and/or unauthenticated entities.
- authenticated users may assume a role that gives them permissions to invoke the api gateway, as well as see information about their own cognito user.
- unauthenticated user may assume a role that gives them no permissions.

In the near future, higher environments will configure IDM as an external identity provider. Ephemeral/dev environments will continue to use only the cognito user pool.
7 changes: 0 additions & 7 deletions docs/docs/services/dashboard.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,11 @@
layout: default
title: dashboard
parent: Services
nav_order: 2
---

# dashboard
{: .no_toc }

## Table of contents
{: .no_toc .text-delta }

- TOC
{:toc}

## Why do I need this?
Part of any good project is a way to determine how well it is working. The purpose of a `CloudWatch Dashboard` is to determine the performance, health, and a variety of other aspects that factor into the product being delivered. What we have done here is provided an easy to use solution that will make creating a dashboard easy and deploying it even easier.

Expand Down
14 changes: 14 additions & 0 deletions docs/docs/services/data.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
layout: default
title: data
parent: Services
---

# data
{: .no_toc }

## Summary
The data service deploys our OpenSearch data layer and supporting infrastructure.

## Detail
OpenSearch, Amazon's managed Elasticsearch offering, was selected as the data layer tech.
2 changes: 1 addition & 1 deletion docs/docs/services/services.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,4 +12,4 @@ permalink: docs/services
Details on each Serverless service
{: .fs-6 .fw-300 }

The {{ site.repo.name }} project is a [serverless monorepo](https://serverless-stack.com/chapters/organizing-serverless-projects.html). It is, for the most part, a collection of standalone Serverless Framework services bound together in a repository. Loose coupling of the services is facilitated using one or several tools, which include CloudFormation outputs, AWS Systems Manager Parameter Store paramters, and AWS Secrets Manager stores. This section will describe each service in a high level of detail.
The {{ site.repo.name }} project is a [serverless monorepo](https://serverless-stack.com/chapters/organizing-serverless-projects.html). It is, for the most part, a collection of standalone Serverless Framework micro services bound together in a repository. Loose coupling of the micro services is facilitated using one or several tools, which include CloudFormation outputs, AWS Systems Manager Parameter Store paramters, and AWS Secrets Manager stores. This section will describe each service in a high level of detail.
11 changes: 2 additions & 9 deletions docs/docs/services/ui-infra.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,23 +2,16 @@
layout: default
title: ui-infra
parent: Services
nav_order: 5
---

# UI Infra
{: .no_toc }

## Table of contents
{: .no_toc .text-delta }

- TOC
{:toc}

## Overview
## Summary
This service provides the appropriate infrastructure for the UI application running on AWS. It creates several resources including an S3 bucket, a bucket policy, a logging bucket, a logging bucket policy, and an IAM role with permissions.


## Components
## Details

- AWS IAM role with permissions for CloudWatch logs and an IAM boundary policy.
- Serverless plugins to help with deploying and managing the infrastructure.
Expand Down
7 changes: 0 additions & 7 deletions docs/docs/services/ui.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,18 +2,11 @@
layout: default
title: ui
parent: Services
nav_order: 4
---

# UI
{: .no_toc }

## Table of contents
{: .no_toc .text-delta }

- TOC
{:toc}

## Overview
This service deploys a static web application to an S3 bucket with a CloudFront distribution in front of it for CDN caching and performance optimization. The template uses the serverless framework and includes several plugins to help with deployment and configuration.

Expand Down
Loading
Loading