Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SWC-7064 - Logic for direct S3 upload #1345

Merged
merged 7 commits into from
Nov 4, 2024

Conversation

nickgros
Copy link
Collaborator

@nickgros nickgros commented Nov 1, 2024

No description provided.

Mock AWS packages instead of the S3 upload service, since it's too complicated to get the right set of polyfills set up in Jest/JSDOM
Comment on lines +32 to +33
"@aws-sdk/client-s3": "^3.682.0",
"@aws-sdk/lib-storage": "^3.682.0",
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In SWC, we are using v2 of the AWS client. Upgrade to v3, which is modular and actively maintained.

Comment on lines +71 to +87
rest.post(
`${backendOrigin}${FILE}/externalFileHandle`,
async (req, res, ctx) => {
const request = await req.json<ExternalFileHandleInterface>()

const response: SynapseApiResponse<ExternalFileHandleInterface> = {
...request,
id: uniqueId(),
etag: 'fake-etag',
createdBy: MOCK_USER_ID.toString(),
createdOn: new Date().toISOString(),
modifiedOn: new Date().toISOString(),
}

return res(ctx.status(201), ctx.json(response))
},
),
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simple handler for POST /externalFileHandle service

@@ -0,0 +1,9 @@
export type FileUploadArgs = {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This type represents the shared arguments for useDirectUploadToS3 in this PR, and another hook to use Synapse multipart upload coming in a later PR.

import { Upload } from '@aws-sdk/lib-storage'
import { uploadToS3 } from './UploadToS3'

jest.mock('@aws-sdk/lib-storage', () => {
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I originally tried to mock the S3 HTTP API using Mock Service Worker. Unfortunately, I ran into issues where the AWS SDK required methods on Blob that JSDOM does not polyfill, and replacing the Blob polyfill caused errors in other tests because other JSDOM polyfills rely on using their Blob polyfill.

@nickgros nickgros merged commit 8528d73 into Sage-Bionetworks:main Nov 4, 2024
24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants