Add a File menu entry to upload files

[jankeromnes]

When you want to upload a file into your Theia IDE, there is no visible option to do that.

Most people will first upload files to somewhere, and then wget or curl or git pull (!) desired files from the Terminal. This is terrible.

Adding a File menu entry that says Upload file... and opens a file picker would be awesome for UX.

[idoprz]

you can drag and drop as a work around.

[jankeromnes]

you can drag and drop as a work around.

Thanks, that's true (dropping into the file tree, not the editor) but this feature is almost impossible to discover without someone telling you about it first.

A menu option would make that feature more discoverable.

[jankeromnes]

I see two potential places where this Upload Files... feature could be exposed:

1. In the File top-menu (right under New File and New Folder, maybe with an extra separator)
2. In the file tree context menu (right under New File, New Folder and Open in Terminal)

Notes:

• For 1. MenuPath === CommonMenus.FILE_NEW === ['menubar', '1_file', '1_new']
• For 2. MenuPath === NavigatorContextMenu.NEW === ['navigator-context-menu', '4_new']

[jankeromnes]

I'll go ahead and implement both (we can always change our minds during code review).

[jankeromnes]

Interesting find: The current file upload implementation uses DataTransferItem.webkitGetAsEntry(), which only works in Firefox, Chrome and Edge. So you can't upload files in IE, Opera or Safari.

https://github.com/theia-ide/theia/blob/5156023a4b0c590b40485f82d5c798c6ef6f8dec/packages/filesystem/src/browser/file-tree/file-tree-model.ts#L191-L196

[akosyakov]

@jankeromnes If you know official stable API which cover all browsers, it would be good to rewrite it. I could not such find or figure out how to upload directories with that.

[jankeromnes]

@akosyakov I usually use https://github.com/felixge/node-formidable for robust cross-browser file upload support. Would that work?

[akosyakov]

@jankeromnes yes, why not, so we won't use our filesystem anymore, but need an http endpoint?

[akosyakov]

You should be aware that adding new dependencies is troublesome right now 😢 We need get CQs for them: https://github.com/theia-ide/theia/wiki/Registering-CQs I hope it would be resolved soon that we can develop fast again. //cc @marcdumais-work

[jgbradley1]

Was a decision ever made on what module should be used for this functionality? The PR seems almost complete if the licensing issue can be resolved.

[jankeromnes]

If there exists a way to transform a input.files (a FileList from a <input type="file" multiple>), into a DataTransferItemList to pass to FileTreeModel.upload(), then yes the pull request is almost finished in adding the menu entries.

However, just like the current drag-an-drop upload, it would still only work in Firefox, Chrome and Edge, see #4088 (comment).

The better solution would be to replace the existing FileTreeModel.upload() code with a library like https://github.com/felixge/node-formidable, because it would make file uploads work across all browsers, but this means significantly more work, and importing a new library into Theia requires extra process.

[akosyakov]

FileTreeModel.upload() cannot handle big files: #969 It's a limitation of how we using web sockets. Need to fix it as well, but we can use node-formidable instead if it is better.

[jgbradley1]

Since this issue has been open for two months, using FileTreeModel.upload() seems like the path of least resistance in order for this feature to move forward. As long as the upload() function isn't modified, changing the underlying code to use node-formidable at a later time would not be a breaking change would it?

For browser compatibility, a browser check could be performed in the code with a warning dialog box notifying the user if they're using an incompatible browser.

I also think mentioning the 60 MB cap should be placed somewhere so users are aware. They can deal with splitting files up outside of Theia as a workaround for now.

[jgbradley1]

If I understand the codebase correctly, it looks like the backend server is an express server. If so, express-formidable might be a better option to use.

[akosyakov]

They are limitations to implementation.

With stable browser APIs:

• only files can be uploaded
• only with simple name (not a relative path)

With webkit: one can also uploads directories, but exclusively, i.e one cannot select files then.

Would it be alright to have two commands Upload Files... (based on stable) and Upload Folders... (available only for webkit)?

cc @jankeromnes @jgbradley1

[jankeromnes]

Would it be alright to have two commands Upload Files... (based on stable) and Upload Folders... (available only for webkit)?

That would work for me, thanks!

Or what about just implementing the Upload Files... menu entry, and skipping Upload Folders... altogether? The underlying browser feature is non-standard, I don't think I've ever needed it such a thing, and all the use cases I can think of can be worked around easily (e.g. by selecting all files in a directory, or by uploading an archive).

[jgbradley1]

In VS Code, there is a single Open... command that allows the user to select either a file or folder. Maybe that behavior should be imitated with the Upload Files... command. Allow a user to select files or folders, perform a check to determine what was chosen, which would determine which upload method is used (stable browser API vs webkit).

[jgbradley1]

Extending the contributions of @jankeromnes, I came up with a very crude implementation here using formidable. You can test it it out with gitpod. It doesn't parse out the uploaded file correctly just yet.

I think the file upload option should use the native file picker dialog box. It feels more intuitive than trying to create and maintain a custom dialog box.

[akosyakov]

Or what about just implementing the Upload Files... menu entry, and skipping Upload Folders... altogether? The underlying browser feature is non-standard, I don't think I've ever needed it such a thing, and all the use cases I can think of can be worked around easily (e.g. by selecting all files in a directory, or by uploading an archive).

I will start with that, but folder hierarchy won't be preserved, i.e. all files will be uploaded with simple names, if there are duplicates the last one wins. One can workaround it by drag & drop in the navigator.

In VS Code, there is a single Open... command that allows the user to select either a file or folder. Maybe that behavior should be imitated with the Upload Files... command. Allow a user to select files or folders, perform a check to determine what was chosen, which would determine which upload method is used (stable browser API vs webkit).

As I explained, It is not feasible in browser context because standard limitations: there is not allow to select files OR folders. VS Code uses native Electron APIs for it.

[jgbradley1]

Okay @akosyakov I understand what you are saying now.

Or what about just implementing the Upload Files... menu entry, and skipping Upload Folders... altogether? The underlying browser feature is non-standard, I don't think I've ever needed it such a thing, and all the use cases I can think of can be worked around easily (e.g. by selecting all files in a directory, or by uploading an archive).

I agree.

[jankeromnes]

🎉 🎉 🎉 Many thanks @akosyakov!

[davide-sergi]

@akosyakov From this discussion, I understood that is not possible to select programmatically (e.g. via command invocation) a file or folder, right? I ask because I'm trying to do so from my plugin