Files sidebar improvements

[josevalim]

Currently the filesystem icons are not immediately clear what they mean. I have discussed with @jonatanklosko but we were unable to find a solution. So let's break down the problem: which information do we want to portray the user (besides the name)?

1. We want to know if the file is a reference (storage/url) or an attachment (in files/)

2. We want to know if the file is local (to the current machine) or remote. This matters because it affects how you will interact to the file. For example, when using Explorer, it may help decide if you want to use file_spec or file_path

3. If the file has already been cached locally or not (we can postpone this one)

Given the above, it seems clear to me we cannot only use icons to portray all of the information above, as it would require 5 different icons. Therefore I think we should break the file listing in two areas. Something like this (my favorite):

FILES

[      Add file      ]

External           [?]

[ ] foo.csv
[ ] bar.csv

Hard copies        [?]

[ ] baz.parquet

An alternative design is:

FILES              [?]

[ ] foo.csv
[ ] bar.csv

[      Add file      ]

Hard copies        [?]

[ ] baz.csv

We can discuss the names later. The icons will be:

• Cloud for any remote entry
• File for any local entry
• Link/chain for URLs

I like this a lot because having a separate section for hard copies will allow us to call more attention to them. Especially because they must always be copied and versioned with the notebook. If there are no hard copies, we can say "There are no associated entries in files/ the directory" and maybe even "There is no files/ directory alongside this notebook".

---

Assuming we go in this direction, we can have two drag and dropable areas. One for external and another for hard copies (each below their respective title). When you upload a file, we will no longer show the pop-up, we can now immediately add the file, which improves the user experience. We will provide a "Rename" action if you want to rename it after the fact.

However, there is one issue: we won't allow names to have spaces or other characters. Is there a reason for so? Given they are always strings, we can probably lift this limitation. If we want to keep it, then we will also need to prune the name.

---

Summary of changes:

• Break sidebar in two areas: references and attachments
• Both areas are drag and dropable and dropping a file no longer open up the modal
• We will have an operation to rename the file
• We will have an operation to copy the original location for references
• We should either relax the constraints on the name or automatically infer a valid name from the file (applies to drag and drop and the modal)
• We will mention attachment on the radio actions

[josevalim]

As discussed with @jonatanklosko, we will call the sections: "References" and "Attachments".

[hugobarauna]

I like this direction!

So, If I got it right, we'd have the following "concept hierarchy."

• Files
  • References
  • Attachments

Also, if I got it right, File Attachments are shared by all notebooks inside the same directory. But File References are not shared; they're per notebook. Is that right?

I want to check if one is shared and the other is not.

[jonatanklosko]

@hugobarauna all files are per notebook. When one notebook has an attachment, other notebooks in the same directory don't see that file, unless you go to "Add file -> From unlisted" :)

[hugobarauna]

@jonatanklosko perfect, that's even better! =D

[zachallaun]

I wonder if some inspiration could be taken from ls -l, which shows symlinks as:

data.csv -> /home/user/foo/some_data.csv
local_data.csv
pic1.jpg -> /home/user/bar/pic.jpg
pic2.jpg -> /home/user/baz/pic.jpg

At a glance, the symlinks are obvious (visually different/longer lines) and give additional useful context, the location/name of the original file.

Adapting that to this use-case, you could imagine something like:

📥 data.csv -> /home/user/foo/some_data.csv
📥 local_data.csv
☁️ pic1.jpg -> *team_storage/*pictures/pic1.jpg
📥 pic2.jpg -> https://github.com/.../some_picture.jpg

(Forgive my use of emoji here, couldn't get anything else to line up 😝).

Basically you could use a "downloaded" icon (📥) to indicate something that's either physically present in files/ or cached locally, and a cloud icon (and possible grayed out a bit) to indicate something that's available but hasn't been pulled down.

This might allow all files to be visible in a single list sorted by the name you might actually reference in your notebook, which is usually what you'd care about (I think), while still making it obvious which are downloaded or otherwise. (You could also imagine, for links to files on the drive, some kind of X some_missing_file.csv -> ??? listing for missing files and a way to re-associate a file; useful for sharing notebooks.)

[josevalim]

At a glance, the symlinks are obvious (visually different/longer lines) and give additional useful context, the location/name of the original file.

I am not sure if we should do it by default but you raise a good point. Today it seems it is not easy to recover the original reference. We should perhaps have another action in the menu called "copy original location".

a cloud icon (and possible grayed out a bit) to indicate something that's available but hasn't been pulled down.

The issue is that you don't need to necessarily pull it down to disk, so we may not want to call attention to this state. If you are working with large data, you would rather stream and process it, than ever download it to disk.

You could also imagine, for links to files on the drive, some kind of X some_missing_file.csv -> ??? listing for missing files and a way to re-associate a file; useful for sharing notebooks.

We already do this!

---

@jonatanklosko, if we officially call it "Attachments", should we change the phrasing here?

• "Copy file contents to the notebook files directory" -> "Save file contents as an attachment in the notebook files directory"

• "Download URL contents to the notebook files directory" -> "Download URL contents as an attachment in the notebook files directory"

?

Let me know what you think, so I update the issue and we can act on it.

[jonatanklosko]

You could also imagine, for links to files on the drive, some kind of X some_missing_file.csv -> ??? listing for missing files and a way to re-associate a file; useful for sharing notebooks.

We already do this!

@josevalim we don't, the exception we handle is for allowing access. Showing an indicator whether the file exists is tricky, because we would need to monitor the file system for changes to be accurate.

if we officially call it "Attachments", should we change the phrasing here?

Sounds good, I would try to make it shorter given it's a radio option, but the best I can think of is dropping "contents", so "Save file as" and "Download file as".

[josevalim]

Thanks for clarifying. I have added a summary of changes to the issue.

[hugobarauna]

Should we also update the copy inside settings accordingly? I mean, instead of "File systems", "File storages"?

[josevalim]

This will be done by Ale since he is working on those areas.