Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

When I set mounted gdrive as download folder for torrent client, nothing is downloaded #169

Closed
Konstantinusz opened this issue Dec 4, 2016 · 12 comments
Closed

When I set mounted gdrive as download folder for torrent client, nothing is downloaded #169

Konstantinusz opened this issue Dec 4, 2016 · 12 comments

Comments

@Konstantinusz
Copy link

I have very low space on VPS, so I tried GDriveFS for mounting my google drive to get some extra disk space. Everything went fine on my Arch Linux box, I was able to copy files to and from the mounted drive. Unfortunately when I have set it as a download directory for my Transmission torrent client, downloads doesn't start. What can I do to make it capable to be download directory of the torrent client?
@dsoprea
Copy link
Owner

dsoprea commented Dec 4, 2016 via email

@Konstantinusz
Copy link
Author

I have set the download directory of the torrent client Transmission to a directory on the remote disk mounted by gdrivefs, but after a while all downloading torrent becomes "stopped" state, and also the drive becomes read-only. I had to umount and remount.

@dsoprea
Copy link
Owner

dsoprea commented Dec 8, 2016 via email

@Konstantinusz
Copy link
Author

Konstantinusz commented Dec 8, 2016 •
edited
Loading

It becomes buggy right after mounting. I tried to copy a 27 KB jpeg file to the drive, I got an error message in Midnight Commander, and a 4096 byte file appeared on the drive, which was the first 4096 byte of the 27 KB file. Then I set the GD_DEBUG environment variable to 1, and this is the log:
http://pastebin.com/KqbcxEwt

@dsoprea
Copy link
Owner

dsoprea commented Dec 8, 2016

A Google search for the error seems to indicate that this is a potential issue with the GD client and the OAuth client integration. I'm looking into it, further.

googleapis/google-api-python-client#299

@Konstantinusz
Copy link
Author

I see, and python2.7 is activated by the program and not python3, isn't it a problem?

@dsoprea
Copy link
Owner

dsoprea commented Dec 8, 2016

The project is only currently Python 2.7-compatible.

It remains to be seen whether the error in the logs (though I have not seen it before) is causing your problem. We've been using the same version of that library for more than two years.

Try to do two things. Maintain the order:

  1. Manually change this to force the "ERROR" log-level and show me if the log still shows those exceptions:

    https://github.com/dsoprea/GDriveFS/blob/master/gdrivefs/config/log.py#L9

    This will tell us if this is an actual exception rather than an ineffectual warning.

  2. I misread the Google project dependencies. Google just requires oauth2client version >1.5 and this error appears to be specific to >=4.0 . I'm speculating that you might just have a higher version on your system because of something else. In this case, you might want to put GDFS into a virtualenv (so it can use its version of the dependencies and whatever else needs a later one won't be disrupted)... Or vice-versa (if you can identify who requires the later version, you can push it (or them?) into a virtualenv where it'll leave the rest of the system alone.

    If you just want to change the system without getting into virtualenvs (not that it's complicated), just kill the current version(s) of oauth2client (run "pip uninstall oauth2client" repeatedly until it says that it's not there) and then install 3.0.0 (the latest version before 4.0.0; "pip install oauth2client==3.0.0"). See if things work better.

If you want something that definitely works in order to compare against, use the Vagrant definition that's included with the project to launch a preconfigured environment ("vagrant up" in the same path as the Vagrantfile).

If it's of any interest to you for reasons that I can't predict, these are the current versions of *oauth2client":

1.0beta5prerelease, 1.0b2, 1.0b3, 1.0b4, 1.0b6, 1.0b7, 1.0b8, 1.0rc1, 1.0rc2, 1.0rc3, 1.0, 1.1, 1.2, 1.3, 1.3.1, 1.3.2, 1.4, 1.4.1, 1.4.2, 1.4.3, 1.4.4, 1.4.5, 1.4.6, 1.4.7, 1.4.8, 1.4.9, 1.4.10, 1.4.11, 1.4.12, 1.5.0, 1.5.1, 1.5.2, 2.0.0, 2.0.0.post1, 2.0.1, 2.0.2, 2.1.0, 2.2.0, 3.0.0, 4.0.0)

@tvierling
Copy link
Contributor

Important sidenote: Using gdrivefs for large amounts of extra space that is randomly accessed (e.g., torrents) is not going to work as well as you'd hope.

gdrivefs works by caching client-side copies of data files, which means that you don't really save disk space on files that are open and being accessed. Any file currently being transferred in or out by a torrent client is open and being accessed, so it will take up its full space on the local system's /tmp directory.

More importantly, torrents tend to be of relatively large (10s or 100s of MB) individual files. If a torrent has been sitting idle for long enough that its files drop out of cache, the first torrent client which connects for it will pull it back out by downloading the entire file back to the local system.

All this means that using gdrivefs for torrent download space is a very, very bad idea. It (and the protocol used by gdrive) is not designed for that usage pattern.

@Konstantinusz
Copy link
Author

Konstantinusz commented Dec 9, 2016 •
edited
Loading

I see, now I know why my 256 MB /tmp became full with no free space. Does the program make temp files with names like "0" "1" "2" etc? I have only 15 GB disk space, 512 MB RAM, 512 MB swap on the VPS, so /tmp is only 256 MB. Maybe this was the cause of the error too, /tmp was simply full with no space left. My other scripts also didn't worked well due to this.

@tvierling
Copy link
Contributor

The temp files are of the pattern /tmp/tmpXXXXXX/AAAAAAAAA_*; the AAAAAAAAA identifies your account, and the _... after it is gdrive's API ID for a given file.

@dsoprea
Copy link
Owner

dsoprea commented Dec 10, 2016 •
edited
Loading

@tvierling As far as random access is concerned, there is actually supposed to be support for downloading specific byte ranges. However, it didn't work properly when I tested it.

https://developers.google.com/drive/v3/web/manage-downloads

@dsoprea
Copy link
Owner

dsoprea commented Dec 23, 2016

@Konstantinusz I'm closing this issue as I've seen no activity on your part.

@dsoprea dsoprea closed this as completed Dec 23, 2016
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tvierling @dsoprea @Konstantinusz