[FEATURE] Allow multiple instances of BDFR to run simultaneously.

[StuffonGithub]

• I am requesting a feature.
• I am running the latest version of BDfR
• I have read the Opening an issue

Description

As requested. 😊
Running another instance of BDFR gives this error:

PermissionError: [WinError 32] The process cannot access the file because it is being used by another process: 'C:\\Users\\USERNAME\\AppData\\Local\\BDFR\\bdfr\\log_output.txt' -> 'C:\\Users\\USERNAME\\AppData\\Local\\BDFR\\bdfr\\log_output.txt.1'

Would be nice if we can run several instances of BDFR at the same time. Thank you!

[gPoupon]

Might just be a Windows only issue. I've run multiple instances from a Linux command line simultaneously without issue.

[Serene-Arc]

@tayyyTF it is only a Windows issue. We spoke about it in a comment chain in #304 but Linux lets you have multiple processes writing to a single file. It makes the log file roughly useless but it doesn't throw an error like on Windows. It's the Linux philosophy: you can do whatever you want, even if it's a stupid idea.

[Serene-Arc]

@StuffonGithub would you be able to post more of the logfile that causes this crash? I need the exact line of code at which it occurs to catch it gracefully.

EDIT: Nevermind I was able to pull it from previous test runs

[Serene-Arc]

@StuffonGithub It should be fixed now. If you use the --log option to specify the log file location, it won't throw you any errors

[StuffonGithub]

@Serene-Arc Oh sorry for missing your request earlier. But glad to know you got it solved nevertheless. 😄

And wow, thanks for the quick implementation! Wasn't expecting you to work on it right away. Haha, thank you. ❤

I updated my BDFR but I'm getting a PermissionError if I include a --log option in the command. I tried all my available drives and tried different directories. I also ran the PS terminal as admin. But they all gave me this error:

PermissionError: [Errno 13] Permission denied: 'directory:\\logs'

And not sure if it's related though, but I seem to still have no BDFR folder in my %APPDATA% directory despite the permission error I pasted on the main topic linking a supposed folder location. Maybe because of limited perms but it just continues the download without bothering to create the log file?

Additionally, I know I haven't tested it yet but does the --log option automatically create a different log file per run? Or do I have to specify a different folder each time? I'm only asking because when I create new .ps1 files, I only tend to just edit the {USERNAME} or the {SUBREDDIT} part. It would be nice if there's a log option that will automatically create the log based on whichever source it's pulling the info from. Or maybe, a simpler option would be able to give logs an auto-naming scheme of sorts like log_{DATE}_{USER/SUBREDDIT} or something to that effect. 😁

Thanks again for the really quick turnaround on this!

[Serene-Arc]

For the first issue, try ./log.txt or something to that effect. It appears that it's not resolving it for some reason to the current folder like it should.

It's odd that there's still no folder. If you go through the logfile, there should be an entry that tells you where it's loading the configuration from. It might tell you where it is. The last place it checks is the default one provided with the package, but if that one is still being used then I'll need to look into why it's not creating the proper folder.

No, so far the idea is that the log file is the same name. That's the only way we can make it rollover properly. Normally it does things like log_output.txt.1 and log_output.txt.2 and so on, then starts overwriting once it gets to a certain point. If we'd named them based on the timestamp for example, it actually would have solved this issue we're talking on, but it would make it harder to clean up old logs. Since the logfile is quite verbose, there was a small danger of the logs taking up more and more hard drive space. For instance, a single run of a session generated a 7 MB logfile, so it adds up quickly.

[StuffonGithub]

Thanks! ./log.txt works. Good thing I can just create a sub-folder in my .ps1 directory and have all the logs stored through there.

Yeah, I still can't find any BDFR folder anywhere outside of where I have stored the BDFR files. I checked the newly created log though and it doesn't seem to show where it's loading the config from.

[2021-04-27 09:44:25,547 - bdfr.downloader - DEBUG] - Setting maximum download wait time to 120 seconds
[2021-04-27 09:44:25,548 - bdfr.downloader - Level 9] - Created download filter
[2021-04-27 09:44:25,548 - bdfr.downloader - Level 9] - Created time filter
[2021-04-27 09:44:25,548 - bdfr.downloader - Level 9] - Created sort filter
[2021-04-27 09:44:25,548 - bdfr.downloader - Level 9] - Create file name formatter
[2021-04-27 09:44:25,548 - bdfr.downloader - DEBUG] - Using unauthenticated Reddit instance
[2021-04-27 09:44:25,549 - bdfr.downloader - Level 9] - Created site authenticator
[2021-04-27 09:44:25,550 - bdfr.downloader - Level 9] - Retrieved subreddits
[2021-04-27 09:44:25,550 - bdfr.downloader - Level 9] - Retrieved multireddits
[2021-04-27 09:44:27,469 - bdfr.downloader - DEBUG] - Retrieving submitted posts of user <USERNAME>
[2021-04-27 09:44:27,470 - bdfr.downloader - Level 9] - Retrieved user data
[2021-04-27 09:44:27,470 - bdfr.downloader - Level 9] - Retrieved submissions for given links
[2021-04-27 09:44:30,551 - bdfr.downloader - DEBUG] - Attempting to download submission <POSTID>
[2021-04-27 09:44:30,552 - bdfr.downloader - DEBUG] - Using Imgur with url <DIRECT LINK>
[2021-04-27 09:44:31,441 - bdfr.downloader - DEBUG] - File <~/redditor/file.jpg> already exists, continuing

I agree that it can tend to bloat the storage with that kind of log naming. Interestingly, it made me check since I never bothered with it but, one of the log folders I had for BDFRv1 is apparently taking up over 10GB of my storage space. Haha. It is evidently inefficient. Nevertheless, I think it's more than fine the way it is now. I'm just realising I can simply rename the log file as .txt or .txt and with the way your logging works, it will work flawlessly and will be easily understood.

[Serene-Arc]

Oops, I forgot that won't work anymore. Because we have to load the number of backup logs from the configuration file, the logfile isn't initialised when that logging message is broadcast. If you run it the BDFR with the -v flag, it will print it out on your screen.

[StuffonGithub]

The - v flag worked and it outputted the config location but weirdly enough, when trying to directly access the printed directory, I'm getting this:

Haha but tbh, it's not a problem for myself. I just thought you'd be interested to know it's happening.

[Serene-Arc]

uhh okay, that's a new one. Windows isn't my forte but thanks, I'll look into it