Rename existing / downloaded files

[Anzuch4ng]

Hi everyone! Is it possible at all to rename already existing files based on their md5 from sqlite archive OR based on {id} field in existing filename from specific website (in this case danbooru)?

Before my config filename was containing only danbooru ID info
"[{id}].{extension}"

and now I have bunch of other tags:
"filename": "[{tags_artist[:2]:J /}] [{tags_copyright[:2]:J /}] [{tags_character[:2]:J /}] [{id}].{extension}"

so the question is upon re-running script - is it possible to detect already existing files in local folder and rename them based on current config filename settings?

My sqlite archive was set to
"archive-format": "{md5}",
"archive": "./db_danbooru.sqlite"

Long time ago I was using "DanbooruDownloader" and it had a feature, where you can scan local folder, and apply new naming scheme based on downloaded info from danbooru/sankaku etc, which obviously not exactly what I need right now, but was the closest thing to solve renaming of existing files.

[Anzuch4ng]

My bad it was not in DanbooruDownloader but in imgbrd-grabber and from what I see this feature is here and still works, but my questions still remains, if it's possible to achieve via gallerydl only.

[mikf]

It can be done, but it is not a builtin feature (yet), so you'll need to run some external code. Take a look at #5846, which is more or less the same issue.

"filename": "[{tags_artist[:2]:J /}] [{tags_copyright[:2]:J /}] [{tags_character[:2]:J /}] [{id}].{extension}"

OK, you'll need a more complex version of the script from #5846 (comment) to handle these formatting options. Give me a sec.

[mikf]

{
    "filename": "[{tags_artist[:2]:J /}] [{tags_copyright[:2]:J /}] [{tags_character[:2]:J /}] [{id}].{extension}",
    "metadata-path": "_path",
    "postprocessors": [
        {
            "name": "python",
            "event": "prepare",
            "function": "/tmp/rename.py:rename"
        }
    ]
}

import os
from gallery_dl import formatter

OLD_FMT = "[{id}].{extension}"
NEW_FMT = "[{tags_artist[:2]:J /}] [{tags_copyright[:2]:J /}] [{tags_character[:2]:J /}] [{id}].{extension}"

fmt_old = formatter.parse(OLD_FMT)
fmt_new = formatter.parse(NEW_FMT)

def rename(metadata):
    pathfmt = metadata["_path"]

    old = fmt_old.format_map(metadata)
    path_old = pathfmt.realdirectory + old

    if os.path.exists(path_old):
        new = fmt_new.format_map(metadata)
        path_new = pathfmt.realdirectory + new
        os.replace(path_old, path_new)

[Anzuch4ng]

Thanks a lot! It does work perfectly.

I was worried that I didn't had any metadata downloaded locally (apart from md5 sqlite archive), but it seems like I don't need them.

The only issue I've encountered so far is that if website filename contains any unsuported ascii characters it gives an error, for example it tries to write new name as "re:zero" where ":" are not replaced with underscores "_" automatically (and probably other unsuported characters as well)

Error from console: [danbooru][error] Unable to download data: OSError: [WinError 123] The filename, directory name, or volume label syntax is incorrect: '[umamusume] [mihono_bourbon_(code:glassage)_(umamusume) mihono_bourbon_(umamusume)] [8003528].png

[mikf]

Right, the function needs to apply clean_segment and clean_path to the filenames to replace invalid characters with _ or delete control characters:

import os
from gallery_dl import formatter

OLD_FMT = "[{id}].{extension}"
NEW_FMT = "[{tags_artist[:2]:J /}] [{tags_copyright[:2]:J /}] [{tags_character[:2]:J /}] [{id}].{extension}"

fmt_old = formatter.parse(OLD_FMT)
fmt_new = formatter.parse(NEW_FMT)

def rename(metadata):
    pathfmt = metadata["_path"]

    old = fmt_old.format_map(metadata)
    old = pathfmt.clean_path(pathfmt.clean_segment(old))
    path_old = pathfmt.realdirectory + old

    if os.path.exists(path_old):
        new = fmt_new.format_map(metadata)
        new = pathfmt.clean_path(pathfmt.clean_segment(new))
        path_new = pathfmt.realdirectory + new
        os.replace(path_old, path_new)

[Anzuch4ng]

And again thanks a lot for your help! Now it works flawlessly :)

[mikf]

I've added a rename post processor that basically uses this script internally. (17f5ba4)

With this, you could have renamed your files with just

gallery-dl --rename "[{id}].{extension}" URL