Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store albumtypes multi-value field consistently in-DB & in-tag, preventing continual file re-tagging #4582

Merged
merged 9 commits into from
Mar 8, 2023

Conversation

jpluscplusm
Copy link
Contributor

@jpluscplusm jpluscplusm commented Dec 12, 2022

Description

This PR proposes a tentative fix for #4528, where everyone with a 1.6.0 version of beets and with any multi-value albumtypes stored in their DB (or acquired after upgrading to 1.6.0+, via new music or data source refreshes) will see Beets continually re-tagging all the files which have multiple albumtypes stored.

Additionally, as per the initial report in #4528, there's some interplay where even single albumtypes values (e.g. album) are getting turned into multi-value tags, and rewritten on every beets write.

I wrote the first version of the fix, which is included in the initial 2 commits of the PR branch. @mkhl was kind enough to review from a more Python-aware perspective, and from a place of having an existing Beets library, and I've realigned the PR branch with their known-better version. (I /don't/ have such a library (I encountered this issue on first setup), so I'm inclined to trust their code more!)

I strongly suggest that this only be merged after tests are added that would have caught this problem before 1.6.0 was released. I've added no tests in this PR as the relevant parts of the Beets test suite quickly put me out of my depth. I really think it's worth getting a failing test in place, before merging this (I'll happily rebase, or anything else needed) as I /believe/ this bug will be causing any Beets user on 1.6.0+ to be re-tagging much/all of their library on each write.

Thanks for an awesome tool, @sampsyo - I hope this PR helps in some small way!

Fixes #4528

To Do

  • Documentation. (If you've add a new command-line flag, for example, find the appropriate page under docs/ to describe it.)
  • Changelog. (Add an entry to docs/changelog.rst near the top of the document.)
  • Tests. (Encouraged but not strictly required.)

@jpluscplusm jpluscplusm changed the title Store albumtypes multi-value field correctly in-DB, preventing continual file re-tagging Store albumtypes multi-value field correctly in-DB & in-tag, preventing continual file re-tagging Dec 12, 2022
@jpluscplusm jpluscplusm changed the title Store albumtypes multi-value field correctly in-DB & in-tag, preventing continual file re-tagging Store albumtypes multi-value field consistently in-DB & in-tag, preventing continual file re-tagging Dec 12, 2022
@CharlesSchimmel
Copy link

CharlesSchimmel commented Dec 14, 2022

I thought I'd give this build a try after stumbling over #4528 and I'm getting the following error when running anything:

Traceback (most recent call last):
  File "/home/user/.local/bin/beet", line 5, in <module>
    from beets.ui import main
  File "/home/user/.local/lib/python3.10/site-packages/beets/ui/__init__.py", line 33, in <module>
    from beets import library
  File "/home/user/.local/lib/python3.10/site-packages/beets/library.py", line 33, in <module>
    from beets import dbcore
  File "/home/user/.local/lib/python3.10/site-packages/beets/dbcore/__init__.py", line 19, in <module>
    from .db import Model, Database
  File "/home/user/.local/lib/python3.10/site-packages/beets/dbcore/db.py", line 29, in <module>
    from beets.dbcore import types
  File "/home/user/.local/lib/python3.10/site-packages/beets/dbcore/types.py", line 255, in <module>
    SEMICOLON_SPACE_DSV = DelimitedString(delim='; ')
TypeError: DelimitedString.__init__() got an unexpected keyword argument 'delim'

Removing the delim= fixes it, I'm pretty rusty with python though so I couldn't say if that's appropriate or what the intent is.

Thanks for working on this issue by the way!

Edit to add: I don't think removing that named parameter was appropriate as I'm getting the same original issue but with semicolons:

Milo - budding ornithologists are weary of tired analogies - nominy
  albumtype: album -> [
  albumtypes: ['album'] -> [; '; a; l; b; u; m; '; ]
Milo - budding ornithologists are weary of tired analogies - pure scientific intelligence (quantum)
  albumtype: album -> [
  albumtypes: ['album'] -> [; '; a; l; b; u; m; '; ]

@jpluscplusm
Copy link
Contributor Author

jpluscplusm commented Dec 14, 2022

@CharlesSchimmel Thank you - you've caught a mistake I made when incorporating some changes from another user, which passed through mostly because I haven't been able to add any tests to this PR, as mentioned above.

I've force-pushed a fix: please do try again and confirm it's working with your media library!

@CharlesSchimmel
Copy link

Thanks! I've built the change, and I'm still getting the original issue except with semicolons:

Milo - So The Flies Don't Come - song about a raygunn (an ode to Driver)
  albumtype: album -> [
  albumtypes: ['album'] -> [; '; a; l; b; u; m; '; ]

@mkhl
Copy link

mkhl commented Dec 14, 2022

@CharlesSchimmel that looks like your tags might have been degraded already; could you check the TXXX/MusicBrainz Album Type frame of one of those files?
mid3v2 /path/to/the.mp3 | grep '^TXXX=MusicBrainz Album Type=' should do that if you have mutagen installed

that file you mentioned should have the type album but i think it might be something like [ / ' / a / l / b / u / m / ' / ] instead

@CharlesSchimmel
Copy link

@mkhl Thanks, I think you're correct. Looks like a lot of my files have releasetype of a;l;b;u;m or [;';a;l;b;u;m;';]

I beet updated a couple files with correct releasetypes and this patch seems to have worked.

@jpluscplusm
Copy link
Contributor Author

jpluscplusm commented Dec 15, 2022

@CharlesSchimmel Thank you for checking further! Did you by any chance mean the albumtypes, not releasetypes, tag?

More generally, this suggests we need a Benevolent Dictator's view on the already-corrupt situation: do we need to recover if the tags are now wrong and also (as I /suspect/ was the case with @CharlesSchimmel's library) the in-DB entries have been corrupted as well?

I might very well be misunderstanding the scope/frequency of the underlying Issue's affected users, but is /automatically/ repairing the albumtypes/type data of folks who've upgraded to 1.6.0+ possible and/or desirable?

@CharlesSchimmel
Copy link

@jpluscplusm Correct, that's what I meant, the releasetype tag seems to be a mirror of the albumtype tag.

Personally, I'm re-importing the problem files with beet import -L 'albumtype::^\[', which only works for tracks in the MusicBrainz db.

@mkhl
Copy link

mkhl commented Dec 15, 2022

I might very well be misunderstanding the scope/frequency of the underlying Issue's affected users, but is /automatically/ repairing the albumtypes/type data of folks who've upgraded to 1.6.0+ possible and/or desirable?

i think that should be out of scope. beets 1.6.0 writes problematic tags to the files but keeps the database intact, and this fix works fine if the database is intact.
to also "corrupt" the database one would have to update it from files with problematic tags, and for those cases reimporting the files will restore the correct data in the database anyway.

@jpluscplusm
Copy link
Contributor Author

PR branch rebased against master, following the addition of the expected test failure in #4586.

Another commit added, changing the test to reflect that it shouldn't be failing after this PR is merged [to the best of my pythony- and beetsy- abilities!]

@wisp3rwind
Copy link
Member

Some test seem to be failing due to not expecting a list-valued albumtypes, could you have a look into this? Probably, this only requires slight modification to those tests. Thanks!

@JOJ0
Copy link
Member

JOJ0 commented Feb 10, 2023

@jpluscplusm would it help if someone would explain how those failing tests could be modified to pass? Is there anything else that would help to kickstart this PR? I think it's a nasty bug and it covers a lot of ground. Thanks for your heroic efforts so far!

@jpluscplusm
Copy link
Contributor Author

@JOJ0 Hello 👋 Yes, any assistance any at all would be very welcome.

I feel very unsure in proposing changes beyond the current state because:

  • I lack the project-knowledge to know how completely we test, and how to test appropriately
  • I'm very concerned about the disparate audiences facing this bug, across axes like how recently they upgraded to 1.6.0+, and if their tags have already/ever been "infected" by this bug.

I really really don't want to be responsible for proposing a "fix" that -- for instance -- only deals with my newcomer/fresh-library use case, whilst leaving long-time users in a potentially worse place.

Any assistance or guidance is very very welcome!

@JOJ0
Copy link
Member

JOJ0 commented Feb 21, 2023

Hi, first of all appologies for letting you hang there for another ten days...

Regarding your worries: At the end of the day, it's the beets project who is responsible because they merged something into the main code. If that breaks something, a contributor can't be made responsilbe for that. Certainly we will be happy if you would still support us and help moving forward with analysis and fixing. My personal opinion is: We are in this together and I only can be grateful that people like you are brave enough to submit a fix for such a tricky problem.

Speaking of, last weekend I fiddled around a little with the currently failing unittests wisp3rwind mentionend above. I was not successful so far but will continue working on it. Did you give it a try yet?

Also I'm still trying to wrap my head around "the problem" as a whole. What I understand so far is:

  • "The problem" is only existing with the albumtypes field which has nothing to do with the albumtype field. They don't interact with each other.
  • "The problem" is only existing when MusicBrainz is used as the metadata source. It's not present when the Discogs plugin or other metadata source plugins is in use. The don't touch the albumtypes field, they use the albumtype field only.
  • This PR fixes "the problem" when data that is correctly existing in the database.
  • The test wisp3wind provided in tests: add a (xfailing) test for issue #4528 #4586 currently proves in master that "the problem" is existing and thanks to your latest commit will prove that "the problem" is fixed with this PR.
  • What will be fixed is the correct writing of the albumtypes field to the file's tag, as well as the correct reading from the tag and writing to the beets database.
  • What will not be fixed is the "magically" correcting of already messed up file tags. For example because a user issued beet update which reads from tag and writes to the database.

Please correct me if I'm technical wrong with any of this or give me your own opinion if you understood differently.

My conclusion now would be:

  • This PR can be merged and it's unlikely that it's bringing more harm.
  • The problem that is persisting is how to fix already messed up database entries (because of beet update).
  • In my opinion a "magic" fix is almost impossible, especially in cases where a user for some reason had issued beet write and beet update several times! There is no way to clean up this mess, except by re-importing and overwriting the albumtypes field with fresh information. See my following post with an example of a quite messed up field that most probably can't be autofixed! Also I'm supporting @mkhl's opinion that this actually is out of scope: Store albumtypes multi-value field consistently in-DB & in-tag, preventing continual file re-tagging #4582 (comment)

Thus, I have the following ideas of how this could be tackled further:

  • Fix the existing tests to pass with the new code,
  • include a detailed tutorial for users on:
    • how a possible "infection" can be detected,
    • how a reimport can be issued to fix the problem,
    • and how it can be verified that the problem won't pop up again in the future.
  • then merge this PR.

Fortunately proper hints around such a tutorial are provided here already: #4582 (comment)

Where would a tutorial like this be placed is what I'm unsure about: Release notes? Add as the final answer in the existing github issue to this PR? Somewhere in the documentation? All of this places, just to make sure ?!

Feel free to give your opinion on these thoughts.

@JOJ0
Copy link
Member

JOJ0 commented Feb 21, 2023

Some database examples from my dev environment. We see that the releases on label "Music man records" are pretty messed up and I don't think that there is any possible auto-fix other then suggesting to reimport:

image

The second "Astrophonica" release and the "Human Imprint" one are, as the uuid-style mb_albumid reveals, tagged using MusicBrainz, the rest is tagged from Discogs. We see that the albumtypes field is empty on those fields. The "Music Man" Records "I think" where tagged with MusicBrainz before and then retagged with Discogs. If that is not true, then my assumption about "the problem is only existing with MB-tagged albums" is not true!

@JOJ0
Copy link
Member

JOJ0 commented Feb 21, 2023

To prove my assumption that Discogs tagged files don't touch the albumtypes field let's do the following experiment. Let's focus on the 3 Music Man Records releases. I just reimported the last one from MusicBrainz (catalognum: MMCD026). We see that albumtype and albumtypes look correctly - The messed up albumtypes field from my previous post is now fixed:

image

Now I reimport from Discogs and get:

image

MM129A is the Catalog number we are taking about and we see that the Discogs metadata source plugin did not care about the albumtypes field, it left it as-is - it still shows what MusicBrainz had put in there before. What it did write is the albumtype field, it now contains 12".

That proves my assumption that at least this source plugin, doesn't touch albumtypes and I suppose Spotify, Deezer plugins and so on, might as well ignore albumtypes BUT that remains to be found out.....@arsaboo you might be my person to have a look into the Spotify plug. Does it touch albumtypes? Cheers!

Final note: Certainly in such a situation a writing to file tags using beet write and then beet update prior to the fix in this PR would still lead to a messed up field as we have seen in my first "picture post" and actually see in theses last pictures with record MM129C.

Hope that clarifies and doesn't confuse further :-)

@mkhl
Copy link

mkhl commented Feb 21, 2023

First of all: Thank you!

Also I'm still trying to wrap my head around "the problem" as a whole. What I understand so far is:

Let me preface with:
My understanding is that there are conflicting forces between

  • the information beets stores about music files in its database
  • the information beets reads from music files
  • the information beets fetches from the metadata source
  • "The problem" is only existing with the albumtypes field which has nothing to do with the albumtype field. They don't interact with each other.

I don't think that's technically correct.
Beets treats these fields as independent in the database, but in music files these are (currently) represented by a single tag, the "MusicBrainz Album Type" TXXX comment.
Example: TXXX=MusicBrainz Album Type=album / live (printed by mid3v2)

I also believe they are separate in the MusicBrainz API, but beets tries to derive the albumtype when reading files as being the first element of albumtypes. This can lead to the secondary problem (that this PR doesn't try to address) where beet write tries to modify the albumtype every time.

  • "The problem" is only existing when MusicBrainz is used as the metadata source. It's not present when the Discogs plugin or other metadata source plugins is in use. The don't touch the albumtypes field, they use the albumtype field only.

I believe that's correct.

  • This PR fixes "the problem" when data that is correctly existing in the database.

Yes it does.

  • What will be fixed is the correct writing of the albumtypes field to the file's tag, as well as the correct reading from the tag and writing to the beets database.

I believe that writing to the file's tag won't be modified, only reading it and matching it against the database get fixed.

* _What will not be fixed_ is the "magically" correcting of already messed up file tags. For example because a user issued `beet update` which reads from tag and writes to the database.

Yes, and we seem to agree that it doesn't need to.

Where would a tutorial like this be placed is what I'm unsure about: Release notes? Add as the final answer in the existing github issue to this PR? Somewhere in the documentation? All of this places, just to make sure ?!

I'd recommend adding it as a page in either the wiki or the documentation, then linking to that page from this PR, from the issue it fixes, and prominently in the release notes.

@Sciencentistguy
Copy link
Contributor

In my opinion a "magic" fix is almost impossible, especially in cases where a user for some reason had issued beet write and beet update several times! There is no way to clean up this mess, except by re-importing and overwriting the albumtypes field with fresh information.

This can be done quite easily, if you don't mind effectively retagging your whole library in the process: beet update, beet mbsync, beet write

@JOJ0 JOJ0 self-assigned this Feb 25, 2023
@JOJ0
Copy link
Member

JOJ0 commented Feb 25, 2023

Some test seem to be failing due to not expecting a list-valued albumtypes, could you have a look into this? Probably, this only requires slight modification to those tests. Thanks!

I finally found time to look into this and I think I have a solution. There is 2 things we need to include into this PR:

  • The new database type for albumtypes now returns a list. We need to make sure that the albumtypes plugin uses it that way. Up to now it did the splitting from ;-separated string to list itself.
  • All the failing unittests for the albumtypes plugin use the method _create_album() which tries to do the translation from list to string itself. We just need to make sure it uses the list we are passing it already.

Copy link
Member

@JOJ0 JOJ0 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've pushed a commit to this PR's branch with the fix I just described. Sorry for hijacking, I hope that's ok! :-)

@@ -55,7 +55,7 @@ def _atypes(self, item: Album):
bracket_r = ''

res = ''
albumtypes = item.albumtypes.split('; ')
albumtypes = item.albumtypes
Copy link
Member

@JOJ0 JOJ0 Feb 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

item.albumtypes will always return a list already,

@@ -106,6 +106,6 @@ def _set_config(self, types: [(str, str)], ignore_va: [str], bracket: str):

def _create_album(self, album_types: [str], artist_id: str = 0):
return self.add_album(
albumtypes='; '.join(album_types),
albumtypes=album_types,
Copy link
Member

@JOJ0 JOJ0 Feb 25, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and also when creating the test-album we can rely on the DelimitedString type to translate to a ;-delimited string for us.

@JOJ0
Copy link
Member

JOJ0 commented Feb 25, 2023

Hi @mkhl, thanks so much for correcting and clarifying, that was tremendously helpful for moving on with this topic. Very much appreciated :-)

As for the location of the "manual fixing advice" I would prefer to add it to the docs instead of the wiki. The wiki is a place that I have to admit I personally haven't looked much into as a beets user. Now that I look at it, there is a lot of interesting and developer-centric things in there (Shame on me for not realising this earlier :-~) :-) )

So one place I would like to add it in the docs is the already mentionend albumtypes plugin docs . A link to the howto would suffice here. But also for users that don't use that plugin a properly fixed albumtypes field in the database is essential. Not sure where that information should be stated...

So I'm still looking for a place in the docs where a tutorial containing of 3 to 4 steps should live......maybe the wiki is a good place for that afterall. All the places we would like to mention it could then link to the wiki. Do you have an idea where you would see such a tutorial? Maybe create something like a "known issues" chapter and add it there?

@JOJ0
Copy link
Member

JOJ0 commented Feb 25, 2023

Sanity Check & Fix for #4528

The resolution for a bug that was introduced in beets 1.6.0 requires manual intervention. The issue was tracked in issue #4528. The bug usually shows itself by trying to repeatedly populate the albumtype field to files on disk when issuing beet write. The issue has been fixed in PR #4582.

When beet update is used to read the albumtype tag back into the database, things get worse and broken data is saved into the database. This is the situation this tutorial addresses.

Have I been infected?

If you've never used beet update you might as well get not results here and are good to go. If you get results, this means that these albums need to be fixed in the database:

beet ls -a 'albumtypes::^\[' data_source:musicbrainz

Fix

The quickest way to fix is by using the mbsync plugin, it will make sure the `albumtypes field is overwritten with data fetched from MusicBrainz:

beet mbsync 'albumtypes::^\[' data_source:musicbrainz

Alternatively, or if you don't use the mbsync plugin, a full reimport of the albums in question would also fix the issue:

beet import -L 'albumtypes::^\[' data_source:musicbrainz

Fix Non-MusicBrainz-tagged albums

There might be albums in your library where initially MusicBrainz was used as the metadata source and later on a retag with another metadata source plugin (Discogs, Spotify, ...) was used. The existence of such broken albums can be verified with:

beet ls -a 'albumtypes::^\[' ^data_source:musicbrainz

The best solution for these broken database fields is to "zero them out". This command will ask for confirmation. Answer yes to really do it:

beet modify -a  'albumtypes::^\[' ^data_source:musicbrainz albumtypes=""

Verify

To verify whether the issue is fixed:
beet write -p

Then try a real write to an album:
beet write <albumname>

Read the tags from disk into the db again:
beet update <albumname>

and have a look at the albumtypes field. It should show a list of strings delimited with ; characters:
beet ls -f '$album: $albumtypes'

@JOJ0
Copy link
Member

JOJ0 commented Feb 25, 2023

In my opinion a "magic" fix is almost impossible, especially in cases where a user for some reason had issued beet write and beet update several times! There is no way to clean up this mess, except by re-importing and overwriting the albumtypes field with fresh information.

This can be done quite easily, if you don't mind effectively retagging your whole library in the process: beet update, beet mbsync, beet write

Hi thanks for the hint. I have to admit I didn't have the mbsync plugin on my radar since I don't use. But yes, essentially it does the same as the desired part of reimporting: It overwrites the albumtypes database field with fresh data from MusicBrainz. Thanks again and as you see above, mbsync will be suggested as the proper fix in above's tutorial :-)

@jpluscplusm
Copy link
Contributor Author

. Sorry for hijacking, I hope that's ok! :-)

1000% ok! Please feel to take this PR anywhere the project needs ... including closing and starting again if that'd best serve the purpose of the fix! I'm just happy if my initial code or thoughts were at all helpful :-)

@judemille
Copy link

Potentially related: I do still notice an issue, where beets will keep overwriting the albumtype field in manners such as album -> live for no good reason. This means there's still continual re-tagging going on, though now not of my whole library.

@JOJ0
Copy link
Member

JOJ0 commented Mar 16, 2023

Hi @judemille, indeed there seems to still be an issue. I can confirm that 'live' doesn't seemt to be recognized, while writing as well as while reading:

$ beet ls -a -f '$album: $albumtypes' cocorosie live
CocoRosie Live (Spaziale Festival): 
$
$ beet write cocorosie live                       
CocoRosie - CocoRosie Live (Spaziale Festival) - 01 - CocoRosie Live Spaziale Festival, Turin, Italy 19.7.2010 (0000/)  [] f:MP3-320kCBR a:70-01/ASIS |EMPTY
  albumtype:  -> live
$
$ beet ls -a -f '$album: $albumtypes' cocorosie live
CocoRosie Live (Spaziale Festival): 

@JOJ0
Copy link
Member

JOJ0 commented Mar 16, 2023

Ah, there is one flaw in my test above. I check for the plural albumtypes field whereas beet write definitely writes the singular one. Checking again and will hopefully soon open a new issue describing the remnaining problem...

@mkhl
Copy link

mkhl commented Mar 16, 2023

FWIW this is the issue I was referring to above:

I also believe they are separate in the MusicBrainz API, but beets tries to derive the albumtype when reading files as being the first element of albumtypes. This can lead to the secondary problem (that this PR doesn't try to address) where beet write tries to modify the albumtype every time.

If an album has multiple types the MusicBrainz API seems to indicate that one of those is the "primary" type, and beets stores that in the db as the albumtype, but parsing the ID3 tag of the file it seems like it just uses the first element of albumtypes as the albumtype.

Writing the albumtype of those files doesn't do anything so beet write will always report trying to write it.

@JOJ0
Copy link
Member

JOJ0 commented Mar 16, 2023

@mkhl yes that is super helpful! Thanks! I have to admit I managed to forget already that we are still not through with this problem and that you already explained that above. Sorry! I will try to write a feature proposal in a new issue and then let's discuss over there what ways we could imagine to get rid of this remaining issue.

@mkhl
Copy link

mkhl commented Mar 16, 2023

No worries, glad the info is helpful!

@JOJ0
Copy link
Member

JOJ0 commented Mar 17, 2023

I used your words @mkhl, hope that is ok! Thanks again! #4715 (comment)

@taylorthurlow
Copy link

taylorthurlow commented Jan 14, 2024

@JOJ0 Your comments (and guide on solving the issue) has been helpful but I think something has changed that has broken your query for square brackets:

beet ls -a 'albumtypes::^\[' data_source:musicbrainz

I get a bunch of results for this query, but after manually inspecting the library database file, I am convinced that [ and ] do not occur anywhere in my albumtypes tag. I'm on the most recent master branch commit as of right now (828c6e7). Perhaps there is some deserialization (into a list) going on before the regex is matching the string, and those brackets come out of nowhere? Just an idea.

I am still battling this tag (and I think a few others) always wanting to write. Looks like #4715 is the ongoing discussion of this one.


EDIT: Hmmm... getting rid of the data_source:musicbrainz part of the query actually causes the search to return zero results. Bug?

@JOJ0
Copy link
Member

JOJ0 commented Jan 15, 2024

@taylorthurlow If you dont have square brackets in your albumtypes, then indeed #4715 is the way to go.

@jpluscplusm jpluscplusm deleted the jcm_fix_albumtypes branch January 31, 2024 18:32
@MarkTerMors
Copy link

Hi
I still have problems fixing flac files. I have managed to fix all my mp3 files as far as I can tell.

mark@musicbox:~$ beet write album:Recycler
ZZ Top - Recycler - Concrete and Steel
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - Lovething
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - Penthouse Eyes
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - Tell It
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - My Head’s in Mississippi
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - Decision or Collision
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - Give It Up
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - 2000 Blues
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - Burger Man
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
ZZ Top - Recycler - Doubleback
  albumtype: a -> album
  albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
mark@musicbox:~$ metaflac --list /media/music/Z/ZZ\ Top/Recycler/01\ Concrete\ and\ Steel.flac | grep TYPE 
    comment[23]: RELEASETYPE=a
    comment[24]: RELEASETYPE=l
    comment[25]: RELEASETYPE=b
    comment[26]: RELEASETYPE=u
    comment[27]: RELEASETYPE=m
    comment[28]: MUSICBRAINZ_ALBUMTYPE=a
    comment[29]: MUSICBRAINZ_ALBUMTYPE=l
    comment[30]: MUSICBRAINZ_ALBUMTYPE=b
    comment[31]: MUSICBRAINZ_ALBUMTYPE=u
    comment[32]: MUSICBRAINZ_ALBUMTYPE=m

@JOJ0
Copy link
Member

JOJ0 commented Mar 2, 2024

How did you fix it? Seems like your db still has broken data. Have you had a look into the db directly with sqlite client (as a last resort solution)

@MarkTerMors
Copy link

MarkTerMors commented Mar 3, 2024

There are no a,l,b,u,m or single letter entries listed if I do:
sqlite> select albumtype from item;
Beets seems to be able to write the correct tags to MP3 files but not to FLAC files.
it tries as evidenced by the
albumtype: a -> album
albumtypes: ['a', 'l', 'b', 'u', 'm'] -> album
but fails to properly update the tags.

@JOJ0
Copy link
Member

JOJ0 commented Mar 5, 2024

Oh sorry for not reading closely enough.

Interesting...which OS? I'll see if I can repro with FLAC files on macOS

@MarkTerMors
Copy link

Debian 12.
Further diagnosis reveals that if I remove the albumtype and albumtypes fields entirely the come back (incorrectly) on re-import.

@JOJ0
Copy link
Member

JOJ0 commented Mar 5, 2024

Hi again,
I can't repro, at least on macOS Catalina, editing the multi-value field albumtypesseems to work as designed:

$ beet edit dark side of italo body heat
Fockewulf 190 - The Dark Side Of Italo Disco - Body Heat (Vocal) - src:Discogs |Italo Disco, New Wave, Disco
  albumtypes: LP; Compilation -> LP; Compilation; NewAlbumtype
➜ continue [E]diting, Apply, Cancel? a
$ mutagen-inspect "/Users/jojo/Music/dev-beets/Compilations/The Dark Side of Italo Disco/01 Body Heat (vocal).flac" | grep -i -e type
RELEASETYPE=LP
RELEASETYPE=Compilation
RELEASETYPE=NewAlbumtype
MUSICBRAINZ_ALBUMTYPE=LP
MUSICBRAINZ_ALBUMTYPE=Compilation
MUSICBRAINZ_ALBUMTYPE=NewAlbumtype

There are no a,l,b,u,m or single letter entries listed if I do:
sqlite> select albumtype from item;

Did you also check select albumtypes from item `?

@JOJ0
Copy link
Member

JOJ0 commented Mar 6, 2024

@MarkTerMors If all else fails and you dont use any plugin that requires albumtype, give this radical approach a try: #5075

@MarkTerMors
Copy link

Given that all else has failed, I deleted everything and set up a new venv with beets installed in it. It works now but I have no idea what was wrong.

@JOJ0
Copy link
Member

JOJ0 commented Mar 7, 2024

Hmm, glad you got it to work. Well, Python venvs sometimes can be broken as hell. I had weird things with them often, where recreating just helped.

daniele-athome pushed a commit to daniele-athome/beets that referenced this pull request Aug 14, 2024
Store `albumtypes` multi-value field consistently in-DB & in-tag, preventing continual file re-tagging
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

albumtypes is converted from a string to a list of individual characters
10 participants