Ignore fields albumtype and albumtypes #5042

arogl · 2023-12-11T22:37:31Z

arogl
Dec 11, 2023

Is there a way for beets to ignore these fields in any of the commands?

I have looked in the the beets.db file and these fields do not contain any of the troublesome [] characters, I have the run a beet write, which goes through and changes all of the tags to ensure none of those characters exist, but when I run a beet write -p it wants to update all of those tags again.

I have used Mp3Tag, ExifTool, and MediaInfo to list all the tags of the file, but they do not display the above tags.

I have started to look at how I can use mutagen to list the tags, but have not put too much effort into that as yet.

This is not just for MusicBrainz sources either.

This is for both mp3 and m4a files.

I am running the latest git version.

Any help or suggestions?

DjSlash · 2023-12-12T08:14:23Z

DjSlash
Dec 12, 2023

If I understand correctly, beets tries to write the fields as [; '; a; l; b; u; m; '; ] ? In #4582, @JOJ0 wrote an excellent comment on how to check for this issue and what to do to fix it.

If that doesn't work, you need to provide more info, like the output of beet version and the output of beet write -p (preferably shortened).

I have started to look at how I can use mutagen to list the tags

With python3-mutagen (that's the package-name on Debian at least) installed, you can use mutagen-inspect [file].

2 replies

JOJ0 Dec 12, 2023
Maintainer

@DjSlash, thanks for trying to help and pointing to that issue but I think your suggestions are not entirely correct. No worries, it is a rather complex issue with a lot of side-effects and partly we have fixed things already.

So what I think is that @arogl does not face the issue you are describing but only the remains of that issue. @arogl I think this is what you are experiencing: #4715 (comment)

JOJ0 Dec 12, 2023
Maintainer

@arogl regarding that issue: Can you help me? I see that I have missed to finalize that report over there. There is some cli outputs missing in my description in #4715. Can you provide them? Just post them here. I'll add them to the issue. Many thanks!

Furthermore I suggest you try to read into what others and me discussed of possible solutions to this problem. I'm feling bad for not moving forward with that issue and it is on my mind. I just can't get into it - other priorities in life and besides those I'm trying to do just low hanging fruit stuff in the beets project (which that issue is not). Anyway, what I try to say: If you have brain capacity left, any help is very much appreciated!

arogl · 2023-12-13T02:58:38Z

arogl
Dec 13, 2023
Author

If I understand correctly, beets tries to write the fields as [; '; a; l; b; u; m; '; ] ? In #4582, @JOJ0 wrote an excellent comment on how to check for this issue and what to do to fix it.

If that doesn't work, you need to provide more info, like the output of beet version and the output of beet write -p (preferably shortened).

I have started to look at how I can use mutagen to list the tags

With python3-mutagen (that's the package-name on Debian at least) installed, you can use mutagen-inspect [file].

I had already applied the fix from here excellent comment

As commented, I run beets from git, updated daily.

beet info shows the fields with the correct information, however something is not updating those fields in the database or back to the files.

The assumption that this only applies to musicbrainz is incorrect, during my sql inspections I found it applied to nearly every data source.

I eventually fixed by making SQL changes:

update items set albumtype=''
update items set albumtypes=''

then a beet write

Now when I run the beet write -p no more updating of those fields
Also beet update -p no longer wants to update the database

I'll try running a beet mbsync and see the consequences

0 replies

arogl · 2023-12-13T06:39:43Z

arogl
Dec 13, 2023
Author

I have run the beet mbsync and added back all the albumtype and albumtypes fields

beet write -p now gives the following output 😒

Roy Acuff and His Crazy Tennesseans - She’s Selling What She Used to Give Away (28 Risqué Hillbilly Songs From the ‘30s) - When Lulu’s Gone
  albumtype: a -> compilation
  albumtypes: ['a', 'l', 'b', 'u', 'm', ';', ' ', 'c', 'o', 'm', 'p', 'i', 'l', 'a', 't', 'i', 'o', 'n'] -> album; compilation

beet info "artist:Acuff and His Crazy Tennesseans" "album:She’s Selling What She Used to Give Away (28 Risqué Hillbilly Songs From the ‘30s)" "title:When Lulu’s Gone"

         acoustid_id: 65f4ff06-9d59-406b-ab34-910586515228
               album: She’s Selling What She Used to Give Away (28 Risqué Hillbilly Songs From the ‘30s)
         albumartist: Various Artists
  albumartist_credit: Various Artists
    albumartist_sort: Various Artists
        albumartists: Various Artists
 albumartists_credit: Various Artists
   albumartists_sort: Various Artists
         albumstatus: Official
           albumtype: a
          albumtypes: a; l; b; u; m; ;;  ; c; o; m; p; i; l; a; t; i; o; n

From thw database

select albumtype, albumtypes from items where artist like '%Roy Acuff and His Crazy Tennesseans%'


albumtype   |  albumtypes
------------|--------------------
compilation	|  album; compilation
compilation	|  album; compilation

mutagen-inspect output

TXXX=MusicBrainz Album Artist Id=89ad4ac3-39f7-470e-963a-56509c546377
TXXX=MusicBrainz Album Id=870fd07e-281b-4d3b-8c02-39acefc34b45
TXXX=MusicBrainz Album Status=Official
TXXX=MusicBrainz Album Type=a / l / b / u / m / ; /   / c / o / m / p / i / l / a / t / i / o / n
TXXX=MusicBrainz Artist Id=19872845-92ad-4e94-9a89-2fa96dc7d702
TXXX=MusicBrainz Release Group Id=7c7b4ab5-e1a5-47ef-ac38-21c471801637
TXXX=MusicBrainz Release Track Id=e8d48945-1fcd-4925-954e-f3dbc8276cc3
TXXX=MusicBrainz Work Id=68e98b69-62eb-48b9-8938-56a26b994417

I also had a proper look with Mp3Tag and it shows the following

I am not sure how beets sees both albumtype and albumtypes fields when only albumtype seems to exist in the file, mutagen and Mp3Tag output

So it looks like the database is correct, but the way the fields are written is broken.

I'll try the the quick fix

0 replies

arogl · 2023-12-13T07:12:05Z

arogl
Dec 13, 2023
Author

OK commented out medifile.py#L1951

Now I have the following output

beet write "artist:Acuff and His Crazy Tennesseans" "album:She’s Selling What She Used to Give Away (28 Risqué Hillbilly S ongs From the ‘30s)" "title:When Lulu’s Gone"

Roy Acuff and His Crazy Tennesseans - She’s Selling What She Used to Give Away (28 Risqué Hillbilly Songs From the ‘30s) - When Lulu’s Gone
  albumtypes: ['a', 'l', 'b', 'u', 'm', ';', ' ', 'c', 'o', 'm', 'p', 'i', 'l', 'a', 't', 'i', 'o', 'n'] -> album; compilation

beet update -p "artist:Acuff and His Crazy Tennesseans" "album:She’s Selling What She Used to Give Away (28 Risqué Hillbilly Songs From the ‘30s)" "title:When Lulu’s Gone"
Show no updates to the database

mutagen-inspect "She's Selling What She Used to Give Away (28 Risqué Hillbilly Songs From the ‘30s) - 01 - Roy Acuff and His Crazy Tennesseans - When Lulu's Gone.mp3"

TXXX=MusicBrainz Album Artist Id=89ad4ac3-39f7-470e-963a-56509c546377
TXXX=MusicBrainz Album Id=870fd07e-281b-4d3b-8c02-39acefc34b45
TXXX=MusicBrainz Album Release Country=DE
TXXX=MusicBrainz Album Status=Official
TXXX=MusicBrainz Album Type=a / l / b / u / m / ; /   / c / o / m / p / i / l / a / t / i / o / n
TXXX=MusicBrainz Artist Id=19872845-92ad-4e94-9a89-2fa96dc7d702

Off to edit mb.py in beets itself

10 replies

arogl Dec 14, 2023
Author

Confirm that database has none of the extra characters in there.

JOJ0 Dec 14, 2023
Maintainer

Sorry, I've not been very helpful here so far. I've only skimmed through all this, since my head is working elsewhere. But so to wrap up: With a non-modified version of mediafile and beets git master you experience:

Cleanup database albumtype and albumtypes.
run beet mbsync
database now has properly formatted albumtype and albumtypes fields.
run beet write -p
you see this effect:

  albumtype: a -> compilation
  albumtypes: ['a', 'l', 'b', 'u', 'm', ';', ' ', 'c', 'o', 'm', 'p', 'i', 'l', 'a', 't', 'i', 'o', 'n'] -> album; compilation

arogl Dec 14, 2023
Author

Steps I took

Cleanup database albumtype and albumtypes, via sql
run beet write
run beet update -p confirms all tags are empty
run beet mbsync
database now has properly formatted albumtype and albumtypes fields.
run beet update -p
I now see this effect:

  albumtype: a -> compilation
  albumtypes: ['a', 'l', 'b', 'u', 'm', ';', ' ', 'c', 'o', 'm', 'p', 'i', 'l', 'a', 't', 'i', 'o', 'n'] -> album; compilation

Modification of mediaile made no difference

Modify mb.py to remove the albumtype field writes the correct albumtypes tag

run beet update -p shows no updates back to the database

JOJ0 Dec 15, 2023
Maintainer

Thanks for the clarification! Very helpful! I just tried to repro these steps with one item. The difference is that my test file has exactly one albumtype, namly album. In this case everything works as expected:

$ sqlite3 -column library.db "update items set albumtype='',albumtypes='' where title like '%be free%';"
$ beet write peas be free
Black Eyed Peas - Behind the Front - Be Free - src:$data_source |Rap, Funk, Hip Hop
  albumtype: album ->
  albumtypes: album ->
$ beet update -p peas be free
$ beet mbsync peas behind the front
Black Eyed Peas - Behind the Front - Be Free - src:MusicBrainz |Rap, Funk, Hip Hop
  albumtypes:  -> album
  albumtype:  -> album
$ sqlite3 -column library.db "select artist,title,albumtype,albumtypes from items where title like '%be free%';"
-- Loading resources from /Users/jojo/.sqliterc
artist           title       albumtype   albumtypes
---------------  ----------  ----------  ----------
Black Eyed Peas  Be Free     album       album
$ beet update -p peas be free
$ mediainfo "/Users/jojo/Music/dev-beets/Black Eyed Peas/Behind the Front/07 Be Free.mp3" | grep "Album Type"
MusicBrainz Album Type                   : album

I wil do exactly that testrun with a file that has multipl albumtypes later today or this weekend...

~~I also just noticed that I've missed the beet write step (second bulltepoint)~~ update: re-ran everything and included this step in cli outputs above -> still working as expected

JOJ0 Dec 15, 2023
Maintainer

Tried to repro with an item that has album; compilation in albumtypes:

$ sqlite3 -column library.db "update items set albumtype='',albumtypes='' where title like '%body heat%';"
-- Loading resources from /Users/jojo/.sqliterc
$ beet write body heat
 Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:$data_source |Pop
  albumtype: album ->
  albumtypes: album; compilation ->
$ beet update -p body heat
$ beet mbsync dark side of italo disco
Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:MusicBrainz |Pop
  albumtype:  -> compilation
  albumtypes:  -> album; compilation
$ sqlite3 -column library.db "select artist,title,albumtype,albumtypes from items where title like '%body heat%';"
-- Loading resources from /Users/jojo/.sqliterc
artist         title              albumtype    albumtypes
-------------  -----------------  -----------  ------------------
Fockewulf 190  Body Heat (vocal)  compilation  album; compilation
$ beet update -p body heat

Works as expected. It is really weird that you still face that bug on your machine. I can't tell you other then: Please double check my outputs - I might have missed something. And check your versions and try repro with just one track as I did. There might be a difference if this is done with one track or with a whole album! I did not test with a whole album. Hope that helps so far to move on with your investigations. Also note: I did not edit anything in my code so far!

arogl · 2023-12-13T22:21:17Z

arogl
Dec 13, 2023
Author

FWIW mediafile is also latest git?

The edited version of mediafile I forked and modified locally, so yes.

1 reply

JOJ0 Dec 15, 2023
Maintainer

My current test environment, just for the sake of completeness:

pip list | grep -e mediafile -e beets
beets                         1.6.1               /Users/jojo/git/beets
beets-beatport4               0.3.2               /Users/jojo/git/beets-beatport4
beets-bpmanalyser             1.3.3
beets-describe                0.0.4
beets-dirfields               1.1                 /Users/jojo/git/beets-dirfields
beets-xtractor                0.3.1               /Users/jojo/git/beets-xtractor
mediafile                     0.12.0

arogl · 2023-12-16T02:30:59Z

arogl
Dec 16, 2023
Author

Tried to repro with an item that has album; compilation in albumtypes:
$ sqlite3 -column library.db "update items set albumtype='',albumtypes='' where title like '%body heat%';"
-- Loading resources from /Users/jojo/.sqliterc
$ beet write body heat
 Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:$data_source |Pop
  albumtype: album ->
  albumtypes: album; compilation ->
$ beet update -p body heat
$ beet mbsync dark side of italo disco
Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:MusicBrainz |Pop
  albumtype:  -> compilation
  albumtypes:  -> album; compilation
$ sqlite3 -column library.db "select artist,title,albumtype,albumtypes from items where title like '%body heat%';"
-- Loading resources from /Users/jojo/.sqliterc
artist         title              albumtype    albumtypes
-------------  -----------------  -----------  ------------------
Fockewulf 190  Body Heat (vocal)  compilation  album; compilation
$ beet update -p body heat
Works as expected. It is really weird that you still face that bug on your machine. I can't tell you other then: Please double check my outputs - I might have missed something. And check your versions and try repro with just one track as I did. There might be a difference if this is done with one track or with a whole album! I did not test with a whole album. Hope that helps so far to move on with your investigations. Also note: I did not edit anything in my code so far!

Can you try a beet write dark side of italo disco after the beet mbsync?

I have just done the same steps and added a beet write and that seems to be updating the tags with all the weirdness I am seeing, but writing the tags correctly.

beet update -p "albumtypes::;" now shows no updates to be completed

2 replies

JOJ0 Dec 19, 2023
Maintainer

Just tried that:

$ beet write dark side of italo disco
The Creatures - The Dark Side of Italo Disco - The Other World Robots (extended version) - src:$data_source |Pop
  albumtype: album -> compilation
Dario Dell'Aere - The Dark Side of Italo Disco - Eagles in the Night - src:$data_source |Pop
  albumtype: album -> compilation
Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:$data_source |Pop
  albumtype: album -> compilation
Sylvi Foster - The Dark Side of Italo Disco - Hookey - src:$data_source |Pop
  albumtype: album -> compilation
Kirlian Camera - The Dark Side of Italo Disco - Blue Room (original extended) - src:$data_source |Pop
  albumtype: album -> compilation
Steel Mind - The Dark Side of Italo Disco - Bad Passion (Flemming Dalum remix) - src:$data_source |Pop
  albumtype: album -> compilation
Xenon - The Dark Side of Italo Disco - Galaxy (club version) - src:$data_source |Pop
  albumtype: album -> compilation

$ beet write dark side of italo disco
The Creatures - The Dark Side of Italo Disco - The Other World Robots (extended version) - src:$data_source |Pop
  albumtype: album -> compilation
Dario Dell'Aere - The Dark Side of Italo Disco - Eagles in the Night - src:$data_source |Pop
  albumtype: album -> compilation
Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:$data_source |Pop
  albumtype: album -> compilation
Sylvi Foster - The Dark Side of Italo Disco - Hookey - src:$data_source |Pop
  albumtype: album -> compilation
Kirlian Camera - The Dark Side of Italo Disco - Blue Room (original extended) - src:$data_source |Pop
  albumtype: album -> compilation
Steel Mind - The Dark Side of Italo Disco - Bad Passion (Flemming Dalum remix) - src:$data_source |Pop
  albumtype: album -> compilation
Xenon - The Dark Side of Italo Disco - Galaxy (club version) - src:$data_source |Pop
  albumtype: album -> compilation

It always shows that it is changing something AND it actually DOES CHANGE the file:

ls -l "/Users/jojo/Music/dev-beets/Compilations/The Dark Side of Italo Disco/01 Body Heat (vocal).flac"
-rw-r--r--  1 jojo  staff  46857531 Dec 19 10:33 /Users/jojo/Music/dev-beets/Compilations/The Dark Side of Italo Disco/01 Body Heat (vocal).flac

$ beet write body heat

Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:$data_source |Pop
  albumtype: album -> compilation

ls -l "/Users/jojo/Music/dev-beets/Compilations/The Dark Side of Italo Disco/01 Body Heat (vocal).flac"
-rw-r--r--  1 jojo  staff  46857531 Dec 19 10:34 /Users/jojo/Music/dev-beets/Compilations/The Dark Side of Italo Disco/01 Body Heat (vocal).flac

So this is actually exactly issue #4648, but I don't see any weird character splitting into lists like you are experiencing. That is what I still don't understand how that's possible....

JOJ0 Dec 19, 2023
Maintainer

Ah and yeah to conclude this test. I don't see any weirdness when trying to import tags to database again:

$ beet write body heat
Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:$data_source |Pop
  albumtype: album -> compilation
$ beet update body heat
$

arogl · 2023-12-20T06:01:30Z

arogl
Dec 20, 2023
Author

Ah and yeah to conclude this test. I don't see any weirdness when trying to import tags to database again:
$ beet write body heat
Fockewulf 190 - The Dark Side of Italo Disco - Body Heat (vocal) - src:$data_source |Pop
  albumtype: album -> compilation
$ beet update body heat
$

Can you try the beet write body heat again?

I have found that the albumtype tag ALWAYS wants to be rewritten, as mutagen or beets is unable to distinguish between the albumtype and albumtypes fields, and the first one it encounters is album rather than live or compilation.

i hope that makes sense.

1 reply

JOJ0 Dec 20, 2023
Maintainer

Yes ot always wants to write ot again, it doesnt go away. That's what I tried to indicate by posting two times beet write in a row here ;-) Sorry if that wasnt clear enough #5042 (reply in thread)

So yes, this now is exactly #4715

Also see my updates in the issue description. I added command outputs and things I learned here. Hope it's correctly and completely describing what's happening. Corrections welcome!

arogl · 2023-12-20T08:50:59Z

arogl
Dec 20, 2023
Author

Yes ot always wants to write ot again, it doesnt go away. That's what I tried to indicate by posting two times beet write in a row here ;-) Sorry if that wasnt clear enough #5042 (reply in thread)

So yes, this now is exactly #4715

Also see my updates in the issue description. I added command outputs and things I learned here. Hope it's correctly and completely describing what's happening. Corrections welcome!

Sorry, so many things happening.

I did still have the albumtype and albumtypes fields with the ;; style, once I cleared the database and did the beet mbysnc things are back to the original issue in #4715

How we solve the albumtype and albumtypes filed issue, I am still not sure

1 reply

JOJ0 Dec 20, 2023
Maintainer

No worries, that's very good news and a relief! I'm glad we finally have a consistent reproduction of the issue. :-)

arogl · 2023-12-21T22:22:56Z

arogl
Dec 21, 2023
Author

I still believe we have issues where users' database are still having the albumtype and albumtypes fields populated with the [a; l; b ; u; m ;; l; i; v; e] styles.

1 reply

arsaboo Dec 21, 2023

Yes, I have them too. I largely use Spotify as my tag source, so I am not sure why I am getting these errors.

Ignore fields albumtype and albumtypes #5042

arogl Dec 11, 2023

Replies: 9 comments · 18 replies

DjSlash Dec 12, 2023

JOJ0 Dec 12, 2023 Maintainer

JOJ0 Dec 12, 2023 Maintainer

arogl Dec 13, 2023 Author

arogl Dec 13, 2023 Author

arogl Dec 13, 2023 Author

arogl Dec 14, 2023 Author

JOJ0 Dec 14, 2023 Maintainer

arogl Dec 14, 2023 Author

JOJ0 Dec 15, 2023 Maintainer

JOJ0 Dec 15, 2023 Maintainer

arogl Dec 13, 2023 Author

JOJ0 Dec 15, 2023 Maintainer

arogl Dec 16, 2023 Author

JOJ0 Dec 19, 2023 Maintainer

JOJ0 Dec 19, 2023 Maintainer

arogl Dec 20, 2023 Author

JOJ0 Dec 20, 2023 Maintainer

arogl Dec 20, 2023 Author

JOJ0 Dec 20, 2023 Maintainer

arogl Dec 21, 2023 Author

arsaboo Dec 21, 2023

arogl
Dec 11, 2023

Replies: 9 comments 18 replies

DjSlash
Dec 12, 2023

JOJ0 Dec 12, 2023
Maintainer

JOJ0 Dec 12, 2023
Maintainer

arogl
Dec 13, 2023
Author

arogl
Dec 13, 2023
Author

arogl
Dec 13, 2023
Author

arogl Dec 14, 2023
Author

JOJ0 Dec 14, 2023
Maintainer

arogl Dec 14, 2023
Author

JOJ0 Dec 15, 2023
Maintainer

JOJ0 Dec 15, 2023
Maintainer

arogl
Dec 13, 2023
Author

JOJ0 Dec 15, 2023
Maintainer

arogl
Dec 16, 2023
Author

JOJ0 Dec 19, 2023
Maintainer

JOJ0 Dec 19, 2023
Maintainer

arogl
Dec 20, 2023
Author

JOJ0 Dec 20, 2023
Maintainer

arogl
Dec 20, 2023
Author

JOJ0 Dec 20, 2023
Maintainer

arogl
Dec 21, 2023
Author