Add TIDAL autotagger plugin #5637

jcjordyn130 · 2025-02-18T22:22:50Z

Description

This is a replacement of the last TIDAL PR I opened back in 2023 (#4641), as in, it is a proper autotagger implementation using the TIDAL API.

Candidate retrieval and support for direct TIDAL album and track URLs are supported.

It also fetches lyrics (both time-synced and not), and embeds it into both the file tags and a LRC file if configured.

Adding testing is a little complicated as testing lyrics support requires a paid account, however, basic metadata queries only require an account with or without an active subscription.

I have been using this plugin for about a week now with my library and it seems to be holding up, obtaining lyrics that are otherwise unavailable through the available lyrics backends.

Poetry files were changed to add optional dependencies for this plugin, which are tidalapi, cachetools (for LFU cache in hot functions), and backoff (to slow down queries when rate limited).

Also, please do let me know if you want me to squash my commits, as I know there are quite a few of them.

To Do

Documentation
~~Changelog~~ (Will add once change is approved)
~~Tests~~

github-actions · 2025-02-18T22:23:12Z

Thank you for the PR! The changelog has not been updated, so here is a friendly reminder to check if you need to add an entry.

- Initial implementation of LRC writing support - Split imported file processing to its' own function

- Actually remove the session file instead of blanking it out

- Check for auto status before loading the TIDAL session

- Implement LFUCache on hot functions - Fix docstrings - Add backoff handler for debugging - Add fatal argument to _load_session to throw UserError - Reduce excessive calls for track metadata retrival - _search_track now returns tidalapi types and no longer grabs the full album metadata - Add user configurable result limits for both metadata and lyrics

- tidalapi was changed from a star import to a regular import - Logic error in _process_item was fixed, was used undefined track

- Fix duplicate track removal from _search_lyrics - Stop adding top_hit result twice in _search_album and _search_track

- Add output when the login was successful

It was never used and it can always be dumped with a regular `cat` command

- Break out _get_lyrics specific metadata searching algorithm into seperate function (_search_from_metadata) - Remove _search_album as it is now unused - Fix candidates function shadowing parameters - Candidate functions now use _search_from_metadata - Candidate functions now handle va_likely

The plugin has been operating on the original files this entire time. Changes: - Add a write event listener to write out sidecar files - Remove file management from the import stage, as this is already handled in Item.write()

This currently only works with synced lyrics and needs to be changed if TIDAL ever changes their lyrics format.

- Change _validate_lyrics to use tidalapi Track length instead of calculating length from lyrics - Change _search_from_metadata to search for alternative artists - Add config toggle for items with no duration to assume valid lyrics or not - Change _search_track so queries are stripped of special characters and medium details, as the TIDAL search engine is picky

Technically, Items can have an infinite number of max artists. Let's add a user-configurable cap to avoid excessive TIDAL API usage.

- Add another implementation of _serach_album - Remove unused variables from LRC timestamp calculation - Change candidates function to use _search_album instead of grabbing a new Album instance from _search_track - Change _search_from_metadata to search for albums as well - Fix logic errors and unused variables in _validate_lyrics

- Move the initial session load into __init__ as it is called before any of the plugin functions - Move query fixing into _tidal_search from the respective _search functions - Reworded UIError message in _load_session

- Ran `poe lint` and fixed all errors - Ran `poe format` - Reworded some docstrings to make them shorter - Split some strings between two lines to avoid going over line length limit

- Move _load_session from __init__ to an import_begin event handler

Turns out, _search_lyrics didn't use its own limit parameter and was directly using self.config.

- _load_session now calls _save_session if the login succeeds and the expiry date is in the past, meaning the token has changed. This saves on excessive authentication traffic.

- Rewrite the main search loop in _search_lyrics to stop searching when valid lyrics are found instead of fetching lyrics for all found tracks _then_ returning

beetsplug/tidal.py

- Add singleton handling in candidates() as searching by track can return more results - Fix item_candidates not returning beets types, therefore, causing the autotagger to crash - Add support for multiple artists, record labels, and cover art in both tracks and albums - Fix top_hit handling in _search_track and _search_album, turns out it can be any tidalapi type even if models is defined

- Fix artists_ids in _track_to_trackinfo and _album_to_albuminfo as it was giving the autotagger intergers when it expected strings, causing it to crash. - Add _parse_copyright to attempt to parse record labels from freeform TIDAL copyright string.

- Add support for TIDAL popularity with tracks and albums - Remove debug logging from _parse_copyright as it was overly verbose, once per track - Move TIDAL specific metadata out of TrackInfo/AlbumInfo constructor - Add support for grabbing cover art from TIDAL with a max resolution limit, as it is lossy encoded and TIDAL provides a few set resolutions for us - Add more TIDAL IDs in albums and tracks, now tracks contain the track id, album id, and primary artist ID - Update documentation for new config tunables

- Add --fetch to fetch lyrics for arbitrary TIDAL track IDs. This is useful when the lyrics are correct, but other sources have better metadata. - Add --refresh to refresh metadata for all TIDAL tagged tracks and albums in the library. Currently this only supports popularity.

snejus

Added a couple of comments. Please add some tests to assert this works fine and to document the JSON data that TIDAL API returns.

Thanks for this great work!

beetsplug/tidal.py

snejus · 2025-03-09T22:40:17Z

beetsplug/tidal.py

+        # Nothing above invalidated the lyrics, assuming valid
+        return True
+
+    def _search_lyrics(self, item, limit=10):


I don't think this is the right place for lyrics search. We have lyrics plugin for this, so I would expect TIDAL to be added as a backend there (in a separate PR).

That's what I thought at first, but I though that from #4641 (comment) that they wanted it to be part of the autotagger.

Anyways, would I be reimplementing parts of this plugin in the lyrics plugin or would I be able to call into this plugin from the lyrics plugin? Either that or monkeypatch the lyrics plugin to add a new backend

snejus · 2025-03-09T22:43:11Z

beetsplug/tidal.py

+
+        return trackinfo
+
+    def _search_from_metadata(self, item, limit=10):


It seems to me that this method will be slow to return results due to the number of search requests it makes. Have you tested it?

They were pretty similar and both ended up calling _tidal_search anyways, so they were abstracted into it. The cache decorator was also adjusted to use the query and return type as a cache key

jcjordyn130 · 2025-03-17T14:04:04Z

No problem, I enjoy contributing to and using beets.

Anyways, as for testing, are you wanting me to probe the TIDAL API directly with requests for testing or to just use the tidalapi wrapper for it?

Additionally, some asserts were added to ensure that _load_session was called and to make MyPy happy. Docstring type comments were removed as they were made redundant by the type hints.

Turns out, I used the wrong quotes in an f-string and it split the string into two... whoops

- Add logger assertion in backoff_handler - Change candidates signature to match the superclass - Fix incorrect signature in commands - Import annotations from __future__ to fix X | Y union shorthand on older Python versions - Fix incorrect assignment to logger in TidalPlugin

The asserts were comparing the object to a type vs checking the result of isinstance

This should hopefully resolve the missing attribute errors from MyPy

jcjordyn130 · 2025-03-18T17:41:44Z

I ask because tidalapi already has tests implemented https://github.com/tamland/python-tidal/tree/master/tests, and using tidalapi types is impossible without a session meaning that the majority of the plugin would need to use integration tests instead of regular tests

I forgot to do this in the last commit and didn't notice until importing a big album that triggered rate limiting

Copilot

Pull Request Overview

This PR implements the new TIDAL autotagger plugin using the TIDAL API to fetch metadata and lyrics, supporting direct TIDAL URLs and multiple features.

Adds optional dependencies (tidalapi, cachetools, backoff) in pyproject.toml
Introduces a dependency group for TIDAL-related libraries

Files not reviewed (2)

docs/plugins/index.rst: Language not supported
docs/plugins/tidal.rst: Language not supported

Comments suppressed due to low confidence (1)

pyproject.toml:152

[nitpick] The dependency group 'tidal' is a bit ambiguous; consider renaming it to a more descriptive name such as 'tidal_autotagger' to clarify its purpose.

tidal = ["tidalapi", "cachetools", "backoff"]

jcjordyn130 added 23 commits February 18, 2025 16:23

Add initial implementation of TIDAL metedata plugin

3189082

Add support for writing lyrics to LRC files

2a71f11

- Initial implementation of LRC writing support - Split imported file processing to its' own function

Fix session file handling with expired tokens

72a0895

- Actually remove the session file instead of blanking it out

Optimize import stage handler when auto is False

7ade534

- Check for auto status before loading the TIDAL session

Convert docstrings to ReST format

2f7ad48

Properly handle corrupted session state file

25063c2

Fix formatting

5a590c4

Fix formatting and logic errors

05e3525

- tidalapi was changed from a star import to a regular import - Logic error in _process_item was fixed, was used undefined track

Reduce TIDAL API load

627a9a0

- Fix duplicate track removal from _search_lyrics - Stop adding top_hit result twice in _search_album and _search_track

Make _login more user friendly

e212ebc

- Add output when the login was successful

Remove the ability to dump session state

ad879f6

It was never used and it can always be dumped with a regular `cat` command

Properly handle library items during import

fe71ee1

The plugin has been operating on the original files this entire time. Changes: - Add a write event listener to write out sidecar files - Remove file management from the import stage, as this is already handled in Item.write()

Add lyrics validation

cfdde8a

This currently only works with synced lyrics and needs to be changed if TIDAL ever changes their lyrics format.

Add limit on max artists to query in _search_from_metadata

1d3c673

Technically, Items can have an infinite number of max artists. Let's add a user-configurable cap to avoid excessive TIDAL API usage.

Simplify session handling

9384ec8

- Move the initial session load into __init__ as it is called before any of the plugin functions - Move query fixing into _tidal_search from the respective _search functions - Reworded UIError message in _load_session

Fix formatting to prepare for upstream PR

0d8f552

- Ran `poe lint` and fixed all errors - Ran `poe format` - Reworded some docstrings to make them shorter - Split some strings between two lines to avoid going over line length limit

Fix plugin CLI when there is no session

a40f0c2

- Move _load_session from __init__ to an import_begin event handler

Add plugin documentation

dfd1b47

Add optional dependencies for plugin to poetry

c1b9f21

jcjordyn130 force-pushed the tidal_metadata branch from 19781aa to c1b9f21 Compare February 18, 2025 22:23

jcjordyn130 mentioned this pull request Feb 18, 2025

lyrics: Add TIDAL backend #4641

Closed

3 tasks

jcjordyn130 force-pushed the tidal_metadata branch from 3b3e506 to 851b2b4 Compare February 18, 2025 22:40

Fix logic error in _search_lyrics

68a1df7

Turns out, _search_lyrics didn't use its own limit parameter and was directly using self.config.

jcjordyn130 force-pushed the tidal_metadata branch from 851b2b4 to 68a1df7 Compare February 18, 2025 22:50

Fix session token renewal

bf70bd2

- _load_session now calls _save_session if the login succeeds and the expiry date is in the past, meaning the token has changed. This saves on excessive authentication traffic.

jcjordyn130 force-pushed the tidal_metadata branch from c434b18 to bf70bd2 Compare March 5, 2025 01:41

Reduce TIDAL lyrics API load

02802dc

- Rewrite the main search loop in _search_lyrics to stop searching when valid lyrics are found instead of fetching lyrics for all found tracks _then_ returning

arsaboo reviewed Mar 5, 2025

View reviewed changes

beetsplug/tidal.py Show resolved Hide resolved

arsaboo reviewed Mar 5, 2025

View reviewed changes

beetsplug/tidal.py Show resolved Hide resolved

jcjordyn130 added 2 commits March 4, 2025 22:49

jcjordyn130 force-pushed the tidal_metadata branch from 7361a14 to 2101ccf Compare March 5, 2025 15:31

jcjordyn130 added 2 commits March 5, 2025 14:00

snejus requested changes Mar 9, 2025

View reviewed changes

Abstract _search_track and _search_album

5cedf66

They were pretty similar and both ended up calling _tidal_search anyways, so they were abstracted into it. The cache decorator was also adjusted to use the query and return type as a cache key

jcjordyn130 force-pushed the tidal_metadata branch from 1d022e6 to 5cedf66 Compare March 17, 2025 13:58

jcjordyn130 added 5 commits March 18, 2025 09:23

Added MyPy type checking hints

67e0f5a

Additionally, some asserts were added to ensure that _load_session was called and to make MyPy happy. Docstring type comments were removed as they were made redundant by the type hints.

Fix syntax error in __init__

986c1f0

Turns out, I used the wrong quotes in an f-string and it split the string into two... whoops

Fix logic errors in session asserts

a83f681

The asserts were comparing the object to a type vs checking the result of isinstance

Remove optional type from tidalapi session

024c663

This should hopefully resolve the missing attribute errors from MyPy

jcjordyn130 added 2 commits March 18, 2025 16:49

Fix session loading with new MyPy type annotations

2357dbd

Fix logging assertion in backoff_handler

cc83f3c

I forgot to do this in the last commit and didn't notice until importing a big album that triggered rate limiting

snejus requested a review from Copilot March 20, 2025 21:27

Copilot AI reviewed Mar 20, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add TIDAL autotagger plugin #5637

Add TIDAL autotagger plugin #5637

jcjordyn130 commented Feb 18, 2025

github-actions bot commented Feb 18, 2025

snejus left a comment

snejus Mar 9, 2025

jcjordyn130 Mar 17, 2025

snejus Mar 9, 2025

jcjordyn130 commented Mar 17, 2025

jcjordyn130 commented Mar 18, 2025

Copilot AI left a comment


		return trackinfo

		def _search_from_metadata(self, item, limit=10):

Add TIDAL autotagger plugin #5637

Are you sure you want to change the base?

Add TIDAL autotagger plugin #5637

Conversation

jcjordyn130 commented Feb 18, 2025

Description

To Do

github-actions bot commented Feb 18, 2025

snejus left a comment

Choose a reason for hiding this comment

snejus Mar 9, 2025

Choose a reason for hiding this comment

jcjordyn130 Mar 17, 2025

Choose a reason for hiding this comment

snejus Mar 9, 2025

Choose a reason for hiding this comment

jcjordyn130 commented Mar 17, 2025

jcjordyn130 commented Mar 18, 2025

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview