-
Notifications
You must be signed in to change notification settings - Fork 2
Grabber Behavior
The Grabber Behavior will monitor one or more channels in a Discord Environment for music, cache that music and index it with its metadata. It's capable of scanning a channel's past message history, converting between mp3, pcm (wav) and flac, normalizing song volume, splitting chapters, cropping and more.
Please examine Grabber's parameters carefully before using. channels is the only mandatory parameter, containing a list of channels for the Behavior's instance to operate on. Sane defaults are provided for the remaining parameters.
However, do note that audio will be cached under songs/, and the size of this directory can grow very large if unrestricted. It's recomended to set maxDiskUsage (in bytes) to prevent this.
The following sources are currently supported:
- Youtube video URLs (all types; this is the recommended source)
- Discord message attachments (supported extensions: mp3, ogg, flac, wav, pcm, wma, aac, m4a)
- Google Drive shared file URLs
- Dropbox download URLs
As of Rowboat 2.0, Bandcamp is no longer supported.
When an appropriate source is detected in a message, the audio will be downloaded and analyzed. Its hash is calculated and a new entry created in the index for it, pre-filled with metadata from the audio source and analysis results.
Indexed data can then be accessed using !song.
By default, Rowboat will provide feedback when the operation completes. If successful, this feedback will contain the song hash, which can then be used as a reference to the song.
To control feedback, use the following prefixes at the beginning of a message sent to a monitored channel:
-
XX: Messages that start with this prefix will not be scanned for audio, and index entries will not be created for them. -
$$: Messages that start with this prefix will be scanned quietly. An index entry will still be created, but there won't be a feedback message. -
!!: Messages that start with this prefix will be scanned and an index entry created; the bot will emit a feedback message containing the result of the operation (this is the default).
Both the prefixes used for each behavior and the default behavior (when no prefix is used) can be changed in the configuration.
In addition to references to song sources, Grabber can also extract metadata from the messages where those references were found. This metadata will be added to the song's index entry, replacing or extending the metadata obtained from the remote source.
These tags use the format {FIELD:CONTENTS} or {FIELD=CONTENTS} and can be placed anywhere in the message to provide data for the corresponding index fields.
- name, n, title, song: Replace the default name of the song (obtained from the source).
- author, artist, band, creator, c: Set an artist or originator for the song.
-
album, a: Set an album for the song. Once an album is set for multiple songs, its contents can be listed with
!song album. - track, t: Set a track number for the song in the album. The track must be a positive integer between 1 and 999.
- tweak: Set a hint that the song should be louder or quieter than the source for optimal listening. By default a number between -4 and 4 (dB).
In addition, any amount of keywords or tags can be associated with a song by including [KEYWORD] with the message. If the source is Youtube, any keywords defined by the uploader will be automatically added to the index.
Grabber is capable of extracting only a subset of the audio from the source or splitting the source into multiple chapters or tracks, each with its own index entry. To make this happen, you must include a crop tag in the message. The following formats are supported:
-
<BEGINNING,END>or<BEGINNING>or<,END>: Crop the source to the given beginning and end timestamps (if the beginning or the end are omited, the source's beginning/end will be used). The timestamps can be separately given inhours:minutes:seconds,minutes:secondsor justseconds. -
<C#>where # is a positive integer: Crop a youtube source to the given chapter number (the source must have chapters). -
<Chapter Name>: Crop a youtube source to the given Chapter Name (the source must have chapters). You must provide the exact name. -
<ALL>: Slice a youtube source with chapters into individual chapters and add them all to the index. The chapter name will be used as the song name and the chapter number will be used as the track number. -
<FILL>: Like<ALL>, but preserves existing entries in an album and only adds to the index any missing tracks, based on album name and track number. Thus, for<FILL>to work properly, it's imperative that you provide an{album: ...}.
For cropping to work, there must always be a name for the crop, normally provided through {name: ...}. The name can be omited if you are replacing an existing index entry (the existing name will be used). If there is no name for the crop, the crop tag will be ignored.
By default, songs are cached using the default format you selected in the configuration. The format tag can be used to change the format of the associated song:
{format: mp3|flac|pcm}
Each format has its advantages and disadvantages:
- mp3: More efficient disk space usage, but uses lossy compression.
- flac: No quality loss, but uses more disk space.
- pcm: Easiest to process, but uses the most disk space.
When sharing a song, you may want to specify that it's a replacement for an existing entry in the index rather than a new one. This can be done to prevent duplicates, and also to update existing entries with better quality versions or repaired versions of previous sources without losing all the associated metadata.
To replace a song, include the following tag in your message:
{replace: HASH}
Where HASH is the hash of the index entry that will be replaced. The song will be indexed as normal and a new hash calculated for it. As much data as possible from the previous index entry will be copied to the new one, and then the previous entry will be removed.
Note that if metadata is provided explicitly in the message through the use of tags, these take precedence and will overwrite the same field from the old index entry.
You may want to share multiple audio sources in the same message. For basic grabbing, as long as the message is scanned, all sources should always be indexed (separately). However, if you want to use any of the tags described in this section, you will probably want to subdivide your message into fragments using the {+} boundary. This will help Rowboat assign each metadatum to the appropriate source. For example:
https://youtu.be/AAAAAAAA {name: Song one} {+} https://youtu.be/BBBBBBBB {name: Song two}
If you don't separate the sources in the message with {+}, or provide multiple sources in the same fragment, each tag will apply to all sources.
Once your song has been indexed, its index entry can always be uniquely referenced using its hash. Hashes have the format:
#STRING, where STRING is a sequence of 32 hexadecimal characters.
However, every command argument that can take a hash can also take a number of other types of song references:
-
-: References the latest song added to the library, if any (not preserved if Rowboat shuts down). -
-1,-2,-3, ...,-N: References the song added to the library N songs ago. -
Search Text: Searches the index metadata for a song. If there is exactly one result, it will be returned and used in the command. Otherwise, Grabber will point out there are multiple results and ask you to refine your search. -
?Search Text: Searches the index metadata for a song. If there are multiple results, a random result will be returned and used in the command.
Grabber extensions can register more song reference parsers (see See Also). You can obtain a list of all registered parsers using !song hash.
This command provides several helpful maintenance operations for your index and cache. Some of these can be destructive and default to requiring administrator permissions.
- grab scan: Manually request a scan of a channel's history (and grab sources from all retrieved messages). This can take quite a while, but is useful in the event Rowboat has been down or for bootstrapping the index.
- grab regrab - Patch up a corrupted library by attempting to reprocess an already indexed song, or the entire index if no hash is provided. This can also take quite a while.
- grab analyze - Redo the analysis step of the grab process for one song or for the entire index. Currently this consists only in loudness (mean volume) calculation. This information is stored in the index.
- grab reformat - Converts a cached song to a different format. This is a local operation; the source won't be retrieved again, and lossy compression won't be undone.
You can also use it to remove songs from the index.
-
grab undo [n] - Undo a single recent grab from this session, either the latest one or n grabs ago. Equivalent to
!grab delete -n - grab delete - Removes a song from the index and cache.
-
grab latest - Get the hash of a song (or any song). Similar to
!song get - hash.
Finally, the following command will display all currently queued Grabber operations, as well as the number of ongoing downloads (if any). This is useful for monitoring the progress of large batches of operations or for diagnosing issues.
- grab tasks
Once your audio is properly indexed, this command can be used for reading its metadata or replacing basic metadata fields.
- song find - Searches the index and returns a page of results with hashes, names and authors.
- song set and song get - Read or write a field in an index entry. Note that the index contains automatically populated fields that can be used with get, but not with set. These are the commands you'll use to set the name, author, album and track post-grabbing.
-
song album - List all the songs in a given album that are known by the index. The result format is similar to
song find, but will include track numbers. - song kw [add|remove] - List, add or remove keywords associated with the song in the index. Keywords are used when searching for the song.
Add-ons for Grabber:
- SongRanking - Adds user preferences to the index and collects user reactions from the Environment.
- Radio - A music radio that plays songs listed in the index.