Skip to content

Fetcher manipulator: RandomSet

Mark Jordan edited this page Mar 10, 2017 · 7 revisions

Overview

Fetcher manipulator that filters out a random set of objects to generate Islandora ingest packages for. Useful for testing and QA purposes.

If used with MIK's --limit parameter, the number of items in the randomized set will never be larger than the value of --limit (which means that the set could include every object). For example, if you ask for a random set of 10 and the value of --limit is 5, the set will contain 5 items.

Toolchains

Can be used within any toolchain (i.e., is not specific to CONTENTdm, CSV, etc.).

Configuration

To register this manipulator in your toolchain, add the following line to the "[MANIPULATORS]" section of your .ini file:

fetchermanipulators[] = "RandomSet|10"

fetchermanipulators[] = "RandomSet|10|randomset.txt"

Parameters

This manipulator takes a single required parameter: the size of the set you want to create, e.g., 10, 25, 100.

It also takes an optional second parameter: a path to a file. If you include this parameter, MIK will write out the IDs of records that made up the random set. This is useful during testing, for example, when you may want to "replay" a random set of objects by using this file as the input for the SpecificSet fetcher manipulator.

Functionality

This fetcher manipulator extracts a random subset of records from a set of records generated by a fetcher. The set of records will not contain any duplicates. The same record(s) may be included in subsequent MIK jobs; no attempt is made to avoid including the same record across jobs.

Clone this wiki locally