Skip to content

v3_posix_tuto

Jump to bottom
Thomas Leibovici edited this page Aug 2, 2016 · 2 revisions

Quick start: Robinhood v3 on a POSIX filesystem

Table of Contents

Installation

This section briefly list the steps for installing and configuring robinhood on a POSIX filesystem. This doesn't deal with policy configuration as they are described in other tutorials.

For more details about installation, software and hardware requirements, tunings, etc. refer to: Admin guide: Installation.

Robinhood

Requirements

  • Robinhood must run on filesystem client (a host where the filesystem is mounted).
  • Robinhood uses a MySQL or MariaDB database storage backend. It is recommended to install the DB server on the same host as robinhood to ensure a minimum latency for DB operations.
Install and start MySQL on RHEL 6: 
 yum install mysql-server
 service mysqld start

Install and start MariaDB on RHEL 7: 

 yum install mariadb-server
 systemctl start mariadb.service

/!\ Default database configuration is not suitable for production and will result in very low performances. See Admin guide: database tunings for recommended database configuration.

Installation

Configuration

  • Create robinhood database, using rbh-config helper (provided by 'robinhood-adm' package).
 rbh-config create_db <db_name>    'localhost' 'rbh_password'
    • A common name for robinhood database name is 'rbh_fsname.
    • Write the selected password to a file only readable by 'root' (600), for example in /etc/robinhood.d/.dbpassword.
  • Create a robinhood configuration file, starting with a simple robinhood template:
 cp /etc/robinhood.d/templates/basic.conf /etc/robinhood.d/<fsname>.conf
  • Edit the configuration file:
    • In 'General' block, set filesystem root path, and the corresponding filesystem type:
 fs_path = "/fs/root";
 fs_type = xfs;
    • In 'ListManager' block, set database connection parameters:
 # database name passed to 'rbh-config create_db'
 db = <db_name>;
 password_file = "/etc/robinhood.d/.dbpassword" ;

It is recommended to define your fileclasses before running the initial filesystem scan:

  • This way, you will get relevent information in 'rbh-report --class-info' report after the initial scan is completed.
  • This will make some optimizations possible for running policies (e.g. skip processing of 'ignored' classes).
Examples: 
 fileclass empty_file {
    definition { type == file and size == 0 }
 }
 fileclass small_file {
    definition { type == file
             and size > 0
             and size <= 32MB }
 }

Feeding robinhood

On a POSIX filesystem (except Lustre), you need to make robinhood scan the filesystem to populate and update its database contents.

Initial scan

  • If you want to run the initial scan in a terminal and see the log messages in this terminal, run:
 robinhood --scan --once -L stderr
  • If you prefer running it in background (and display messages into robinhood log):
 robinhood --scan --once -d

Regular scans

On a POSIX filesystem, you need to regularly scan the filesystem to update robinhood DB contents.

  • To run a scan once and exit, execute:
 robinhood --scan --once -L stderr
  • To start a robinhood daemon that regularly scans the filesystem to update the DB:
    • In robinhood configuration, specify a 'scan_interval', for example:
 FS_Scan {
    # daily scan
 	scan_interval = 1d;
 }
    • Edit /etc/sysconfig/robinhood to indicate the robinhood daemon must scan the filesystem:
 RBH_OPT="--scan"
    • Start robinhood service:
 # on RHEL 6:
 service robinhood start
 # on RHEL 7:
 systemctl start robinhood.service

Managing multiple filesystems

On RHEL7, if you want to manage several filesystems on the same robinhood host, use 'robinhood@' service instead.

  • Per-filesystem service is managed by systemctl [start|stop|status|restart|...] robinhood@''fsname''
  • Per-filesystem service configuration is in /etc/sysconfig/robinhood.''fsname''

Monitoring scan progress

You can monitor scan progress by looking at robinhood statistics (dumped every 15min by default): 

 grep STATS /var/log/robinhood.log

Filesystem reports

  • rbh-report provides overall reports about filesystem contents (users and groups usage, file size profile, fileclasses...)
  • rbh-find implements classic 'find' command, except that it queries robinhood database instead of the filesystem, which makes it faster. Moreover, it provides specific options to query entries per policy status and other policy specific attributes.
  • rbh-du is a enhanced version of classic 'du' command. It queries robinhood database instead of the filesystem, which makes it faster. It can also report details about entry types, count, etc.
rbh-report examples: 
 # filesystem entries:
 # rbh-report --fs-info
 type    ,    count,   volume, avg_size
      dir,  1780074,  8.02 GB,  4.72 KB
     file, 21366275, 91.15 TB,  4.47 MB
  symlink,   496142, 24.92 MB,       53

 # user info, split by group
 # rbh-report -u bar -S
 user , group,  type,  count,  spc_used,   avg_size
 bar  , proj1,  file,      4,  40.00 MB,   10.00 MB
 bar  , proj2,  file,   3296, 947.80 MB,  273.30 KB
 bar  , proj3,  file, 259781, 781.21 GB,    3.08 MB

 # file size profile for a given user
 # rbh-report -u foo --szprof
 user, type,  count,    volume,  avg_size,   0,  1~31,  32~1K-, 1K~31K, 32K~1M-, 1M~31M, 32M~1G-, 1G~31G, 32G~1T-, +1T
 foo ,  dir,     48,   1.48 MB,  31.67 KB,   0,     0,       0,     26,      22,      0,       0,      0,       0,   0
 foo , file,  11055, 308.16 GB,  28.54 MB,   2,     0,      14,     23,    5276,   5712,       9,     17,       2,   0

 # top disk space consumers
 # rbh-report --top-users
 rank, user    , spc_used,  count, avg_size
   1, usr0021 , 11.14 TB, 116396, 100.34 MB
   2, usr3562 ,  5.54 TB,    575,   9.86 GB
   3, usr2189 ,  5.52 TB,   9888, 585.50 MB
   4, usr2672 ,  3.21 TB, 238016,  14.49 MB
   5, usr7267 ,  2.09 TB,   8230, 266.17 MB
 ...

But also:

  • --top-size Report largest files in the filesystem.
  • --entry-info Report all information about a given entry.
  • Run rbh-report --help to get the full list of available reports.
rbh-find example: 
 # rbh-find /myfs/dir -u root -size +32M -mtime +1h -ost 2 -ls

rbh-du examples: 

 # rbh-du -H -u foo /myfs/dir/dir.3
 45.0G /myfs/dir/dir.3

Back to wiki home

Clone this wiki locally