Skip to content

MatrixDevTop

Jump to bottom
MichaelGoodman edited this page May 16, 2011 · 45 revisions

Matrix Development

This wiki is a reference for Grammar Matrix developers. Users of the Grammar Matrix should start from MatrixTop.

Below we have information about the [#repository SVN repository], [#configuration configuration], the [#directory directory structure], and [#links other links].

Matrix Developers links

Anchor(links)

A collection of documentation regarding the Matrix project, aimed at Matrix developers.

Matrix Repository

Anchor(repository)

svn co svn://lemur.ling.washington.edu/shared/matrix matrix

Note that you need to be granted permission to read or write to/from this repository. Speak to David Brodbeck (linghelp@) or Emily Bender (ebender@) about getting permissions. (Both email addresses above have u.washington.edu as the domain)

Configuration

Anchor(configuration)

CUSTOMIZATIONROOT

Anchor(customizationroot)

Several matrix.py commands (such as "install", "vivify", and "regression-test") require the environment variable CUSTOMIZATIONROOT to be set, with the value being the "gmcs" directory of the Matrix branch being used. In most cases this can, and should, be set as an option to matrix.py (--customizationroot (short form: -C)). For example:

python matrix.py --customizationroot=gmcs/ regression-test

or

python matrix.py -C gmcs/ regression-test

If necessary, the variable may be set with an export command, which would be valid for the current session. This export command may be placed in ~/.bashrc so the variable is set each time a session is started. Assuming the directory matrix/ exists is the home directory, the export commands would look like this for trunk:

export CUSTOMIZATIONROOT=~/matrix/trunk/gmcs

or like this for a branch:

export CUSTOMIZATIONROOT=~/matrix/branches/mybranch/gmcs

Matrix.py

Anchor(matrixpy)

matrix.py is a top-level script used mainly by developers to do a wide range of tasks, such as customization, validation, unit-testing, regression-testing, and installing. The command python matrix.py --help prints the following (as of 2011.04.26):

Usage: matrix.py [OPTION] COMMAND [ARGS...]

OPTIONS:
    --customizationroot (-C) PATH
                Set CUSTOMIZATIONROOT to PATH.
    --cheap-hack
                Add a blank morphological rule to irules.tdl (if it is
                empty) to workaround a bug in Cheap.
    --warning (-w)
                Print warnings when running validate.
    --help (-h) [COMMAND]
                Print a usage message about COMMAND (if specified) or
                else all commands and examples.
    
COMMANDS:
    customize (c) PATH [DEST]
                Customize the grammar at PATH, with the output written to
                DEST or the directory at PATH. PATH points to a choices
                file or a directory that contains a choices file.
    customize-and-flop (cf) PATH [DEST]
                Customize and flop the grammar at PATH, with the output
                written to DEST or the directory at PATH. PATH points to a
                choices file or a directory that contains a choices file.
    validate (v) PATH
                Validate the choices file at PATH.
    regression-test (r) [TEST]
                Run regression test TEST (if specified) or else all tests.
    regression-test-add (ra) CHOICES TXTSUITE
                Add CHOICES (a choices file) and TXTSUITE (a text file
                containing test sentences) as a new regression test. Both
                CHOICES and TXTSUITE are filenames, not paths, and the
                respective files should exist in the scratch directory
                (gmcs/regression_tests/scratch/).
    regression-test-update (ru) TEST
                Update the gold standard of TEST to use the results of the
                current system.
    unit-test (u)
                Run all unit tests.
    install (i) PATH
                Install a custom instance of the Grammar Matrix
                Customization System and Questionnaire at the PATH
                specified on the default server (Homer).
    vivify (v)
                Install a new version of the Grammar Matrix Customization
                System and Questionnaire to the live site after verifying
                the code has been tested and committed to SVN.

EXAMPLES:
  matrix.py customize ../choices/Finnish
  matrix.py cf ../choices/Finnish
  matrix.py v ../choices/Finnish
  matrix.py --customizationroot=gmcs/ r
  matrix.py -C gmcs/ ra Cree_choices Cree_test_suite
  matrix.py -C gmcs/ install my_matrix
  matrix.py -C gmcs/ vivify

Directory Structure

Anchor(directory)

If you get the repository using the command [#repository above], you will see three subdirectories: branches, tags, and trunk. Each of these contain a copy of the Matrix code (a "branch"), but "trunk" is the official version, "tags" contains frozen snapshots (e.g. the version used for a dissertation, etc.), and "branches" contains development versions. The directory structure detailed below explains the directories and files for a single branch (e.g. in trunk):

doc/         [publications and documents related to the system]
gmcs/        [most of the code resides here]
    80proof/          [grammars used in the 80proof paper]
    lib/              [Python libraries for internal code]
        hierarchy.py            [module for type hierarchies]
        tdlhierarchy.py         [module for tdl hierarchies]
    linglib/          [Libraries for linguistic phenomena]
        tests/                  [unit tests for linglib]
        agreement_features.py
        argument_optionality.py
        auxiliaries.py
        case.py
        coordination.py
        direct_inverse.py
        features.py
        lexbase.py              [classes shared by lexicon.py and morphotactics.py]
        lexical_items.py        [should probably be merged with lexicon.py]
        lexicon.py
        morphotactics.py
        negation.py
        parameters.py
        verbal_features.py
        word_order.py
        yes_no_questions.py
    regression-tests/ [code and resources for regression testing]
    sample-choices/   [sample choices files displayed on the live site]
    sql_profiles/     [code and resources for MatrixTDB]
    templates/        [inputs for test-by-generation]
    tests/            [unit tests for the general system]
    web/              [intended place for web questionnaire related files]
    choices.py        [classes for working with choices files]
    customize.py      [primary code for customizing grammars]
    def_check.py      [module to check matrixdef]
    deffile.py        [module to interpret matrixdef for web presentation]
    generate.py       [module to aid in test-by-generation]
    __init__.py       [gmcs packaging module]
    matrix.cgi        [CGI script to handle web requests]
    matrix.css        [style file for the questionnaire]
    matrixdef         [website content definition]
    matrix.js         [website-related functions]
    profiles.py       [MatrixTDB ... might be from old version]
    randgram.py       [out-of-date script to create a semi-random grammar]
    tdl.py            [module for dealing with TDL files]
    tdltest.py        [code to test tdl.py. Should probably be folded into unit tests]
    utils.py          [module with various helper functions]
    validate.py       [module to validate a choices file]
gmmt/        [resources for the "massively multilingual translation" task]
lisp/        [various lisp scripts for Developers]
matrix-core/ [the Matrix grammar files]
modules/     [snippets of TDL and notes about analyses]
install      [bash script for installing the code (e.g. to the live site)]
matrix.py    [Python script for running customize, tests, install, etc.]

Home | Forum | Discussions | Events

Clone this wiki locally