Skip to content
This repository has been archived by the owner on Apr 13, 2019. It is now read-only.
/ Observatory-retired Public archive

A Python based dashboard for the Rensselaer Center for Open Source Software

License

Notifications You must be signed in to change notification settings

rcos/Observatory-retired

Repository files navigation

Observatory

A Python based dashboard for the Rensselaer Center for Open Source Software

Licensing

Observatory is licensed under the ISC License, which is equivalent to the two-clause BSD license.

Features

Observatory is intended to track a group of open source projects across their source code repositories and blogs. It also includes links to the websites and wikis each project, but doesn't scan those for any updates.

Observatory can directly access repositories (for git, hg, and svn), so the "branches don't work" problem has been eliminated. With this also comes a full redesign. Individual commits are visible in detail, and each project has its own page, which shows authors, smaller contributors, blog posts, commits, and screenshots. Additionally, there is a "feed" view that shows all of the most recent actions.

Setup

Observatory is pretty easy to set up. Excluding the wsgi setup that you would want for a production environment (resources on which are best found elsewhere, most of the dependencies) are packed in the observatory/lib directory.

Observatory requires Python 2.7+ (not Python 3+) and is built on the Django web framework. It also depends on the several packages and version control systems described in the requirements.txt file in the observatory directory.

If you want to get started quickly, then install vagrant and puppet librarian and invoke the make test command. This will produce a working demo site that is listening on localhost:8000. For more details on configuration consult the example puppet configuration in puppet/base.pp. Puppet modules will be installed in the modules directory.

Fetching

Since it deals primarily with external data sources, Observatory uses a set of scripts to pull data down and store it in the local database. There scripts are located in the observatory/dashboard/fetch directory. There are two primary scripts, fetch_blogs.py and fetch_repositories.py, the purpose of each being fairly obvious. For development purposes, there is also a fetch.py script (located up a directory from the others) which runs both fetch scripts.

The primary reason for splitting up the fetching process into two scripts is that previously, the update process was somewhat unpleasant for users: either click the update link (which didn't always work), or wait up to an hour. While blogs update infrequently, commits (should) happen much more frequently. Therefore, it is ideal to run the repository fetching script more often than the blog fetching one. Both scripts tend to run fairly quickly (with the exception of the initial import).

As threads and Python don't get along so well, fetching instead uses multiple processes (via the subprocess module). The amount of processes can be configured in the settings.py file. The usual rules about processes per CPU core don't necessarily apply here, because a lot of the time (especially for blog fetching) is spent waiting for servers to respond.

Development

If you would like to help out with Observatory, that is probably a good thing. You can either use the fork + pull request feature of Github, or just send in patches (use Issues or email).

To set up a development environment is very easy. Just follow the instructions in the "Setup" above using vagrant. Note that you must run make test every time you make a change as we copy the source directory, rather than symlink it, for security reasons.

If you don't want to use vagrant the following instructions will prove useful.

There are two scripts that are useful: (Scripts need to be run from the Observatory directory.)

  • observatory/dashboard/demo.py loads up a demo set of projects and users.
  • observatory/dashboard/fetch.py runs the repository and blog fetch scripts.

(Optional) If you don't want the demo set, you need to use syncdb manually:

  • observatory/manage.py syncdb creates some of the necessary database tables.
  • manage.py migrate dashboard creates some of the necessary database tables.
  • manage.py migrate todo creates the rest of the tables.

Once these have been run, the database will be populated with the most recent commits and blog posts for the demo projects. Then, you can just run the server with ./manage.py runserver in the observatory directory, and access it (typically) at http://localhost:8000/projects.

If you want to provide external access to the server, this can be done by running the server with ./manage.py runserver 0.0.0.0:[port number] where [port number] is any open port you want to bind to observatory.

If you're looking for something to work on, check out the Issues page or ask me, via email or on #rcos on irc.freenode.net.

Page Descriptions

Projects

The front page of Observatory, gives a quick list of all projects listed in Observatory with links to each project page, blog, and repository. A status icon is next to each project title in the form of a emoticon used to denote how recently the project has been updated. At the top of the page, a graph breaks down the overall status of all projects by displaying how many projects, repositories, and blogs have been updated within the past week.

Feed

An aggregate feed of all blog posts and commits to any project listed in Observatory. Contains the information of the commit or an excerpt of the blog post as well as who published it to which project.

Posts

An aggregate feed of all full blog posts for the projects in Observatory.

People

A listing of all developers logged in the Obseratory system. Clicking on a name will bring you to the user's page.

User Page

Contains contact email of the developer and a list of all projects started by them under the project heading. Listed under Contributed to is all the projects they committed changes to the project's repo. Only projects currently in the Observatory system will be listed here.

Project Page

Provides a listing of all authors and contributors to the project. Only users listed within the Observatory system will be added, not anyone who commits to the repo. At the bottom there are also small feeds for commits and blog posts.