Skip to content

Commit

Permalink
Switch to mkdocs and update documentation.
Browse files Browse the repository at this point in the history
  • Loading branch information
tibuch committed Sep 26, 2024
1 parent 1bf98e1 commit 58b0eeb
Show file tree
Hide file tree
Showing 40 changed files with 2,447 additions and 531 deletions.
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
# GitHub syntax highlighting
pixi.lock linguist-language=YAML linguist-generated=true
31 changes: 16 additions & 15 deletions .github/workflows/documentation.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,23 @@ on: [push, pull_request, workflow_dispatch]
permissions:
contents: write
jobs:
docs:
deploy:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-python@v3
- name: Install dependencies
- uses: actions/checkout@v4
- name: Configure Git Credentials
run: |
pip install sphinx furo myst-parser sphinx-inline-tabs sphinx-copybutton
- name: Sphinx build
run: |
sphinx-build docs/source docs/_build
- name: Deploy
uses: peaceiris/actions-gh-pages@v3
if: ${{ github.event_name == 'push' && github.ref == 'refs/heads/main' }}
git config user.name github-actions[bot]
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
- uses: actions/setup-python@v5
with:
python-version: 3.x
- run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
- uses: actions/cache@v4
with:
publish_branch: gh-pages
github_token: ${{ secrets.GITHUB_TOKEN }}
publish_dir: docs/_build/
force_orphan: true
key: mkdocs-material-${{ env.cache_id }}
path: .cache
restore-keys: |
mkdocs-material-
- run: pip install mkdocs-material mkdocs-git-committers-plugin-2 mkdocs-git-revision-date-localized-plugin mkdocstrings mkdocstrings-python
- run: mkdocs gh-deploy --force
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -209,3 +209,6 @@ $RECYCLE.BIN/

# Windows shortcuts
*.lnk
# pixi environments
.pixi
*.egg-info
1 change: 1 addition & 0 deletions .pre-commit-config.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ repos:
- id: end-of-file-fixer
- id: trailing-whitespace
- id: check-yaml # checks for correct yaml syntax for github actions ex.
args: [--unsafe]
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.6.3
hooks:
Expand Down
26 changes: 1 addition & 25 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,30 +1,6 @@
<!-- start introduction -->
# Image Processing & Analysis Project Template
Image Processing and Analysis (IPA), transforming raw image data into quantitative measurements, is an important part of life sciences. However, as projects progress, IPA pipelines grow increasingly complex, involving the management of multiple tools, versions, and dependencies, which can be a challenging task. To tackle this complexity, we introduce the IPA Project [Copier](https://copier.readthedocs.io/en/stable/) Template. This template simplifies the initiation of new IPA projects and streamlines the reorganization of existing ones, contributing to research that is more Findable, Accessible, Interoperable, and Reusable (FAIR) promoting efficiency, reproducibility, and reliability in life sciences research.

## Usage
* **Newcomers:**
If you're new to IPA, this template provides a structured starting point to simplify your project setup.
* **Experienced Users:**
Seasoned practitioners will appreciate the quicker setup for new projects.

## Features
* Easy setup and configuration.
* Standardized directory structure for consistency.
* Configured [MkDocs](https://www.mkdocs.org/) documentation template.
* Configured `environment.yaml` to get started with image processing and analysis in Python.
<!-- end introduction -->

## Quick Start
1. Create a Copier environment:
```bash
mamba create -n copier python=3.9 copier -c conda-forge
mamba activate copier
```
2. Copy the template:
```bash
copier copy git+https://github.com/fmi-faim/ipa-project-template faim_demo-project
```
Welcome to the Image Processing & Analysis Project Template. The project structure initialized by this template organizes data and software in a way that supports growth of your research project in size as well as in complexity without sacrificing clarity.

For detailed instructions, please refer to the [documentation](https://fmi-faim.github.io/ipa-project-template/setup.html).

Expand Down
File renamed without changes.
29 changes: 29 additions & 0 deletions docs/index.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,29 @@
{!README.md!lines=2-3}

## Prerequisites
In our project template we use [pixi.sh](https://pixi.sh) as environment manager and [copier](https://copier.readthedocs.io/en/stable/) to generate a copy of the project template. The installation instructions for pixi can be found [here](https://pixi.sh/latest/#installation).
After successful installation of pixi and restarting your terminal, you can type `pixi info` to get the current version of pixi.

??? info "Update pixi"
If you have installed pixi already, please make sure to update it to the latest version by running:
```bash
pixi self-update
```

Now install copier as a global package with pixi:
```bash
pixi global install copier
```

## Generate an IPA Project with Copier
Copier will ask you a series of questions whose answers will be used to generate a tailored IPA project for you. To start the process run the following and replace `my-ipa-project` with the name of your project directory:
```bash
pixi x copier copy git+https://github.com/fmi-faim/ipa-project-template my-ipa-project
```

!!! success "Congratulations!"
You have created a new personalized IPA project! Now you can change into the project root-directory with
```bash
cd my-ipa-project
```
or open the directory in your file browser.
34 changes: 34 additions & 0 deletions docs/ipa_structure.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# IPA Project Structure
The IPA projects structure aims to support the whole development cycle of your project. It is designed to be flexible and scalable, allowing you to grow your project in size and complexity without sacrificing clarity.

<figure markdown="span">
![Project Directory Structure](resources/directory_structure_overview.svg)
<figcaption>Overview of the IPA project structure. Everything related to your project is located within the __project\_root__ directory.</figcaption>
</figure>

The project template is organized to support clear separation between developing a workflow and running a workflow on data. Furthermore, there is a clear separation between __raw\_data__, __processed\_data__, and __results__. Separating these concepts helps to keep the project organized and supports reproducibility.

The projects structure is meant to be used with a version control system like git. But we configured the template such that all data directories are excluded from version control.

## Data
The __raw\_data__ directory is intended to hold the raw data as saved by the acquisition systems. In __processed\_data__ outputs generated by scripts in the __source__ directory are stored. These data could for example be segmentation masks or tables of extracted measurements. The __results__ directory usually contains a condensed version of __processed\_data__, like figures, tables, and final reports.

!!! note Raw Data
If you are working on a unix based system, we recommend to create symbolic links to your raw data in the `raw_data` directory. This way you can keep the original data in its original location and still have it accessible in your project.

## Workflow Development
When developing a new image processing or analysis step you most likely need to test things out first. The __sandbox__ directory is meant for this purpose. It is a place to test out new ideas and develop new processing steps. Anything inside the __sandbox__ is not considered final and is not used to generate __processed\_data__ or __results__.

Once a processing step is stable, and you want to use it to generate reproducible outputs, you will move it to the __source__ directory. The __source__ directory contains all scripts that are used to generate __processed\_data__ and __results__. Moving a script from __sandbox__ to __source__ also means that you should start documenting the script and make it configurable. While you can hard-code paths and parameters in __sandbox__, you should use configuration files in __source__ to facilitate re-using the same script on different data.

## Applying a Workflow
With a versioned processing script ready in __source__ it is time to apply it to your data. To keep track of configuration files and outputs generated by the script, we use the __runs__ directory. Where you should create a new sub-directory for each dataset. Inside this sub-directory the config files for the scripts are stored and executed from. The outputs are stored in a sub-directory of __processed\_data__ or __results__ with the same name as the run-directory.

!!! info Versioning
The runs directory is included in version control but is up to you to commit them after each run. This way you can keep track of which config file was used with which version of the code to produce a given output.

## Documentation
In __docs__ a mkdocs project is set up to render a nice website with your documentation. The documentation is meant to be used to describe the project and the processing steps in detail, in particular how to execute them.

!!! info Render Documentation
To render the documentation locally you can run `pixi run show_docs`. This will start a local server and you can view the documentation in your browser. To render a static version use the command `pixi run build_docs` and your documentation will be available in the __root\_dir/site__ directory.
Loading

0 comments on commit 58b0eeb

Please sign in to comment.