Create a guideline for Project Layout

[sentenz]

Project Layout Guide

• 1. Category
  • 1.1. C/C++
    • 1.1.1. Library
    • 1.1.2. Application
  • 1.2. Go
    • 1.2.1. Package
    • 1.2.2. Application
  • 1.3. Python
    • 1.3.1. Module
    • 1.3.2. Application
  • 1.4. Ansible
    • 1.4.1. Collection
    • 1.4.2. Playbook
  • 1.5. Terraform
    • 1.5.1. Module
    • 1.5.2. Project
• 2. References

1. Category

1.1. C/C++

The Pitchfork Layout (PFL) is a convention for arranging source, build, and resource files in a file system to support uniformity, comprehensibility, and partitioning.

1.1.1. Library

1. Layout and Structure

   NOTE Replace <...> brackets with the library-specific information.

   <library>/
   │
   . `Hierarchical Structure`
   │
   ├── inc/
   │   └── <library>/
   │       └── file.h
   │
   ├── src/
   │   │
   │   . `Functional-based Structure`
   │   │
   │   ├── file.c
   │   ├── file_test.cc
   │   ├── CMakeLists.txt
   │   └── README.md
   │
   ├── external/
   │   ├── <third-party>/
   │   └── README.md
   │
   ├── test/
   │   ├── performance/
   │   │   ├── benchmark_test.cc
   │   │   └── CMakeLists.txt
   │   ├── CMakeLists.txt
   │   └── README.md
   │
   ├── examples/
   │   ├── example.c
   │   ├── CMakeLists.txt
   │   └── README.md
   │
   ├── docs/
   │   ├── decisions/
   │   │   └── adr-<topic>.md
   │   └── README.md
   │
   ├── build/
   │   ├── debug/
   │   │   └── lib/
   │   │       └── libprojectd.a
   │   └── release/
   │       └── lib/
   │           └── libproject.a
   │
   ├── scripts/
   │   ├── setup.sh
   │   └── README.md
   │
   ├── CMakeLists.txt
   ├── CMakePresets.json
   ├── LICENSE
   └── README.md

2. Files and Folders

1.1.2. Application

1. Layout and Structure

   NOTE Replace <...> brackets with the application-specific information.

   /<project>
   │
   . `Hierarchical Structure`
   │
   ├── /internal
   │   │
   │   . `Layered Structure`
   │   │
   │   ├── /presentation
   │   │   ├── /components
   │   │   │   └── component.js
   │   │   ├── /screens
   │   │   │   └── screen.js
   │   │   ├── /styles
   │   │   │   └── style.css
   │   │   └── README.md
   │   │
   │   ├── /application
   │   │   ├── /dtos
   │   │   │   ├── product_dto.hpp
   │   │   │   ├── product_dto.cpp
   │   │   │   ├── product_dto_test.cc
   │   │   │   └── CMakeLists.txt
   │   │   ├── /events
   │   │   │   ├── order_updated_event.hpp
   │   │   │   ├── order_updated_event.cpp
   │   │   │   ├── order_updated_event_test.cc
   │   │   │   └── CMakeLists.txt
   │   │   ├── /handlers
   │   │   │   ├── update_order_handler.hpp
   │   │   │   ├── update_order_handler.cpp
   │   │   │   ├── update_order_handler_test.cc
   │   │   │   └── CMakeLists.txt
   │   │   ├── /services
   │   │   │   ├── product_service.hpp
   │   │   │   ├── product_service.cpp
   │   │   │   ├── product_service_test.cc
   │   │   │   └── CMakeLists.txt
   │   │   ├── CMakeLists.txt
   │   │   └── README.md
   │   │
   │   ├── /domain
   │   │   ├── /models
   │   │   │   ├── model.hpp
   │   │   │   ├── model.cpp
   │   │   │   ├── model_test.cc
   │   │   │   └── CMakeLists.txt
   │   │   ├── /services
   │   │   │   ├── service.hpp
   │   │   │   ├── service.cpp
   │   │   │   ├── service_test.cc
   │   │   │   └── CMakeLists.txt
   │   │   ├── /repositories
   │   │   │   ├── user_repository.hpp
   │   │   │   ├── user_repository.cpp
   │   │   │   ├── user_repository_test.cc
   │   │   │   └── CMakeLists.txt
   │   │   ├── CMakeLists.txt
   │   │   └── README.md
   │   │
   │   ├── /infrastructure
   │   │   ├── /database
   │   │   │   ├── /repositories
   │   │   │   │   │── repository.hpp
   │   │   │   │   │── repository.cpp
   │   │   │   │   │── repository_test.cc
   │   │   │   │   │── repository_interface.hpp
   │   │   │   │   └── CMakeLists.txt
   │   │   │   ├── /schema
   │   │   │   │   └── model_schema.sql
   │   │   │   ├── /migrations
   │   │   │   │   └── 20210422123456-migration.sql
   │   │   │   └── CMakeLists.txt
   │   │   ├── /mqtt
   │   │   │   │── mqtt_service.hpp
   │   │   │   │── mqtt_service.cpp
   │   │   │   │── mqtt_service_test.cc
   │   │   │   │── mqtt_service_interface.hpp
   │   │   │   └── CMakeLists.txt
   │   │   │── /crypto
   │   │   │   │── crypto_service.hpp
   │   │   │   │── crypto_service.cpp
   │   │   │   │── crypto_service_test.cc
   │   │   │   │── crypto_service_interface.hpp
   │   │   │   └── CMakeLists.txt
   │   │   ├── CMakeLists.txt
   │   │   └── README.md
   │   │
   │   ├── CMakeLists.txt
   │   └── README.md
   │
   ├── /external
   │   ├── /postgresql
   │   ├── /mosquitto
   │   ├── /mbedtls
   │   ├── CMakeLists.txt
   │   └── README.md
   │
   ├── /test
   │   ├── /performance
   │   │   ├── benchmark_test.cc
   │   │   └── CMakeLists.txt
   │   ├── /integration
   │   │   ├── top_down_test.cc
   │   │   └── CMakeLists.txt
   │   ├── /e2e
   │   │   ├── scenario_test.cc
   │   │   └── CMakeLists.txt
   │   ├── CMakeLists.txt
   │   └── README.md
   │
   ├── /examples
   │   ├── example.c
   │   ├── CMakeLists.txt
   │   └── README.md
   │
   ├── /tools
   │   ├── /cmake
   │   └── README.md
   │
   ├── /docs
   │   ├── /adr
   │   │   ├── merging-strategy-adr.md
   │   │   └── branching-strategy-adr.md
   │   ├── api.md
   │   └── README.md
   │
   ├── /build
   │   ├── /bin
   │   │   └── project.exe
   │   ├── /lib
   │   │   └── project.a
   │   └── /cmake
   │
   ├── /scripts
   │   ├── setup.sh
   │   └── README.md
   │
   ├── /data
   │   ├── dataset.csv
   │   └── README.md
   │
   ├── CMakeLists.txt
   ├── LICENSE
   └── README.md

2. Files and Folders

1.2. Go

1.2.1. Package

1. Layout and Structure

   NOTE Replace <...> brackets with the package-specific information.

   <package>/
   │
   . `Hierarchical Structure`
   │
   ├── cmd/
   │   └── app/
   │       └── main.go
   │
   ├── internal/
   │   └── pkg/
   │       └── helper.go
   │
   ├── pkg/
   │   └── <package>/
   │       └── <package>.go
   │
   ├── LICENSE
   ├── go.mod
   ├── go.sum
   └── README.md

2. Files and Folders

1.2.2. Application

Go Project Layout for applications. It is a set of common historical and emerging project layout patterns in the Go.

NOTE It's not an official standard defined by the core Go dev team.

1. Layout and Structure

   NOTE Replace <...> brackets with the application-specific information.

2. Files and Folders

   • /cmd

     Contains the main application entry point files for the project, with the directory name matching the name for the binary.

   • /internal

     Private application and library code.

   • /pkg

     Library code to use by external applications.

   • /vendor

     Application dependencies.

   • /configs

     Configuration file templates or default configs.

   • /init

     System init (systemd, upstart, sysv) and process manager/supervisor (runit, supervisord) configs.

   • /scripts

     Scripts to perform various build, install, analysis, etc operations.

   • /build

     Packaging and Continuous Integration.

   • /deployments

     IaaS, PaaS, system and container orchestration deployment configurations and templates (docker-compose, kubernetes/helm, mesos, terraform, bosh).

   • /test

     Additional external test apps and test data.

   • /docs

     Design and user documents (in addition to godoc generated documentation).

   • /tools

     Supporting tools for this project.

   • /examples

     Examples for your applications and/or public libraries.

   • /external

     External helper tools, forked code and other 3rd party utilities (e.g. Swagger UI).

   • /githooks

     Git hooks files.

   • /assets

     Other assets to go along with your repository (images, logos, etc).

   • /website

     Place to put the project's website data if you are not using GitHub pages.

   • /api

     Protocol definition files.

   • /web

     Web application specific components: static web assets, server side templates and SPAs.

   • go.mod

     Package dependency and version management.

   • go.sum

     TODO

1.3. Python

Python Project Layout using best practices.

1.3.1. Module

1. Layout and Structure

   NOTE Replace <...> brackets with the module-specific information.

2. Files and Folders

1.3.2. Application

1. Layout and Structure

   NOTE Replace <...> brackets with the application-specific information.

2. Files and Folders

   • /module

   • /docs

     Package reference documentation.

   • /tests

     Package unit tests, execution tests, integration tests.

   • /bin

     Holds executable files.

   • setup.py

     Package and distribution management.

   • requirements.txt

     Package dependency and version management.

1.4. Ansible

1.4.1. Collection

An Ansible Collection packages and distributes roles, modules and plugins. Organizing Ansible Collection content under a directory structure, it is crucial to follow a specific layout to ensure that Ansible can recognize and properly utilize the collection.

1. Layout and Structure

   The project path is based on the collections/ansible_collections keywords and the Fully Qualified Collection Name (FQCN) <namespace>.<collection>, in accordance with the naming convention.

   NOTE None of the directories are required.

   NOTE Replace <...> brackets with the project-specific information.

   <project>-ansible-collection/
   └── collections/
       └── ansible_collections/
           └── <namespace>/
               └── <collection>/
                   │
                   ├── meta/
                   │   └── runtime.yml
                   │
                   ├── plugins/
                   │   ├── action/
                   │   ├── become/
                   │   ├── cache/
                   │   ├── callback/
                   │   ├── cliconf/
                   │   ├── connection/
                   │   ├── filter/
                   │   ├── httpapi/
                   │   ├── inventory/
                   │   ├── lookup/
                   │   ├── module_utils/
                   │   ├── modules/
                   │   │   ├── __init__.py
                   │   │   └── <module>.py
                   │   ├── netconf/
                   │   ├── shell/
                   │   ├── strategy/
                   │   ├── terminal/
                   │   ├── test/
                   │   └── vars/
                   │
                   ├── roles/
                   │   └── <role>/
                   │       ├── tasks/
                   │       │   └── main.yml
                   │       ├── handlers/
                   │       │   └── main.yml
                   │       ├── templates/
                   │       │   └── config.j2
                   │       ├── files/
                   │       │   └── docker-compose.yml
                   │       ├── vars/
                   │       │   └── main.yml
                   │       ├── defaults/
                   │       │   └── main.yml
                   │       ├── meta/
                   │       │   └── main.yml
                   │       └── README.md
                   │
                   ├── playbooks/
                   │   ├── files/
                   │   ├── vars/
                   │   ├── templates/
                   │   └── tasks/
                   │
                   ├── docs/
                   │
                   ├── tests/
                   │
                   ├── galaxy.yml
                   └── README.md

2. Files and Folders

   • galaxy.yml

     The source of the metadata file used to generate a collection on Galaxy and to build a collection artifact.

   • Modules

   • /meta

   • runtime.yml

   • plugins/

     The Collections plugins directory can be used to ship various plugins inside an Ansible collection. Each plugin is placed in a folder that is named after the type of plugin it is in. It can also include the module_utils and modules directory that would contain module utils and modules respectively.

   • /modules

     At least one plugin required.

   • __init__.py

     A required empty file to initialize namespace and allow Python to import the files.

   • /inventory

     At least one plugin required.

   • Roles

   • Playbooks

   • /docs

     Describes the use of the roles, plugins and role requirements provided by the collection.

1.4.2. Playbook

Ansible Playbook directory layout organizes tasks in roles, with a inventory file for each environment and a playbooks.

1. Layout and Structure

   NOTE Replace <...> brackets with the project-specific information.

   <project>-ansible-playbook/
   ├── collections/
   │   └── ansible_collections/
   │       └── <namespace>/
   │           └── <collection>/
   │               ├── docs/
   │               ├── plugins/
   │               │   ├── modules/
   │               │   ├── lookup/
   │               │   └── ...
   │               ├── roles/
   │               │   ├── <role>/
   │               │   └── ...
   │               ├── tests/
   │               ├── galaxy.yml
   │               └── README.md
   │
   ├── roles/
   │   ├── <role>/
   │   │   ├── tasks/
   │   │   │   └── main.yml
   │   │   ├── handlers/
   │   │   │   └── main.yml
   │   │   ├── templates/
   │   │   │   └── config.j2
   │   │   ├── files/
   │   │   │   ├── bar.txt
   │   │   │   └── foo.sh
   │   │   ├── vars/
   │   │   │   └── main.yml
   │   │   ├── defaults/
   │   │   │   └── main.yml
   │   │   ├── meta/
   │   │   │   └── main.yml
   │   │   └── README.md
   │   └── ...
   │
   ├── playbooks/
   │   ├── <playbook>.yml
   │   └── ...
   │
   ├── inventory/
   │   ├── dev/
   │   │   ├── hosts
   │   │   ├── group_vars/
   │   │   │   ├── <group>.yml
   │   │   │   └── ...
   │   │   └── host_vars/
   │   │       ├── <host>.yml
   │   │       └── ...
   │   ├── stage/
   │   └── prod/
   │
   ├── site.yml
   ├── requirements.yml
   ├── ansible.cfg
   └── README.md

2. Files and Folders

1.5. Terraform

1.5.1. Module

Terraform modules define self-contained, reusable resources of Infrastructure-as-Code (IaC). The Terraform module structure for reusable modules distributed in separate repositories.

1. Layout and Structure

   NOTE Replace <...> brackets with the module-specific information.

   terraform-<provider>-<project>/
   ├── modules/
   │   ├── <module-a>/
   │   │   ├── main.tf
   │   │   ├── variables.tf
   │   │   ├── outputs.tf
   │   │   ├── versions.tf
   │   │   └── README.md
   │   └── <module-b>/
   │       ├── main.tf
   │       ├── variables.tf
   │       ├── outputs.tf
   │       ├── versions.tf
   │       └── README.md
   │
   ├── examples/
   │   ├── simple/
   │   │   ├── main.tf
   │   │   ├── variables.tf
   │   │   ├── outputs.tf
   │   │   ├── versions.tf
   │   │   └── README.md
   │   └── complete/
   │       ├── main.tf
   │       ├── variables.tf
   │       ├── outputs.tf
   │       ├── versions.tf
   │       └── README.md
   │
   ├── tests/
   │   ├── unit/
   │   │   ├── bucket_name.tftest.hcl
   │   │   ├── input_validation.tftest.hcl
   │   │   └── provider.tftest.hcl
   │   ├── integration/
   │   │   └── modules.tftest.hcl
   │   └── README.md
   │
   ├── main.tf
   ├── variables.tf
   ├── outputs.tf
   ├── versions.tf
   ├── LICENSE
   └── README.md

2. Files and Folders

1.5.2. Project

1. Layout and Structure

   NOTE Replace <...> brackets with the project-specific information.

   terraform-<project>/
   ├── modules/
   │   ├── <storage>/
   │   │   ├── main.tf
   │   │   ├── variables.tf
   │   │   └── outputs.tf
   │   ├── <network>/
   │   │   ├── main.tf
   │   │   ├── variables.tf
   │   │   └── outputs.tf
   │   └── <compute>/
   │       ├── main.tf
   │       ├── variables.tf
   │       └── outputs.tf
   │
   ├── environments/
   │   ├── dev/
   │   │   ├── main.tf
   │   │   ├── variables.tf
   │   │   └── backend.tf
   │   ├── stage/
   │   │   ├── main.tf
   │   │   ├── variables.tf
   │   │   └── backend.tf
   │   └── prod/
   │       ├── main.tf
   │       ├── variables.tf
   │       └── backend.tf
   │
   ├── main.tf
   ├── variables.tf
   ├── outputs.tf
   ├── providers.tf
   ├── versions.tf
   ├── backend.tf
   └── README.md

2. Files and Folders

3. Examples and Explanations

   The Terraform project sets up a basic AWS infrastructure. This includes a VPC with public and private subnets and an EC2 instance on the public subnet.

   • main.tf

     The root main.tf file contains the primary Terraform configuration and calls the modules.

     module "network" {
       source = "./modules/network"
     
       vpc_cidr = var.vpc_cidr
     }
     
     module "compute" {
       source = "./modules/compute"
     
       subnet_id     = module.network.public_subnet_id
       instance_type = var.instance_type
     }

   • variables.tf

     Defines all the input variables for the root module.

     variable "vpc_cidr" {
       description = "The CIDR block for the VPC"
       type        = string
       default     = "10.0.0.0/16"
     }
     
     variable "aws_region" {
       description = "The AWS region to deploy resources"
       type        = string
       default     = "us-east-1"
     }
     
     variable "instance_type" {
       description = "Type of the EC2 instance"
       type        = string
     }

   • outputs.tf

     Specifies the outputs from the root module.

     output "vpc_id" {
       description = "The ID of the VPC"
       value       = module.network.vpc_id
     }
     
     output "public_subnet_id" {
       description = "The ID of the public subnet"
       value       = module.network.public_subnet_id
     }
     
     output "instance_public_ip" {
       description = "The public IP of the EC2 instance"
       value       = module.compute.instance_public_ip
     }

   • providers.tf

     Configures the providers used in the project.

     provider "aws" {
       region = var.aws_region
     }

   • versions.tf

     Sets the required Terraform and provider versions.

     terraform {
       required_version = ">= 1.0.0"
     
       required_providers {
         aws = {
           source  = "hashicorp/aws"
           version = "~> 4.0"
         }
       }
     }

   • backend.tf

     Configures the backend of the State Management for remote state storage.

     NOTE Since the state is stored remotely in an S3 bucket, ensure to enable versioning and server-side encryption on the bucket.

     terraform {
       backend "s3" {
         bucket = "my-terraform-state-bucket"
         key    = "global/terraform.tfstate"
         region = var.aws_region
       }
     }

   • modules/network/main.tf

     Defines the VPC and subnet resources.

     resource "aws_vpc" "this" {
       cidr_block = var.vpc_cidr
     
       tags = {
         Name = "my-vpc"
       }
     }
     
     resource "aws_subnet" "public" {
       vpc_id                  = aws_vpc.this.id
       cidr_block              = cidrsubnet(var.vpc_cidr, 8, 0)
       availability_zone       = data.aws_availability_zones.available.names[0]
       map_public_ip_on_launch = true
     
       tags = {
         Name = "public-subnet"
       }
     }
     
     data "aws_availability_zones" "available" {
       state = "available"
     }

   • modules/network/variables.tf

     Declares variables used in the network module.

     variable "vpc_cidr" {
       description = "CIDR block for the VPC"
       type        = string
     }

   • modules/network/outputs.tf

     Defines outputs from the network module.

     output "vpc_id" {
       description = "The ID of the VPC"
       value       = aws_vpc.this.id
     }
     
     output "public_subnet_id" {
       description = "The ID of the public subnet"
       value       = aws_subnet.public.id
     }

   • modules/compute/main.tf

     Creates an EC2 instance.

     resource "aws_instance" "web" {
       ami                    = data.aws_ami.amazon_linux.id
       instance_type          = var.instance_type
       subnet_id              = var.subnet_id
       associate_public_ip_address = true
     
       tags = {
         Name = "web-server"
       }
     }
     
     data "aws_ami" "amazon_linux" {
       most_recent = true
       owners      = ["amazon"]
     
       filter {
         name   = "name"
         values = ["amzn2-ami-hvm-*-x86_64-gp2"]
       }
     }

   • modules/compute/variables.tf

     Declares variables used in the compute module.

     variable "subnet_id" {
       description = "The ID of the subnet to deploy the instance"
       type        = string
     }
     
     variable "instance_type" {
       description = "Type of the EC2 instance"
       type        = string
     }

   • modules/compute/outputs.tf

     Defines outputs from the compute module.

     output "instance_id" {
       description = "The ID of the EC2 instance"
       value       = aws_instance.web.id
     }
     
     output "instance_public_ip" {
       description = "The public IP address of the EC2 instance"
       value       = aws_instance.web.public_ip
     }

   • environments/dev/main.tf

     References the root module with development-specific variables.

     module "root" {
       source = "../../"
     
       vpc_cidr      = var.vpc_cidr
       aws_region    = var.aws_region
       instance_type = var.instance_type
     }

   • environments/dev/variables.tf

     Provides variable values specific to the dev environment.

     NOTE Avoid including sensitive data, use environment variables or a secret management tool to handle sensitive data.

     variable "vpc_cidr" {
       description = "The CIDR block for the VPC."
       type        = string
       default     = "10.1.0.0/16"
     }
     
     variable "aws_region" {
       description = "The AWS region where resources will be deployed."
       type        = string
       default     = "us-west-2"
     }
     
     variable "instance_type" {
       description = "The type to provide an EC2 instance resource."
       type        = string
       default     = "t2.micro"
     }

   • environments/dev/backend.tf

     Configures the backend for the dev environment.

     terraform {
       backend "s3" {
         bucket = "my-terraform-state-bucket"
         key    = "dev/terraform.tfstate"
         region = var.aws_region
       }
     }

   • environments/prod/main.tf

     References the root module with production-specific variables.

     module "root" {
       source = "../../"
     
       vpc_cidr      = var.vpc_cidr
       aws_region    = var.aws_region
       instance_type = var.instance_type
     }

   • environments/prod/variables.tf

     Provides variable values specific to the prod environment.

     NOTE Avoid including sensitive data, use environment variables or a secret management tool to handle sensitive data.

     variable "vpc_cidr" {
       description = "The CIDR block for the VPC."
       type        = string
       default     = "10.2.0.0/16"
     }
     
     variable "aws_region" {
       description = "The AWS region where resources will be deployed."
       type        = string
       default     = "us-east-1"
     }
     
     variable "instance_type" {
       description = "The type to provide an EC2 instance resource."
       type        = string
       default     = "t3.large"
     }

   • environments/prod/backend.tf

     Configures the backend for the prod environment.

     terraform {
       backend "s3" {
         bucket = "my-terraform-state-bucket"
         key    = "prod/terraform.tfstate"
         region = var.aws_region
       }
     }

2. References

• Sentenz Project Layout article.