S3 Manager 💾 ☁️ 👨‍💼

S3 Manager is robust, powerful and an all-in-one tool that provides a number of operations to enhance your experience with AWS S3 buckets. It offers an array of features from setting up file cleaning rules, search string in files, manipulating tags, and much more. It's designed to help you automate and optimize your operations, lowering your public cloud costs in the process.

Available Subcommands

S3 Manager provides the following subcommands:

• clean
• search
• tags
• versioning
• bucketpolicy
• transferacceleration

Configuration

Usage:
  s3-manager [flags]
  s3-manager [command]

Available Commands:
  bucketpolicy         Shows/sets the bucket policy configuration of the target bucket
  clean                Finds and clears desired files by a pre-configured rule set
  completion           Generate the autocompletion script for the specified shell
  help                 Help about any command
  search               Searches the files which has desired substrings in it
  tags                 Shows/sets the tagging configuration of the target bucket
  transferacceleration Shows/sets the transfer acceleration configuration of the target bucket
  versioning           Shows/sets the versioning configuration of the target bucket

Flags:
  --access-key string         Access key credential to access S3 bucket, this value also can be passed via "AWS_ACCESS_KEY" environment variable (default "")
  --banner-file-path string   Relative path of the banner file (default "banner.txt")
  --bucket-name string        Name of the target bucket on S3, this value also can be passed via "AWS_BUCKET_NAME" environment variable (default "")
  -h, --help                  Help for s3-manager
  --region string             Region of the target bucket on S3, this value also can be passed via "AWS_REGION" environment variable (default "")
  --secret-key string         Secret key credential to access S3 bucket, this value also can be passed via "AWS_SECRET_KEY" environment variable (default "")
  --verbose                   Verbose output of the logging library (default false)
  -v, --version               Version for s3-manager

Use "s3-manager [command] --help" for more information about a command.

Installation

Binary

Binary can be downloaded from Releases page.

Homebrew

This project can also be installed with Homebrew:

$ brew tap bilalcaliskan/tap
$ brew install bilalcaliskan/tap/s3-manager

Examples

Here is the couple of examples:

# bucket cleaning with specified ruleset
$ export AWS_ACCESS_KEY=${YOUR_ACCESS_KEY}
$ export AWS_SECRET_KEY=${YOUR_SECRET_KEY}
$ export AWS_REGION=${YOUR_REGION}
$ export AWS_BUCKET_NAME=${YOUR_BUCKET_NAME}
$ s3-manager clean --min-size-mb=1 --max-size-mb=1000 --keep-last-n-files=2 --sort-by=lastModificationDate

# set bucket versioning as enabled
$ s3-manager versioning set enabled --access-key ${YOUR_ACCESS_KEY} --secret-key ${YOUR_SECRET_KEY} --bucketName ${TARGET_BUCKET_NAME} --region ${TARGET_REGION}

# text search
$ s3-manager search text "catch me if you can" --access-key asdasfasfasfasfasfas --secret-key asdasfasfasfasfasfas --bucket-name demo-bucket --region us-east-2

Every subcommand has its own usage examples, please refer to s3-manager [command] --help for more information about a command.

Development

This project requires below tools while developing:

• Golang 1.21
• pre-commit
• golangci-lint - required by pre-commit
• gocyclo - required by pre-commit

After you installed pre-commit and the rest, simply run below command to prepare your development environment:

$ make pre-commit-setup