Skip to content

Commit

Permalink
add docker container & github actions
Browse files Browse the repository at this point in the history
  • Loading branch information
PNixx committed Jul 18, 2024
1 parent 87a0d5f commit 63fd022
Show file tree
Hide file tree
Showing 13 changed files with 2,437 additions and 176 deletions.
7 changes: 6 additions & 1 deletion .editorconfig
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,12 @@ indent_style = space
indent_size = 4
tab_width = 4

[{*.php,*.phtml,run}]
[*.yml]
indent_style = space
indent_size = 2
tab_width = 2

[{*.php,*.phtml,run,*.md,*.xml}]
indent_size = 2
max_line_length = 210
tab_width = 2
Expand Down
39 changes: 39 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
name: Publish Docker image

on:
release:
types: [published]

jobs:
push_to_registry:
name: Push Docker image to Docker Hub
runs-on: [self-hosted]
permissions:
packages: write
contents: read

steps:
- name: Check out the repo
uses: actions/checkout@v4

- name: Log in to Docker Hub
uses: docker/login-action@f4ef78c080cd8ba55a85445d5b36e214a81df20a
with:
username: ${{ secrets.DOCKER_USERNAME }}
password: ${{ secrets.DOCKER_PASSWORD }}

- name: Extract metadata (tags, labels) for Docker
id: meta
uses: docker/metadata-action@9ec57ed1fcdbf14dcef7dfbe97b2010124a938b7
with:
images: pnixx/clickhouse-queue

- name: Build and push Docker image
id: push
uses: docker/build-push-action@3b5e8027fcad23fda98b2e3ac259d8d67585f671
with:
context: .
file: ./Dockerfile
push: true
tags: ${{ steps.meta.outputs.tags }}
labels: ${{ steps.meta.outputs.labels }}
47 changes: 47 additions & 0 deletions .github/workflows/testing.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
name: Testing

on:
push:
branches: [ "main" ]
pull_request:
branches: [ "main" ]

jobs:
tests:
name: Testing
runs-on: ubuntu-latest

strategy:
fail-fast: true
max-parallel: 1
matrix:
php-version: [ '8.2', '8.3' ]
clickhouse: [ '24.6' ]

services:
rabbitmq:
image: itzg/rabbitmq-stomp
ports:
- 61614:61613
clickhouse:
image: clickhouse/clickhouse-server:${{ matrix.clickhouse }}
ports:
- 18123:8123

steps:
- uses: actions/checkout@v4

- name: Set up PHP ${{ matrix.php-version }}
uses: shivammathur/setup-php@v2
with:
php-version: ${{ matrix.php-version }}
extensions: sockets,pcntl,event

- name: Composer install
run: composer install

- name: Testing
run: php vendor/phpunit/phpunit/phpunit --display-warnings
env:
RABBIT_STOMP_PORT: 61614
CLICKHOUSE_PORT: 18123
1 change: 0 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,5 @@ vendor/
config/secrets.ini
config/test_secrets.ini
log/
composer.lock
bin/
tmp/
15 changes: 15 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
FROM php:8.3-cli

RUN docker-php-ext-install pcntl sockets bcmath exif
RUN apt-get update && apt-get -y install openssl libcurl4-openssl-dev pkg-config libssl-dev libevent-dev

RUN pecl channel-update pecl.php.net && pecl install event-3.1.1 && docker-php-ext-enable --ini-name zz-event.ini event

COPY --from=composer /usr/bin/composer /usr/bin/composer

WORKDIR /var/www/clickhouse-queue
COPY . /var/www/clickhouse-queue/

RUN composer install

ENTRYPOINT ["php", "examples/run.php", "start"]
25 changes: 25 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,23 @@ This worker will listen the queue `clickhouse` and will inserting bulk data to C
php examples/run.php start
```

## Docker

```bash
docker build --rm -t clickhouse-queue .
docker run --name clickhouse-queue clickhouse-queue
```

Environment variables:
* `CLICKHOUSE_HOST` - ClickHouse host for inserts, default `localhost`
* `CLICKHOUSE_PORT` - ClickHouse port for inserts, default `8123`
* `CLICKHOUSE_DB` - ClickHouse database for inserts, default `default`
* `RABBIT_HOST` - RabbitMQ host for subscribe, default `localhost`
* `RABBIT_STOMP_PORT` - RabbitMQ STOMP port, default `61613`
* `RABBIT_QUEUE` - RabbitMQ queue for buffer data
* `MAX_DELAY` - time flush data to ClickHouse, default `5`
* `MAX_FILE_SIZE` - max buffer size before flush to ClickHouse, default `5242880`

## Message structure

Header `table`: to insert data into the specified table
Expand All @@ -49,6 +66,14 @@ Body: JSON as string row data
{"column1": "data1", "column2": "data2"}
```

## Development

Testing GitHub actions:

```bash
act
```

## Donations

Donations to this project are going directly to [PNixx](https://github.com/PNixx), the original author of this project:
Expand Down
5 changes: 4 additions & 1 deletion composer.json
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,10 @@
"workerman/stomp": "^v1.0",
"workerman/http-client": "^v2.1",
"psr/log": "^3.0",
"revolt/event-loop": "^1.0"
"revolt/event-loop": "^1.0",
"ext-pcntl": "*",
"ext-sockets": "*",
"ext-event": "*"
},
"autoload": {
"psr-4": {
Expand Down
Loading

0 comments on commit 63fd022

Please sign in to comment.