Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

TT-8735, initial commit #6317

Draft
wants to merge 28 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
d90e444
TT-8735, initial commit
andrei-tyk May 30, 2024
bfff91f
TT-8735, make lint
andrei-tyk May 30, 2024
bdcbb08
TT-8735, added separate nfpms entry
andrei-tyk May 30, 2024
57ad665
TT-8735, added GOEXPERIMENT=boringcrypto env var to test if tests pas…
andrei-tyk Jun 3, 2024
805d3f9
TT-8735, inverted order of fips matrix tests to see if CI pipeline ac…
andrei-tyk Jun 3, 2024
84f7d4c
TT-8735, added env var to build.sh so that it detects the GOEXPERIMEN…
andrei-tyk Jun 3, 2024
4b1ab01
TT-8735, added build tags for boring crypto to plugin compiler build.sh
andrei-tyk Jun 4, 2024
f4443cd
TT-8735, check to see if fips matrix is actually being run
andrei-tyk Jun 4, 2024
5a8ecfb
TT-8735, trial with tags output
andrei-tyk Jun 4, 2024
a221f94
TT-8735, trial with tags output2
andrei-tyk Jun 4, 2024
7fd6134
TT-8735, fips only trial
andrei-tyk Jun 4, 2024
d48f361
TT-8735, added fips build tags to go plugins directories
andrei-tyk Jun 4, 2024
ad65f95
TT-8735, added fips build tags to go plugins directories 2
andrei-tyk Jun 4, 2024
25a8a36
TT-8735, trial with clean go.mod
andrei-tyk Jun 4, 2024
328c0c7
TT-8735, trial with clean go.mod3
andrei-tyk Jun 4, 2024
d0711aa
TT-8735, trial for typo
andrei-tyk Jun 4, 2024
5d26465
TT-8735, trial with replace
andrei-tyk Jun 4, 2024
65c4b68
TT-8735, trial with various go.mod changes
andrei-tyk Jun 4, 2024
396593d
TT-8735, test with boringcrypto
andrei-tyk Jun 4, 2024
502003f
TT-8735, trial with boringcrypto in main release pipeline
andrei-tyk Jun 4, 2024
4a99748
TT-8735, trial with boringcrypto in main release pipeline4
andrei-tyk Jun 5, 2024
b780dd5
TT-8735, removed useless/incorrect tags
andrei-tyk Jun 5, 2024
7bc93f3
TT-8735, trial for release.yml
andrei-tyk Jun 5, 2024
fe7a84c
TT-8735, deleted hybrid and put correct id in dockers build
andrei-tyk Jun 6, 2024
328c415
TT-8735, CI push correct names
andrei-tyk Jun 6, 2024
576e9d9
TT-8735, correct gateway image and plugin compiler image, should fail…
andrei-tyk Jun 6, 2024
0d272d9
TT-8735, trial with plugin compiler fips image
andrei-tyk Jun 6, 2024
b78b1cc
TT-8735, reverted test-plugin/go.mod changes to ensure there's no com…
andrei-tyk Jun 6, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .github/workflows/ci-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@ jobs:
redis-version: [5, 7]
python-version: ["3.11"]
go-version: [1.21.x]
fips: [1]

steps:
- name: Checkout Tyk
Expand All @@ -43,6 +44,11 @@ jobs:
with:
go-version: ${{ matrix.go-version }}

- name: Setup Golang FIPS
if: ${{ contains(matrix.fips, '1') }}
run: |
echo "GOEXPERIMENT=boringcrypto" >> $GITHUB_ENV

- name: Setup Python
uses: actions/setup-python@v4
with:
Expand Down
4 changes: 3 additions & 1 deletion .github/workflows/plugin-compiler-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -65,7 +65,9 @@ jobs:
platforms: linux/amd64
push: true
labels: ${{ steps.set-metadata.outputs.labels }}
tags: ${{ steps.set-metadata.outputs.tags }}
tags: |
${{ steps.set-metadata.outputs.tags }}
${{ steps.login-ecr.outputs.registry }}/tyk-plugin-compiler:sha-${{ github.sha }}-fips-amd64
build-args: |
BASE-IMAGE=tykio/golang-cross:${{ env.GOLANG_CROSS }}
GITHUB_SHA=${{ github.sha }}
Expand Down
16 changes: 12 additions & 4 deletions .github/workflows/release-tests.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,9 +28,11 @@ jobs:
run: task opentelemetry

ci-tests:
name: CI Tests
name: 'CI Tests with fips: ${{ matrix.fips }}'
runs-on: ubuntu-latest

strategy:
matrix:
fips: [1]
steps:
- uses: actions/checkout@v4
with:
Expand All @@ -39,12 +41,18 @@ jobs:
- id: ecr-login
uses: ./.github/actions/ecr-login

- name: Setup Golang FIPS
if: ${{ contains(matrix.fips, '1') }}
run: |
echo "GOEXPERIMENT=boringcrypto" >> $GITHUB_ENV

- name: Run /ci/tests
shell: bash
env:
GITHUB_TAG: ${{ github.ref }}
GATEWAY_IMAGE: ${{ steps.ecr-login.outputs.registry }}/tyk:sha-${{ github.sha }}
PLUGIN_COMPILER_IMAGE: ${{ steps.ecr-login.outputs.registry }}/tyk-plugin-compiler:sha-${{ github.sha }}
GATEWAY_IMAGE: ${{ steps.ecr-login.outputs.registry }}/tyk:sha-${{ github.sha }}-fips-amd64
PLUGIN_COMPILER_IMAGE: ${{ steps.ecr-login.outputs.registry }}/tyk-plugin-compiler:sha-${{ github.sha }}-fips-amd64
GOEXPERIMENT: ${{ env.GOEXPERIMENT }}
run: |
set -eaxo pipefail
for d in ci/tests/*/
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -143,7 +143,11 @@ jobs:
for arch in amd64 arm64; do
docker tag tykio/tyk-gateway:${build_tag}-${arch} ${tag}-${arch} && docker push ${tag}-${arch}
done
for arch in amd64; do
docker tag tykio/tyk-gateway:${build_tag}-fips-${arch} ${tag}-fips-${arch} && docker push ${tag}-fips-${arch}
done
docker manifest create ${tag} ${tag}-amd64 ${tag}-arm64 && docker manifest push ${tag}
docker manifest create ${tag}-fips ${tag}-fips-amd64 && docker manifest push ${tag}-fips
done
- uses: actions/upload-artifact@v4
if: ${{ matrix.golang_cross == '1.21-bullseye' }}
Expand Down
10 changes: 7 additions & 3 deletions bin/ci-tests.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,12 +17,16 @@ set -e

# set tags for the CI tests run / plugin builds

tags="goplugin dev"
tags="goplugin"

if [[ "$GOEXPERIMENT" == "boringcrypto" ]]; then
tags+=" boringcrypto"
fi

# build Go-plugin used in tests
echo "Building go plugin"
go build -tags "${tags}" -buildmode=plugin -o ./test/goplugins/goplugins.so ./test/goplugins
go build -tags "${tags}" -buildmode=plugin -race -o ./test/goplugins/goplugins_race.so ./test/goplugins
GOEXPERIMENT=boringcrypto go build -tags "${tags}" -buildmode=plugin -o ./test/goplugins/goplugins.so ./test/goplugins
GOEXPERIMENT=boringcrypto go build -tags "${tags}" -buildmode=plugin -race -o ./test/goplugins/goplugins_race.so ./test/goplugins

for pkg in ${PKGS}; do
coveragefile=`echo "$pkg" | awk -F/ '{print $NF}'`
Expand Down
104 changes: 104 additions & 0 deletions ci/goreleaser/goreleaser.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,8 +51,50 @@ builds:
goarch:
- s390x
binary: tyk
- id: fips-std-linux
flags:
- -tags=goplugin,boringcrypto
ldflags:
- -X github.com/TykTechnologies/tyk/internal/build.Version={{.Version}}
- -X github.com/TykTechnologies/tyk/internal/build.Commit={{.FullCommit}}
- -X github.com/TykTechnologies/tyk/internal/build.BuildDate={{.Date}}
- -X github.com/TykTechnologies/tyk/internal/build.BuiltBy=goreleaser
env:
- GOEXPERIMENT=boringcrypto
goos:
- linux
goarch:
- amd64
binary: tyk
dockers:
# Build tykio/tyk-gateway, docker.tyk.io/tyk-gateway/tyk-gateway (amd64)
- ids:
- fips
image_templates:
- "tykio/tyk-gateway:{{.Tag}}-fips-amd64"
- "docker.tyk.io/tyk-gateway/tyk-gateway:{{.Tag}}-fips-amd64"
build_flag_templates:
- "--build-arg=PORTS=8080"
- "--platform=linux/amd64"
- "--label=org.opencontainers.image.created={{.Date}}"
- "--label=org.opencontainers.image.title={{.ProjectName}}"
- "--label=org.opencontainers.image.revision={{.FullCommit}}"
- "--label=org.opencontainers.image.version={{.Version}}"
use: buildx
goarch: amd64
goos: linux
dockerfile: ci/Dockerfile.std
extra_files:
- "ci/install/"
- "README.md"
- "LICENSE.md"
- "apps/app_sample.json"
- "templates"
- "middleware"
- "event_handlers/sample"
- "policies"
- "coprocess"
- "tyk.conf.example"
- ids:
- std
image_templates:
Expand Down Expand Up @@ -150,22 +192,27 @@ docker_manifests:
- name_template: tykio/tyk-gateway:{{ .Tag }}
image_templates:
- tykio/tyk-gateway:{{ .Tag }}-amd64
- tykio/tyk-gateway:{{ .Tag }}-fips-amd64
- tykio/tyk-gateway:{{ .Tag }}-arm64
- name_template: tykio/tyk-gateway:v{{ .Major }}.{{ .Minor }}{{.Prerelease}}
image_templates:
- tykio/tyk-gateway:{{ .Tag }}-amd64
- tykio/tyk-gateway:{{ .Tag }}-fips-amd64
- tykio/tyk-gateway:{{ .Tag }}-arm64
- name_template: tykio/tyk-gateway:v{{ .Major }}{{.Prerelease}}
image_templates:
- tykio/tyk-gateway:{{ .Tag }}-amd64
- tykio/tyk-gateway:{{ .Tag }}-fips-amd64
- tykio/tyk-gateway:{{ .Tag }}-arm64
- name_template: tykio/tyk-hybrid-docker:{{ .Tag }}
image_templates:
- tykio/tyk-hybrid-docker:{{ .Tag }}-amd64
tykio/tyk-hybrid-docker:{{ .Tag }}-fips-amd64
- tykio/tyk-hybrid-docker:{{ .Tag }}-arm64
- name_template: docker.tyk.io/tyk-gateway/tyk-gateway:{{ .Tag }}
image_templates:
- docker.tyk.io/tyk-gateway/tyk-gateway:{{ .Tag }}-amd64
- docker.tyk.io/tyk-gateway/tyk-gateway:{{ .Tag }}-fips-amd64
- docker.tyk.io/tyk-gateway/tyk-gateway:{{ .Tag }}-arm64
nfpms:
- id: std
Expand Down Expand Up @@ -227,6 +274,63 @@ nfpms:
signature:
key_file: tyk.io.signing.key
type: origin
- id: fips
vendor: "Tyk Technologies Ltd"
homepage: "https://tyk.io"
maintainer: "Tyk <[email protected]>"
description: Tyk Open Source API Gateway written in Go, supporting REST, GraphQL, TCP and gRPC protocols
package_name: tyk-gateway
file_name_template: "{{ .ConventionalFileName }}"
builds:
- fips-std-linux
formats:
- deb
- rpm
contents:
- src: "README.md"
dst: "/opt/share/docs/tyk-gateway/README.md"
- src: "ci/install/*"
dst: "/opt/tyk-gateway/install"
- src: ci/install/inits/systemd/system/tyk-gateway.service
dst: /lib/systemd/system/tyk-gateway.service
- src: ci/install/inits/sysv/init.d/tyk-gateway
dst: /etc/init.d/tyk-gateway
- src: /opt/tyk-gateway
dst: /opt/tyk
type: "symlink"
- src: "LICENSE.md"
dst: "/opt/share/docs/tyk-gateway/LICENSE.md"
- src: "apps/app_sample.*"
dst: "/opt/tyk-gateway/apps"
- src: "templates/*.json"
dst: "/opt/tyk-gateway/templates"
- src: "templates/playground/*"
dst: "/opt/tyk-gateway/templates/playground"
- src: "middleware/*.js"
dst: "/opt/tyk-gateway/middleware"
- src: "event_handlers/sample/*.js"
dst: "/opt/tyk-gateway/event_handlers/sample"
- src: "policies/*.json"
dst: "/opt/tyk-gateway/policies"
- src: "coprocess/*"
dst: "/opt/tyk-gateway/coprocess"
- src: tyk.conf.example
dst: /opt/tyk-gateway/tyk.conf
type: "config|noreplace"
scripts:
preinstall: "ci/install/before_install.sh"
postinstall: "ci/install/post_install.sh"
postremove: "ci/install/post_remove.sh"
bindir: "/opt/tyk-gateway"
rpm:
scripts:
posttrans: ci/install/post_trans.sh
signature:
key_file: tyk.io.signing.key
deb:
signature:
key_file: tyk.io.signing.key
type: origin
publishers:
- name: tyk-gateway-unstable
env:
Expand Down
5 changes: 4 additions & 1 deletion ci/images/plugin-compiler/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -25,12 +25,15 @@ RUN --mount=type=cache,mode=0755,target=/go/pkg/mod go mod download
ADD . $TYK_GW_PATH

# Provide a gateway test binary for testing plugin loading.
RUN --mount=type=cache,mode=0755,target=/go/pkg/mod GOBIN=/usr/local/bin go install -tags=goplugin -trimpath .
RUN --mount=type=cache,mode=0755,target=/go/pkg/mod GOBIN=/usr/local/bin go install -tags=goplugin,boringcrypto -trimpath .

ARG GITHUB_SHA
ARG GITHUB_TAG
ENV GITHUB_SHA ${GITHUB_SHA}
ENV GITHUB_TAG ${GITHUB_TAG}
ENV GOEXPERIMENT ${GOEXPERIMENT}

RUN echo "GOEXPERIMENT: $GOEXPERIMENT"

COPY ci/images/plugin-compiler/data/build.sh /build.sh
RUN chmod +x /build.sh
Expand Down
10 changes: 9 additions & 1 deletion ci/images/plugin-compiler/data/build.sh
Original file line number Diff line number Diff line change
Expand Up @@ -145,7 +145,15 @@ if [[ "$DEBUG" == "1" ]] ; then
git diff --cached
fi

CC=$CC CGO_ENABLED=1 GOOS=$GOOS GOARCH=$GOARCH go build -buildmode=plugin -trimpath -o $plugin_name
tags="goplugin"

#if [[ "$GOEXPERIMENT" == "boringcrypto" ]]; then
tags+=" boringcrypto"
#fi

echo "Tags: $tags"

CC=$CC GOEXPERIMENT=boringcrypto CGO_ENABLED=1 GOOS=$GOOS GOARCH=$GOARCH go build -buildmode=plugin -tags=goplugin,boringcrypto -trimpath -o $plugin_name

set +x

Expand Down
6 changes: 6 additions & 0 deletions ci/smoke-tests/plugin-aliasing/foobar-plugin/fips.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
//go:build boringcrypto
// +build boringcrypto

package main

import _ "crypto/tls/fipsonly"
2 changes: 0 additions & 2 deletions ci/smoke-tests/plugin-aliasing/foobar-plugin/go.mod
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
module github.com/TykTechnologies/tyk/smoke-tests/plugin-compiler/foobar-plugin

go 1.21

require github.com/kr/pretty v0.3.1 // indirect
6 changes: 6 additions & 0 deletions ci/smoke-tests/plugin-aliasing/helloworld-plugin/fips.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
//go:build boringcrypto
// +build boringcrypto

package main

import _ "crypto/tls/fipsonly"
4 changes: 2 additions & 2 deletions ci/smoke-tests/plugin-aliasing/helloworld-plugin/go.mod
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
module github.com/TykTechnologies/tyk/smoke-tests/plugin-compiler/helloworld-plugin

go 1.21
go 1.21.0

require github.com/kr/pretty v0.3.1 // indirect
toolchain go1.21.4
2 changes: 1 addition & 1 deletion ci/tests/plugin-compiler/test.sh
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@ trap "docker compose down --remove-orphans" EXIT
# Clean up loose .so files, rebuild plugin and normalize plugin name.
PLUGIN_SOURCE_PATH=$PWD/testdata/test-plugin
rm -fv $PLUGIN_SOURCE_PATH/*.so || true
docker run --rm -v $PLUGIN_SOURCE_PATH:/plugin-source $PLUGIN_COMPILER_IMAGE plugin.so
docker run --rm -v $PLUGIN_SOURCE_PATH:/plugin-source -e GOEXPERIMENT='boringcrypto' $PLUGIN_COMPILER_IMAGE plugin.so
cp $PLUGIN_SOURCE_PATH/*.so $PLUGIN_SOURCE_PATH/plugin.so

docker compose up -d --wait --force-recreate || { docker compose logs gw; exit 1; }
Expand Down
6 changes: 6 additions & 0 deletions ci/tests/plugin-compiler/testdata/basic-plugin/fips.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
//go:build boringcrypto
// +build boringcrypto

package main

import _ "crypto/tls/fipsonly"
4 changes: 3 additions & 1 deletion ci/tests/plugin-compiler/testdata/basic-plugin/go.mod
Original file line number Diff line number Diff line change
@@ -1,3 +1,5 @@
module example.com/basic-plugin

go 1.21
go 1.21.0

toolchain go1.21.4
6 changes: 6 additions & 0 deletions ci/tests/plugin-compiler/testdata/complex-plugin/fips.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
//go:build boringcrypto
// +build boringcrypto

package main

import _ "crypto/tls/fipsonly"
5 changes: 4 additions & 1 deletion ci/tests/plugin-compiler/testdata/complex-plugin/go.mod
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
module example.com/basic-plugin

go 1.19
go 1.21.0

toolchain go1.21.4

Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
//go:build boringcrypto
// +build boringcrypto

package main

import _ "crypto/tls/fipsonly"
6 changes: 6 additions & 0 deletions ci/tests/plugin-compiler/testdata/test-plugin/fips.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
//go:build boringcrypto
// +build boringcrypto

package main

import _ "crypto/tls/fipsonly"
2 changes: 1 addition & 1 deletion ci/tests/plugin-compiler/testdata/test-plugin/go.mod
Original file line number Diff line number Diff line change
Expand Up @@ -8,4 +8,4 @@ require (
github.com/kr/pretty v0.2.1
)

replace github.com/jensneuse/graphql-go-tools => github.com/TykTechnologies/graphql-go-tools v1.6.2-0.20210609111804-af8c15678972
replace github.com/jensneuse/graphql-go-tools => github.com/TykTechnologies/graphql-go-tools v1.6.2-0.20210609111804-af8c15678972
Loading
Loading