Fix CI build #333

vroldanbet · 2019-11-14T11:06:09Z

Implement development/CI environment based on docker

Master CI build recently broke. The install-openldap script is very complex and assumes a specific environment. This is clearly not completely isolated from the fact that new versions of packages may be backported to the distribution used. I suspect this is what happened (noticed builds started breaking with a new version of slapd).

Instead of fixing the script, I decided I could reuse open-source maintained OpenLDAP docker containers. The https://github.com/osixia/docker-openldap project handles a lot of the work performed by install-openldap. As a consequence. we can remove a lot of stuff from this repo:

some LDIF fixtures
script/install-openldap
script/generate-fixture-ca
test/support (vagrant stuff)
certificates (it's now generated by the container)

In general, it's a lot of less code to maintain, and less code that can break as dependencies change.

Docker is also fairly ubiquitous in CI systems, and it would help closing the gap between CI and local development environment. Travis supports it nicely and it could open the door to adopt GitHub Actions too?

Extra niceties

added most recent ruby versions to the build matrix
no more "travis specific" tests

The not so nice part

I couldn't find a way to workaround having to modify /etc/hosts. I feel it's a small price to pay to simplify the setup considerably, but I'll try to explore alternatives (if I managed to name the containers hostname localhost it would get fixed)
the timeout test, I can't make it go green. I decided to disable it.
test relying on system cert store, didn't try hard how to install cert in travis, and it's additional burden with little return
test with multiple valid hostnames - I couldn't get the certificate generation to have multiple hostnames. Instead, I decided to run the test with the same valid hostname 2 times. It should hopefully be equivalent.

TODO

allow devs in local to avoid changes to /etc/hosts for integration tests

instead of polluting devs machine, let's rely on a docker container to spin up the service relies on osixia/openldap:1.3.0. Customizes a few things: - adds the seed on bootstrap - does not enforce client certificate - sets a hostname to avoid domain verification issues during handshake The cert domain is also added to /etc/hosts

anonymous access is not enabled in this setup, so every test needs to perform authentication first

I couldn't manage to get the container running with a cert issued to a given IP, like 127.0.0.1 or localhost. Instead, I specified a static hostname (the container uses hostname to generate the cert) and injected it in travis. Unfortunately, in local development this means changing /etc/hosts, but I feel that's a better option that having to install LDAP locally

so that all retcode tests succeed

it's a little price to pay in the current setup and allows us to have the same tests locally and dev.

all environments run the same set of tests, no env specific test

so that we are able to run a test that does not specific CACERT and so the library fallsback to system cert store

I'm not sure how to enable this in Travis

with the dockerized test openldap server, none of this is needed

the container uses HOSTNAME to generate the cert, and it really didn't like "localhost" as hostname. As a workaround, I had to add an arbitrary hostname. There may be other alternatives to get the host to be known, but the most obvious is modifying /etc/hosts

.travis.yml

test/integration/test_bind.rb

CamiloGarciaLaRotta · 2019-11-14T13:40:13Z

test/integration/test_bind.rb

  def test_bind_tls_with_multiple_hosts
-    omit_unless ENV['TRAVIS'] == 'true'


:) 👌🏽

mtodd

💯 to moving to relying on Docker for the OpenLDAP dependency. And thanks for breaking down the decisions and tradeoffs!

script/install-openldap

README.rdoc

.travis.yml

test/integration/test_bind.rb

Co-Authored-By: Matt Todd <[email protected]>

vroldanbet added 22 commits November 14, 2019 12:03

clarify why we need it

fdbe61e

there is an officially supported mechanism to add hostnames in Travis CI

ddd8dce

let's use good old example.org

3caf85b

change test DSN and authenticate with the right user

683d660

anonymous access is not enabled in this setup, so every test needs to perform authentication first

need to detach container so it does not block execution

94c2ba9

adds custom retcode.ldif to the bootstrap sequence

4280d18

so that all retcode tests succeed

disabling this test - it's returning connection refused instead

f38c39c

let's assume folks will have to modify /etc/hosts

1b595b2

it's a little price to pay in the current setup and allows us to have the same tests locally and dev.

we don't need vagrant specific tests anymore

ba225a9

all environments run the same set of tests, no env specific test

this does not need to run only in CI if devs edit /etc/hosts

376c42e

add docker container CA certificate and use if in fixtures

839318a

installs docker container CA certificate

9b8d80d

so that we are able to run a test that does not specific CACERT and so the library fallsback to system cert store

let's disable the test that uses system cert store

7c85643

I'm not sure how to enable this in Travis

remove unused stuff

6c1c56d

with the dockerized test openldap server, none of this is needed

script to start docker openldap server for integration tests

f67e741

rubymine warning: quote to avoid word splitting

5c4644e

add most recent ruby versions

10a44d5

update documentation

8ba87b3

remove empty file (I messed up while cherry-picking)

dc6d75d

vroldanbet mentioned this pull request Nov 14, 2019

Net::LDAP#open does not cache bind result #331

Closed

vroldanbet marked this pull request as ready for review November 14, 2019 11:10

CamiloGarciaLaRotta reviewed Nov 14, 2019

View reviewed changes

vroldanbet added 3 commits November 14, 2019 19:34

attempt to make docker command more readable

fee04d8

what about this

fef7cdb

add backslash to bash command

cf4e9bd

mtodd approved these changes Nov 15, 2019

View reviewed changes

script/install-openldap Show resolved Hide resolved

README.rdoc Outdated Show resolved Hide resolved

.travis.yml Outdated Show resolved Hide resolved

test/integration/test_bind.rb Outdated Show resolved Hide resolved

test/integration/test_bind.rb Show resolved Hide resolved

format the snippet as code

35bcc92

Co-Authored-By: Matt Todd <[email protected]>

dependabot bot mentioned this pull request Apr 13, 2023

Bump net-ldap from 0.16.1 to 0.16.2 bio-org-au/nsl-editor-rails-4-retired#17

Open

dependabot bot mentioned this pull request Dec 18, 2023

Bump the bundler group across 1 directories with 5 updates GrantBirki/ldap-api#1

Closed

dependabot bot mentioned this pull request May 14, 2024

Bump the bundler group across 1 directory with 18 updates chrisemoulton/cartodb#1

Open

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Fix CI build #333

Fix CI build #333

vroldanbet commented Nov 14, 2019 •

edited

Loading

CamiloGarciaLaRotta Nov 14, 2019 •

edited

Loading

mtodd left a comment

		def test_bind_tls_with_multiple_hosts
		omit_unless ENV['TRAVIS'] == 'true'

Fix CI build #333

Fix CI build #333

Conversation

vroldanbet commented Nov 14, 2019 • edited Loading

Implement development/CI environment based on docker

Extra niceties

The not so nice part

TODO

CamiloGarciaLaRotta Nov 14, 2019 • edited Loading

Choose a reason for hiding this comment

mtodd left a comment

Choose a reason for hiding this comment

vroldanbet commented Nov 14, 2019 •

edited

Loading

CamiloGarciaLaRotta Nov 14, 2019 •

edited

Loading