Git support for LambdaCD

Provides Git support for LambdaCD. Will replace the lambdacd.steps.git namespace in the the lambdacd-git library.

Status

Usage

;; project.clj
:dependencies [[lambdacd-git "<most recent version>"]]
;; import:
(:require [lambdacd-git.core :as lambdacd-git])

Complete Example

You'll find a complete example here: example/simple_pipeline.clj

Waiting for a commit

(defn wait-for-commit-on-master [args ctx]
  (lambdacd-git/wait-for-git ctx "[email protected]:flosell/testrepo"
                             ; how long to wait when polling. optional, defaults to 10000
                             :ms-between-polls 1000
                             ; which refs to react to. optional, defaults to refs/heads/master
                             :ref "refs/heads/master"))

; you can also pass in a regex:
(defn wait-for-commit-on-feature-branch [args ctx]
  (lambdacd-git/wait-for-git ctx "[email protected]:flosell/testrepo"
                             :ref #"refs/heads/feature-.*"))

; you can also pass in a function
(defn wait-for-commit-on-any-tag [args ctx]
  (lambdacd-git/wait-for-git ctx "[email protected]:flosell/testrepo"
                             :ref (fn [ref] (.startsWith ref "refs/tags/"))))

Using Web- or Post-Commit Hooks instead of Polling

wait-for-git can be notified about changes in a repository through an HTTP endpoint. To use this, you need to add it to your existing ring-handlers:

(ring-server/serve (routes
                         (ui/ui-for pipeline)
                         (core/notifications-for pipeline)) ; <-- THIS
                       {:open-browser? false
                        :port          8082})

This adds an HTTP endpoint that can receive POST requests on <host>/notify-git?remote=<remote>, e.g. http://localhost:8082/[email protected]:flosell/testrepo

Notice that this doesn't necessarily trigger a new build. Similar to how Jenkins works, a notification just forces a check for changes.

Cloning a Repository

(defn clone [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (:cwd args)))

(def pipeline-structure
  `(; ...
    (with-workspace
      clone
      do-something)))

; Works well with wait-for-git: 
; If no revision is given (e.g. because of manual trigger), clone falls back to the head of the master branch

(defn clone [args ctx]
  (lambdacd-git/clone ctx repo (:revision args) (:cwd args)))

(def pipeline-structure
  `((either
      wait-for-manual-trigger
      wait-for-git)
     (with-workspace
       clone

       do-something)))

Get details on commits since last build

(defn clone [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (:cwd args)))

(def pipeline-structure
  `(wait-for-git
    (with-workspace
      clone
      lambdacd-git/list-changes
      do-something)))

Working with more than one repository

You can have clone steps that clone into different subdirectories:

(defn clone-foo [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (str (:cwd args) "/" "foo")))
(defn clone-bar [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (str (:cwd args) "/" "bar")))

(def pipeline-structure
  `(; ... 
    (with-workspace
      clone-foo
      clone-bar
      do-something)))

If you want to use this in combination with wait-for-git, you need to detect which commit to use. For details, see example/multi_repo_pipeline.clj

Tagging versions

You can tag any revision:

(defn deploy-to-live [args ctx]
  ;do deploy
  (let [cwd (:cwd args)]
    (lambdacd-git/tag-version ctx cwd repo "HEAD" (str "live-" version))))

(def pipeline-structure
  `(; ...
    (with-workspace
      ;...
      deploy-to-live
      do-something)))

Configuration (versions >= 0.3.0)

Certain values can be configured using LambdaCDs config-map. The following example shows the default:

(let [config  {:git {:timeout              20                               ; the timeout for remote operations in seconds
                     :credentials-provider (CredentialsProvider/getDefault) ; the credentials-provider to use for HTTPS clones (e.g. UsernamePasswordCredentialsProvider)
                     :ssh {:use-agent                true                         ; whether to use an SSH agent
                           :known-hosts-files        ["~/.ssh/known_hosts" 
                                                      "/etc/ssh/ssh_known_hosts"] ; which known-hosts files to use for SSH connections 
                           :identity-file            nil                          ; override the normal SSH behavior and explicitly specify a key to use
                           :strict-host-key-checking nil}}}]                      ; override the normal SSH behavior and explicitly set the StrictHostKeyChecking setting. Off by default, can be set to yes,no or ask
  (lambdacd/assemble-pipeline pipeline-structure config))

The same parameters can also be used as parameters to git operations directly, e.g. if different repositories need different credentials:

(defn clone [args ctx]
  (lambdacd-git/clone ctx repo branch-or-tag-or-commit-hash (:cwd args) 
                      :credentials-provider (UsernamePasswordCredentialsProvider. (System/getenv "LAMBDACD_GIT_TESTREPO_USERNAME")
                                                                                  (System/getenv "LAMBDACD_GIT_TESTREPO_PASSWORD"))))

Configuration (versions < 0.3.0)

SSH Configuration

LambdaCD-Git honors the default SSH Config files from ~/.ssh/config(/etc/ssh/ssh_config currently not supported (see #23)). Use this to configure things like StrictHostKeyChecking or the IdentityFile. Alternatively, some options can be configured using ssh-init.

Authentication

Git over SSH

LambdaCD Git automatically picks up SSH-Keys in the default locations, e.g. ~/.ssh/id_rsa. SSH Agents are also supported.

Git over HTTPS

Authentication for HTTPS is supported using an instance of the JGit CredentialsProvider.

Import e.g. UsernamePasswordCredentialsProvider into your namespace:

(ns <your-pipeline-name>.core
  ;...
  (:import org.eclipse.jgit.transport.UsernamePasswordCredentialsProvider)
  ;...
  )

Add an instance to the LambdaCD config:

(let [config {:home-dir "/some/path"
              :git {:credentials-provider (UsernamePasswordCredentialsProvider. "some-username" "some-password")}}]
              ; ... 
              )

Customize SSH Client

Some features of lambdacd-git (ssh-agent support, extended known_hosts support) require customizations to JGits singleton SSH session factory. Call init-ssh! once, e.g. in your -main function:

(defn -main [& args]
  ; ...
  (lambdacd-git/init-ssh!)
  ; ...
  )

Development

Call ./go

License

Copyright © 2015 Florian Sellmayr

Distributed under the Apache License 2.0