CI for llnode?

[bnoordhuis]

Testing llnode on OS X + Linux with a range of node.js and lldb versions would help catch snafus like nodejs/llnode#109.

On OS X, I think testing against the lldb that Apple ships with the latest Xcode is good enough. On Linux, a good matrix would be the cross product of:

1. lldb 3.8, 3.9 and 4.0
2. node.js 4, 6 and 8

I'm happy to help with the setup.

[gibfahn]

I started to set up a couple of jobs:

llnode-pipeline triggers llnode-continuous-integration with the different versions of node you want to test.

I haven't run them (sure to be lots of bugs), and I assume the Linux ones will fail. I've given @nodejs/post-mortem-admins and @bnoordhuis access, so feel free to modify.

We'll need to install llnode on ubuntu, so that'll require a PR to the ansible/ scripts.

I'll sort the pipeline out tomorrow. I tried testing on my local machine, but I'll have to get past the configure error first:

➜  llnode git:(pr-109) ❯ npm i

> [email protected] preinstall /Users/gib/wrk/com/llnode
> node scripts/configure.js

Build dir is: /Users/gib/wrk/com/llnode
xcode-select: error: tool 'xcodebuild' requires Xcode, but active developer directory '/Library/Developer/CommandLineTools' is a command line tools instance
Unable to locate lldb binary. llnode installation failed.

➜  llnode git:(pr-109) ❯ which lldb                                                                                                                                                        ~/wrk/com/llnode
/usr/bin/lldb

➜  llnode git:(pr-109) ❯ lldb --version                                                                                                                                                    ~/wrk/com/llnode
lldb-370.0.42
  Swift-3.1

[gibfahn]

Note that there's also llnode, which it looks like @rvagg was setting up in November 2016. Looks like it's using docker, but I think it's a little out of date now. Happy to move to this style if it makes more sense (gave @nodejs/post-mortem-admins access to that too).

Scripts from the llnode job:

## TESTING WITH NODE.JS v7 ----------------------------------------------

docker run -i --rm --security-opt=seccomp:unconfined llnode-base:xenial /bin/bash -c '

curl -sL https://deb.nodesource.com/setup_6.x | bash -
apt-get install nodejs -y

echo + node -p process.versions
node -p process.versions

cd /home/iojs && su iojs -c "

echo + git clone https://github.com/'${github_user}'/'${github_repo}'.git llnode
git clone https://github.com/'${github_user}'/'${github_repo}'.git llnode
echo + cd llnode
cd llnode
echo + git checkout '${github_ref}' -f
git checkout '${github_ref}' -f
echo + git clone https://chromium.googlesource.com/external/gyp.git tools/gyp
git clone https://chromium.googlesource.com/external/gyp.git tools/gyp
echo + npm install
npm install
echo + make _travis
make _travis

"

'

## TESTING WITH NODE.JS v7 ----------------------------------------------

docker run -i --rm --security-opt=seccomp:unconfined llnode-base:xenial /bin/bash -c '

curl -sL https://deb.nodesource.com/setup_7.x | bash -
apt-get install nodejs -y

echo + node -p process.versions
node -p process.versions

cd /home/iojs && su iojs -c "

echo + git clone https://github.com/'${github_user}'/'${github_repo}'.git llnode
git clone https://github.com/'${github_user}'/'${github_repo}'.git llnode
echo + cd llnode
cd llnode
echo + git checkout '${github_ref}' -f
git checkout '${github_ref}' -f
echo + git clone https://chromium.googlesource.com/external/gyp.git tools/gyp
git clone https://chromium.googlesource.com/external/gyp.git tools/gyp
echo + npm install
npm install
echo + make _travis
make _travis

"

'

[bnoordhuis]

Thanks Gibson, nice work. Maiden run: https://ci.nodejs.org/view/post-mortem/job/llnode-pipeline/1/ - doesn't seem to work yet though but it makes up for that in speed (23 ms!)

Naive question perhaps but how do I go about debugging that? Or do I just need to wait a little longer?

[rnchamberlain]

@gibfahn many thanks for setting this up. Just curious, why the extra 'pipeline' build? For node-report we have just a single build.

[gibfahn]

Just curious, why the extra 'pipeline' build? For node-report we have just a single build.

If you want to trigger the build for Node 4, 6, 8 then you have to run the build three times. I'm lazy so I just have a pipeline that triggers things.

Maiden run: https://ci.nodejs.org/view/post-mortem/job/llnode-pipeline/1/ - doesn't seem to work yet though but it makes up for that in speed (23 ms!)

Yep, I need to sort out the pipeline. I'll try to get to that in a bit.

[hhellyer]

@gibfahn There is a travis build here https://travis-ci.org/nodejs/llnode - we just accepted a PR to make it point at the right place which has caused me to notice it's broken. I'm working on fixes:

nodejs/llnode#110
nodejs/llnode#111

(Disclaimer, I didn't setup travis so I'm not sure how it relates to everything else.)

[bnoordhuis]

@hhellyer Travis doesn't have access to the organization (disallowed, it requests private access) so I don't think that's ultimately going to work.

[mhdawson]

It is better to test with the Node.js community CI as it also gives us the option of full platform coverage.

[bnoordhuis]

@gibfahn Were you able to make progress on this? (Can't check myself, CI is currently under lock-down.)

[gibfahn]

Okay, I've gotten the pipeline to work. It looks a bit mental but I'm not aware of an easier way of creating an arbitrary number of jobs.

If you're just running against one version of Node just run llnode-continuous-integration. If you want to run this on a PR (for example), then using the pipeline should be easier.

Next steps are to actually fix the job itself. That's where it might be more of a @bnoordhuis or @nodejs/post-mortem issue.

[gibfahn]

I note that the only members of @nodejs/post-mortem-admins are @mhdawson and @rnchamberlain. I think we should expand the membership to include people who are interested in setting up and maintaining the Jenkins jobs.

I'd suggest @indutny @bnoordhuis and @hhellyer , but I haven't been very involved in llnode so there may well be others I've missed.

[bnoordhuis]

@gibfahn I just tried to start https://ci.nodejs.org/view/post-mortem/job/llnode-pipeline/ but it says "master is offline."

I'd suggest @indutny @bnoordhuis and @hhellyer , but I haven't been very involved in llnode so there may well be others I've missed.

Yes, that should cover it.

[joaocgreis]

"master is offline" is because of #825 (comment) , I can fix it.

[joaocgreis]

Done, did it by bringing master back online. @rvagg @gibfahn apparently, pipeline jobs do not have a "Preference of node" or "Restrict where this project can be run" options, so it'll only run on master. So, unless I missed something, we'll need master online to run pipeline jobs.

[bnoordhuis]

Thanks @joaocgreis, I can confirm that it's working now.

[joyeecheung]

@gibfahn The Jenkins job is currently broken, I think it's probably because the build flow of llnode has changed (although the test suite itself was a bit flaky). I've got the Travis working in nodejs/llnode#144, a fix to Jenkins would probably involve getting the scripts up-to-date similar to what's done in https://github.com/joyeecheung/llnode/blob/ae83fe8384fb37f6039b4fe2c9f07c3e852727e9/.travis.yml (we might want a makefile target for Jenkins).

Can I have the permission to configure the Jenkins job? I would like to make it work with Node core PRs as well. (refs: nodejs/post-mortem#50 (comment))

[mhdawson]

@joyeecheung is this the one that you need access to: https://ci.nodejs.org/view/post-mortem/job/llnode-pipeline/ ?

[gibfahn]

I gave nodejs/build permission to access llnode-pipeline and llnode-continuous-integration, which should be enough.

[Trott]

Does https://ci.nodejs.org/job/llnode-continuous-integration/ meet the need here? Or is there more to be done?

[bnoordhuis]

Does https://ci.nodejs.org/job/llnode-continuous-integration/ meet the need here?

@joyeecheung @mmarchini You're probably better qualified to answer that, you've been involved with llnode more than me lately.

[mmarchini]

Last time I tried https://ci.nodejs.org/job/llnode-continuous-integration/ it wasn't working properly.

TBH I think we can close this. llnode test suite (which is using Travis) has been quite useful lately and covers a lot of cases (multiple Node.js versions on Linux and OS X). Keeping it on a 3rd party CI is one less thing for the build wg to worry about.

I'm closing this issue, feel free to reopen it if you believe we should still have a Jenkins CI for llnode.