Functional test part1

[dhermes]

This rolls back #259 (otherwise inserts fail) and can't be merged until #121 is fixed.

This is a first attempt at #106 and only scratches the surface.

Before deep diving into writing all the tests, I wanted to get a feel for how we wanted it to look.

@silvolu this will require setting some environ variables in Travis. Do I have the ability to change this or should I just use you as a relay?

Also @tseaver and @silvolu this is a good place to decide what we'd like to do with demo.key.

[coveralls]

Coverage remained the same when pulling 4c34706 on dhermes:functional-test-part1 into fab0237 on GoogleCloudPlatform:master.

[silvolu]

You should be able to add the environment variables using the Travis UI, but let me know if that is not the case.
For the key, don't use demo.key for the regression testing; use encrypted files. I can set that up for you if you want, after we merge this initial core, which looks good to me.

[dhermes]

Great to hear. I'd love a tiny walk-through, possible via screen sharing?

I'm certainly not going to use demo.key. IMO we should destroy it and update the docs on running the demo as well as making the demo code fail with a message explaining what needs to be added.

I will hold off on this until #121 is fixed or do we want to revert the broken code from #259?

@tseaver any thoughts?

[silvolu]

My 2 cents on reverting #259: unless a fix is imminent, we should revert.

[tseaver]

I have reverted it.

[dhermes]

Thanks for reverting. I rebased this PR so now it could be merged (per @silvolu LGTM). But I'll hold off if @tseaver has any comments.

[tseaver]

My major worry is how we keep the functests from bitrotting -- when are they run?

[dhermes]

I am happy to put them in the Travis build but it will start to take a LONG time. I don't know if there are best practices around this or if people have opinions?

I think that's not so big a deal since people can do lot's of things locally.

Before we can ask Travis to run tox -e regression we need to set some environment variables and an encrypted key.

[silvolu]

The plan is to run the functests after every merge to master, as a safety net that can catch breakings like #259.

[coveralls]

Coverage remained the same when pulling 929618b on dhermes:functional-test-part1 into 8b97707 on GoogleCloudPlatform:master.

[dhermes]

@tseaver so this will get run in .travis.yml along with the other "before success" scripts.

I can add this in a forthcoming PR after the necessary settings are in place on the Travis side of things or we can put it in this PR. By putting it in this PR, it will block on @silvolu and I syncing up to get Travis set up correctly.

[tseaver]

I see. LGTM.

[coveralls]

Coverage remained the same when pulling 2c101b4 on dhermes:functional-test-part1 into 7f6e4b1 on GoogleCloudPlatform:master.