implement an http resource

[purpleidea]

We should have an http resource. Alternate names for it welcome. I considered calling it "rest", but I think http is best.

It should probably support long-polling to watch. It could use the Poll metaparam.

It should store the output as an Output var to be used by send/recv https://ttboj.wordpress.com/2016/12/07/sendrecv-in-mgmt/ so that it can be put into a file.

It could probably take a map[string]string as input (eg: a query string). It should probably support at least GET and POST.

Correct state determination could be based on some sort of matching of the results. Eg: status: 200, or value == success. More ideas needed here.

In some ways, it could be thought of as the "exec" of network resources, and in a positive manner. This is meant to replace traditional exec { "wget foo > /var/bar": } patterns that puppet users are familiar with.

It will probably get used in composite resources to implement higher level resources that act over an API.

Comments welcome!

(Disclaimer: Updated since original posting!)

[ffrank]

FWIW, without data passing, this could really only store HTTP responses to files. In this case, it would probably be awkward that this is an alternative way to manage files.

[purpleidea]

On Wed, Sep 28, 2016 at 6:02 PM, Felix Frank [email protected] wrote:

FWIW, without data passing, this could really only store HTTP responses to files. In this case, it would probably be awkward that this is an alternative way to manage files.

Without data passing it could still take a file path and store the
output there... Not saying I necessarily prefer that, just being open
to different ideas.

[purpleidea]

@ChrisMcKenzie Am particularly excited. Feel free to push a non-compiling WIP branch in case you want me to look at something early.

[ChrisMcKenzie]

I am starting to get a hold on the Res interface, I am trying to think about how the Watch might be implemented I feel like in this case it should be a noop, originally I though about polling but that seems like it would cause issues!

[purpleidea]

@ChrisMcKenzie What do you think about:

It should probably support long-polling to watch.

As a backup:

It could use the Poll metaparam.

[ChrisMcKenzie]

I think I like the metaparam approach best as it can be disabled if need be.

[purpleidea]

Since I did some reading of the HTTP libs recently, I thought I should mention that this should probably look at *Modified-Since headers perhaps.

[purpleidea]

FWIW, this should probably be called http:client. We have the http:server stuff merged, so this makes sense. Having a client resource that can download a file for us would be really neat.

[purpleidea]

@frebib asks in channel:

This is an interesting one, and raises a possible question with mgmt's design. What is the difference between a http:client resource with send/recv and a http client function in mcl? Could you argue that they do the same thing? A resource by nature is to "enforce a state", but it's being somewhat abused here to do something else. It's widely accepted that having multiple ways to do the same thing are frowned upon, so should this be a good example of where to define what a resource should and shouldn't be used for?

Here's my answer:

This is an excellent point.

So there are some strong reasons why I think we need an http:client resource. I don't think it breaks the idea of "state enforcement", but rather extends it past the legacy state enforcement ideas we're used to. I've demonstrated other examples of this with the dhcp:server resource and so on.

But it's true that an http_client function would have some value too. I think that the function is low priority for two reasons:

1. Failing functions aren't handled as well as they could be long-term. Once we have more language features, that will be more valuable. It's easy for an http endpoint to fail. Resources are the real "effectors" of things.

2. You can simulate the function aspect (and this might be the correct long-term way of getting data into a function, TBD) by having an http:client resource download to a file, and then the os.readfile function read that in. That would work today and give us resource meta param and programmatic control of how and when we download things.

3. Resources (like this one) can use send->recv, and so far that's where I expect most use will be. (Other than just writing to a file directly.)