Support optional full flux input

[fsteeg]

As a first step, it should be possible to paste and run a Flux like this:

"http://lobid.org/gnd/1160720738"
| open-http(accept="application/json")
| as-lines
| decode-json
| encode-json(prettyPrinting="true")
| print
;

No fix, no reading or writing local files. This should be run as it is.

Next steps would be to run workflows with embedded fix (fill the fix in UI with embedded content?) or fix files (show warning and use fix from UI?), as well as writing to files (write to temp file as we currently do for the input?).

If the flux does not provide it's own input we'd like to retain the current behavior. To detect that might be a little tricky since flux files can start with variables. We could use something like PG_DATA to refer to the UI input data in the Playground, which could be detected/replaced when building the actual flux to run, e.g. for the current sample:

PG_DATA
|as-lines
|decode-formeta
|fix
|stream-to-xml(rootTag="collection")

Functional review: @TobiasNx
Code review: @fsteeg

[katauber]

@fsteeg and I discussed offline:

• We will not implement filling the fix editor in the UI with the embedded fix content. The embedded fix content will stay in the flux editor as typed/pasted.
• The example above for an embedded fix is old, so I used for testing this example
• Using files in the flux editor will led to warnings when trying to process
• writing the result to a file via playground will leave the result in the playground empty and the server answers with the corresponding file
• A missing ; in the flux file throws no exception, I created an issue

[katauber]

• Using files in the flux editor will led to warnings when trying to process

Further we decided that we will not implement special warnings if files are used in the flux, because when processing there will be a corresponding error message that this file can not be found.

[katauber]

• writing the result to a file via playground will leave the result in the playground empty and the server answers with the corresponding file

the server does not answer with a file, the result will be in the result field, same as using print

[katauber]

Deployed to https://test.metafacture.org/playground/

[TobiasNx]

Greyed ares are good! And good development :)

Some things I stumbled upon:

• I assumed that the PG_Data would start with open-file as first function. But perhaps I am conceptually confused.
• local files can't be open, e.g. file:///home/tobias/Dokumente/marcrel.json and https://test.metafacture.org/playground/?flux=%22file%3A///home/tobias/Dokumente/marcrel.json%22%0A%7C+open-file%0A%7C+as-lines%0A%7C+decode-json%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%3B&active-editor=fix is that something that it should be able to do?

PS: for another ticket. FLUX editor suggests FIX commands, when I type "open-fil":

[katauber]

• I assumed that the PG_Data would start with open-file as first function. But perhaps I am conceptually confused.

Internally the function 'open-file' is used on the server, because the input of the data editor is stored in a temp file. But I think the user does not have to know where or how the input data is stored. Important is that you can use the content of the data field

I do not know if I am fully convinced by this, since this is about the full flux input. If someone would copy the flux the open-file would have to be added. Perhaps the variable PG_Data already hides the way the data is stored well enough.
But let us see if this will be a problem later on.

PS: for another ticket. FLUX editor suggests FIX commands, when I type "open-fil":

Did you test an inline fix before? I guess you wrote this fix command before in the flux editor. The monaco editor remembers every word you typed, so the editor don't know the difference between a fix or flux function (or something else).

I used your test case:

https://test.metafacture.org/playground/?flux=%22http%3A//lobid.org/gnd/1160720738%22%0A%7C+open-http%28accept%3D%22application/json%22%29%0A%7C+as-lines%0A%7C+decode-json%0A%7C+fix%28%22%0Amove_field%28preferredName%2C+prefLabel%29%0A%22%29%0A%7C+encode-json%28prettyPrinting%3D%22true%22%29%0A%7C+print%3B&active-editor=fix

Seems to be an result of this.

+1

[katauber]

Deployed to https://metafacture.org/playground/