How To: Add built in functions

Builtin functions like map, sort and so on are defined in builtin.c. Many are written in jq itself, while others have to be implemented in C.

If at all possible, new builtins should be written in jq code. If this is impossible, it's better to write a minimal C builtin for the currently-impossible part and then write the real function in jq.

There are three types of builtins: jq-defined, bytecoded, and C-coded. Bytecoded builtins are outside the scope of this wiki -- they are very intimately tied to the jq VM, parser, and compiler.

Builtins written in jq are just that, with all the power of jq. Builtins written in C do not take closures as arguments, just values, therefore C-coded builtins should usually be wrapped with a jq-coded builtin. Bytecoded builtins can have very strange semantics, but again, they are out of scope here.

Builtins written in jq

Add new jq-coded builtins to src/builtin.jq.

That's it. If you've never seen the def syntax before, you can check it out in the jq manual.

Of course, after you do this you need to write docs and tests, which in practice means adding a paragraph to the manual source, which is a YAML file. Examples from the manual are extracted and used for testing when you run make check.

Builtins written in C

Do something like commit 48be232

A builtin can be written in C by defining it as a static function in builtin.c, and adding an entry to the array function_list like:

{(cfunction_ptr)f_keys, "keys", 1}

f_keys is the name of the function, by convention they start with f_. The number at the end is the number of arguments including the implicit input argument: keys is called from jq passing no explicit arguments and operates on the program's input.

The function must be declared to take a jq_state * argument, the appropriate number of arguments of type jv, and return a jv. It should consume all arguments (to know what "consume" means, and to figure out what you can do with a jv, read C-API:-jv).

Builtin functions written in C can't call back to jq like map does. If this is what you want to do, you should split your builtin into a small C builtin and a wrapper jq builtin which calls it. For an example of this, look at the implementation of sort_by, which is written in jq but calls _sort_by_impl.

Again, write docs and tests when you're done.

Bytecoded builtins

Mostly you should never need to add bytecoded builtins. The primitives that jq provides should be sufficient to build a wide array of jq-coded functions. However, if you need to add a bytecoded builtin, you should do something like commit 5a863bf (which also adds syntax).