| --- |
| headline: jq 1.4 Manual |
| |
| history: | |
| |
| *The manual for the development version of jq can be found |
| [here](/jq/manual).* |
| |
| body: | |
| |
| A jq program is a "filter": it takes an input, and produces an |
| output. There are a lot of builtin filters for extracting a |
| particular field of an object, or converting a number to a string, |
| or various other standard tasks. |
| |
| Filters can be combined in various ways - you can pipe the output of |
| one filter into another filter, or collect the output of a filter |
| into an array. |
| |
| Some filters produce multiple results, for instance there's one that |
| produces all the elements of its input array. Piping that filter |
| into a second runs the second filter for each element of the |
| array. Generally, things that would be done with loops and iteration |
| in other languages are just done by gluing filters together in jq. |
| |
| It's important to remember that every filter has an input and an |
| output. Even literals like "hello" or 42 are filters - they take an |
| input but always produce the same literal as output. Operations that |
| combine two filters, like addition, generally feed the same input to |
| both and combine the results. So, you can implement an averaging |
| filter as `add / length` - feeding the input array both to the `add` |
| filter and the `length` filter and dividing the results. |
| |
| But that's getting ahead of ourselves. :) Let's start with something |
| simpler: |
| |
| manpage_intro: | |
| jq(1) -- Command-line JSON processor |
| ==================================== |
| |
| ## SYNOPSIS |
| |
| `jq` [<options>...] <filter> [<files>...] |
| |
| `jq` can transform JSON in various ways, by selecting, iterating, |
| reducing and otherwise mangling JSON documents. For instance, |
| running the command `jq 'map(.price) | add'` will take an array of |
| JSON objects as input and return the sum of their "price" fields. |
| |
| By default, `jq` reads a stream of JSON objects (whitespace |
| separated) from `stdin`. One or more <files> may be specified, in |
| which case `jq` will read input from those instead. |
| |
| The <options> are described in the [INVOKING JQ] section, they |
| mostly concern input and output formatting. The <filter> is written |
| in the jq language and specifies how to transform the input |
| document. |
| |
| ## FILTERS |
| |
| manpage_epilogue: | |
| ## BUGS |
| |
| Presumably. Report them or discuss them at: |
| |
| https://github.com/jqlang/jq/issues |
| |
| ## AUTHOR |
| |
| Stephen Dolan `<mu@netsoc.tcd.ie>` |
| |
| sections: |
| - title: Invoking jq |
| body: | |
| |
| jq filters run on a stream of JSON data. The input to jq is |
| parsed as a sequence of whitespace-separated JSON values which |
| are passed through the provided filter one at a time. The |
| output(s) of the filter are written to standard out, again as a |
| sequence of whitespace-separated JSON data. |
| |
| Note: it is important to mind the shell's quoting rules. As a |
| general rule it's best to always quote (with single-quote |
| characters) the jq program, as too many characters with special |
| meaning to jq are also shell meta-characters. For example, `jq |
| "foo"` will fail on most Unix shells because that will be the same |
| as `jq foo`, which will generally fail because `foo is not |
| defined`. When using the Windows command shell (cmd.exe) it's |
| best to use double quotes around your jq program when given on the |
| command-line (instead of the `-f program-file` option), but then |
| double-quotes in the jq program need backslash escaping. |
| |
| You can affect how jq reads and writes its input and output |
| using some command-line options: |
| |
| * `--version`/`-V`: |
| |
| Output the jq version and exit with zero. |
| |
| * `--slurp`/`-s`: |
| |
| Instead of running the filter for each JSON object in the |
| input, read the entire input stream into a large array and run |
| the filter just once. |
| |
| * `--online-input`/`-I`: |
| |
| When the top-level input value is an array produce its elements |
| instead of the array. This allows on-line processing of |
| potentially very large top-level arrays' elements. |
| |
| * `--raw-input`/`-R`: |
| |
| Don't parse the input as JSON. Instead, each line of text is |
| passed to the filter as a string. If combined with `--slurp`, |
| then the entire input is passed to the filter as a single long |
| string. |
| |
| * `--null-input`/`-n`: |
| |
| Don't read any input at all! Instead, the filter is run once |
| using `null` as the input. This is useful when using jq as a |
| simple calculator or to construct JSON data from scratch. |
| |
| * `--compact-output` / `-c`: |
| |
| By default, jq pretty-prints JSON output. Using this option |
| will result in more compact output by instead putting each |
| JSON object on a single line. |
| |
| * `--color-output` / `-C` and `--monochrome-output` / `-M`: |
| |
| By default, jq outputs colored JSON if writing to a |
| terminal. You can force it to produce color even if writing to |
| a pipe or a file using `-C`, and disable color with `-M`. |
| |
| * `--ascii-output` / `-a`: |
| |
| jq usually outputs non-ASCII Unicode codepoints as UTF-8, even |
| if the input specified them as escape sequences (like |
| "\u03bc"). Using this option, you can force jq to produce pure |
| ASCII output with every non-ASCII character replaced with the |
| equivalent escape sequence. |
| |
| * `--unbuffered`: |
| |
| Flush the output after each JSON object is printed (useful if |
| you're piping a slow data source into jq and piping jq's |
| output elsewhere). |
| |
| * `--sort-keys` / `-S`: |
| |
| Output the fields of each object with the keys in sorted order. |
| |
| * `--raw-output` / `-r`: |
| |
| With this option, if the filter's result is a string then it |
| will be written directly to standard output rather than being |
| formatted as a JSON string with quotes. This can be useful for |
| making jq filters talk to non-JSON-based systems. |
| |
| * `-f filename` / `--from-file filename`: |
| |
| Read filter from the file rather than from a command line, like |
| awk's -f option. You can also use '#' to make comments. |
| |
| * `-e` / `--exit-status`: |
| |
| Sets the exit status of jq to 0 if the last output value was |
| neither `false` nor `null`, 1 if the last output value was |
| either `false` or `null`, or 4 if no valid result was ever |
| produced. Normally jq exits with 2 if there was any usage |
| problem or system error, 3 if there was a jq program compile |
| error, or 0 if the jq program ran. |
| |
| * `--arg name value`: |
| |
| This option passes a value to the jq program as a predefined |
| variable. If you run jq with `--arg foo bar`, then `$foo` is |
| available in the program and has the value `"bar"`. |
| |
| * `--argfile name filename`: |
| |
| This option passes the first value from the named file as a |
| value to the jq program as a predefined variable. If you run jq |
| with `--argfile foo bar`, then `$foo` is available in the |
| program and has the value resulting from parsing the content of |
| the file named `bar`. |
| |
| - title: Basic filters |
| entries: |
| - title: "`.`" |
| body: | |
| |
| The absolute simplest (and least interesting) filter |
| is `.`. This is a filter that takes its input and |
| produces it unchanged as output. |
| |
| Since jq by default pretty-prints all output, this trivial |
| program can be a useful way of formatting JSON output from, |
| say, `curl`. |
| |
| examples: |
| - program: '.' |
| input: '"Hello, world!"' |
| output: ['"Hello, world!"'] |
| |
| - title: "`.foo`, `.foo.bar`" |
| body: | |
| |
| The simplest *useful* filter is `.foo`. When given a |
| JSON object (aka dictionary or hash) as input, it produces |
| the value at the key "foo", or null if there's none present. |
| |
| If the key contains special characters or starts with a digit, |
| you need to surround it with double quotes like this: `."foo$"`. |
| |
| A filter of the form `.foo.bar` is equivalent to `.foo|.bar`. |
| |
| examples: |
| - program: '.foo' |
| input: '{"foo": 42, "bar": "less interesting data"}' |
| output: ['42'] |
| - program: '.foo' |
| input: '{"notfoo": true, "alsonotfoo": false}' |
| output: ['null'] |
| - program: '.["foo"]' |
| input: '{"foo": 42}' |
| output: ['42'] |
| |
| - title: "`.foo?`" |
| body: | |
| |
| Just like `.foo`, but does not output an error when `.` is not an |
| object. |
| |
| examples: |
| - program: '.foo?' |
| input: '{"foo": 42, "bar": "less interesting data"}' |
| output: ['42'] |
| - program: '.foo?' |
| input: '{"notfoo": true, "alsonotfoo": false}' |
| output: ['null'] |
| - program: '.["foo"]?' |
| input: '{"foo": 42}' |
| output: ['42'] |
| - program: '[.foo?]' |
| input: '[1,2]' |
| output: ['[]'] |
| |
| - title: "`.[<string>]`, `.[<number>]`, `.[<number>:<number>]`" |
| body: | |
| |
| You can also look up fields of an object using syntax like |
| `.["foo"]` (`.foo` above is a shorthand version of this). This |
| one works for arrays as well, if the key is an |
| integer. Arrays are zero-based (like javascript), so `.[2]` |
| returns the third element of the array. |
| |
| The `.[10:15]` syntax can be used to return a subarray of an |
| array or substring of a string. The array returned by |
| `.[10:15]` will be of length 5, containing the elements from |
| index 10 (inclusive) to index 15 (exclusive). Either index may |
| be negative (in which case it counts backwards from the end of |
| the array), or omitted (in which case it refers to the start |
| or end of the array). |
| |
| The `?` "operator" can also be used with the slice operator, |
| as in `.[10:15]?`, which outputs values where the inputs are |
| slice-able. |
| |
| examples: |
| - program: '.[0]' |
| input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}]' |
| output: ['{"name":"JSON", "good":true}'] |
| |
| - program: '.[2]' |
| input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}]' |
| output: ['null'] |
| |
| - program: '.[2:4]' |
| input: '["a","b","c","d","e"]' |
| output: ['["c", "d"]'] |
| |
| - program: '.[2:4]' |
| input: '"abcdefghi"' |
| output: ['"cd"'] |
| |
| - program: '.[:3]' |
| input: '["a","b","c","d","e"]' |
| output: ['["a", "b", "c"]'] |
| |
| - program: '.[-2:]' |
| input: '["a","b","c","d","e"]' |
| output: ['["d", "e"]'] |
| |
| - title: "`.[]`" |
| body: | |
| |
| If you use the `.[index]` syntax, but omit the index |
| entirely, it will return *all* of the elements of an |
| array. Running `.[]` with the input `[1,2,3]` will produce the |
| numbers as three separate results, rather than as a single |
| array. |
| |
| You can also use this on an object, and it will return all |
| the values of the object. |
| |
| examples: |
| - program: '.[]' |
| input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}]' |
| output: |
| - '{"name":"JSON", "good":true}' |
| - '{"name":"XML", "good":false}' |
| |
| - program: '.[]' |
| input: '[]' |
| output: [] |
| |
| - program: '.[]' |
| input: '{"a": 1, "b": 1}' |
| output: ['1', '1'] |
| |
| - title: "`.[]?`" |
| body: | |
| |
| Like `.[]`, but no errors will be output if . is not an array |
| or object. |
| |
| - title: "`,`" |
| body: | |
| |
| If two filters are separated by a comma, then the |
| input will be fed into both and there will be multiple |
| outputs: first, all of the outputs produced by the left |
| expression, and then all of the outputs produced by the |
| right. For instance, filter `.foo, .bar`, produces |
| both the "foo" fields and "bar" fields as separate outputs. |
| |
| examples: |
| - program: '.foo, .bar' |
| input: '{"foo": 42, "bar": "something else", "baz": true}' |
| output: ['42', '"something else"'] |
| |
| - program: ".user, .projects[]" |
| input: '{"user":"stedolan", "projects": ["jq", "wikiflow"]}' |
| output: ['"stedolan"', '"jq"', '"wikiflow"'] |
| |
| - program: '.[4,2]' |
| input: '["a","b","c","d","e"]' |
| output: ['"e"', '"c"'] |
| |
| - title: "`|`" |
| body: | |
| The | operator combines two filters by feeding the output(s) of |
| the one on the left into the input of the one on the right. It's |
| pretty much the same as the Unix shell's pipe, if you're used to |
| that. |
| |
| If the one on the left produces multiple results, the one on |
| the right will be run for each of those results. So, the |
| expression `.[] | .foo` retrieves the "foo" field of each |
| element of the input array. |
| |
| examples: |
| - program: '.[] | .name' |
| input: '[{"name":"JSON", "good":true}, {"name":"XML", "good":false}]' |
| output: ['"JSON"', '"XML"'] |
| |
| - title: Types and Values |
| body: | |
| |
| jq supports the same set of datatypes as JSON - numbers, |
| strings, booleans, arrays, objects (which in JSON-speak are |
| hashes with only string keys), and "null". |
| |
| Booleans, null, strings and numbers are written the same way as |
| in javascript. Just like everything else in jq, these simple |
| values take an input and produce an output - `42` is a valid jq |
| expression that takes an input, ignores it, and returns 42 |
| instead. |
| |
| entries: |
| - title: "Array construction: `[]`" |
| body: | |
| |
| As in JSON, `[]` is used to construct arrays, as in |
| `[1,2,3]`. The elements of the arrays can be any jq |
| expression. All of the results produced by all of the |
| expressions are collected into one big array. You can use it |
| to construct an array out of a known quantity of values (as |
| in `[.foo, .bar, .baz]`) or to "collect" all the results of a |
| filter into an array (as in `[.items[].name]`) |
| |
| Once you understand the "," operator, you can look at jq's array |
| syntax in a different light: the expression `[1,2,3]` is not using a |
| built-in syntax for comma-separated arrays, but is instead applying |
| the `[]` operator (collect results) to the expression 1,2,3 (which |
| produces three different results). |
| |
| If you have a filter `X` that produces four results, |
| then the expression `[X]` will produce a single result, an |
| array of four elements. |
| |
| examples: |
| - program: "[.user, .projects[]]" |
| input: '{"user":"stedolan", "projects": ["jq", "wikiflow"]}' |
| output: ['["stedolan", "jq", "wikiflow"]'] |
| - title: "Objects: `{}`" |
| body: | |
| |
| Like JSON, `{}` is for constructing objects (aka |
| dictionaries or hashes), as in: `{"a": 42, "b": 17}`. |
| |
| If the keys are "sensible" (all alphabetic characters), then |
| the quotes can be left off. The value can be any expression |
| (although you may need to wrap it in parentheses if it's a |
| complicated one), which gets applied to the {} expression's |
| input (remember, all filters have an input and an |
| output). |
| |
| {foo: .bar} |
| |
| will produce the JSON object `{"foo": 42}` if given the JSON |
| object `{"bar":42, "baz":43}`. You can use this to select |
| particular fields of an object: if the input is an object |
| with "user", "title", "id", and "content" fields and you |
| just want "user" and "title", you can write |
| |
| {user: .user, title: .title} |
| |
| Because that's so common, there's a shortcut syntax: `{user, title}`. |
| |
| If one of the expressions produces multiple results, |
| multiple dictionaries will be produced. If the input's |
| |
| {"user":"stedolan","titles":["JQ Primer", "More JQ"]} |
| |
| then the expression |
| |
| {user, title: .titles[]} |
| |
| will produce two outputs: |
| |
| {"user":"stedolan", "title": "JQ Primer"} |
| {"user":"stedolan", "title": "More JQ"} |
| |
| Putting parentheses around the key means it will be evaluated as an |
| expression. With the same input as above, |
| |
| {(.user): .titles} |
| |
| produces |
| |
| {"stedolan": ["JQ Primer", "More JQ"]} |
| |
| examples: |
| - program: '{user, title: .titles[]}' |
| input: '{"user":"stedolan","titles":["JQ Primer", "More JQ"]}' |
| output: |
| - '{"user":"stedolan", "title": "JQ Primer"}' |
| - '{"user":"stedolan", "title": "More JQ"}' |
| - program: '{(.user): .titles}' |
| input: '{"user":"stedolan","titles":["JQ Primer", "More JQ"]}' |
| output: ['{"stedolan": ["JQ Primer", "More JQ"]}'] |
| |
| - title: Builtin operators and functions |
| body: | |
| |
| Some jq operators (for instance, `+`) do different things |
| depending on the type of their arguments (arrays, numbers, |
| etc.). However, jq never does implicit type conversions. If you |
| try to add a string to an object you'll get an error message and |
| no result. |
| |
| entries: |
| - title: "Addition: `+`" |
| body: | |
| |
| The operator `+` takes two filters, applies them both |
| to the same input, and adds the results together. What |
| "adding" means depends on the types involved: |
| |
| - **Numbers** are added by normal arithmetic. |
| |
| - **Arrays** are added by being concatenated into a larger array. |
| |
| - **Strings** are added by being joined into a larger string. |
| |
| - **Objects** are added by merging, that is, inserting all |
| the key-value pairs from both objects into a single |
| combined object. If both objects contain a value for the |
| same key, the object on the right of the `+` wins. (For |
| recursive merge use the `*` operator.) |
| |
| `null` can be added to any value, and returns the other |
| value unchanged. |
| |
| examples: |
| - program: '.a + 1' |
| input: '{"a": 7}' |
| output: ['8'] |
| - program: '.a + .b' |
| input: '{"a": [1,2], "b": [3,4]}' |
| output: ['[1,2,3,4]'] |
| - program: '.a + null' |
| input: '{"a": 1}' |
| output: ['1'] |
| - program: '.a + 1' |
| input: '{}' |
| output: ['1'] |
| - program: '{a: 1} + {b: 2} + {c: 3} + {a: 42}' |
| input: 'null' |
| output: ['{"a": 42, "b": 2, "c": 3}'] |
| |
| - title: "Subtraction: `-`" |
| body: | |
| |
| As well as normal arithmetic subtraction on numbers, the `-` |
| operator can be used on arrays to remove all occurrences of |
| the second array's elements from the first array. |
| |
| examples: |
| - program: '4 - .a' |
| input: '{"a":3}' |
| output: ['1'] |
| - program: . - ["xml", "yaml"] |
| input: '["xml", "yaml", "json"]' |
| output: ['["json"]'] |
| |
| - title: "Multiplication, division, modulo: `*`, `/`, `%`" |
| body: | |
| |
| These operators only work on numbers, and do the expected. |
| |
| Multiplying a string by a number produces the concatenation of |
| that string that many times. |
| |
| Dividing a string by another splits the first using the second |
| as separators. |
| |
| Multiplying two objects will merge them recursively: this works |
| like addition but if both objects contain a value for the |
| same key, and the values are objects, the two are merged with |
| the same strategy. |
| |
| examples: |
| - program: '10 / . * 3' |
| input: '5' |
| output: ['6'] |
| - program: '. / ", "' |
| input: '"a, b,c,d, e"' |
| output: ['["a","b,c,d","e"]'] |
| - program: '{"k": {"a": 1, "b": 2}} * {"k": {"a": 0,"c": 3}}' |
| input: 'null' |
| output: ['{"k": {"a": 0, "b": 2, "c": 3}}'] |
| |
| - title: "`length`" |
| body: | |
| |
| The builtin function `length` gets the length of various |
| different types of value: |
| |
| - The length of a **string** is the number of Unicode |
| codepoints it contains (which will be the same as its |
| JSON-encoded length in bytes if it's pure ASCII). |
| |
| - The length of an **array** is the number of elements. |
| |
| - The length of an **object** is the number of key-value pairs. |
| |
| - The length of **null** is zero. |
| |
| examples: |
| - program: '.[] | length' |
| input: '[[1,2], "string", {"a":2}, null]' |
| output: ['2', '6', '1', '0'] |
| |
| - title: "`keys`" |
| body: | |
| |
| The builtin function `keys`, when given an object, returns |
| its keys in an array. |
| |
| The keys are sorted "alphabetically", by unicode codepoint |
| order. This is not an order that makes particular sense in |
| any particular language, but you can count on it being the |
| same for any two objects with the same set of keys, |
| regardless of locale settings. |
| |
| When `keys` is given an array, it returns the valid indices |
| for that array: the integers from 0 to length-1. |
| |
| examples: |
| - program: 'keys' |
| input: '{"abc": 1, "abcd": 2, "Foo": 3}' |
| output: ['["Foo", "abc", "abcd"]'] |
| - program: 'keys' |
| input: '[42,3,35]' |
| output: ['[0,1,2]'] |
| |
| - title: "`has`" |
| body: | |
| |
| The builtin function `has` returns whether the input object |
| has the given key, or the input array has an element at the |
| given index. |
| |
| `has($key)` has the same effect as checking whether `$key` |
| is a member of the array returned by `keys`, although `has` |
| will be faster. |
| |
| examples: |
| - program: 'map(has("foo"))' |
| input: '[{"foo": 42}, {}]' |
| output: ['[true, false]'] |
| - program: 'map(has(2))' |
| input: '[[0,1], ["a","b","c"]]' |
| output: ['[false, true]'] |
| |
| - title: "`del`" |
| body: | |
| |
| The builtin function `del` removes a key and its corresponding |
| value from an object. |
| |
| examples: |
| - program: 'del(.foo)' |
| input: '{"foo": 42, "bar": 9001, "baz": 42}' |
| output: ['{"bar": 9001, "baz": 42}'] |
| - program: 'del(.[1, 2])' |
| input: '["foo", "bar", "baz"]' |
| output: ['["foo"]'] |
| |
| - title: "`to_entries`, `from_entries`, `with_entries(f)`" |
| body: | |
| |
| These functions convert between an object and an array of |
| key-value pairs. If `to_entries` is passed an object, then |
| for each `k: v` entry in the input, the output array |
| includes `{"key": k, "value": v}`. |
| |
| `from_entries` does the opposite conversion, and |
| `with_entries(f)` is a shorthand for `to_entries | map(f) | |
| from_entries`, useful for doing some operation to all keys |
| and values of an object. |
| |
| examples: |
| - program: 'to_entries' |
| input: '{"a": 1, "b": 2}' |
| output: ['[{"key":"a", "value":1}, {"key":"b", "value":2}]'] |
| - program: 'from_entries' |
| input: '[{"key":"a", "value":1}, {"key":"b", "value":2}]' |
| output: ['{"a": 1, "b": 2}'] |
| - program: 'with_entries(.key |= "KEY_" + .)' |
| input: '{"a": 1, "b": 2}' |
| output: ['{"KEY_a": 1, "KEY_b": 2}'] |
| |
| |
| - title: "`select`" |
| body: | |
| |
| The function `select(foo)` produces its input unchanged if |
| `foo` returns true for that input, and produces no output |
| otherwise. |
| |
| It's useful for filtering lists: `[1,2,3] | map(select(. >= 2))` |
| will give you `[2,3]`. |
| |
| examples: |
| - program: 'map(select(. >= 2))' |
| input: '[1,5,3,0,7]' |
| output: ['[5,3,7]'] |
| |
| |
| - title: "`arrays`, `objects`, `iterables`, `booleans`, `numbers`, `strings`, `nulls`, `values`, `scalars`" |
| body: | |
| |
| These built-ins select only inputs that are arrays, objects, |
| iterables (arrays or objects), booleans, numbers, strings, |
| null, non-null values, and non-iterables, respectively. |
| |
| examples: |
| - program: '.[]|numbers' |
| input: '[[],{},1,"foo",null,true,false]' |
| output: ['1'] |
| |
| - title: "`empty`" |
| body: | |
| |
| `empty` returns no results. None at all. Not even `null`. |
| |
| It's useful on occasion. You'll know if you need it :) |
| |
| examples: |
| - program: '1, empty, 2' |
| input: 'null' |
| output: ['1', '2'] |
| - program: '[1,2,empty,3]' |
| input: 'null' |
| output: ['[1,2,3]'] |
| |
| - title: "`map(f)`" |
| body: | |
| |
| For any filter `f`, `map(f)` will run that filter for each |
| element of the input array, and produce the outputs a new |
| array. `map(.+1)` will increment each element of an array of numbers. |
| |
| `map(f)` is equivalent to `[.[] | f]`. In fact, this is how |
| it's defined. |
| |
| examples: |
| - program: 'map(.+1)' |
| input: '[1,2,3]' |
| output: ['[2,3,4]'] |
| |
| - title: "`paths`" |
| body: | |
| |
| Outputs the paths to all the elements in its input (except it |
| does not output the empty list, representing . itself). |
| |
| `paths` is equivalent to |
| |
| def paths: path(recurse(if (type|. == "array" or . == "object") then .[] else empty end))|select(length > 0); |
| |
| examples: |
| - program: '[paths]' |
| input: '[1,[[],{"a":2}]]' |
| output: ['[[0],[1],[1,0],[1,1],[1,1,"a"]]'] |
| |
| - title: "`leaf_paths`" |
| body: | |
| |
| Outputs the paths to all the leaves (non-array, non-object |
| elements) in its input. |
| |
| examples: |
| - program: '[leaf_paths]' |
| input: '[1,[[],{"a":2}]]' |
| output: ['[[0],[1,1,"a"]]'] |
| |
| - title: "`add`" |
| body: | |
| |
| The filter `add` takes as input an array, and produces as |
| output the elements of the array added together. This might |
| mean summed, concatenated or merged depending on the types |
| of the elements of the input array - the rules are the same |
| as those for the `+` operator (described above). |
| |
| If the input is an empty array, `add` returns `null`. |
| |
| examples: |
| - program: add |
| input: '["a","b","c"]' |
| output: ['"abc"'] |
| - program: add |
| input: '[1, 2, 3]' |
| output: ['6'] |
| - program: add |
| input: '[]' |
| output: ["null"] |
| |
| - title: "`any`" |
| body: | |
| |
| The filter `any` takes as input an array of boolean values, |
| and produces `true` as output if any of the elements of |
| the array are `true`. |
| |
| If the input is an empty array, `any` returns `false`. |
| |
| examples: |
| - program: any |
| input: '[true, false]' |
| output: ["true"] |
| - program: any |
| input: '[false, false]' |
| output: ["false"] |
| - program: any |
| input: '[]' |
| output: ["false"] |
| |
| - title: "`all`" |
| body: | |
| |
| The filter `all` takes as input an array of boolean values, |
| and produces `true` as output if all of the elements of |
| the array are `true`. |
| |
| If the input is an empty array, `all` returns `true`. |
| |
| examples: |
| - program: all |
| input: '[true, false]' |
| output: ["false"] |
| - program: all |
| input: '[true, true]' |
| output: ["true"] |
| - program: all |
| input: '[]' |
| output: ["true"] |
| |
| - title: "`range`" |
| body: | |
| |
| The `range` function produces a range of numbers. `range(4;10)` |
| produces 6 numbers, from 4 (inclusive) to 10 (exclusive). The numbers |
| are produced as separate outputs. Use `[range(4;10)]` to get a range as |
| an array. |
| |
| examples: |
| - program: 'range(2;4)' |
| input: 'null' |
| output: ['2', '3'] |
| - program: '[range(2;4)]' |
| input: 'null' |
| output: ['[2,3]'] |
| |
| - title: "`floor`" |
| body: | |
| |
| The `floor` function returns the floor of its numeric input. |
| |
| examples: |
| - program: 'floor' |
| input: '3.14159' |
| output: ['3'] |
| |
| - title: "`sqrt`" |
| body: | |
| |
| The `sqrt` function returns the square root of its numeric input. |
| |
| examples: |
| - program: 'sqrt' |
| input: '9' |
| output: ['3'] |
| |
| - title: "`tonumber`" |
| body: | |
| |
| The `tonumber` function parses its input as a number. It |
| will convert correctly-formatted strings to their numeric |
| equivalent, leave numbers alone, and give an error on all other input. |
| |
| examples: |
| - program: '.[] | tonumber' |
| input: '[1, "1"]' |
| output: ['1', '1'] |
| |
| - title: "`tostring`" |
| body: | |
| |
| The `tostring` function prints its input as a |
| string. Strings are left unchanged, and all other values are |
| JSON-encoded. |
| |
| examples: |
| - program: '.[] | tostring' |
| input: '[1, "1", [1]]' |
| output: ['"1"', '"1"', '"[1]"'] |
| |
| - title: "`type`" |
| body: | |
| |
| The `type` function returns the type of its argument as a |
| string, which is one of null, boolean, number, string, array |
| or object. |
| |
| examples: |
| - program: 'map(type)' |
| input: '[0, false, [], {}, null, "hello"]' |
| output: ['["number", "boolean", "array", "object", "null", "string"]'] |
| |
| - title: "`sort`, `sort_by`" |
| body: | |
| |
| The `sort` functions sorts its input, which must be an |
| array. Values are sorted in the following order: |
| |
| * `null` |
| * `false` |
| * `true` |
| * numbers |
| * strings, in alphabetical order (by unicode codepoint value) |
| * arrays, in lexical order |
| * objects |
| |
| The ordering for objects is a little complex: first they're |
| compared by comparing their sets of keys (as arrays in |
| sorted order), and if their keys are equal then the values |
| are compared key by key. |
| |
| `sort_by` may be used to sort by a particular field of an |
| object, or by applying any jq filter. `sort_by(foo)` |
| compares two elements by comparing the result of `foo` on |
| each element. |
| |
| examples: |
| - program: 'sort' |
| input: '[8,3,null,6]' |
| output: ['[null,3,6,8]'] |
| - program: 'sort_by(.foo)' |
| input: '[{"foo":4, "bar":10}, {"foo":3, "bar":100}, {"foo":2, "bar":1}]' |
| output: ['[{"foo":2, "bar":1}, {"foo":3, "bar":100}, {"foo":4, "bar":10}]'] |
| |
| - title: "`group_by`" |
| body: | |
| |
| `group_by(.foo)` takes as input an array, groups the |
| elements having the same `.foo` field into separate arrays, |
| and produces all of these arrays as elements of a larger |
| array, sorted by the value of the `.foo` field. |
| |
| Any jq expression, not just a field access, may be used in |
| place of `.foo`. The sorting order is the same as described |
| in the `sort` function above. |
| |
| examples: |
| - program: 'group_by(.foo)' |
| input: '[{"foo":1, "bar":10}, {"foo":3, "bar":100}, {"foo":1, "bar":1}]' |
| output: ['[[{"foo":1, "bar":10}, {"foo":1, "bar":1}], [{"foo":3, "bar":100}]]'] |
| |
| - title: "`min`, `max`, `min_by`, `max_by`" |
| body: | |
| |
| Find the minimum or maximum element of the input array. The |
| `_by` versions allow you to specify a particular field or |
| property to examine, e.g. `min_by(.foo)` finds the object |
| with the smallest `foo` field. |
| |
| examples: |
| - program: 'min' |
| input: '[5,4,2,7]' |
| output: ['2'] |
| - program: 'max_by(.foo)' |
| input: '[{"foo":1, "bar":14}, {"foo":2, "bar":3}]' |
| output: ['{"foo":2, "bar":3}'] |
| |
| - title: "`unique`" |
| body: | |
| |
| The `unique` function takes as input an array and produces |
| an array of the same elements, in sorted order, with |
| duplicates removed. |
| |
| examples: |
| - program: 'unique' |
| input: '[1,2,5,3,5,3,1,3]' |
| output: ['[1,2,3,5]'] |
| |
| - title: "`unique_by`" |
| body: | |
| |
| The `unique_by(.foo)` function takes as input an array and produces |
| an array of the same elements, in sorted order, with |
| elements with a duplicate `.foo` field removed. Think of it as making |
| an array by taking one element out of every group produced by |
| `group_by`. |
| |
| examples: |
| - program: 'unique_by(.foo)' |
| input: '[{"foo": 1, "bar": 2}, {"foo": 1, "bar": 3}, {"foo": 4, "bar": 5}]' |
| output: ['[{"foo": 1, "bar": 2}, {"foo": 4, "bar": 5}]'] |
| - program: 'unique_by(length)' |
| input: '["chunky", "bacon", "kitten", "cicada", "asparagus"]' |
| output: ['["bacon", "chunky", "asparagus"]'] |
| |
| |
| - title: "`reverse`" |
| body: | |
| |
| This function reverses an array. |
| |
| examples: |
| - program: 'reverse' |
| input: '[1,2,3,4]' |
| output: ['[4,3,2,1]'] |
| |
| - title: "`contains`" |
| body: | |
| |
| The filter `contains(b)` will produce true if b is |
| completely contained within the input. A string B is |
| contained in a string A if B is a substring of A. An array B |
| is contained in an array A is all elements in B are |
| contained in any element in A. An object B is contained in |
| object A if all of the values in B are contained in the |
| value in A with the same key. All other types are assumed to |
| be contained in each other if they are equal. |
| |
| examples: |
| - program: 'contains("bar")' |
| input: '"foobar"' |
| output: ['true'] |
| - program: 'contains(["baz", "bar"])' |
| input: '["foobar", "foobaz", "blarp"]' |
| output: ['true'] |
| - program: 'contains(["bazzzzz", "bar"])' |
| input: '["foobar", "foobaz", "blarp"]' |
| output: ['false'] |
| - program: 'contains({foo: 12, bar: [{barp: 12}]})' |
| input: '{"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]}' |
| output: ['true'] |
| - program: 'contains({foo: 12, bar: [{barp: 15}]})' |
| input: '{"foo": 12, "bar":[1,2,{"barp":12, "blip":13}]}' |
| output: ['false'] |
| |
| - title: "`indices(s)`" |
| body: | |
| |
| Outputs an array containing the indices in `.` where `s` |
| occurs. The input may be an array, in which case if `s` is an |
| array then the indices output will be those where all elements |
| in `.` match those of `s`. |
| |
| examples: |
| - program: 'indices(", ")' |
| input: '"a,b, cd, efg, hijk"' |
| output: ['[3,7,12]'] |
| - program: 'indices(1)' |
| input: '[0,1,2,1,3,1,4]' |
| output: ['[1,3,5]'] |
| - program: 'indices([1,2])' |
| input: '[0,1,2,3,1,4,2,5,1,2,6,7]' |
| output: ['[1,8]'] |
| |
| - title: "`index(s)`, `rindex(s)`" |
| body: | |
| |
| Outputs the index of the first (`index`) or last (`rindex`) |
| occurrence of `s` in the input. |
| |
| examples: |
| - program: 'index(", ")' |
| input: '"a,b, cd, efg, hijk"' |
| output: ['3'] |
| - program: 'rindex(", ")' |
| input: '"a,b, cd, efg, hijk"' |
| output: ['12'] |
| |
| - title: "`startswith`" |
| body: | |
| |
| Outputs `true` if . starts with the given string argument. |
| |
| examples: |
| - program: '[.[]|startswith("foo")]' |
| input: '["fo", "foo", "barfoo", "foobar", "barfoob"]' |
| output: ['[false, true, false, true, false]'] |
| |
| - title: "`endswith`" |
| body: | |
| |
| Outputs `true` if . ends with the given string argument. |
| |
| examples: |
| - program: '[.[]|endswith("foo")]' |
| input: '["foobar", "barfoo"]' |
| output: ['[false, true]'] |
| |
| - title: "`ltrimstr`" |
| body: | |
| |
| Outputs its input with the given prefix string removed, if it |
| starts with it. |
| |
| examples: |
| - program: '[.[]|ltrimstr("foo")]' |
| input: '["fo", "foo", "barfoo", "foobar", "afoo"]' |
| output: ['["fo","","barfoo","bar","afoo"]'] |
| |
| - title: "`rtrimstr`" |
| body: | |
| |
| Outputs its input with the given suffix string removed, if it |
| starts with it. |
| |
| examples: |
| - program: '[.[]|rtrimstr("foo")]' |
| input: '["fo", "foo", "barfoo", "foobar", "foob"]' |
| output: ['["fo","","bar","foobar","foob"]'] |
| |
| - title: "`explode`" |
| body: | |
| |
| Converts an input string into an array of the string's |
| codepoint numbers. |
| |
| examples: |
| - program: 'explode' |
| input: '"foobar"' |
| output: ['[102,111,111,98,97,114]'] |
| |
| - title: "`implode`" |
| body: | |
| |
| The inverse of explode. |
| |
| examples: |
| - program: 'implode' |
| input: '[65, 66, 67]' |
| output: ['"ABC"'] |
| |
| - title: "`split`" |
| body: | |
| |
| Splits an input string on the separator argument. |
| |
| examples: |
| - program: 'split(", ")' |
| input: '"a, b,c,d, e"' |
| output: ['["a","b,c,d","e"]'] |
| |
| - title: "`join`" |
| body: | |
| |
| Joins the array of elements given as input, using the |
| argument as separator. It is the inverse of `split`: that is, |
| running `split("foo") | join("foo")` over any input string |
| returns said input string. |
| |
| examples: |
| - program: 'join(", ")' |
| input: '["a","b,c,d","e"]' |
| output: ['"a, b,c,d, e"'] |
| |
| |
| - title: "`recurse`" |
| body: | |
| |
| The `recurse` function allows you to search through a |
| recursive structure, and extract interesting data from all |
| levels. Suppose your input represents a filesystem: |
| |
| {"name": "/", "children": [ |
| {"name": "/bin", "children": [ |
| {"name": "/bin/ls", "children": []}, |
| {"name": "/bin/sh", "children": []}]}, |
| {"name": "/home", "children": [ |
| {"name": "/home/stephen", "children": [ |
| {"name": "/home/stephen/jq", "children": []}]}]}]} |
| |
| Now suppose you want to extract all of the filenames |
| present. You need to retrieve `.name`, `.children[].name`, |
| `.children[].children[].name`, and so on. You can do this |
| with: |
| |
| recurse(.children[]) | .name |
| |
| examples: |
| - program: 'recurse(.foo[])' |
| input: '{"foo":[{"foo": []}, {"foo":[{"foo":[]}]}]}' |
| output: |
| - '{"foo":[{"foo":[]},{"foo":[{"foo":[]}]}]}' |
| - '{"foo":[]}' |
| - '{"foo":[{"foo":[]}]}' |
| - '{"foo":[]}' |
| |
| - title: "`recurse_down`" |
| body: | |
| |
| A quieter version of `recurse(.[])`, equivalent to: |
| |
| def recurse_down: recurse(.[]?); |
| |
| - title: "`..`" |
| body: | |
| |
| Short-hand for `recurse_down`. This is intended to resemble |
| the XPath `//` operator. Note that `..a` does not work; use |
| `..|a` instead. |
| |
| examples: |
| - program: '..|.a?' |
| input: '[[{"a":1}]]' |
| output: ['1'] |
| |
| - title: "String interpolation: `\\(exp)`" |
| body: | |
| |
| Inside a string, you can put an expression inside parens |
| after a backslash. Whatever the expression returns will be |
| interpolated into the string. |
| |
| examples: |
| - program: '"The input was \(.), which is one less than \(.+1)"' |
| input: '42' |
| output: ['"The input was 42, which is one less than 43"'] |
| |
| - title: "Convert to/from JSON" |
| body: | |
| |
| The `tojson` and `fromjson` builtins dump values as JSON texts |
| or parse JSON texts into values, respectively. The tojson |
| builtin differs from tostring in that tostring returns strings |
| unmodified, while tojson encodes strings as JSON strings. |
| |
| examples: |
| - program: '[.[]|tostring]' |
| input: '[1, "foo", ["foo"]]' |
| output: ['["1","foo","[\"foo\"]"]'] |
| - program: '[.[]|tojson]' |
| input: '[1, "foo", ["foo"]]' |
| output: ['["1","\"foo\"","[\"foo\"]"]'] |
| - program: '[.[]|tojson|fromjson]' |
| input: '[1, "foo", ["foo"]]' |
| output: ['[1,"foo",["foo"]]'] |
| |
| - title: "Format strings and escaping" |
| body: | |
| |
| The `@foo` syntax is used to format and escape strings, |
| which is useful for building URLs, documents in a language |
| like HTML or XML, and so forth. `@foo` can be used as a |
| filter on its own, the possible escapings are: |
| |
| * `@text`: |
| |
| Calls `tostring`, see that function for details. |
| |
| * `@json`: |
| |
| Serialises the input as JSON. |
| |
| * `@html`: |
| |
| Applies HTML/XML escaping, by mapping the characters |
| `<>&'"` to their entity equivalents `<`, `>`, |
| `&`, `'`, `"`. |
| |
| * `@uri`: |
| |
| Applies percent-encoding, by mapping all reserved URI |
| characters to a `%xx` sequence. |
| |
| * `@csv`: |
| |
| The input must be an array, and it is rendered as CSV |
| with double quotes for strings, and quotes escaped by |
| repetition. |
| |
| * `@sh`: |
| |
| The input is escaped suitable for use in a command-line |
| for a POSIX shell. If the input is an array, the output |
| will be a series of space-separated strings. |
| |
| * `@base64`: |
| |
| The input is converted to base64 as specified by RFC 4648. |
| |
| This syntax can be combined with string interpolation in a |
| useful way. You can follow a `@foo` token with a string |
| literal. The contents of the string literal will *not* be |
| escaped. However, all interpolations made inside that string |
| literal will be escaped. For instance, |
| |
| @uri "https://www.google.com/search?q=\(.search)" |
| |
| will produce the following output for the input |
| `{"search":"what is jq?"}`: |
| |
| "https://www.google.com/search?q=what%20is%20jq%3f" |
| |
| Note that the slashes, question mark, etc. in the URL are |
| not escaped, as they were part of the string literal. |
| |
| examples: |
| - program: '@html' |
| input: '"This works if x < y"' |
| output: ['"This works if x < y"'] |
| |
| - program: '@sh "echo \(.)"' |
| input: "\"O'Hara's Ale\"" |
| output: ["\"echo 'O'\\\\''Hara'\\\\''s Ale'\""] |
| |
| - title: Conditionals and Comparisons |
| entries: |
| - title: "`==`, `!=`" |
| body: | |
| |
| The expression 'a == b' will produce 'true' if the result of a and b |
| are equal (that is, if they represent equivalent JSON documents) and |
| 'false' otherwise. In particular, strings are never considered equal |
| to numbers. If you're coming from JavaScript, jq's == is like |
| JavaScript's === - considering values equal only when they have the |
| same type as well as the same value. |
| |
| != is "not equal", and 'a != b' returns the opposite value of 'a == b' |
| |
| examples: |
| - program: '.[] == 1' |
| input: '[1, 1.0, "1", "banana"]' |
| output: ['true', 'true', 'false', 'false'] |
| |
| - title: if-then-else-end |
| body: | |
| |
| `if A then B else C end` will act the same as `B` if `A` |
| produces a value other than false or null, but act the same |
| as `C` otherwise. |
| |
| Checking for false or null is a simpler notion of |
| "truthiness" than is found in JavaScript or Python, but it |
| means that you'll sometimes have to be more explicit about |
| the condition you want: you can't test whether, e.g. a |
| string is empty using `if .name then A else B end`, you'll |
| need something more like `if (.name | length) > 0 then A else |
| B end` instead. |
| |
| If the condition `A` produces multiple results, then `B` is evaluated |
| once for each result that is not false or null, and `C` is evaluated |
| once for each false or null. |
| |
| More cases can be added to an if using `elif A then B` syntax. |
| |
| examples: |
| - program: |- |
| if . == 0 then |
| "zero" |
| elif . == 1 then |
| "one" |
| else |
| "many" |
| end |
| input: '2' |
| output: ['"many"'] |
| |
| - title: "`>`, `>=`, `<=`, `<`" |
| body: | |
| |
| The comparison operators `>`, `>=`, `<=`, `<` return whether |
| their left argument is greater than, greater than or equal |
| to, less than or equal to or less than their right argument |
| (respectively). |
| |
| The ordering is the same as that described for `sort`, above. |
| |
| examples: |
| - program: '. < 5' |
| input: '2' |
| output: ['true'] |
| |
| - title: "`and`, `or`, `not`" |
| body: | |
| |
| jq supports the normal Boolean operators `and`, `or`, `not`. |
| They have the same standard of truth as if expressions - |
| `false` and `null` are considered "false values", and |
| anything else is a "true value". |
| |
| If an operand of one of these operators produces multiple |
| results, the operator itself will produce a result for each input. |
| |
| `not` is in fact a builtin function rather than an operator, |
| so it is called as a filter to which things can be piped |
| rather than with special syntax, as in `.foo and .bar | |
| not`. |
| |
| These three only produce the values `true` and `false`, and |
| so are only useful for genuine Boolean operations, rather |
| than the common Perl/Python/Ruby idiom of |
| "value_that_may_be_null or default". If you want to use this |
| form of "or", picking between two values rather than |
| evaluating a condition, see the `//` operator below. |
| |
| examples: |
| - program: '42 and "a string"' |
| input: 'null' |
| output: ['true'] |
| - program: '(true, false) or false' |
| input: 'null' |
| output: ['true', 'false'] |
| - program: '(true, true) and (true, false)' |
| input: 'null' |
| output: ['true', 'false', 'true', 'false'] |
| - program: '[true, false | not]' |
| input: 'null' |
| output: ['[false, true]'] |
| |
| - title: "Alternative operator: `//`" |
| body: | |
| |
| A filter of the form `a // b` produces the same |
| results as `a`, if `a` produces results other than `false` |
| and `null`. Otherwise, `a // b` produces the same results as `b`. |
| |
| This is useful for providing defaults: `.foo // 1` will |
| evaluate to `1` if there's no `.foo` element in the |
| input. It's similar to how `or` is sometimes used in Python |
| (jq's `or` operator is reserved for strictly Boolean |
| operations). |
| |
| examples: |
| - program: '.foo // 42' |
| input: '{"foo": 19}' |
| output: ['19'] |
| - program: '.foo // 42' |
| input: '{}' |
| output: ['42'] |
| |
| - title: Advanced features |
| body: | |
| Variables are an absolute necessity in most programming languages, but |
| they're relegated to an "advanced feature" in jq. |
| |
| In most languages, variables are the only means of passing around |
| data. If you calculate a value, and you want to use it more than once, |
| you'll need to store it in a variable. To pass a value to another part |
| of the program, you'll need that part of the program to define a |
| variable (as a function parameter, object member, or whatever) in |
| which to place the data. |
| |
| It is also possible to define functions in jq, although this is |
| is a feature whose biggest use is defining jq's standard library |
| (many jq functions such as `map` and `select` are in fact written |
| in jq). |
| |
| Finally, jq has a `reduce` operation, which is very powerful but a |
| bit tricky. Again, it's mostly used internally, to define some |
| useful bits of jq's standard library. |
| |
| entries: |
| - title: Variables |
| body: | |
| |
| In jq, all filters have an input and an output, so manual |
| plumbing is not necessary to pass a value from one part of a program |
| to the next. Many expressions, for instance `a + b`, pass their input |
| to two distinct subexpressions (here `a` and `b` are both passed the |
| same input), so variables aren't usually necessary in order to use a |
| value twice. |
| |
| For instance, calculating the average value of an array of numbers |
| requires a few variables in most languages - at least one to hold the |
| array, perhaps one for each element or for a loop counter. In jq, it's |
| simply `add / length` - the `add` expression is given the array and |
| produces its sum, and the `length` expression is given the array and |
| produces its length. |
| |
| So, there's generally a cleaner way to solve most problems in jq than |
| defining variables. Still, sometimes they do make things easier, so jq |
| lets you define variables using `expression as $variable`. All |
| variable names start with `$`. Here's a slightly uglier version of the |
| array-averaging example: |
| |
| length as $array_length | add / $array_length |
| |
| We'll need a more complicated problem to find a situation where using |
| variables actually makes our lives easier. |
| |
| |
| Suppose we have an array of blog posts, with "author" and "title" |
| fields, and another object which is used to map author usernames to |
| real names. Our input looks like: |
| |
| {"posts": [{"title": "First post", "author": "anon"}, |
| {"title": "A well-written article", "author": "person1"}], |
| "realnames": {"anon": "Anonymous Coward", |
| "person1": "Person McPherson"}} |
| |
| We want to produce the posts with the author field containing a real |
| name, as in: |
| |
| {"title": "First post", "author": "Anonymous Coward"} |
| {"title": "A well-written article", "author": "Person McPherson"} |
| |
| We use a variable, $names, to store the realnames object, so that we |
| can refer to it later when looking up author usernames: |
| |
| .realnames as $names | .posts[] | {title, author: $names[.author]} |
| |
| The expression `exp as $x | ...` means: for each value of expression |
| `exp`, run the rest of the pipeline with the entire original input, and |
| with `$x` set to that value. Thus `as` functions as something of a |
| foreach loop. |
| |
| Variables are scoped over the rest of the expression that defines |
| them, so |
| |
| .realnames as $names | (.posts[] | {title, author: $names[.author]}) |
| |
| will work, but |
| |
| (.realnames as $names | .posts[]) | {title, author: $names[.author]} |
| |
| won't. |
| |
| examples: |
| - program: '.bar as $x | .foo | . + $x' |
| input: '{"foo":10, "bar":200}' |
| output: ['210'] |
| |
| - title: 'Defining Functions' |
| body: | |
| |
| You can give a filter a name using "def" syntax: |
| |
| def increment: . + 1; |
| |
| From then on, `increment` is usable as a filter just like a |
| builtin function (in fact, this is how some of the builtins |
| are defined). A function may take arguments: |
| |
| def map(f): [.[] | f]; |
| |
| Arguments are passed as filters, not as values. The |
| same argument may be referenced multiple times with |
| different inputs (here `f` is run for each element of the |
| input array). Arguments to a function work more like |
| callbacks than like value arguments. |
| |
| If you want the value-argument behaviour for defining simple |
| functions, you can just use a variable: |
| |
| def addvalue(f): f as $value | map(. + $value); |
| |
| With that definition, `addvalue(.foo)` will add the current |
| input's `.foo` field to each element of the array. |
| |
| examples: |
| - program: 'def addvalue(f): . + [f]; map(addvalue(.[0]))' |
| input: '[[1,2],[10,20]]' |
| output: ['[[1,2,1], [10,20,10]]'] |
| - program: 'def addvalue(f): f as $x | map(. + $x); addvalue(.[0])' |
| input: '[[1,2],[10,20]]' |
| output: ['[[1,2,1,2], [10,20,1,2]]'] |
| |
| - title: Reduce |
| body: | |
| |
| The `reduce` syntax in jq allows you to combine all of the |
| results of an expression by accumulating them into a single |
| answer. As an example, we'll pass `[3,2,1]` to this expression: |
| |
| reduce .[] as $item (0; . + $item) |
| |
| For each result that `.[]` produces, `. + $item` is run to |
| accumulate a running total, starting from 0. In this |
| example, `.[]` produces the results 3, 2, and 1, so the |
| effect is similar to running something like this: |
| |
| 0 | (3 as $item | . + $item) | |
| (2 as $item | . + $item) | |
| (1 as $item | . + $item) |
| |
| examples: |
| - program: 'reduce .[] as $item (0; . + $item)' |
| input: '[10,2,5,3]' |
| output: ['20'] |
| |
| |
| - title: Assignment |
| body: | |
| |
| Assignment works a little differently in jq than in most |
| programming languages. jq doesn't distinguish between references |
| to and copies of something - two objects or arrays are either |
| equal or not equal, without any further notion of being "the |
| same object" or "not the same object". |
| |
| If an object has two fields which are arrays, `.foo` and `.bar`, |
| and you append something to `.foo`, then `.bar` will not get |
| bigger. Even if you've just set `.bar = .foo`. If you're used to |
| programming in languages like Python, Java, Ruby, JavaScript, |
| etc. then you can think of it as though jq does a full deep copy |
| of every object before it does the assignment (for performance, |
| it doesn't actually do that, but that's the general idea). |
| |
| entries: |
| - title: "`=`" |
| body: | |
| |
| The filter `.foo = 1` will take as input an object |
| and produce as output an object with the "foo" field set to |
| 1. There is no notion of "modifying" or "changing" something |
| in jq - all jq values are immutable. For instance, |
| |
| .foo = .bar | .foo.baz = 1 |
| |
| will not have the side-effect of setting .bar.baz to be set |
| to 1, as the similar-looking program in JavaScript, Python, |
| Ruby or other languages would. Unlike these languages (but |
| like Haskell and some other functional languages), there is |
| no notion of two arrays or objects being "the same array" or |
| "the same object". They can be equal, or not equal, but if |
| we change one of them in no circumstances will the other |
| change behind our backs. |
| |
| This means that it's impossible to build circular values in |
| jq (such as an array whose first element is itself). This is |
| quite intentional, and ensures that anything a jq program |
| can produce can be represented in JSON. |
| |
| - title: "`|=`" |
| body: | |
| As well as the assignment operator '=', jq provides the "update" |
| operator '|=', which takes a filter on the right-hand side and |
| works out the new value for the property being assigned to by running |
| the old value through this expression. For instance, .foo |= .+1 will |
| build an object with the "foo" field set to the input's "foo" plus 1. |
| |
| This example should show the difference between '=' and '|=': |
| |
| Provide input '{"a": {"b": 10}, "b": 20}' to the programs: |
| |
| .a = .b |
| .a |= .b |
| |
| The former will set the "a" field of the input to the "b" field of the |
| input, and produce the output {"a": 20}. The latter will set the "a" |
| field of the input to the "a" field's "b" field, producing {"a": 10}. |
| |
| - title: "`+=`, `-=`, `*=`, `/=`, `%=`, `//=`" |
| body: | |
| |
| jq has a few operators of the form `a op= b`, which are all |
| equivalent to `a |= . op b`. So, `+= 1` can be used to increment values. |
| |
| examples: |
| - program: .foo += 1 |
| input: '{"foo": 42}' |
| output: ['{"foo": 43}'] |
| |
| - title: Complex assignments |
| body: | |
| Lots more things are allowed on the left-hand side of a jq assignment |
| than in most languages. We've already seen simple field accesses on |
| the left hand side, and it's no surprise that array accesses work just |
| as well: |
| |
| .posts[0].title = "JQ Manual" |
| |
| What may come as a surprise is that the expression on the left may |
| produce multiple results, referring to different points in the input |
| document: |
| |
| .posts[].comments |= . + ["this is great"] |
| |
| That example appends the string "this is great" to the "comments" |
| array of each post in the input (where the input is an object with a |
| field "posts" which is an array of posts). |
| |
| When jq encounters an assignment like 'a = b', it records the "path" |
| taken to select a part of the input document while executing a. This |
| path is then used to find which part of the input to change while |
| executing the assignment. Any filter may be used on the |
| left-hand side of an equals - whichever paths it selects from the |
| input will be where the assignment is performed. |
| |
| This is a very powerful operation. Suppose we wanted to add a comment |
| to blog posts, using the same "blog" input above. This time, we only |
| want to comment on the posts written by "stedolan". We can find those |
| posts using the "select" function described earlier: |
| |
| .posts[] | select(.author == "stedolan") |
| |
| The paths provided by this operation point to each of the posts that |
| "stedolan" wrote, and we can comment on each of them in the same way |
| that we did before: |
| |
| (.posts[] | select(.author == "stedolan") | .comments) |= |
| . + ["terrible."] |
