How to parse JSON after cURL

If you use cURL for requests to an API with JSON response in the body, you know that feeling of frustration at the sight of solid text in the console.

Fig. 1. Terminal

I am using cURL as a tool for testing REST API — making HTTP/HTTPS requests. In most cases, the response body contains JSON. I assume that the reader knows what cURL (wiki) is for and is familiar with JSON (wiki) format.

All the methods described below will work if the cURL output contains only valid JSON ⇒ do not use -i option (it shows the HTTP response headers), which will add additional data to the output except the response’s body like this:

Fig. 2. Response headers are included in the cURL output

Otherwise, you will not be able to parse the response by any tool:

Fig. 3. Invalid JSON is not parseable

For all examples, I used macOS Big Sur and its default Terminal; all additional tools can be installed through homebrew.

List of choices:

• Python
• jq
• fx
• jless

Python

If you do not want to install any additional tool — Python is your choice, of course, if your OS comes with it. It has a built-in JSON encoder and decoder, which can be called from the command line through the pipe:

curl 'https://api.openweathermap.org/data/2.5/onecall?lat=40.1811&lon=44.5136&appid={api_key}' | python -m json.tool

The response will be in human-readable form:

Fig. 4. | python -m json.tool

Additional reading:

• How can I pretty-print JSON in a shell script
• Parsing JSON files using Python

jq

Jq is a flexible command-line JSON processor and the most popular solution:

curl 'https://api.openweathermap.org/data/2.5/onecall?lat=40.1811&lon=44.5136&appid={api_key}' | jq

The response will look nice and with syntax highlighting:

Fig. 5. | jq

The advantage of the tool is an extended filtration system. For example, you can immediately get the value of the required key:

curl 'https://api.openweathermap.org/data/2.5/onecall?lat=40.1811&lon=44.5136&appid={api_key}' | jq .timezone

Fig. 6. | jq .json_key

Additional reading:

• Parse JSON data using jq and curl from command line
• Working with JSON using jq
• Parsing JSON with jq
• jq Manual

fx

Fx is a command-line JSON viewer and manipulation tool — after getting the JSON, you can navigate through it:

Fig. 7. fx navigation through JSON

The highlighted response can be obtained without interactive mode by . option:

curl 'https://api.openweathermap.org/data/2.5/onecall?lat=40.1811&lon=44.5136&appid={api_key}' | fx .

Fig. 8. | fx .

Despite that fx is less popular than jq, it also supports filters:

curl 'https://api.openweathermap.org/data/2.5/onecall?lat=40.1811&lon=44.5136&appid={api_key}' | fx .current.humidity

Fig. 9. | fx .json_key

Additional reading:

• Discover how to use fx effectively, a JSON manipulation command line tool
• Getting started with FX: Powerful and handy JSON manipulation from the command line

jless

Jless is the newest command-line JSON viewer with a bunch of vim-inspired commands (like :q to exit) for navigation.

After request, you will immediately get into the beautiful interactive mode, where you can move and search through JSON:

curl 'https://api.openweathermap.org/data/2.5/onecall?lat=40.1811&lon=44.5136&appid={api_key}' | jless

Fig. 10. | jless

Jless is also a worthy tool to work with local JSON files.

Additional reading:

• Jless, a command-line JSON viewer

Earlier this year the maintainer of cURL announced the beginning of implementation of JSON support. I hope in the near future we could work with JSON output out of the cURL’s box.

Additional reading:

• How to start using Curl and why
• An Introduction to JSON
• Parsing JSON with Unix tools