flywithu/Learning-Bitcoin-from-the-Command-Line
1
1
Fork 0
mirror of https://github.com/ChristopherA/Learning-Bitcoin-from-the-Command-Line.git synced 2026-04-13 00:22:25 +00:00
Code Issues Releases Wiki Activity

edits/updates

Browse Source
This commit is contained in:
Shannon Appelcline 2026-02-26 12:02:25 -10:00 committed by GitHub
parent cef132dbad
commit 34db2c5164
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 42 additions and 36 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

78
04_2a_Interlude_Using_JQ.md
View File

@ -1,17 +1,18 @@
# Interlude: Using JQ # Interlude: Using JQ
Creating a raw transaction revealed how more complex bitcoin-cli results can't easily be saved into command-line variables. The answer is JQ, which allows you to filter out individual elements from more complex JSON data. Creating a raw transaction revealed how more complex `bitcoin-cli` results can't easily be saved into command-line variables. The answer is JQ, which allows you to filter out individual elements from more complex JSON data.
## Install JQ ## Install JQ
For modern versions of Debian, you should be able to install JQ using `apt-get`: For modern versions of Debian, you should be able to install JQ using `apt-get`:
``` ```
# apt-get install jq $ sudo apt-get install jq
``` ```
> :book: ***What is JQ?*** The repository explains it best, saying "jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text."
If that works, you're done! If that works, you're done!
> :book: ***What is JQ?*** The repository explains it best, saying "jq is like sed for JSON data - you can use it to slice and filter and map and transform structured data with the same ease that sed, awk, grep and friends let you play with text."
Otherwise, you can download JQ from a [Github repository](https://stedolan.github.io/jq/). Just download a binary for Linux, OS X, or Windows, as appropriate. Otherwise, you can download JQ from a [Github repository](https://stedolan.github.io/jq/). Just download a binary for Linux, OS X, or Windows, as appropriate.
Once you've downloaded the binary, you can install it on your system. If you're working on a Debian VPS as we suggest, your installation will look like this: Once you've downloaded the binary, you can install it on your system. If you're working on a Debian VPS as we suggest, your installation will look like this:
@ -27,29 +28,29 @@ In the previous section, the use of `signrawtransaction` offered an example of n
``` ```
$ bitcoin-cli signrawtransactionwithwallet $rawtxhex $ bitcoin-cli signrawtransactionwithwallet $rawtxhex
{ {
"hex": "02000000013a6e4279b799791049e1826602e84d2e36797e2005887b98c3ecf16b01b7f361010000006a4730440220335d15a2a2ca3ce6a302ce041686739d4a38eb0599a5ea08305de71965268d05022015f77a33cf7d613015b2aba5beb03088033625505ad5d4d0624defdbea22262b01210278608b54b8fb0d8379d3823d31f03a7c6ab0adffb07dd3811819fdfc34f8c132ffffffff01409c0000000000001976a914e7c1345fc8f87c68170b3aa798a956c2fe6a9eff88ac00000000", "hex": "02000000000101fd296670b9d95353a5a9c1659ef9ad563e714f6fb9e9cc3f4c265ecd9afd7ff60100000000fdffffff01b882010000000000160014f1ad72d46aa89d5e622e42bbbb75064eebd9d1010247304402202cc47fe2a529a14765010845eaf393becd91e95df7ba5c142f33a0246382c107022065d7abafe6144e6af899d3311ba59a7c3c9fe8b01df6f022f28ca81fc1eac181012102b412b4329e450c854293c4c4f9d6e716aa9b6f7c7aa622500ddb15f28f215e0400000000",
"complete": true "complete": true
} }
``` ```
Fortunately, JQ can easily capture data of that sort! Fortunately, JQ can easily capture data of that sort!
To use JQ, run `jq` at the backend of a pipe, and always use the standard invocation of `jq -r '.'`. The `-r` tells JQ to produce raw output, which will work for command-line variables, while the `.` tells jq to output. We protect that argument in `' '` because we'll need that protection later as our `jq` invocations get more complex. To use JQ, run `jq` at the back end of a pipe, and always use the standard invocation of `jq -r '.'`. The `-r` tells JQ to produce raw output, which will work for command-line variables, while the `.` tells jq to output. We protect that argument in `' '` because we'll need that protection later as our `jq` invocations get more complex.
To capture a specific value from a JSON object, you just list the key after the `.`: To capture a specific value from a JSON object, you just list the key after the `.`:
``` ```
$ bitcoin-cli signrawtransactionwithwallet $rawtxhex | jq -r '.hex' $ bitcoin-cli signrawtransactionwithwallet $rawtxhex | jq -r '.hex'
02000000013a6e4279b799791049e1826602e84d2e36797e2005887b98c3ecf16b01b7f361010000006a4730440220335d15a2a2ca3ce6a302ce041686739d4a38eb0599a5ea08305de71965268d05022015f77a33cf7d613015b2aba5beb03088033625505ad5d4d0624defdbea22262b01210278608b54b8fb0d8379d3823d31f03a7c6ab0adffb07dd3811819fdfc34f8c132ffffffff01409c0000000000001976a914e7c1345fc8f87c68170b3aa798a956c2fe6a9eff88ac00000000 0200000001fd296670b9d95353a5a9c1659ef9ad563e714f6fb9e9cc3f4c265ecd9afd7ff60100000000fdffffff01b882010000000000160014f1ad72d46aa89d5e622e42bbbb75064eebd9d10100000000
``` ```
With that tool in hand, you can capture information from JSON objects to command-line variables: With that tool in hand, you can capture information from JSON objects to command-line variables:
``` ```
$ signedtx=$(bitcoin-cli signrawtransactionwithwallet $rawtxhex | jq -r '.hex') $ signedtx=$(bitcoin-cli signrawtransactionwithwallet $rawtxhex | jq -r '.hex')
$ echo $signedtx $ echo $signedtx
02000000013a6e4279b799791049e1826602e84d2e36797e2005887b98c3ecf16b01b7f361010000006a4730440220335d15a2a2ca3ce6a302ce041686739d4a38eb0599a5ea08305de71965268d05022015f77a33cf7d613015b2aba5beb03088033625505ad5d4d0624defdbea22262b01210278608b54b8fb0d8379d3823d31f03a7c6ab0adffb07dd3811819fdfc34f8c132ffffffff01409c0000000000001976a914e7c1345fc8f87c68170b3aa798a956c2fe6a9eff88ac00000000 0200000001fd296670b9d95353a5a9c1659ef9ad563e714f6fb9e9cc3f4c265ecd9afd7ff60100000000fdffffff01b882010000000000160014f1ad72d46aa89d5e622e42bbbb75064eebd9d10100000000
``` ```
You can then use those variables easily and without error: You can then use those variables easily and without error:
``` ```
$ bitcoin-cli sendrawtransaction $signedtx $ bitcoin-cli sendrawtransaction $signedtx
3f9ccb6e16663e66dc119de1866610cc4f7a83079bfec2abf0598ed3adf10a78 8a0d9ab73d81a1ce043d1ede0e737136e6d3352d3e0bc9590f9e8bbd91036dc2
``` ```
## Use JQ to Access Single JSON Object Values in an Array by Key ## Use JQ to Access Single JSON Object Values in an Array by Key
@ -61,23 +62,26 @@ When working with a JSON array, the first thing you need to do is tell JQ which
``` ```
$ bitcoin-cli listunspent | jq -r '.[1]' $ bitcoin-cli listunspent | jq -r '.[1]'
{ {
"txid": "91261eafae15ea53dedbea7c1db748c52bbc04a85859ffd0d839bda1421fda4c", "txid": "8a0d9ab73d81a1ce043d1ede0e737136e6d3352d3e0bc9590f9e8bbd91036dc2",
"vout": 0, "vout": 0,
"address": "mjehC2KHzXcBDcwTd4LhZ2GzyzrZ3Kd3ff", "address": "tb1q7xkh94r24zw4uc3wg2amkagxfm4an5gpwrgsnl",
"label": "", "label": "",
"scriptPubKey": "76a9142d573900aa357a38afd741fbf24b075d263ea6e088ac", "scriptPubKey": "0014f1ad72d46aa89d5e622e42bbbb75064eebd9d101",
"amount": 0.00022, "amount": 0.00099000,
"confirmations": 9, "confirmations": 2,
"spendable": true, "spendable": true,
"solvable": true, "solvable": true,
"desc": "pkh([d6043800/0'/0'/3']0278608b54b8fb0d8379d3823d31f03a7c6ab0adffb07dd3811819fdfc34f8c132)#nhjc3f8y", "desc": "wpkh([b8309bae/84h/1h/0h/0/5]027650c44c6f71d50a173f3f18858490cf26d2661a1d9929deba9aca57c76d1e5b)#7ss7jfnn",
"parent_descs": [
"wpkh([b8309bae/84h/1h/0h]tpubDDpSvPDUjstxFUEWzHkaL4qykf8vjNCspm8SZ26Z1wgPFbd63AdYrn4bDpEGPT1giJ6gcLW8Xou8fnhi35DJrUza9ikgu5dg2mDkd8jQpA6/0/*)#gduft8tw"
],
"safe": true "safe": true
} }
``` ```
You can then capture an individual value from that selected array by (1) using a pipe _within_ the JQ arguments; and then (2) requesting the specific value afterward, as in the previous example. The following would capture the `txid` from the 1st JSON object in the JSON array produced by `listunspent`: You can then capture an individual value from that selected array by (1) using a pipe _within_ the JQ arguments; and then (2) requesting the specific value afterward, as in the previous example. The following would capture the `txid` from the 1st JSON object in the JSON array produced by `listunspent`:
``` ```
$ bitcoin-cli listunspent | jq -r '.[1] | .txid' $ bitcoin-cli listunspent | jq -r '.[1] | .txid'
91261eafae15ea53dedbea7c1db748c52bbc04a85859ffd0d839bda1421fda4c 8a0d9ab73d81a1ce043d1ede0e737136e6d3352d3e0bc9590f9e8bbd91036dc2
``` ```
Carefully note how the `' 's` go around the whole JQ expression _including_ the pipe. Carefully note how the `' 's` go around the whole JQ expression _including_ the pipe.
@ -86,7 +90,7 @@ This method can be used to fill in variables for a UTXO that you want to use:
$ newtxid=$(bitcoin-cli listunspent | jq -r '.[1] | .txid') $ newtxid=$(bitcoin-cli listunspent | jq -r '.[1] | .txid')
$ newvout=$(bitcoin-cli listunspent | jq -r '.[1] | .vout') $ newvout=$(bitcoin-cli listunspent | jq -r '.[1] | .vout')
$ echo $newtxid $ echo $newtxid
91261eafae15ea53dedbea7c1db748c52bbc04a85859ffd0d839bda1421fda4c 8a0d9ab73d81a1ce043d1ede0e737136e6d3352d3e0bc9590f9e8bbd91036dc2
$ echo $newvout $ echo $newvout
0 0
``` ```
@ -99,8 +103,8 @@ Voila! We could now create a new raw transaction using our 1st UTXO as an input,
Instead of accessing a single, specific value in a specific JSON object, you could instead access all of a specific value across all the JSON objects. This is done with `.[]`, where no index is specified. For example, this would list all unspent funds: Instead of accessing a single, specific value in a specific JSON object, you could instead access all of a specific value across all the JSON objects. This is done with `.[]`, where no index is specified. For example, this would list all unspent funds:
``` ```
$ bitcoin-cli listunspent | jq -r '.[] | .amount' $ bitcoin-cli listunspent | jq -r '.[] | .amount'
0.0001 0.01895537
0.00022 0.00099000
``` ```
## Use JQ for Simple Calculations by Key ## Use JQ for Simple Calculations by Key
@ -110,10 +114,11 @@ $ bitcoin-cli listunspent | jq -r '.[] | .amount'
At this point, you can start using JQ output for simple math. For example, adding up the values of those unspent transactions with a simple `awk` script would give you the equivalent of `getbalance`: At this point, you can start using JQ output for simple math. For example, adding up the values of those unspent transactions with a simple `awk` script would give you the equivalent of `getbalance`:
``` ```
$ bitcoin-cli listunspent | jq -r '.[] | .amount' | awk '{s+=$1} END {print s}' $ bitcoin-cli listunspent | jq -r '.[] | .amount' | awk '{s+=$1} END {print s}'
0.00032 0.0199454
$ bitcoin-cli getbalance $ bitcoin-cli getbalance
0.00032000 0.01994537
``` ```
(Note the slight difference due to different numbers of significant digits! A `printf` in the `awk` could have improved that, but by default it only goes out to six significant digits; when `bitcoin-cli` offers a value natively, it'll be more accurate with less work!)
## Use JQ to Display Multiple JSON Object Values in an Array by Multiple Keys ## Use JQ to Display Multiple JSON Object Values in an Array by Multiple Keys
@ -124,44 +129,45 @@ JQ can easily capture individual elements from JSON objects and arrays and place
For example, you might want to see a listing of all your UTXOs (`.[]`) and get a listing of all of their most important information (`.txid, .vout, .amount`): For example, you might want to see a listing of all your UTXOs (`.[]`) and get a listing of all of their most important information (`.txid, .vout, .amount`):
``` ```
$ bitcoin-cli listunspent | jq -r '.[] | .txid, .vout, .amount' $ bitcoin-cli listunspent | jq -r '.[] | .txid, .vout, .amount'
ca4898d8f950df03d6bfaa00578bd0305d041d24788b630d0c4a32debcac9f36 f67ffd9acd5e264c3fcce9b96f4f713e56adf99e65c1a9a55353d9b9706629fd
0 0
0.0001 0.01895537
91261eafae15ea53dedbea7c1db748c52bbc04a85859ffd0d839bda1421fda4c 8a0d9ab73d81a1ce043d1ede0e737136e6d3352d3e0bc9590f9e8bbd91036dc2
0 0
0.00022 0.00099000
``` ```
This makes it easy to decide which UTXOs to spend in a raw transaction, but it's not very pretty. This makes it easy to decide which UTXOs to spend in a raw transaction, but it's not very pretty.
Fortunately, JQ also lets you be fancy. You can use `{}`s to create new JSON objects (either for additional parsing or for pretty output). You also get to define the name of the new key for each of your values. The resulting output should be much more intuitive and less prone to error (though obviously, less useful for dumping info straight into variables). Fortunately, JQ also lets you be fancy. You can use `{}`s to create new JSON objects (either for additional parsing or for pretty output). You also get to define the name of the new key for each of your values. The resulting output should be much more intuitive and less prone to error (though obviously, less useful for dumping info straight into variables).
The following example shows the exact same parsing of `listunspent`, but with the each old JSON object rebuilt as a new, abridged JSON object, with all of the new values named with their old keys: The following example shows the exact same parsing of `listunspent`, but with each old JSON object rebuilt as a new, abridged JSON object, with all of the new values named with their old keys:
``` ```
$ bitcoin-cli listunspent | jq -r '.[] | { txid: .txid, vout: .vout, amount: .amount }' $ bitcoin-cli listunspent | jq -r '.[] | { txid: .txid, vout: .vout, amount: .amount }'
{ {
"txid": "ca4898d8f950df03d6bfaa00578bd0305d041d24788b630d0c4a32debcac9f36", "txid": "f67ffd9acd5e264c3fcce9b96f4f713e56adf99e65c1a9a55353d9b9706629fd",
"vout": 0, "vout": 0,
"amount": 0.0001 "amount": 0.01895537
} }
{ {
"txid": "91261eafae15ea53dedbea7c1db748c52bbc04a85859ffd0d839bda1421fda4c", "txid": "8a0d9ab73d81a1ce043d1ede0e737136e6d3352d3e0bc9590f9e8bbd91036dc2",
"vout": 0, "vout": 0,
"amount": 0.00022 "amount": 0.00099000
} }
``` ```
You could of course rename your new keys as you see fit. There's nothing magic in the original names: You could of course rename your new keys as you see fit. There's nothing magic in the original names:
``` ```
$ bitcoin-cli listunspent | jq -r '.[] | { tx: .txid, output: .vout, bitcoins: .amount }' $ bitcoin-cli listunspent | jq -r '.[] | { tx: .txid, output: .vout, bitcoins: .amount }'
{ {
"tx": "ca4898d8f950df03d6bfaa00578bd0305d041d24788b630d0c4a32debcac9f36", "tx": "f67ffd9acd5e264c3fcce9b96f4f713e56adf99e65c1a9a55353d9b9706629fd",
"output": 0, "output": 0,
"bitcoins": 0.0001 "bitcoins": 0.01895537
} }
{ {
"tx": "91261eafae15ea53dedbea7c1db748c52bbc04a85859ffd0d839bda1421fda4c", "tx": "8a0d9ab73d81a1ce043d1ede0e737136e6d3352d3e0bc9590f9e8bbd91036dc2",
"output": 0, "output": 0,
"bitcoins": 0.00022 "bitcoins": 0.00099000
} }
``` ```
## Use JQ to Access JSON Objects by Looked-Up Value ## Use JQ to Access JSON Objects by Looked-Up Value
@ -266,7 +272,7 @@ $ echo ${usedvout[1]}
``` ```
The only real trick here is how we saved the information using the bash shell. Rather than saving to a variable with `$(command)`, we instead saved to an array with `($(command))`. We were then able to access the individual bash array elements with a `${variable[n]}` construction. We could instead access the whole array with `${variable[@]}`. (Yeah, no one ever said bash was pretty.) The only real trick here is how we saved the information using the bash shell. Rather than saving to a variable with `$(command)`, we instead saved to an array with `($(command))`. We were then able to access the individual bash array elements with a `${variable[n]}` construction. We could instead access the whole array with `${variable[@]}`. (Yeah, no one ever said bash was pretty.)
> :warning: **WARNING:** Always remember that a UTXO is a transaction _plus_ a vout. We missed the vout the first time we wrote this JQ example, and it stopped working when we ended up with a situation where we'd been sent two `vouts` from the same transaction. > :warning: **WARNING: txid+vout=UTXO.** Always remember that a UTXO is a transaction _plus_ a vout. We missed the vout the first time we wrote this JQ example, and it stopped working when we ended up with a situation where we'd been sent two `vouts` from the same transaction.
### Retrieve the Related Object(s) ### Retrieve the Related Object(s)
@ -361,7 +367,7 @@ $ bitcoin-cli decoderawtransaction $rawtxhex | jq -r '.vout [] | .value' | awk
``` ```
To complete the transaction fee calculation, you subtract the .vout .amount (1.045) from the .vin .amount (1.3). To complete the transaction fee calculation, you subtract the .vout .amount (1.045) from the .vin .amount (1.3).
To do this, you'll need to install `bc`: To do this, you'll need to install `bc` if you haven't already:
``` ```
$ sudo apt-get install bc $ sudo apt-get install bc
``` ```
@ -377,7 +383,7 @@ $ echo $(printf '%.8f-%.8f' $btcin $btcout_f) | /usr/bin/bc
``` ```
And that's also a good example of why you double-check your fees: we'd intended to send a transaction fee of 5,000 satoshis, but sent 255,000 satoshis instead. Whoops! And that's also a good example of why you double-check your fees: we'd intended to send a transaction fee of 5,000 satoshis, but sent 255,000 satoshis instead. Whoops!
> :warning: **WARNING:** The first time we wrote up this lesson, we genuinely miscalculated our fee and didn't see it until we ran our fee calculator. It's *that* easy, then your money is gone. (The example above is actually from our second iteration of the calculator, and that time we made the mistake on purpose.) > :warning: **WARNING: Money Goes Poof.** The first time we wrote up this lesson, we genuinely miscalculated our fee and didn't see it until we ran our fee calculator. It's *that* easy, then your money is gone. (The example above is actually from our second iteration of the calculator, and that time we made the mistake on purpose.)
For more JSON magic (and if any of this isn't clear), please read the [JSON Manual](https://stedolan.github.io/jq/manual/) and the [JSON Cookbook](https://github.com/stedolan/jq/wiki/Cookbook). We'll be regularly using JQ in future examples. For more JSON magic (and if any of this isn't clear), please read the [JSON Manual](https://stedolan.github.io/jq/manual/) and the [JSON Cookbook](https://github.com/stedolan/jq/wiki/Cookbook). We'll be regularly using JQ in future examples.