19 KiB
12.1: Accessing Bitcoind with Curl
NOTE: This is a draft in progress, so that I can get some feedback from early reviewers. It is not yet ready for learning.
bitcoin-cli, the heart of chapters 3-6, is ultimately just a wrapper. It's a way to interface with bitcoind from the command line, providing simplified access to its many RPC commands. But RPC can, of course, be accessed directly. That's what this chapter is about: directly connecting to RPC with the curl command.
Know Your Curl
curl, short for "see URL", is a command-line tool that allows you to directly access URLs in a programmatic way. It's an easy way to interact with servers that listen to ports on the internet and that speak a variety of protocols, among them bitcoind. It also comes as a library for many programming languages, such as C, Java, PHP, and Python. So, once you know how to work with Curl, you'll have a strong foundation for using a lot of different APIs — some of which we'll touch on in the next few chapters.
In order to use curl with bitcoind, you must know three things: the standard format, the user name and password, and the correct port.
Know Your Format
The bitcoin-cli commands are all linked to the RPC commands in bitcoind. That makes the transition from using bitcoin-cli to using curl very simply. In fact, if you look at any of the help pages for bitcoin-cli, you'll see that they list not only the bitcoin-cli command, but also a parallel curl command. For example, here is bitcoin-cli getinfo help:
$ bitcoin-cli help getmininginfo
getmininginfo
Returns a json object containing mining-related information.
Result:
{
"blocks": nnn, (numeric) The current block
"currentblocksize": nnn, (numeric) The last block size
"currentblockweight": nnn, (numeric) The last block weight
"currentblocktx": nnn, (numeric) The last block transaction
"difficulty": xxx.xxxxx (numeric) The current difficulty
"errors": "..." (string) Current errors
"networkhashps": nnn, (numeric) The network hashes per second
"pooledtx": n (numeric) The size of the mempool
"chain": "xxxx", (string) current network name as defined in BIP70 (main, test, regtest)
}
Examples:
> bitcoin-cli getmininginfo
> curl --user myusername --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getmininginfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:8332/
And there's the curl command, at the end of the help screen! This somewhat lengthy command has four major parts: (1) a listing of your user name; (2) a --data-binary flag; (3) a JSON object that tells bitcoind what to do; and (4) an HTTP header that includes the URL to talk to. When you are working with curl, most of this will stay the same from command to command, other than the method and params entries in the JSON array. However, you need to know how to fill in your username and your URL address in order to make it work in the first place!
Whenever you're unusure about how to directly access an RPC command via curl, just look at the bitcoin-cli help and go from there.
Know Your User Name
In order to speak with the bitcoind port, you need a user name and password. These were created as part of your initial Bitcoin setup, and can be found in ~/.bitcoin/bitcoin.conf.
For example, here's our current setup:
$ more ~/.bitcoin/bitcoin.conf
server=1
dbcache=1536
par=1
blocksonly=1
maxuploadtarget=137
maxconnections=16
rpcuser=bitcoinrpc
rpcpassword=d8340efbcd34e312044c8431c59c792c
txindex=1
testnet=1
Clearly, our user is bitcoinrpc and our password is d8340efbcd34e312044c8431c59c792c.
The secure way to access your user name is as follows:
$ curl --user bitcoinrpc --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getmininginfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:8332/
Enter host password for user 'bitcoinrpc':
As noted, you will be prompted for your password.
The insecure way to do so is as follows:
$ curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getmininginfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:8332/
WARNING: Entering your password on the command line may put your password into the process table and/or save it into a history. It's not recommended, except for testing on testnet. If you want to do it anywhere else, make sure you know what you're doing!
Know Your Port
The port should be easy, it's the http://127.0.0.1:8332/ shown in the tutorials, right? Not exactly. It depends on whether you're using testnet or mainnet.
MAINNET VS. TESTNET. To access RPC commands on the mainnet, use port 8332, but to do so on testnet, use port 18332.
Know Your Command & Parameters
By now it may be obvious that there are only two elements of the standard curl command which tend to change.
The first is method, which is the RPC method being used. This should generally match with the command names you've been feeding into bitcoin-cli for ages.
The second is params, which is a JSON array of parameters. These are the same as the arguments (or named arguments) that you've been using. They're also the most confusing part of curl, as we'll further discuss when we send a transaction, in large part because they're a structured array rather than a simple list.
Here's what some parameter arrays will look like:
[]— An empty array["000b4430a7a2ba60891b01b718747eaf9665cb93fbc0c619c99419b5b5cf3ad2"]— An array with data["'$signedhex'"]— An array with a variable[6, 9999999]— An array with two parameters[''[ { "txid": "'$utxo_txid'", "vout": '$utxo_vout' } ]'', ''{ "'$recipient'": 0.298, "'$changeaddress'": 1.0}'']— An array with an array and an ojbect
With that in hand, you should now be able to curl out to the bitcoind:
$ curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getmininginfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/
{"result":{"blocks":1128599,"currentblocksize":0,"currentblockweight":0,"currentblocktx":0,"difficulty":1,"errors":"Warning: unknown new rules activated (versionbit 28)","networkhashps":8658807967387.751,"pooledtx":0,"chain":"test"},"error":null,"id":"curltest"}
We provided the method, getmininginfo, and the null set [] for parameters, and everything else was the standard JSON input.
You'll note that the result is another JSON array, which is kind of ugly to read if you're using curl by hand. That's why bitcoin-cli and curl libraries exist. Fortunately, you can clean it up very simply by piping it through jq:
$ curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getmininginfo", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.'
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 354 100 277 100 77 65546 18220 --:--:-- --:--:-- --:--:-- 92333
{
"result": {
"blocks": 1128609,
"currentblocksize": 0,
"currentblockweight": 0,
"currentblocktx": 0,
"difficulty": 2777757.750498331,
"errors": "Warning: unknown new rules activated (versionbit 28)",
"networkhashps": 8795436830406.6,
"pooledtx": 0,
"chain": "test"
},
"error": null,
"id": "curltest"
}
You'll see a bit of connectivity reporting as the data is downloaded, then when that data hits jq, everything will be output in a correctly indented form. (We'll be omiting the download information in future example.)
WARNING: If you get a result like "Failed to connect to 127.0.0.1 port 8332: Connection refused", be sure that a line like
rpcallowip=127.0.0.1is in your ~/.bitcoin/bitcoin.conf. If things still don't work, be sure that you're allowing access to port 18332 (or 8332) from localhost. Our standard setup from Chapter Two: Creating a Bitcoin-Core VPS should do all of this.
Manipulate Your Wallet
Though you're accessing bitcoind directly, you'll still get access to wallet functionality, because that's largely stored in bitcoind itself.
Look Up Addresses
The getaddressesbyaccount RPC lists all of your current addresses:
$ curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getaddressesbyaccount", "params": [""] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.'
{
"result": [
"mg7YqyvK8HUFvpgZ5iYTfZ5vjfaJWnNTd9",
"mkMkhbUzcSPdEHUoRQkBKHe8otP1SzWWeb",
"moXjRArk4rWUYZvAiDHnMXQHmwyf3TtAb8",
"mp3qqSTEYqhm3iuo9nM5GnzCGuoEvb9KrK",
"msW5tzCMJRECSATENN11ATmxx3PLwjGEaW",
"msZ5bMbZC4HQPRuJCoivDWYExAB7ssE5o8",
"mx3wUEVp526gkqzMteK6NwsPffgtFB3CGX",
"mxwLPpo2X1VXXAiLHNVb55H4JASeLMDm7A",
"n2AfUVXq9MzsJohCjCKNXKj2SYHfdbrcqy",
"n2fGxDXKozkZjpi9DqW93NZ65TnQDqd49V",
"n4cqjJE6fqcmeWpftygwPoKMMDva6BpyHf",
"2Mzw7WBvh9RAQ4ssKqxyNyP7L9NAojLqSW8",
"2NAGfA4nW6nrZkD5je8tSiAcYB9xL2xYMCz",
"2NDx9HTeLagiEqWrYvBuPkG3gwdhQ32Jz4B"
],
"error": null,
"id": "curltest"
}
This is our first example of a real parameter, "". This is the account parameter, but all of our addresses are in the default account, so nothing special was required here.
The result is a list of all the addresses that have been used by this wallet ... some of which presumably contain funds.
Look Up Funds
The listunspent RPC lists out the funds that you have available:
$ curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "listunspent", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.'
{
"result": [
{
"txid": "6a184a2f07fa30189f4831d6f041d52653a103b3883d2bec2f79187331fd7f0e",
"vout": 0,
"address": "mnsdQUxpGa6UsroUJhJGupPrPcuvQbPzmV",
"scriptPubKey": "76a91450b1d90a130c4f3f1e5fbfa7320fd36b7265db0488ac",
"amount": 1.2985,
"confirmations": 1260,
"spendable": true,
"solvable": true
},
{
"txid": "966099e0fd330007eac0394d97a36985980ff971d64b865c5d474f1931eeec3a",
"vout": 0,
"address": "2NAGfA4nW6nrZkD5je8tSiAcYB9xL2xYMCz",
"account": "",
"scriptPubKey": "a914babf9063cee8ab6e9334f95f6d4e9148d0e551c287",
"amount": 1.2985,
"confirmations": 3681,
"spendable": false,
"solvable": false
},
{
"txid": "e31f747cb97a364a1a00b56a3d9d43a97c6e02938a47e599ad54d7cf710f056f",
"vout": 0,
"address": "2NDx9HTeLagiEqWrYvBuPkG3gwdhQ32Jz4B",
"account": "",
"redeemScript": "52210307fd375ed7cced0f50723e3e1a97bbe7ccff7318c815df4e99a59bc94dbcd81921029bf628adf0698a089137294975c8589cf0cc6be050f92d944faaa8c8a623430352ae",
"scriptPubKey": "a914e31fa4be1f718e068ba5cbb2f12a59bba66a2da687",
"amount": 1.3,
"confirmations": 6515,
"spendable": false,
"solvable": true
},
{
"txid": "88e5d5f3077517d76f5a61491fa52e6aaae078c52bc62d849f09507ef0cfada2",
"vout": 0,
"address": "n4cqjJE6fqcmeWpftygwPoKMMDva6BpyHf",
"account": "",
"scriptPubKey": "76a914fd67e8a7c7813e7a5c376eb71074f373d924d96888ac",
"amount": 0.47,
"confirmations": 38043,
"spendable": true,
"solvable": true
},
{
"txid": "7e8abac5040a14f74fc8c44300c89cf93ce7f59f1e1c1d1711fdddd7012941c1",
"vout": 0,
"address": "moXjRArk4rWUYZvAiDHnMXQHmwyf3TtAb8",
"account": "",
"scriptPubKey": "76a91457e6ac3cfbc9c34c292ca59f105a2d7a819db7a488ac",
"amount": 0.65,
"confirmations": 6586,
"spendable": true,
"solvable": true
}
],
"error": null,
"id": "curltest"
}
This is almost exactly the same output that you get when you type bitcoin-cli listunspent, showing how closely tied the two interfaces are. If no cleanup or extra help is needed, then bitcoin-cli just outputs the RPC. Easy!
Create an Address
So you know where your funds are, as you're aware the next step in creating a transaction is to create a change address. By now you've probably got the hang of this, and you know that for simple RPC commands, all you need to do is adjust the method is the curl command:
$ curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getrawchangeaddress", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.'
{
"result": "mznccEt2ozGFN6oVaYU5BGgTzcdH8Zj8wf",
"error": null,
"id": "curltest"
}
At this point, we can even revert to our standard practice of saving results to variables:
$ changeaddress=$(curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getrawchangeaddress", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.result')
$ echo $changeaddress
mmDWRH3CbeXwCqBwdHCj7E9d3oWTuuizxc
No need to worry about the downloading info. It'll go to STDERR and be displayed on your screen, while the results go to STDOUT and are saved in your variable.
Create a Transaction
You're now ready to create a transaction with curl.
Ready Your Variables
Just as with bitcoin-cli, in order to create a transaction using the RPC directly, you should first save your variables. The only change here is that you always need to pipe through the .result tag before you do anything else. This example sets up our variables for using the 1.2985 BTC in funds listed in the first unspent transaction above:
$ utxo_txid=$(curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "listunspent", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.result | .[0] | .txid')
$ utxo_vout=$(curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "listunspent", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.result | .[0] | .vout')
recipient=mwCwTceJvYV27KXBc3NJZys6CjsgsoeHmf
$ changeaddress=$(curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "getrawchangeaddress", "params": [] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.result')
$ echo $utxo_txid
6a184a2f07fa30189f4831d6f041d52653a103b3883d2bec2f79187331fd7f0e
$ echo $utxo_vout
0
$ echo $recipient
mwCwTceJvYV27KXBc3NJZys6CjsgsoeHmf
$ echo $changeaddress
mmDWRH3CbeXwCqBwdHCj7E9d3oWTuuizxc
Create the Transaction
The transaction created with curl is very similar to the transaction created with bitcoin-cli, but a few subtle differences:
$ curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "createrawtransaction", "params": [''[ { "txid": "'$utxo_txid'", "vout": '$utxo_vout' } ]'', ''{ "'$recipient'": 0.298, "'$changeaddress'": 1.0}'']}' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.'
{
"result": "02000000010e7ffd317318792fec2b3d88b303a15326d541f0d631489f1830fa072f4a186a0000000000ffffffff0240b6c601000000001976a914ac19d3fd17710e6b9a331022fe92c693fdf6659588ac00e1f505000000001976a9143e84156731d67c80c3dff6c1cc3b4f58460e642388ac00000000",
"error": null,
"id": "curltest"
}
The heart of the transaction is, of course, the params JSON array, which we're putting to full use for the first time.
Note that the entire params is lodged in []s to mark an outermost array that you don't usually see.
We've also varied up the quoting, to lead and end each array and object within the params array with '' instead of our traditional '''. That's because the entire set of JSON arguments already has a ' around it. As usual, just take a look at the bizarre shell quoting an get used to it.
However, there's one last thing of note in this example, and it can be maddening if you miss it. When we executed this command with bitcoin-cli the JSON array of inputs and the JSON object of outputs were separated by a space. Now, because they're part of that outermost JSON array, they're separated by a comma (,). Miss that and you'll get a continuous string of parse errors without much additional information.
WARNING: Ever having troubles debugging your
curl? Add the argument--trace-ascii /tmp/foo. Full information on what's being sent to the server will be saved in/tmp/foo(or whatever file name you provide).
Having verified that things work, you probably want to save this into a variable:
$ hexcode=$(curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "createrawtransaction", "params": [''[ { "txid": "'$utxo_txid'", "vout": '$utxo_vout' } ]'', ''{ "'$recipient'": 0.298, "'$changeaddress'": 1.0}'']}' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.result')
Sign and Send
Signing and sending is an easy use of the signrawtransaction and sendrawtransaction functions:
$ signedhex=$(curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "signrawtransaction", "params": ["'$hexcode'"] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.result | .hex')
$ curl --user bitcoinrpc:d8340efbcd34e312044c8431c59c792c --data-binary '{"jsonrpc": "1.0", "id":"curltest", "method": "sendrawtransaction", "params": ["'$signedhex'"] }' -H 'content-type: text/plain;' http://127.0.0.1:18332/ | jq -r '.'
{
"result": "000b4430a7a2ba60891b01b718747eaf9665cb93fbc0c619c99419b5b5cf3ad2",
"error": null,
"id": "curltest"
}
Summary: Accessing Bitcoind with Curl
Having finished this section, you may feel that accessing bitcoind via curl is very much like accessing it through bitcoin-cli ... but more cumbersome. And, you'd be right. bitcoin-cli has pretty complete RPC functionality, so anything that you do through curl you can probably do through bitcoin-cli.
What is the power of curl? Most obviously, curl takes out one level of indirection. Instead of working with bitcoin-cli which sends RPC commands to bitcoind, you're sending those RPC commands directly. This allows for more robust programming, because you don't have to worry about what unexpected things bitcoin-cli might do or how it might change over time. However, you're also talking your first steps toward using a more comprehensive programming language than the poor options offered by a shell script. But for that, you'll need to use a curl library within a more familiar language like JavaScript or C. In other words, command-line curl was just the first step, the basis to better explain what you're doing as you move forward.