Introduction

Bitcoin was the very first non-centralized crypto currency that has been made public by the mysterious entity Satoshi Nakamoto.  Its general principles are announced in a seminal paper in january 2009.  In fact, Satoshi refers to a post of more than 10 years before by Wei Dai where in fact, the basic principle is already more or less sketched.  Satoshi set out to fill in the technical and conceptual details.  In its paper, Satoshi doesn't give all details, but outlines the lines along which the technical system of bitcoin is going to operate.

Bitcoin being the very first actually operational crypto currency, Satoshi had to make technical, but also conceptual choices within the grand lines he outlined.  Many of these choices have been of prophetic power.  However, some choices have been less fortunate.  Amongst the unforeseen consequences of these choices, we mention the most important ones in my eyes:

  • the centralisation of mining
  • the failed anonymity
  • the highly wasteful mining process, in resources and electricity
  • the lack of fungible coins
  • the small number of transactions per second and/or the bloating of the block chain
  • the violence of the block reward halving
  • the half-baked status of the language in which forms of smart contracts can or cannot be expressed

Newer crypto currencies have introduced several technological advances that tackle many of these issues.  Most of these crypto currencies, however, don't enjoy the success of bitcoin even though one can appreciate their technical improvements.  However, bitcoin encountered an unforeseen difficulty.   The monetary belief system that sustains bitcoin is based upon the verifiability of the accepted rules.  But this means that the rules are almost impossible to change, without affecting fundamentally the monetary belief system.  Even if most people using bitcoin will acknowledge some of these problems, there is in fact no way to change the rules without at the same time killing the belief that the rules are here to stay.  Every change of a protocol is difficult once the protocol is established, but in the mean time we have HTML 5.   However, the necessity of the monetary belief in eternal rules makes it much harder to change the rules of bitcoin than the HTML protocol.

So, of all crypto currencies, bitcoin is probably the least evolved, but nevertheless the most successful as of this day.  That is not to denigrate the work of Satoshi, on the contrary.  Satoshi is the only inventor of a crypto currency who had to do so without example or real world return of experience, but, contrary to other protocols, the nature itself of the monetary belief makes that the initial rules can almost not be changed.

Nevertheless, it has turned out that the bitcoin system is amazingly versatile, and workarounds from within the system have been found to several issues.

The technical side of bitcoin is very involved, and consists essentially of two sides: the block chain, and the peer-to-peer network protocol.  However, the core part is the block chain, as this is "here to stay".   In as much as one might decide to change the network protocol, this wouldn't have any implication on the belief system of bitcoin, as the thing to be verified is the block chain.  The communications are of course gone, but the block chain is the core part.  So we will limit ourselves to the analysis of the block chain here, and it will be the occasion to indicate the essential workings of the bitcoin system.

Digital structure of the block chain

The digital structure of the block chain consists of two layers: the block, and the transaction.  Because of the cryptographic nature of the crypto currency bitcoin, the exact description of each bit is necessary in order to define technically how bitcoin works.   The block chain is a numbered list of blocks, and each block is a well-defined list of bits.  Changing one single bit in this list would change all the cryptographic hashes, so although it may seem to be overly technically detailed, without the exact description of every single bit, the description is not complete.  In fact, the detail of description can be stopped on the level of bytes.

Block structure

A block is an ordered list of bytes, of which the description now follows.

The overall structure of a block is this:

  1. A magic number, consisting of 4 bytes.  It is equal to 0xF9BEB4D9 in hexadecimal notation (first byte is 0xF9, second is 0xBE...)
  2. 4 bytes, representing the number of bytes that remain in the block from the next byte onward
  3. A block header, making up 80 bytes
  4. A variant-size integer representation of the number of transactions that will follow, which take between 1 and 9 bytes
  5. the list of transactions (transactions themselves are also ordered lists of bytes)

The 32-bit unsigned integer is chunked in bytes in the little-endian way (that means that the most significant bits will be written in the last byte).  This is BTW also why sometimes one says that the magic number is 0xD9B4BEF9, because when interpreted as a 32-bit unsigned integer, and then written down in little-endian byte order, we obtain exactly what the protocol foresees: namely that the very first byte of a block is 0xF9 and so on.

We will address the block header afterwards, but let us first look at point 4: the variable-size integer representation.  It goes as follows.  If the number is smaller than 253, then the representation is one single byte, the unsigned integer representation of the number.

If the number is equal or larger than 253, but smaller or equal than 65535, then the representation will include 3 bytes.  The first byte will be 0xFD, and the two that follow will be the unsigned integer representation of the number (which can fit in 16 bits).  The byte order is officially specified as that of uint16_t, but that byte order is machine dependent, so there is an ambiguity here, but it doesn't matter, as that number never enters any specific crypto algorithm.  If the block is transmitted, it will be in little endian order.

If the number is larger than 65535, but smaller than 4 billion, then 5 bytes will be used: 0xFE followed by the unsigned representation with 4 bytes, as in uint32_t.  Finally, if the number is larger than 4 billion, then 9 bytes will be used: 0xFF followed by the unsigned representation with 8 bytes, as in uint64_t.

The block header is a very specific structure, and is one of the important cryptographic elements of the bitcoin system.

The block header itself, made up of exactly 80 bytes, has the following structure:

  1. the bitcoin block version number, made up of 4 bytes
  2. the hash of the previous block header in the block chain, 32 bytes
  3. the hash of the Merkle root of the transactions in this block, 32 bytes
  4. the current time stamp in seconds since 1970.  4 bytes
  5. the current target, 4 bytes
  6. the nonce

The 80 byte block is what will be hashed, and on which proof of work will be delivered.  The link with the hash of the previous block is what links the block chain and the hash of the Merkle tree of transactions fixes every byte in the transactions.  The nonce is the proof of work.

Because the block header is what will ultimately make up the links in the block chain, the securing of the transactions' invariability, and the irreversibility of the proof of work, in other words, the graving in stone of the consensus, it is of utmost importance that every single bit in the block header is well-determined.  The integers are stored in little-endian convention.   As such, the version number is usually first 0x01, then 0x00 three times, which comes down to the unsigned integer 0x00 00 00 01 in "normal" (that is, big-endian) hexadecimal notation.

The hashes are also stored in "little endian" notation, in as much as we interpret a hash as a (big) number.  Because the proof of work will require the block header hash to be a small number, given by the difficulty, this will come down in requiring the last bytes of the 32 hash bytes to be zero (while in normal big-endian notation of a number, it will be the leading bytes that are zero of course).  The byte order of the hashes is in fact the byte order of the standard definition of the hash function, which is a double application of the sha256 standard algorithm.

The current time stamp is in little-endian format.

The current target is a special "floating point" number, that has a exponent of one byte, and a mantissa of 3 bytes, which is a signed representation.  The fourth byte represents actually how many significant digits the target has, when it is represented in basis 256.  This number is augmented by 1 if the highest non-zero digit is larger than 127.  The three "mantissa" bytes are the three highest "significant" digits (including a leading zero if we added a 1 to the exponent).  The mantissa is stored in reverse order (little endian).  This floating point number is the target, that is, the upper limit below which the hash has to fall, hence indicating the difficulty.

Note that the twice iterated sha-256 hash function, applied to the block header, should result in a hash smaller than the target.

Note that the nonce is actually too small.  It has only 4 bytes, which allows 232 = about 4 billion possibilities (different hashes by changing the nonce).  If the difficulty is much larger than 4 billion (which it is, now, already since a long time), then most block headers will never be able to provide a good hash by changing the nonce, as it was originally designed.   One can win a few extra "nonce" bits by modifying the time stamp, but that is not sufficient either.  The solution is to modify fields in the transactions, which make a new Merkle tree hash.  This is a design error in bitcoin (the nonce is much too short), because the idea was that only the block header needed to be modified in order to do the proof of work.  On the other hand, that fact by itself (that only the header enters into the game) can also be considered a design error, as it leads to advantages of scale and centralisation of mining.   In any case, as the rules have been frozen, there's not much one can do about it, as argued earlier on.  In fact, even from the start, the nonce has been too small, because the starting difficulty of bitcoin has been defined as one out of 232.  The starting difficulty was already one out of 4 billion, which means that one needed on average 4 billion different nonces to have success.

Let us look at the genesis block of bitcoin:

F9 BE B4 D9  -- Magic number
1D 01 00 00 -- Number of bytes in the block
01 00 00 00 -- version number
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -- previous block hash (there isn't any)
3B A3 ED FD 7A 7B 12 B2 7A C7 2C 3E 67 76 8F 61 7F C8 1B C3 88 8A 51 32 3A 9F B8 AA 4B 1E 5E 4A -- Merkle hash
29 AB 5F 49 -- the current time in seconds since 1970
FF FF 00 1D -- the target in compact floating point notation
1D AC 2B 7C -- the nonce
01 -- one transaction follows.
01 00 00 00
01
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF FF FF FF
4D
04 FF FF 00 1D 01 04 45 54 68 65 20 54 69 6D 65 73 20 30 33 2F 4A 61 6E 2F 32 30 30 39 20 43 68 61 6E 63 65 6C 6C 6F 72 20 6F 6E 20 62 72 69 6E 6B 20 6F 66 20 73 65 63 6F 6E 64 20 62 61 69 6C 6F 75 74 20 66 6F 72 20 62 61 6E 6B 73
FF FF FF FF
01
00 F2 05 2A 01 00 00 00
43
41 04 67 8A FD B0 FE 55 48 27 19 67 F1 A6 71 30 B7 10 5C D6 A8 28 E0 39 09 A6 79 62 E0 EA 1F 61 DE B6 49 F6 BC 3F 4C EF 38 C4 F3 55 04 E5 1E C1 12 DE 5C 38 4D F7 BA 0B 8D 57 8A 4C 70 2B 6B F1 1D 5F AC
00 00 00 00

The genesis block of bitcoin has the normal structure of a block.  We put the header in bold and the transaction list (containing one transaction) in italic.

Transaction structure

The list of transactions is a succession of transactions.  Each transaction takes on the following form:

  1. The version number, 4 bytes, little endian order of an unsigned integer
  2. The number of inputs in the transaction, given by a variant integer representation, from 1 to 9 bytes
  3. The list of inputs
  4. The number of outputs of the transaction, given by a variant integer representation, from 1 to 9 bytes
  5. The list of outputs
  6. The lock time, 4 bytes, in little-endian notation.

Input structure

A transaction can have several inputs, or only one.  Each of these inputs takes on the following form:

  1. The previous transaction hash which has the output that is now the input (32 bytes)
  2. The previous transaction index (4 bytes)
  3. The length of the script that will follow, as a variable-length integer (1 to 9 bytes)
  4. The scriptsig, with the number of bytes given in 3.
  5. the sequence number (4 bytes, in little endian order)

There is a special transaction, which is the first transaction in the list, which has as "input" the generation of new bitcoins.  Instead of a scriptsig, which is meaningless, as there was no script of a previous output to satisfy, the space for the script is still there, but can be anything.  This is a place where arbitrary data can be written on the block chain.  it is called the coinbase.  Note that the outputs of this creation transaction are normal outputs, which the block creator decides upon (as the coins are his).  In fact, these outputs automatically collect also all the fees, that is, the difference between the sum of all outputs, and all inputs in all transactions.

The coinbase is also used as an extra nonce, because every time the coinbase changes, this will change the Merkle hash in the header, and hence gives another way of changing the header and hence its hash, in order to try to satisfy the difficulty.

Output structure

A transaction can have several outputs and each of these takes on the following form:

  1. The value (the amount of "satoshis", that is the minimal unit of bitcoin) as an unsigned 64-bit integer in little endian order
  2. The length of the script that will follow as a variable-length integer (1 to 9 bytes)
  3. The output script, which can take 2 forms: a transaction output script, or a pubkey script hash, with the number of bytes given in 2.

 Let us look again at the genesis block of bitcoin, of what follows after the header.  Now, the genesis block had only one transaction, and that must be the generation transaction.

01 00 00 00 -- the version number of the transaction
01 -- the number of inputs. This is only the fake input because it is a generation transaction. Next will follow the first (and only) input
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 -- the previous transaction hash, which has no meaning here
FF FF FF FF -- the index of the output in the previous transaction, which has no meaning either

4D -- the length of the scriptsig, but for a generation transaction, this is just the coinbase with random input:
04 FF FF 00 1D 01 04 45 54 68 65 20 54 69 6D 65 73 20 30 33 2F 4A 61 6E 2F 32 30 30 39 20 43 68
61 6E 63 65 6C 6C 6F 72 20 6F 6E 20 62 72 69 6E 6B 20 6F 66 20 73 65 63 6F 6E 64 20 62 61 69 6C
6F 75 74 20 66 6F 72 20 62 61 6E 6B 73

FF FF FF FF -- the sequence number
01 -- the number of outputs. Next will follow the first (and only) output
00 F2 05 2A 01 00 00 00 -- the amount of satoshis, here, the creation amount of satoshis, which was 50 bitcoin, or 5 000 000 000 satoshis, or 1 2A 05 F2 00 in hexadecimal notation
43 -- the length of the script that will follow
41 04 67 8A FD B0 FE 55 48 27 19 67 F1 A6 71 30 B7 10 5C D6 A8 28 E0 39 09 A6 79 62 E0 EA 1F 61
DE B6 49 F6 BC 3F 4C EF 38 C4 F3 55 04 E5 1E C1 12 DE 5C 38 4D F7 BA 0B 8D 57 8A 4C 70 2B 6B F1
1D 5F AC -- the output script

00 00 00 00 -- the lock time

The funny thing with the genesis block is that Satoshi has used the coinbase to put an ASCII message:

1D 01 04 45 54 68 65 20 54 69 6D 65 73 20 30 33 2F 4A 61 6E 2F 32 30 30 39 20 43 68 61 6E 63 65 
6C 6C 6F 72 20 6F 6E 20 62 72 69 6E 6B 20 6F 66 20 73 65 63 6F 6E 64 20 62 61 69 6C 6F 75 74 20
66 6F 72 20 62 61 6E 6B 73

can be translated in ASCII and this gives the following result:

The Times 03/Jan/2009 Chancellor on brink of second bailout for banks

There was indeed an article in the journal The Times on the third of January 2009, that said exactly that.  The meaning of including this is double.   First, it proves that the genesis block was constructed after that date (which is important to show that the difficulty, which is related to real time, is correct) ; and of course, it graves this message in stone for ever ; in as much as one would think that the fiat monetary system is totally corrupt and that this kind of bailout proves it, one of the reasons of existence of bitcoin is indicated here.

Scripting

If bitcoin were just a bare-bones crypto currency with the sole aim of transactions of coins, the following section wouldn't have any meaning.  But bitcoin also included a first attempt at smart contracts

The way it is done in bitcoin is essentially that a smart contract is a transaction that is executed whenever a combination of conditions is satisfied.  Of course, a simple transaction can be seen as an elementary form of smart contract: there is only one condition to be satisfied, and that is that the former owner proves that he's the former owner and shows that he agrees with the transaction. 

The general principle is the following: There is a script in an output of a transaction, and there is a sigscript in the input of a next transaction that refers to that output.  Verification is done as follows: the sigscript and the output script are concatenated in that order (in fact, there is a security issue with that, so in reality it is more complicated, but conceptually, that's the idea - the actual original bitcoin code did exactly that, and the more complicated way has been implemented to avoid the security issue).  Then this is executed.  If the result of that execution is "true" then that part of the transaction is verified ; if it returns "false", then the transaction is not acceptable.

The bitcoin scripting language is a fairly simple one.  It are instructions that do operations on what is called a stack, and can generate a return value true or false.  The instructions themselves are single bytes, which can be followed by operands.    In a way, one can see the output script as a challenge, and the signature script in an input as taking up the challenge.   The scripting language has also access to the block chain and the current transaction.  Simple as it is, the bitcoin scripting language has an enormous amount of possibilities, but of course the very simplest one is a normal transaction from one owner to another.  If it were only for this, no scripting language were necessary, but as there is a scripting language, the simple transaction is also expressed as a function of it.

The simple transaction has the following output script:

OP_DUP OP_HASH160 <pubKeyHash> OP_EQUALVERIFY OP_CHECKSIG

and the sigscript takes on the following form:

<sig> <pubKey>

As such, when we combine both in order to verify the input of a transaction, the following happens:

<sig> <pubKey> OP_DUP OP_HASH160 <pubKeyHash> OP_EQUALVERIFY OP_CHECKSIG

The script is a concatenation of the signature script and the original challenge script.

First, the signature is pushed upon the stack.  Next the public key (of the owner) is pushed on the stack.

The OP_DUP instruction duplicates the last item on the stack, which is the public key.  So now our stack looks like:

<sig> <pubkey> <pubkey>

The OP_HASH160 instruction calculates the 160-bit hash of the last item on the stack,

As such, the stack now looks like:

<sig> <pubkey> <hash160 of pubkey>

Next, the <pubKeyHash> is pushed on the stack (from the output script).  The stack now looks like:

<sig> <pubkey> <hash160 of pubkey> <pubKeyHash>

The operation OP_EQUALVERIFY checks whether the two last items on the stack are the same.  If they are not, this instruction returns a "false" and halts the script.  In this case, we check whether the hash from the public key in the input of a transaction matches the required hash in the corresponding output of an earlier transaction.  if it is OK, the script continues.  We now have on the stack:

<sig> <pubkey>

The instruction OP_CHECKSIG verifies the signature of the transaction that is to be checked.   Now of course there is a problem of principle here, as the current transaction contains the signature itself, which is of course unknown until one has calculated the signature.  So things are in fact slightly more complicated.

What happens is that the current transaction (that is to be checked)gets all its input sig scripts removed (but all output scripts are kept).   The script length variable integers are also put to 0 (and eventually shortened to 1 byte if ever they were longer).   Next, the old output script (we are actually executing) of the specific transaction output is copied in the place of the sigscript of the input we're checking, and the variable length integer of the script is again adapted.  The signature itself has an extra byte which specifies the hash type, and this single byte is changed into a 4-byte hash type code.  These 4 bytes are appended to the modified transaction string.

This string is then hashed (the hash is a double SHA256).  Finally, it is checked whether the signature provided in the sigscript is the signature of this hash, with the public key provided in the sigscript using elliptic curve public key cryptography.

Let us look at an example.

We take a block from 2011:

F9 BE B4 D9 A1 23 00 00 
-- header 01 00 00 00 98 D9 02 9C EB 0E F7 BB 3A F1 35 E7 7E 26 DA F9 03 56 0C 7F
14 DE 1D EB 5F 4B 00 00 00 00 00 00 07 DE 60 9F 04 A4 BE 2C 1B 23 E0 93 DB 7E 8E D5 08 47 7B 0B
E7 32 65 F7 1E 8F 8C BF 45 34 9C 9E 84 37 B3 4D AC B5 00 1B 0C 17 39 BC 1B
-- coinbase transaction 01 00 00 00 01 00 00
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 FF FF
FF FF 07 04 AC B5 00 1B 01 0B FF FF FF FF 01 40 34 15 2A 01 00 00 00 43 41 04 C5 8F 32 C8 E2 F7
24 EB E7 BF 94 C2 45 1C EE 28 BA 07 03 0A 2C AC EC C7 1C 28 14 00 1A 51 E6 6C B0 EC AF AA 15 2F
74 8E DB 8E A0 7C B2 76 AB 2F B3 43 8C 15 06 8D 96 52 26 65 DE 10 CB 3B 48 A7 AC 00 00 00 00
01 00 00 00 -- version
01 -- one input
78 3F 1D 91 43 77 F8 E7 19 37 A0 4B 9A 4D 22 D8 EC 16 E5 14 AD 35 70 8C 80 2D E6 09 C3 0F 3C 1F -- hash of transaction
00 00 00 00 -index of output of transaction
8B -- length of scriptsig
48 30 45 02 21 00 CF 7B F3 A0 D4 E8 FC E0 71 2C 48 1F 20 A3 4D 26 6A DE CB 3A 8D 2B 69 95 D4 DD
C2 AE CA EA F9 F3 02 20 6E 92 39 F1 36 B2 81 42 37 97 87 1A AE 7C 40 40 93 9F 4B 82 0E 9D 27 CF
76 42 FF 67 BA E8 C6 3D 01 41 04 86 79 12 72 F2 37 A7 C8 00 6F 54 AB 11 D8 26 B3 54 72 98 9F C3
3A 7D 02 4A 57 A9 3E 25 EB 7E D0 EA 9B D9 05 E7 CC 98 F4 5D 64 07 5A D0 EF 9D AD 73 8F 87 90 BE
83 F6 87 8B 95 67 0C 36 02 15 1D -- the scriptsig of the first input
FF FF FF FF -- the sequence number of this input
02 -- the number of outputs in this transaction
00 CA 9A 3B 00 00 00 00 -- first output amount, exactly 10 bitcoin
19 -- length of the output script for the first output
76 A9 14 61 2C D0 3B B0 A9 FB 52 60 34 72 88 FF FE 14 13 1D 4F C5 90 88 AC -- the output script of the first output
00 BD 0A 42 00 00 00 00 -- the amount of the second output: 11.08 bitcoin
19 -- the length of the output script for the second output
76 A9 14 F8 30 46 C8 6F C7 2D D6 95 EA 0C 14 F7 83 A9 F8 8C 9C 55 40 88 AC
00 00 00 00 -- the lock time of this transaction
01 00 00 00 -- the version of the following transaction
02 -- two inputs
86 46 F8 D8 1E 99 4E 10 CE 37 69 70 3D AE F6 38 76 F6 D7 96 AD 54 A8 8B AE F3 29 70 08 A4 32 5D
-- hash of the transaction containing the output that is spend in the first input
01 00 00 00 -- index of that output
8B -- length of the scriptsig for the first input
48 30 45 02 20 4B 5C 60 42 12 D9 B3 E2 61 C4 3A 5D FD 09 D7 CA AF FF 72 DA 44 4B 86 5F 0E AC 1C
BC 6C 89 D4 B8 02 21 00 DB 5D AF CC 67 9B 78 F9 A6 D4 08 CC AC BB 47 E8 92 FE 86 62 C2 99 95 A3
D4 74 BD 40 CC D8 EE 0C 01 41 04 E6 79 D4 D7 5E FF C8 6F A0 50 E9 BC 8B 19 DA 65 8E 8C 61 A4 09
67 72 23 FD 86 5B 62 7D 9B 46 4C EB 13 02 E1 61 85 1B 5B CD 13 AD 20 6D 26 C5 8C 02 E0 2D 7F 22
A9 7D 1A 64 A4 DE E9 53 10 B5 B7 -- scriptsig of the first input
FF FF FF FF -- sequence number
CD CB B0 EA 3D 58 5E CA 95 10 A0 4A 01 C8 2F FC EC 34 0F 3B 3C 03 45 36 09 9E A8 76 39 22 28 2E
-- hash of the transaction containing the output for the second input
01 00 00 00 -- index of that output within that transaction
8C -- length of the scriptsig for the second input
49 30 46 02 21 00 BC AF FF F5 20 90 3C 67 D4 B2 77 91 FE 96 11 50 7E 43 B5 F2 3C F7 73 2C 12 52
12 B0 65 B4 A2 C9 02 21 00 BF C3 14 5C 51 73 8A 20 84 43 4C 09 2F 38 48 FC D3 27 FE 85 C5 33 CB
91 86 EE EA 55 80 7A 1D 48 01 41 04 91 9A 77 6A 35 A7 2A 2D 11 E3 F7 BB E7 BF 50 6E 52 A8 66 86
B4 7D 25 C7 55 77 94 EF 08 F2 59 6A 1E F0 75 67 60 3B 99 8E E4 8F B3 95 C6 6C 28 FB 42 14 14 29
FB C0 D1 E3 BB 49 6D 9F FC E0 F8 4B -- scriptsig
FF FF FF FF -- sequence number
02 -- the number of outputs in this transaction (here, 2)
00 F2 05 2A 01 00 00 00 -- amount of the first output (50 bitcoin)
19 -- length of the first output script
76 A9 14 8C B5 0A C1 25 3B C2 21 DD 67 3F 66 35 E6 70 33 66 0E 53 73 88 AC -- first output script
C0 B0 60 06 00 00 00 00 -- amount of the second output (1.07 bitcoin)
19 -- length of the second output script
76 A9 14 A3 08 8B 97 43 B7 7D AC 81 96 2F CC 37 DB BF 42 3E CC 40 A4 88 AC -- second output script
00 00 00 00 -- lock time
01 00 00 00 -- version of the following transaction
01 -- one input
...

The block is longer, about 9KB, but we truncated it to a small part that we're interested in, with two transactions to look at.  The italic part is what comes before the transaction list, containing the block header.  The 1B byte indicates that the transaction list will contain 27 transactions (of which the first is the coinbase transaction, where new bitcoins are given to the block creator).  We look at the two genuine transactions that follow.  The first has one input and two outputs, the second has two inputs and two outputs.

Let us have a look at the second output script of the first transaction we are considering.  It is 25 bytes long:

76 A9 14 F8 30 46 C8 6F C7 2D D6 95 EA 0C 14 F7 83 A9 F8 8C 9C 55 40 88 AC 

What does this script contain ?  It is a standard simple transaction "challenge", which is of the form:

OP_DUP OP_HASH160 <pubKeyHash> OP_EQUALVERIFY OP_CHECKSIG

Can we recognize the parts ?

OP_DUP has the opcode 0x76.  OP_HASH160 has the opcode 0xA9

Indeed, we find 76 A9 as the first two bytes in the script.

OP_CHECKSIG has the opcode 0xAC which is the last byte in the script, and OP_EQUALVERIFY has the opcode 0x88 which is the previous byte.

The (underlined) 0x14) means that the next 20 bytes (14 in hexadecimal) are to be pushed onto the stack.

The raw bitcoin address of the recipient is:

F8 30 46 C8 6F C7 2D D6 95 EA 0C 14 F7 83 A9 F8 8C 9C 55 40

This is not exactly the address that is used, because the address that is used contains a check sum and a version indicator, which has been left off the raw command in the block chain.

We can now recognize simple transaction outputs: they start always with 3 bytes: 76 A9 14 and they end with 88 AC.  In between is the raw bitcoin address of the recipient, made up of 20 bytes.

Let us now look at something somewhat more complicated: the simple sigscript.  Let us look at the first and only input of the first transaction we are considering.

The scriptsig is much longer than the output script as it contains 139 bytes:

48 30 45 02 21 00 CF 7B F3 A0 D4 E8 FC E0 71 2C 48 1F 20 A3 4D 26 6A DE CB 3A 8D 2B 69 95 D4 DD 
C2 AE CA EA F9 F3 02 20 6E 92 39 F1 36 B2 81 42 37 97 87 1A AE 7C 40 40 93 9F 4B 82 0E 9D 27 CF
76 42 FF 67 BA E8 C6 3D 01 41 04 86 79 12 72 F2 37 A7 C8 00 6F 54 AB 11 D8 26 B3 54 72 98 9F C3
3A 7D 02 4A 57 A9 3E 25 EB 7E D0 EA 9B D9 05 E7 CC 98 F4 5D 64 07 5A D0 EF 9D AD 73 8F 87 90 BE
83 F6 87 8B 95 67 0C 36 02 15 1D

Can we understand its structure ?

If this scriptsig takes up the challenge of a simple transaction output script, it should be of the form:

<sig> <pubKey>

Let us see if we can understand that.

One should push first the signature on the stack, and next, the public key.  "pushing on the stack" starts with a byte giving the number of bytes to be pushed.  So here, that looks like 0x48 which is 72 bytes.

 The signature looks then like:

30 45 02 21 00 CF 7B F3 A0 D4 E8 FC E0 71 2C 48 1F 20 A3 4D 26 6A DE CB 3A 8D 2B 69 95 D4 DD C2 
AE CA EA F9 F3 02 20 6E 92 39 F1 36 B2 81 42 37 97 87 1A AE 7C 40 40 93 9F 4B 82 0E 9D 27 CF 76
42 FF 67 BA E8 C6 3D 01

Next, one should push the public key on the stack.  0x41 (or 65 bytes)

The public key then looks like

04 86 79 12 72 F2 37 A7 C8 00 6F 54 AB 11 D8 26 B3 54 72 98 9F C3 3A 7D 02 4A 57 A9 3E 25 EB 7E 
D0 EA 9B D9 05 E7 CC 98 F4 5D 64 07 5A D0 EF 9D AD 73 8F 87 90 BE 83 F6 87 8B 95 67 0C 36 02 15
1D

We must be on the right track, because the number of bytes is at least OK !

The signature is the actual elliptic curve signature, plus one byte, that is usually 1, as is the case here.  This "1" indicates the hash type.  The actual structure of the signature is also somewhat involved, as it contains several elements in DER format.  We will come to that later.

The public key consists of two 32-byte words, which make up the (x,y) pair on the elliptic curve.  So we expect the public key to be 64 bytes long.   For an unclear reason, in bitcoin, this concatenation of two 32 byte words is preceded by an extra byte: 0x04.  The underlined part is the x-coordinate of the public key point. The remainder is the y-point (which is essentially redundant, as it can be restored from the x-point up to a single ambiguity of sign).

It is important to note that in order for an input to be valid, that is, to be accepted as the succession of an output somewhere (given by the hash of the transaction containing it, and the index number), it is not sufficient to take up successfully the challenge of the output.  It must also be the very first time in the block chain that this challenge is taken up.   This was the whole point of having a block chain in the first place, to avoid double spending.

This means that an output has to be spend in one single go, completely.

It could have been done differently, but in bitcoin, an output can only be used once, and in its entity.  So if Joe paid 50 bitcoin to Jack, and hence Jack has the secret key that can take up the challenge of the output of this transaction from Joe to Jack, worth 50 bitcoin, if Jack wants to pay 0.05 bitcoin to Alice to buy a coffee, he needs to spend the 50 bitcoin, because he will be able to use this output only once.  As such, Jack will construct a transaction with two outputs: one output worth 0.05 bitcoin with a challenge that Alice can solve (so this is his payment to Alice), and a second output with 49.95 bitcoin to another challenge he can solve, so that's a payment to himself.

Such a payment to oneself because the output to be spend is higher than what one wants to pay, is called a change payment.  It looks like a good idea, but it compromises entirely the anonymity of bitcoin.  Indeed, up to the payment from Joe to Jack, only Joe and Jack knew about these 50 bitcoin.  But now that Alice knows Jack (he bought a coffee) and sees his transaction, she knows not only that Jack received 50 bitcoin from someone, but she also knows the change address.  If Jack next buys a 30 bitcoin pair of earrings for his lover, then Alice knows that Jack has been spending 30 bitcoins.  Imagine that the guy selling the earrings comes and takes a coffee at Alice's coffee bar, and pays Alice with these 30 coins and a change address, then Alice knows that Jack has been paying this gentleman for something, worth 30 bitcoin even though she has nothing to do with that.   It turns out that the partial knowledge one can gain from the relationship between certain transactions and certain real world persons can be exploited to gain an understanding of many ownership relationships, and the anonymity of bitcoin is hence largely broken due to this "change" necessity.

Transaction hashes

In order to specify a former output, an input specifies a "transaction hash" and the "index in that transaction" of the concerned output.  The transaction hash is the double application of SHA256 to the entire transaction, starting with the version number, including the inputs and outputs, and finally including the lock time, exactly as it appears in the block chain.  However, the hash itself appears nowhere.  So it is the responsibility of the verifier, in order to find out which transaction one is talking about, to make a lookup table of all the hashes of all the known transactions on the block chain and their place on the block chain.  Once such a dictionary is set up, given a hash, the transaction can be found.   It would, at first sight, be easier to specify the block number, and the number of the transaction in the block, but the point is that the transaction itself is independent of exactly where it is in the block chain, so the transaction indicator should be independent of the exact location on the block chain.  As such, in order to be able to use the block chain, one has to construct a dictionary with hashes of transactions. 

Bitcoin addresses

When considering simple transactions, there is an aspect to the bitcoin system which looks, at first sight, like something like a bank account, but in fact the resemblance is only superficial.  Nevertheless, the confusion can easily set in and even regulators fall for this confusion.  Let it be clear: there is no such thing as a bitcoin bank account.   When someone wants to transfer bitcoins to you - because you deliver them goods or services for instance - then you should provide them with a challenge only you can take upThat is exactly what a "bitcoin address" is: a challenge you've prepared and which only you can take up.

There is no reason why another transaction to you should be attached to the same challenge.  Indeed, it is in general much better to use a new challenge for every transaction in your direction although nothing forbids you to do so.  If you always use the same challenge for all transactions in your direction, then this challenge starts to look like a bank account.    However, it is true that the naming of this challenge, namely a "bitcoin address" works a lot the confusion in hand that a challenge is a kind of account with an address to which one sends bitcoins.

For a simple transaction, the challenge is to provide a signature (with an ECDSA private key) of a message (namely a modified transaction) such that the corresponding public key hashes to the challenge.  This is easy to do if you start out with the private key in some way, from which you can generate the public key and the challenge hash.  This is cryptographically impossible to do if you didn't start out with a private key and it wasn't you who provided the challenge hash.  And that's the idea: that the provider of the challenge is the only one possessing the tools to take up the challenge (and in doing so, prove that he's the owner of the bitcoins).

So the normal way of building a challenge that only you can solve in bitcoin, is this:

  1. Generate a key pair, of a secret key, and a public key, in the elliptic group with the generator, indicated by bitcoin.  The public key consists of two 32-byte words (one representing the  x coordinate, and the other, the y coordinate).
  2. Concatenate the two parts of the public key, to form a 64-byte word, and and in front the byte 0x04.
  3. Calculate the SHA256 hash of this 65 byte block, resulting in a 64 byte hash
  4. Calculate the RIPEMD-160 hash of this 64 byte hash, resulting in a 20 byte hash

This 20-byte hash is the challenge to which you will be able to respond with the secret key of point 1.

However, in order to transmit any address (which the receiver has to do to the sender), some extra things are added.  So a bitcoin address that is given to someone, "secures" this raw 20-byte bitcoin address as follows:

  1. Add in front a network ID byte to the string.  For the bitcoin main net, this is 0x00.  For the testnet, this is 0x6F.
  2. Take twice the SHA256 hash of this 21-byte word, and conserve the first 4 bytes of this hash.  This is the check sum, that is to be appended at the end of the 21-byte string.
  3. Convert the 25-byte string, as a number, into the base-58 check encoding: the bitcoin address is ready.

Note that, in order to take up the challenge of the given 20 bytes, one needs the original secret key.  However, nothing stops you from inventing a random 20-byte string (and adding the decoration with the 0x00 at the beginning and the 4-byte check sum at the end if you want to transmit it).  If someone makes a valid transaction to that random string as a challenge, then, because nobody knows any secret key that goes with it, these bitcoins are then lost forever.  The only way to unlock them would be to find a secret key that generates the same challenge, which is considered cryptographically impossible

The other point to notice is that there's something strange with the way this has been done in bitcoin.  One would think that the cryptographic difficulty of the public key system is of the order of 256 bits, given that one uses an elliptic curve with 256 bits (32 bytes).  But the fact that the only actual challenge is reduced to 20 bytes, means that the actual cryptographic difficulty is in fact only 160 bits.  Indeed, there are of the order of 296 different private/public key pairs that will result in exactly the same RIPEMD-160 hash.  Each of these different key pairs will be able to take up the challenge.  So it is kind of strange that one goes through the difficulty of using a 256 bit ECDSA system of which one has reduced the security by a factor of 296.  One could have used directly a curve with 160 bits.

Conclusion and outlook

Exploring the structure of the block chain has allowed us to look at many technical details of the bitcoin system.  We can now "read" the block chain, and understand most of the bytes that appear in it (we haven't treated the more complicated script cases, but at least, we can recognize where the script is)  We understand now everything concerning the data structures of simple transactions (the only thing that is necessary to make bitcoin a basic crypto currency).

Next to the block chain structure, is of course the peer-to-peer communication protocol which we didn't touch upon, and we also didn't delve into the cryptographic details of the bitcoin system (especially the elliptic curve cryptography).  

Cryptographically, the bitcoin system uses the following building blocks:

  • the hash function SHA-256, which is used as transaction hashes, as proof of work and block header hash, and in the fabrication of a bitcoin address.  It is also used in the Merkle tree of transactions, to compile a transaction list hash in the block header.
  • the hash function RIPEMD-160, which is used to calculate "addresses".
  • ECDSA, elliptic curve digital signature algorithm

In the scripting language a few other cryptographic systems are present that could be used.

One of the main difficulties discovered with bitcoin is the centralisation of "mining" (that is, the process of providing proof of work to obtain new coins).  It is thought that this is the result of possible economies of scale, and entrance barriers.  Initially, bitcoin was intended to be mined by everybody.  Everybody with a PC would be able to mine, and the seigniorage, if any, would be distributed amongst all users.  However, the "proof of work" function in bitcoin is too simple: only the header is involved (one only has to find a header hash), and the only way the older blocks enter is through their fixed hash.   As such, the development of special-purpose electronics that can do this at huge scale was possible: ASIC miners.  This is so efficient, that no PC can compete with them, and their existence has driven the difficulty to astronomic heights.  On the other hand, given that an ASIC miner needs a certain investment, and becomes more and more efficient if the scale is driven up, only huge mining factories are now the norm, and one can count the number of significant miners on one's fingers.   An oligarchy of miners can hence decide any future change to the bitcoin protocol.  That was not the idea.

Another difficulty is the following.   Given that any bitcoin on the block chain has a transaction history dating back to its creation, and that this transaction history can be followed, not all bitcoins are equal.  One might decide that bitcoins that have been gone through transactions with challenges owned by, say, drugs dealers, or Germans, are not accepted any more by miners (of which there are not so many).  Even though of course nobody can find out from the challenge (the bitcoin address) who you are in principle, when you have to give a bitcoin address to someone, to get paid, or when you pay someone, then your partner knows you, and knows the related bitcoin addresses.  It has been shown that the networks of transactions and the partial knowledge of ownership, is in fact very powerful in revealing who is owner of what challenge.  The main mechanism is the "change" mechanism, which links together many transactions of the same owner.  One single leak of knowledge of his identity, and the whole net of transactions can be assigned to that identity, and identity leaks of course each time one acquires, or one spends bitcoins.