Transaction data vs metadata

With Bitcoin as well as most "Crypto 1.0" currencies, the transactions are simple and elegant. You specify which coins you're spending, what are the redemption requirements, and that's about it. A transaction either goes through, is included in a block and can be spent, or it never gets included and can be safely ignored. However, when we look at the more complex Crypto 2.0 systems, things start to get more complicated - there are many more states a transaction can be in, and we require additional metadata to figure out what really happened.

The way Ripple handles its transactions is a good example of the data vs metadata. When submitting a transaction, we submit its data - our intent of what we want the transaction to do. For example:

{
"TransactionType" : "Payment",
"Account" : "rf1BiGeXwwQoi8Z2ueFYTEXSwuJYfV2Jpn",
"Destination" : "ra5nK24KXen9AHvsdFTKHSANinZseWnPcX",
"Amount" : {
"currency" : "USD",
"value" : "1",
"issuer" : "rf1BiGeXwwQoi8Z2ueFYTEXSwuJYfV2Jpn"
},
"Fee": "10",
"Flags": 2147483648,
"Sequence": 2,
}

Indicates that we want to send 1USD from rf1... to ra5... and we pay the fee of 10. Now, when we submit the actual transaction, we also see its metadata:

{

"id": 6,

"status": "success",

"type": "response",

"result": {

"Account": "rf1BiGeXwwQoi8Z2ueFYTEXSwuJYfV2Jpn",

"Amount": {

"currency": "USD",

"issuer": "rf1BiGeXwwQoi8Z2ueFYTEXSwuJYfV2Jpn",

"value": "1"

"Destination": "ra5nK24KXen9AHvsdFTKHSANinZseWnPcX",

"Fee": "10",

"Flags": 2147483648,

"Sequence": 2,

"SigningPubKey": "03AB40A0490F9B7ED8DF29D246BF2D6269820A0EE7742ACDD457BEA7C7D0931EDB",

"TransactionType": "Payment",

"TxnSignature": "3045022100D64A32A506B86E880480CCB846EFA3F9665C9B11FDCA35D7124F53C486CC1D0402206EC8663308D91C928D1FDA498C3A2F8DD105211B9D90F4ECFD75172BAE733340",

"date": 455224610,

"hash": "33EA42FC7A06F062A7B843AF4DC7C0AB00D6644DFDF4C5D354A87C035813D321",

"inLedger": 7013674,

"ledger_index": 7013674,

"meta": {

"AffectedNodes": [

{

"ModifiedNode": {

"FinalFields": {

"Account": "rf1BiGeXwwQoi8Z2ueFYTEXSwuJYfV2Jpn",

"Balance": "99999980",

"Flags": 0,

"OwnerCount": 0,

"Sequence": 3

"LedgerEntryType": "AccountRoot",

"LedgerIndex": "13F1A95D7AAB7108D5CE7EEAF504B2894B8C674E6D68499076441C4837282BF8",

"PreviousFields": {

"Balance": "99999990",

"Sequence": 2

"PreviousTxnID": "7BF105CFE4EFE78ADB63FE4E03A851440551FE189FD4B51CAAD9279C9F534F0E",

"PreviousTxnLgrSeq": 6979192

}

{

"ModifiedNode": {

"FinalFields": {

"Balance": {

"currency": "USD",

"issuer": "rrrrrrrrrrrrrrrrrrrrBZbvji",

"value": "2"

"Flags": 65536,

"HighLimit": {

"currency": "USD",

"issuer": "rf1BiGeXwwQoi8Z2ueFYTEXSwuJYfV2Jpn",

"value": "0"

"HighNode": "0000000000000000",

"LowLimit": {

"currency": "USD",

"issuer": "ra5nK24KXen9AHvsdFTKHSANinZseWnPcX",

"value": "100"

"LowNode": "0000000000000000"

"LedgerEntryType": "RippleState",

"LedgerIndex": "96D2F43BA7AE7193EC59E5E7DDB26A9D786AB1F7C580E030E7D2FF5233DA01E9",

"PreviousFields": {

"Balance": {

"currency": "USD",

"issuer": "rrrrrrrrrrrrrrrrrrrrBZbvji",

"value": "1"

}

"PreviousTxnID": "7BF105CFE4EFE78ADB63FE4E03A851440551FE189FD4B51CAAD9279C9F534F0E",

"PreviousTxnLgrSeq": 6979192

}

"TransactionIndex": 0,

"TransactionResult": "tesSUCCESS"

"validated": true

}

Which specifies, among other things, AffectedNodes - the actual state change exacted on the system. It specifies the balance change of multiple addresses the transaction rippled through. This can be especially important when there are multiple paths a transaction could take, possibly even spanning many different currencies and entities.

How Ripple Works - Gateways and Pathways

All in all:

Transaction data specifies what we want the system to do
Transaction metadata specifies what did happen in the system

Lets look at a few examples of why this distinction is important.

Blockchain interpretation in Bitcoin 2.0

Some people use the term "Bitcoin 2.0" and "Crypto 2.0" interchangeably. I personally make the distinction of using the first term only when referring to systems built on top of Bitcoin itself - Mastercoin/Omni, Counterparty, Colored Coins, etc., while using the second term for all cryptocurrency systems allowing one to issue custom currencies (which includes Bitcoin 2.0s as well as systems like Ripple, Ethereum, etc.).

The distinction is important here because Bitcoin 2.0 systems inherently have no control over which of their transactions are included in the Bitcoin blockchain they are using. Unlike Bitcoin, two conflicting transactions can be included in the block and the system has to be able to interpret them correctly. Without transaction metadata, it is hard to tell at a glance whether a transaction is valid and spendable, or whether it is a double-spend and should be ignored.

A cautionary tale of the partial payment flag

In 2014 JustCoin, a Ripple gateway, got into a lot of trouble due to a small feature in Ripple very few people noticed before then - the partial payment flag. When a transaction is created with that flag, it signals to the network "I want to pay the person as much as I can up to the limit specified", rather than "I want to pay the person exactly this much". So for example if my transaction data specifies the amount I'm paying to be 1'000'000USD, but my balance is only 10USD, without the partial payment flag the transaction would fail, and with the flag it would succeed but only give a person 10USD.

The big problem JustCoin ran into was that the transaction data still would quote the big number, even if very little was sent, and only by examining the metadata would they be able to see how much money was actually sent. This meant their attackers could rack up bogus deposits and cash out of the gateway, leaving it short on funds.

Transaction successful, payment failed

While working on a Ripple gateway in the past, I got to explore a few different end states a transaction can end up in - a transaction can be not included in a block and be in an undefined state, it can be included in a block and be successfully applied, included in a block and partially applied (partial payment, open exchange), or it can be included in a block but still fail. In the context of Bitcoin, the last state would be unthinkable.

A scenario where a transaction is not applied to a block is similar to Bitcoin's unconfirmed transaction - it is in a state of limbo. However, with Bitcoin one can still spend other outputs without worrying about the transactions interfering with one another - each transaction output can be spent independently. For systems relying on address balances rather than transaction outputs, a dangling transaction can be a blocker. This is why professional transactions are sent with an expiration date (after which the transaction will definitely fail and not do anything), as well as sending a NOP transaction to overwrite the expired transactions to allow everything else to go through.

Successful transactions that applied fully are pretty straightforward - everything went through (or as much as could in case of partial payment flags).

Open-ended transactions are initialized by one transaction, but end up being fulfilled by other transactions. This applies mostly to the decentralized exchange offers - it's similar to placing a bid on a market and waiting for one or more asks to fulfill it. The transaction only closes when it is fully fulfilled, or it becomes invalid due to low balance, etc.

Failed transactions being included in a block mostly apply to Bitcoin 2.0s and balance-based cryptocurrencies. Those transactions either are inevitable - any valid Bitcoin transaction can be included in a Bitcoin block, but it can be an invalid Omni transaction -, or they are there for simplicity's sake - to do nothing, consume a sequence number and allow next transactions to be committed.

Conclusions

Transaction data specifies what we want the system to do, while transaction metadata specifies what did happen in the system. While not as important to Bitcoin and other first generation cryptos, the transaction metadata becomes more and more important for more complex Crypto 2.0 systems.

Transaction data vs metadata - interpreting what has happened