It has been 4 years after completing my PhD in 2012. At the time, I thought “enough of studying, let’s start to work”. The journey began and I realized that there are still so much things that I didn’t know and apparently I enjoys learning still. In any field, the knowledge is vast and depending on how far you want to go, there are always stuffs to learn. And I do feel strongly that we should not taking things for granted as it was told, but asking “what-why-how-why not” for those that are of our interest.
This blog has been my personal notes. And it will remain the same, but with the additional part to record my learning journey. About things that I’m interested to learn. Let’s see…
To gather my comments on solving problems in general, inspired by the training that I has attended last week. I will skip the two steps of 1) Problem statement and 2) Formalize problem. These steps are crucial for solving any problem, but will not be covered here.
In my view, many problems can be seen as finding input $latex X$ such that output $latex Y = F(X)$ satisfies certain conditions. For example, essentially, root cause analysis for a problem of f(X) = incorrect (in contrast to f(X) = expected) is to identify subset of $latex X$ that causes the incorrect results.
Note that the problem statement and formalization is important as it will determine how $latex X$ should be represented:
Linear: vectors / matrix
Convolution: complex data structure / layers
Solving approach: I think there are two main approaches:
Analytical
Logical analysis: based on the logical rules of causality $latex X \rightarrow Y$ is known. For given output Y, one can identify possible root causes X and do back-testing using the rules to validate. Known as: fish-bone techniques.
Differential thinking: one does not know about the blackbox function $latex X \rightarrow Y$ but knows how Y changes when X changes by a $latex \Delta X$
Recursive thinking: one does not know about the blackbox function $latex X_n \rightarrow Y_n$ but know how to link the problem to $latex X_{n-1} \rightarrow Y_{n-1}$ and can solve the problem at certain lower level. Known as: divide-and-conquer.
Equivalent mapping: one can relate this problem to a equivalent, more familiar and solvable problems.
Stochastic
Mix and match solutions coming from different related problems under different domains.
Distributed ledger/database of ownership: remove completely the role of centralize custodian
Time-stamping (double-spending): a mechanism to avoid A transfer same “coin” to both B and C. Without centralized trusted custodians, due to the nature of digital communication, it is more feasible to happen. Imagine: without centralized custodian, how to sell same house to two different buyers?
Consensus via proof-of-work: Once a transaction (ownership transfer) is submitted (broadcasting), it must be validated and recorded in the “distributed ledger” database. Assume everyone owns the same ledger copies, now some receive a broadcast of transferring from A to B: $latex Coin_{k+1}=Sign(Hash(Coin_k, P_B), S_A)$ and some receive a broadcast of transferring to C: $latex Coin_{k+1}=Sign(Hash(Coin_k, P_C), S_A)$. How everyone reach a consensus of which transfer comes first and invalidating the second one?
First, each new transaction, once validated, will be in a new block that linked to previous blocks in a recursive manner. The new block has a header hash that is created based on the new transaction & a random number (nounce) & the hash of previous block. In this way, the blocks are linked sequentially. It means that if one ledger is tampered in any block, one can recompute its hash and verify with other ledgers to identify its invalidity.
Another trick is that the random number is mined such that the hash begins with certain 0-zeros. Meaning that there are computational effort spent to create new block, i.e., proof of work. Also, in some ways, the one who spent efforts to create new block, i.e., validator node, will be rewarded with new coins as incentives. There will be a “race” among node validating the same or different block of new transactions to find “the nounce”.
Lastly, once one validator node successfully creates a new block, it will broadcasting the new hash (and block?) so that other can update their ledger with this new node and move on the validating the new block. One principle is that the longer chain should be considered as a “truth”. It implies that if one tries to tamper the existing block, it would need to create blocks faster against the “race” of other honest nodes. If the “race” is fierce, the chain quickly gets longer, dimishing the chance to tamper data.
Imagine: we have 10 personels as validators. I announce to all the network that I will sell my house to B (message 1) and at the same time sell to C (message 2). Assuming that at one time, 6 personels are validating message 1 and remained 4 personels validating message 2. All can validate that I own the house based on the current state of ledger, hence accepting both message 1 or 2, just the matter of which one comes first (time-stamping). The proof of work is that a validator need to do 100 push-up before accepting my message. Only who does the job fastest (physical limit) will be able to convince other validator and be rewarded. Later if one want to change record that I transfer the house to C instead by compromising one validator, this one needs to convince (all) other validators to agree to his up-to-date and longest ledger. The longer the chain means he would need to do push up more times and still the fastest! The proof of work is to make this task impossible, statistically.
One question remained is how to validate a transaction against the current state of ledger? Meaning that if I hear A transfer to B, how can I validate A’s ownership of the asset against my best knowledge? To be explored.
Second concern is exactly on the modification of existing data. How would that be achieved? By B transfer back to A and then A to C again?
Lastly, enhancing the ownership hash to something more complex while keeping blockchain as the infrastructure, we get to smart contract and more fancy stuffs.
(*) mainly after reading the original paper “Bitcoin: A peer-to-peer electronic cash system” by Satoshi Nakamoto
I was confused at first to understand the two concepts of bitcoin and blockchain. Slowly, I start to realized the two pillars of bitcoin (or any blockchain applications would required): crytocurrency & blockchain (*)
Which problems that crytocurrency has tackled?
Ownership without custodian/ centralized party
Based on the application of public-key cryptography: system that uses pairs of keys: public keys that may be disseminated widely paired with private keys which are known only to the owner.
There are two functions that can be achieved: 1) using a public key to verify the sender (digital signature); or 2) encrypting a message with a public key to ensure that only the holder of the paired private key can decrypt it (public encryption).
Example: Transfering from A to B of digital coin $latex Coin_{k+1}=Sign(Hash(Coin_k, P_B), S_A)$ in which $latex P_B$, $latex S_A$ are the public key and private key of B and A respectively. Imagine it is like you put the receipient name on a dollar note and sign on it to transfer the ownership.
By this mechanism, everyone can verify A transfer to B “something” using A’s public key and only B can decrypt and hash/sign next time if B wants to transfer to C.
Not sure how but we need to also ensure that everyone can validate that this asset no longer belongs to A but owned by B by querying the ledger?
(*) mainly after reading the original paper “Bitcoin: A peer-to-peer electronic cash system” by Satoshi Nakamoto
Harder trades/goods movements + value of pound (people’s belief) drops => Difficult import & cheaper export => Higher inflation (still difficult to say! – a value of pound become less in future)
Due to inflation, mortgage cost increases – depending on monetary control.
Hence house price drop
Trade tariff with EU (no idea on this…)
Cross-border business: Corporation/company leaving UK due to restriction from the separation.
UK Equity – short term drops due to behavioral finance. Long term depends on company’s business sensitivity to Import/Export
Revised Trade Agreement with EU and other countries
What is block-chain?
Block-chain is a technology to store information with 3 key characteristics:
Distributed (so called “distributed ledger”)
Secure: only the owner & authorized parties can read the relevant records and not from others. No administrator rights.
Non-reversible: data/information is stored in the system in the form of blocks that linked to each others in the chronological order. It is not possible to modify previous blocks once it is committed.
Why block-chain?
Borrow words from “Global Insight: Blockchain in Banking:Disruptive Threat or Tool?“. I prefer the approach of viewing Blockchain as a new database technology rather than other interpretations like peer-to-peer secured information sharing which has been applied for sending money (bitcoins).
“…Because, in fact, blockchain technology – although when you get into the technology itself, there is some complexity there and some ingeniousness – is nothing more than a new form of database … approach to database architecture, that is. Fundamentally, an improvement over the way that, traditionally, databases have been designed and used in the past.
So, what’s a database? You know, it’s a place where we store important information that we want to refer back to for various purposes, at some point in the future. Visually, you can imagine it as a cylindrical thing. It’s owned by someone, it’s administered by someone, it typically exists in a physical location. The administrator of the database is very interested in investing and ensuring that the perimeter security to that cylinder that protects the contents is strong, impenetrable. The lucky ones are the ones that know that it turned out that penetration happened, and then there’s the rest of us who just don’t know yet. And, typically, the information that’s kept in databases is raw information, not encrypted, not further protected other than the perimeter security which, increasingly proves to be penetrable. So when someone gets in there, that isn’t authorised to do that, the consequences can be dire, whether it’s the theft of someone’s identity for theft of funds or what have you, you read about this every day. So that’s traditional database architecture, and pretty much everybody keeps databases because they have important information about their activity, their client’s activity, that they need to keep a record of, and as a matter of fulfilling their fiduciary responsibilities, they need to keep that record.
Interaction between entities who keep their own separate databases requires a process of taking information from that database, sending it off to another party, for whatever the relevant purpose is, independently validating, verifying and then, of course, when the two sets of information aren’t consistent, reconciling the reasons for the differences. And, simply put, that reconciliation process between different renditions of the same data that needs to be communicated between parties accounts for tens of billions of dollars of cost every year, in the financial manufacturing process.
So that’s the way things have traditionally worked in financial database architecture for years, and along comes a new technology, this is blockchain technology or distributed ledger technology, which makes a few fundamental changes to what I just described. The first thing is that instead of this cylinder in one place, you have a network with multiple nodes, and you have a technology which enables the multiple nodes in that network to be accessed by those with a need and right to know it, and for ensuring that the data, the important information that we’re all referring back to, can be kept in sync and replicated continuously, on a real-time, or as near as need to real-time basis, between parties.
And protecting all of this is the fact that the data itself is encrypted, and the identity of the users or those who have the right, need and right, to access that information, is subject to a sophisticated identity management regime involving, again, cryptographic tools, so that it is very possible to independently verify whether or not an actor is who they say they are and has the right to access that information. And, simply put, what that allows you to do, for the first time, is responsibly share, mutualise, common infrastructure and use this database environment as a place to keep a record of important transactional information that is one record, a golden or prime record, that can be shared by multiple, independent entities, that is protected through cryptography, so that only those with a need and right to know, only the piece of information in that environment that they have the need and right to know, can do that.”
How Blockchain works?
There is so much “noises” around the topic of “explaining” how Blockchain works and I’m still in the process of distilling the information. I will start with the view of distributed database and then discuss on the applications around bitcoins, payments in existing currency, trade confirmations, reconciliation in general to some fancier concepts like clearing, “smart contract“(*/**). A bit ambitious I guess…
What I often forget?
What we mean exactly by distributed? “Everyone keep the DB on their personal computer?”- really?
Database run with administrative cost / maintenance cost. Who paid and who gets rewards?
Delay in validating transactions
Cost (electricity / computation power) to validate transactions
Intrinsic value of distributed ledger – what are the benefits?
In a cryptocurrency system, miners are incentivized to create blocks to collect two types of rewards: a pre-defined per-block award, and fees offered within the transactions themselves, payable to any miner who successfully confirms the transaction. (Wikipedia)
Video references:
Bitcoin: Transaction block chains: The mechanics of a bitcoin transaction block chain, which is a construct that is generated by bitcoin miners and functions as a global ledger for recording and validating bitcoins.