• Call Us: +1 240 200 6131 789
  • - Mail Us: hello@ourdomain
  • Falls Church, Virginia USA
Linkedin
Online Courses
Free Courses

How Sharding and Gasper work in Ethereum

Note

If you are new to the blockchain technology, taking our Introduction to Blockchain Technology self-paced course is highly recommended. Also, for a comprehensive coverage of blockchain development in Ethereum or mastering Solidity programming, taking our below self paced courses is highly recommended:

Recap

In our previous article (How Plasma Chains Work in Ethereum), we covered how Plasma Chains work in Ethereum.

In this article, we learn how Sharding and Gasper work in Ethereum.

Ethereum sharding and Casper

As we mentioned earlier, sharding has been implemented in many distributed database systems. Essentially, sharding is a particular method for horizontally partitioning a large dataset within a database. More specifically, the database is broken into little pieces called shards. When aggregated together, they form the original database. In the following diagram, one large dataset can be sliced horizontally into two or more partitions, and each partition may be stored in separate database instances:

Ethereum blockchain development

 

In a decentralized blockchain network, the network consists of a series of nodes connected in a peer-to-peer format, with no central authority. As is the case with current blockchain system, each node stores all states of the network and processes all of the transactions.

In order to match transaction scalability and throughput from Mastercard or Visa, Ethereum will have to process large volumes of transactions much faster. One logical way we discussed earlier is to increase the block size, but this may not be able to fundamentally address the scalability issues, since every node still needs to validate and verify a large block of more transactions. They will need more computing resources to handle the increased load. That is where sharding comes to help.

Not only the large volume of transactions can be sharded, but the entire peer-to-peer network of nodes can be grouped into smaller groups, also called shards. This greatly improves the parallel execution in the network, and scalability.

Imagine at any period, in order to process a large volume of transactions, you would need to have a really large block to be able to hold all the transactions submitted in this period. Instead of creating a larger physical block, what if you slice them into different smaller blocks, as in the following diagram, and have smaller blocks stored in the subset of network nodes? Similar to database sharding, in order to find all transactions and verify any particular transactions, you have to aggregate the transactions from all shards for that period. When you aggregate all transactions, you will have a virtual large block:

Ethereum blockchain development

It looks like a simple concept, but complexity arises when dealing with maintaining the integrity and security of the Ethereum world state and transaction ledger across a subset of network partitions. Like any other blockchains, Ethereum maintains transactions and their ordering in the chain of blocks through a consensus mechanism. Furthermore, Ethereum also manages the account as a state object in the work state. A state object makes things much more complicated.

Let us take a look at how the transactions and data work in Ethereum sharding. As depicted in the following diagram, let us say the network is split into multiple groups and each group is an Ethereum shard. Transactions pertaining to each shard will be collected and added to the shard chain. A group of validators in the collator pool, stake their coins and become the validators in the PoS system. Let’s take a look at the following diagram:

 

Ethereum blockchain development

Shard chain is similar to blockchain in concept. The miner’s role in Ethereum is split into three separate roles: collation proposer, collator, and executor. A collation is the same as a block in the blockchain. As the following diagram shows, at any time period, a collation proposer collects all transactions, and adds them to a proposed collation. Data blobs from transactions are packaged together into the collation body, a Merkle tree of the data blobs, and other shard identifier information is packaged as the collation header.

The collation proposer proposes the proposed collation to the collator, which is randomly selected from the validator pool. The collator’s role in this process is to validate the proposed collations and pass the collation header to smart contracts through the executor on the main chain. The collation headers from all shards will be added to blocks on the main blockchain:

Ethereum blockchain development

The smart contract is also called a validator manager contract (VMC), as depicted in the preceding diagram. It is a shard-specific smart contract, deployed on the main chain. The VMC main role is to maintain the entire sharding systems and act as the connection between the main chain and the shard chain it links to. Its responsibilities include the following:

  • Validate and maintain collation headers from each shard
  • Manage a pool of collation proposers
  • Manage a pool of collators and hold the collator’s stakes to enforce PoS and perform random sampling of collators for each shard
  • Shuffle and pseudo-randomly select an eligible collator for each shard
  • Process cross-shard transactions and record all cross-shard receipts

Cross-shard transactions, as discussed earlier, are one of the challenges that the sharding system faces. When a transaction happens across different shards, it is typically implemented through the cross-shard communications of transactions and receipts.

For example, suppose Alice from shard 1 needs to pay 100ETH to Bob on shard 2. The single transaction will be broken into two separate transactions: one is a debit transaction to draw 100ETH from Alice’s account on shard 1, and another one is the credit transaction of 100ETH made to Bob on shard 2. The following are the steps involved:

  • One debit transaction of 100ETH deducted from Alice’s account on shard 1 is submitted to the main chain. A receipt is generated once the transaction is verified.
  • Wait until the debit transaction is finalized.
  • Submit the second credit transaction to deposit 100ETH to Bob’s account on shard 2, along with the original receipt from the debit transaction.
  • A second receipt is generated once the debit transaction is processed. The receipt is sent back to Alice on shard 1, so that further account action can be taken.

 

Sharding, along with the PoS consensus mechanism, has the potential to drastically improve Ethereum’s scalability and increase the transaction throughput in Ethereum. There are too many moving parts in combining both layer 1 scalability solutions and creating a solid implementation. It has been handled in multiple phases, we will go over the current Ethereum 2.0 roadmap in later articles to discuss which different components should be implemented and when.

 

Next Article

In our next article (How Proof-of-Stack Consensus Works in Ethereum), we discuss how the Proof-of-Stack consensus is used in Ethereum.

This article is written in collaboration with Brian Wu who is a leading author of “Learn Ethereum: Build your own decentralized applications with Ethereum and smart contracts” book. He has written 7 books on blockchain development.

Resources

Free Webinars on Blockchain

Here is the list of our free webinars that are highly recommended:

 

Free Courses

Here is the list of our 10 free self-paced courses that are highly recommended:

 

Self-Paced Blockchain Courses

If you like to learn more about Hyperledger Fabric, Hyperledger Sawtooth, Ethereum or Corda, taking the following self-paced classes is highly recommended:

  1. Intro to Blockchain Technology
  2. Blockchain Management in Hyperledger for System Admins
  3. Hyperledger Fabric for Developers
  4. Intro to Blockchain Cybersecurity
  5. Learn Solidity Programming by Examples
  6. Introduction to Ethereum Blockchain Development
  7. Learn Blockchain Dev with Corda R3
  8. Intro to Hyperledger Sawtooth for System Admins

 

Live Blockchain Courses

If you want to master Hyperledger Fabric, Ethereum or Corda, taking the following live classes is highly recommended:

 

Articles and Tutorials on Blockchain Technology

If you like to learn more about blockchain technology and how it works, reading the following articles is highly recommended:

 

Articles and Tutorials on Ethereum and Solidity

If you like to learn more about blockchain development in Ethereum with Solidity, reading the following articles and tutorials is highly recommended:

 

Articles and Tutorials on Hyperledger Family

If you like to learn more about blockchain development with Hyperledger, reading the following articles and tutorials is highly recommended:

 

Articles and Tutorials on R3 Corda

If you like to learn more about blockchain development on Corda , reading the following articles and tutorials is highly recommended:

 

Articles and Tutorials on Other Blockchain Platforms

If you like to learn more about blockchain development in other platforms, reading the following articles and tutorials is highly recommended:

Tags: