15/07/22
Crust Network: IPFS-based Decentralized Cloud Storage
Category
Written by
Read time
8 min
Crust Network aims to be the preferred storage network for Web3, acting as an interconnectivity and compatibility layer between blockchains and its own decentralized storage blockchain that interfaces with IPFS for file storage. Crust Network is built on the Substrate framework, primarily known from the Polkadot ecosystem.
Token
The CRU token is a utility token with the main functions of purchasing storage services and maintaining blockchain consensus through staking.
Storage Technology & Consensus Mechanisms
Crust Network is built on IPFS and participates in the public IPFS DHT. This means Crust Network IPFS nodes can be found by, and store CIDs and data of, non-Crust IPFS nodes. Crust further operates a network of IPFS public gateways to transmit and receive files.
IPFS itself, however, does not incentivize long-term storage of data nor does it incentivize the transmission or replication of data. Simply speaking, IPFS can be considered a data transfer and retrieval framework. This is where the Crust Network protocol comes in. The Crust Network protocol essentially manages and incentivizes two networks simultaneously that nodes can mine;
- the Crust Network blockchain (aka “verification network”),
- the storage network, which uses IPFS
Furthermore, Crust Network has two consensus mechanisms that run in parallel and require different proofs form the network:
- Meaningful Proof of Work (MPoW): provides technical assurance for the trusted execution of code in the Trusted Execution Environment (TEE)
- Guaranteed Proof of Stake (GPoS): manages storage orders and maintains blockchain consensus
MPoW is responsible for the generation of two proof types that require network consensus, namely environment proof and workload proof, and runs within a node’s TEE. Simply put, the TEE is a hardware computing module on physical devices that is separated from the core processing units of the device. This module allows for verifiably untampered computing and can be used to retrieve information about the system or be used for complex calculations. The only limitation of joining the Crust Network therefore becomes whether the mining hardware supports TEE.
The environment proof ensures that new nodes that join the network have a valid TEE instance, by using a public key generated within the TEE and other nodes on the network executing a remote attestation procedure. This uses the public key and data produced by the new node and compares this against an expected result. If the results match, the new node passes the environment proof and the node identity and the TEE public key will be recorded on the Crust Network blockchain.
The workload proof deals with both the storage of new files, and also handles random data checks executed by the network. When a user stores a file on Crust, an operation within the TEE called the inspector will split the file into multiple pieces, which are each encrypted and hashed into a Merkle tree. As a result, in a node’s external storage a node can only see encrypted file fragments and is not able to reconstruct the files stored on their own. When random data checks are initiated, the inspector algorithm within the TEE retrieves a random Merkle tree hash and relevant file fragment which is decrypted and re-hashed. Finally, the new hash is compared against the expected hash. The inspector not only checks file fragments stored on the nodes, but also the storage capacity of the node. Together these actions form a work report which is signed, verified by other nodes and recorded on the blockchain. If the file fragment cannot be retrieved or otherwise misbehaves, staked CRU is confiscated with a portion of the confiscated tokens being burned and the remainder being allocated as block rewards – more details regarding staking in Guaranteed Proof of Stake explanation further below.
While in this model Proof of Work calculations are still executed, only meaningful calculations based on changes to the node’s storage state and validated through TEE are executed – hence the name Meaningful Proof of Work.
The workload proof also has a Proof of Running Tracking (PoRT) algorithm which enables the tracking of computational workloads. Although not activated on the Crust Mainnet at this point in time, this algorithm can enable decentralized computing.
The Guaranteed Proof of Stake (GPoS) consensus algorithm, which is essentially a proof of stake algorithm where nodes can only stake CRU tokens if they have provided storage capacity to the network, whereas staking capacity increases based on the total storage capacity provided to the network and proof that the nodes actively process storage orders from the network. In the network, MPoW generates environment and workload reports which are submitted to verifier nodes in the network as transactions, and using the GPoS consensus mechanism these verifier nodes then generate blocks and write them to the Crust Network blockchain. It should be noted here that by definition, verifier nodes must also provide storage resources (hence Guaranteed Proof of Stake).
Within the GPoS consensus mechanism, a further node type is distinguished: candidate nodes. These are verifier nodes that are not qualified for block generation due to insufficient staked CRU tokens. At the end of each block generation cycle, candidate nodes may become verifier nodes if the meet the required staking quota.
Finally, the last two participants in GPoS are guarantors and storage users. Guarantors can provide their CRU tokens to nodes on the network to help them increase their stake in order to reach the required staking quota to become a verifier node. By doing this, guarantors can earn guarantee income. Storage users are users on the network that pay in CRU to store files.
Below is a summary of the network actors and token flows:
- Verifier nodes:
- Block packaging rewards (network token emissions, storage fees reward pool), enabled through sufficient token staking
- Block cycle rewards (network token emissions)
- Storage income (miner storage rewards)
- Candidate nodes:
- Block cycle rewards (network token emissions)
- Storage income (network token emissions)
- Guarantors:
- Guarantee income (from verifier node block packaging rewards)
- Users:
- Storage fees spending (users’ storage fees spending split into miner storage rewards (20%) and staking reward pool (80%))
In summary, the MPoW algorithm ensures storage operations are validly and verifiably executed at a node level, and the GPoS algorithm is responsible for accepting and bundling storage work reports from nodes in the form of transactions and writing them to blocks. GPoS is hence the mechanism that brings together the various actors in the network.
On top of the MPoW and GPoS consensus mechanisms lives the Decentralized Storage Market (DSM), which is an algorithm that lives is executed on the nodes and is responsible for storage pricing, processing storage orders and file retrieval operations.
When a user wants to store a file on a node, the user is given a dynamically calculated storage price based on storage requirements and details of the data to be stored. The user then submits the storage request to the network with storage fees attached. The network accepts the files and transmits them to a node through IPFS, which saves the files locally using the TEE and MPoW algorithm. The storage proof is then generated (including proof of files stored and remaining storage capacity), verified by other nodes on the network and submitted to the GPoS algorithm. The GPoS algorithm locks the valid proofs to the blockchain and pays out relevant rewards to nodes that hold the files.
To ensure nodes do not only store files and are unwilling to transmit files when requested by nodes or users, a nodes ability to receive storage orders is linked to their willingness to transmit files. Crust implements the IPFS BitSwap credit system between nodes, in which nodes track the number of storage requests that were fulfilled by other nodes to them. Over time, nodes will either build credits (data shared) or build debt (data received). Selfish nodes that build debt with other nodes will eventually not receive any further files until they have replenished their credit. This essentially incentivizes nodes to always try to maintain a high credit, because only nodes with sufficient credit are awarded new storage orders. Also, before a retrieval request is actioned the node will check the Crust Network blockchain to see whether the file is indeed a valid file on the network. This ensures nodes’ bandwidth isn’t wasted. It should be noted here that only the four fastest merchants that can provide valid storage proofs are given storage rewards, further incentivizing fast node bandwidth.
When a file is retrieved on IPFS the file fragments are retrieved from a number of nodes to speed up the download process and are reconstructed on the node which issued the retrieval request. If a user requests a file through a gateway, the gateway reconstructs the file before passing it on to the user using the HTTP protocol (as described in the IFPS section).
Saving the real-time status of all files onto the Crust blockchain allows for the DSM to count all saved instances of files across the entire network, as well as retrieve their storage locations. As a result, the network is able to calculate the replication status of all files. Currently a mechanism to improve the replication status and hence the resilience of data on the network is being designed that will allow for dynamic replenishment of files on the network.
Summarizing the above; MPoW ensures file storage operations are valid, GPoS ensures at a network level actors are rewarded for their various roles in the network, and DSM acts as user/application interface and the coordinating mechanism between GPoS and MPoW.
Storage Process & Pricing Mechanism
In Crust Network, a storage user connects directly with Crust Network’s Decentralized Storage Market (DSM), which dynamically calculates the network rate for storage based on file size, quantity and certain network metrics. On the DSM, the users directly settle storage fees based on their storage requirements and store files directly. The fees involved for storing on Crust include:
- Byte fee: fee levied on storage size and quantity of files
- On-chain state fee: fee for every storage order placed
- Basic fee: fee charged for the duration of the contract in 6-month increments
Formula:
Whereas storage duration is calculated in 6-month increments.
On Crust Network, the storage costs are based on network activity and denominated in CRU. If more files are stored, the price increases and if fewer files are stored the price decreases – similar to the relationship between Ethereum’s base fee price for gas units. That means if the price of the CRU token changes over time, the USD cost for storage will increase, while generally it can be expected that storage fees remain more stable.
Tokenomics
Initial pool of 20 million CRU, split as follows:
- 5 million CRU for community development (25%)
- 2 million CRU for ecological growth (10%)
- 5 million CRU reserved for investors (25%)
- 4 million CRU for the technical team (20%)
- 4 million CRU foundation reservation (20%)
New tokens are issued on an annual basis with an annually reducing inflation coefficient. All newly emitted tokens are mining rewards for active nodes.
References
All references were accessed between May 5th and 31st, 2022.
Crust Network (2020) Crust Technical White Paper v1.9.9. Available at: https://gw.crustapps.net/ipfs/QmP9WqDYhreSuv5KJWzWVKZXJ4hc7y9fUdwC4u23SmqL6t
Crust Network (2021) Crust Token Metrics & Economies. Available at: https://medium.com/crustnetwork/crust-token-metrics-economics-84592efc6d1f
Crust Network (2021) Economy Whitepaper. Available at: https://gw.crustapps.net/ipfs/QmRYJN6V5BzwnXp7A2Avcp5WXkgzyunQwqP3Es2Q789phF
Crust Network (n.d.) DSM. Available at: https://wiki.crust.network/docs/en/DSM
Crust Network Subscan (n.d.) Storage Price Calculator. Available at: https://crust.subscan.io/tools/storage_calculator
SHARE THIS PIECE
Related content
In this piece @lingchenjaneliu explores the staking market landscape, including key actors and top players, as well as latest developments in Proof-of-Stake. Special thanks to @0xPhillan for ...
The year of 2022 was a year of great highs and new lows for the Web3 industry. Read more about the defining events of 2022!
Metaverse Concept and History The term "metaverse" was invented by American author Neal Stephenson in his science fiction novel titled Snow Crash, published in 1992. ...
Short answer: Account abstraction creates a new account type which exists as a smart contract. By having the account exist as a smart contract, transaction ...