Protocol updates
What's new on the infrastructure updates
Last updated
What's new on the infrastructure updates
Last updated
The following description of updates should be considered a summary, and it assumes some knowledge of how the DKG V6 system works. The details of each of the features will be further explained in the dedicated documentation pages.
The major improvement V8 brings is orders of magnitude higher throughput. Improving from V6, which peaked at 100k Knowledge Assets per day, we expect OriginTrail V8 to be able to reach up to 1000x that number, enabling Internet-scale neuro-symbolic AI.
The two major features enabling scalability are:
Batch minting of Knowledge Assets, which enables minting hundreds of Knowledge Assets in 1 transaction (compared to previously minting just a single Knowledge Asset), as well as easier creation of whole collections of Knowledge Assets. Together with the new Edge Node’s Knowledge Mining API, it will now be very easy to create hundreds of Knowledge Assets from various data inputs such as PDFs, CSVs, JSON, and many others. The capability comes together with an updated version of Knowledge Assets, explained further below.
Random sampling proof system, which makes the protocol much less gas intensive through an optimized proof system and lower complexity of implementation. This means the:
V6 service agreements are being replaced by a random sampling algorithm, which periodically proves randomized elements of the DKG by Core Nodes, which significantly lowers the protocol blockchain footprint. This will unlock significant capacity on all the blockchains connected to the DKG, significantly lowering protocol required node transactions.
Neighborhoods are superseded by sectors: In V6 nodes have been organized in a p2p hash ring with fluid ranges called “neighborhoods”. A neighborhood was formed of 20 nodes closest in the hash ring (in terms of hash distance) to the Merkle root hash of the currently published Knowledge Asset. In V8, nodes are no longer grouped in neighborhoods, but rather in fixed “sectors”. A sector is implemented as a fixed set of Core Nodes (with hash distance no longer being relevant), and all the nodes active in V6 (with over 50k TRAC stake) will be automatically transferred to the initial V8 sector. Sectors will scale over time by fractal division (once a sector grows too big, it will be split in half). All nodes in a sector host the entire sector of the DKG. It is expected that the initial V8 sector will be the only sector and will not split for some time (several months, or even years), and once more sectors are available on the network, it will be possible to have Core Nodes join multiple sectors.
Epochs: Knowledge Asset lifetime and publishing fees will now follow synchronized epochs, rather than each Knowledge Asset Service Agreement having its own “timeline of epochs”. This makes the system simpler to track, implement, and understand. Together with this update, epochs in V8 will also be shortened from 3 months to 1 month.
Efficient reward collection: Nodes will collect rewards more gas-efficiently, as they will be able to collect fees for multiple proofs at once, saving on gas costs. Node runners will be able to configure the frequency of reward collection for their node
DKG V6 introduced Knowledge Assets for the first time - a revolutionary AI primitive initially implemented as 'containers of knowledge'. More specifically, a Knowledge Asset in DKG V6 is implemented as a binding between a triple store named graph, containing as many triples as one wants, and an on-chain ERC721 NFT containing the Knowledge Asset Merkle root. The Knowledge Asset is addressed by the use of a special kind of ownable URI - a Universal Asset Locator, or UAL, which is provably globally unique, is a qualified Decentralized Identifier (DID), and is transferrable (via ERC721).
Knowledge Assets have been deployed productively in various contexts, and have yielded two key feedback points from builders:
For most of the builders, it was more intuitive to think in terms of having one Knowledge Asset correspond to one real-world entity (building, train, wheel, song, etc.). Having the ability to put multiple entities in a “container of knowledge” (Knowledge Asset in V6) was often not used because of this intuition, and likely as it requires a deeper understanding of the underlying implementation, such as the concept of named graphs.
More importantly, from a semantic perspective, pointing to a UAL of a named graph is much less practical than referring to a UAL of an actual knowledge graph (KG) entity when building a knowledge graph or a paranet.
Combining those two learnings with the scalability improvements, the DKG V8 brings an updated version of Knowledge Assets, which are now knowledge graph entities. Named graphs are now used to implement “Knowledge Collections” (equivalent to NFT collections) that can still be referenced by a UAL, and each resource in the graph becomes a Knowledge Asset, represented by an ERC1155Delta standard token on-chain. This satisfies both feedback points 1 and 2, making Knowledge Assets more intuitive, semantically useful, and easier to operate with.
Given the scalability updates and the new paradigm introduced with Knowledge Assets in V8, it is expected that the cost of publishing per Knowledge Asset will go down significantly.
The V8 staking system will also be improved in many ways, more specifically in terms of visibility and transparency of the system on-chain, as well as user experience and interface improvements.
The V8 staking system is inspired by Uniswap V3 implementation, just as the V6 system was inspired by Uniswap V2 by adopting the concept of “node share tokens”. In V6, node share tokens are ERC20 tokens that represent the “share” of one’s delegated TRAC stake in a node - they are minted when TRAC is delegated to a node and burned when one withdraws it.
In V8, the share tokens are going away, being replaced by a more optimal system. This will however require a migration of staking tokens by all V6 delegators. The updated Staking Dashboard will facilitate this migration, which will require delegators to submit several transactions that will burn their share tokens and account for the stake in the new V8 smart contracts. The V8 staking mechanism remains fully non-custodial.
IMPORTANT: Note that this migration can be done at any time - if you are a stake delegator in V6, your TRAC stake will still be accruing rewards and accounted for in the contracts after the V8 update. The migration is only required for delegators to be able to interact with the new V8 system (to delegate TRAC, withdraw rewards, etc.).
The new Staking Dashboard will also provide new statistics and ways to measure your TRAC stake performance, according to the updated tokenomics described in OT-RFC-21, and will be upgraded, specifically during the Tuning period.
The DKG V8 introduces a new concept of a DKG Edge Node, designed to run the DKG on edge devices such as laptops, phones, and others. DKG Core Nodes on the other hand are designed to host the DKG and form a secure network backbone. The new Edge Node services include neuro-symbolic services such as the Knowledge Mining API which facilitates Knowledge Asset graph creation pipelines, the DRAG API which provides a framework to perform Decentralized Retrieval Augmented Generation (DRAG) on the DKG, multi-modal LLM interfaces, a customizable UI, a publishing service and more.
Edge and Core Nodes share most of their code and services, and as a builder, you can extend and combine them in different ways, including creating your custom services either for your node, or paranet. The key differences between Core and Edge Nodes are in their mode of operation:
DKG Edge Node:
It is designed for privacy-preserving applications, where private Knowledge Assets, the use of local LLMs, and knowledge pipelines enable the utmost privacy-preserving environment.
Is not intended to host the public DKG and, therefore, does not require a TRAC stake collateral. The Edge Node is therefore also not capable of collecting publishing fees & network rewards.
Does not require a high uptime as the DKG Core Node does.
Can be used as a substrate to quickly create neuro-symbolic AI applications based on the DKG paranets.
Can be “converted” into a Core Node (interesting for builders of Paranets who would like to recuperate some of the publishing fees by running a Core Node), assuming the right conditions
DKG Core Node:
It is designed to host the public DKG and provide high availability, so is incentivized to provide high uptime.
Requires a minimum of 50,000 TRAC token stake to be eligible for joining the network, however, with a higher stake Core Nodes increase their reward chances.
Can be used as a “gateway node” for publishing, and with a higher publishing rate, the chances of network rewards grow.
Performs the network services for a fee, determined globally by the pricing mechanism from all the individual Core Node asks. However, nodes with lower asks (less “greedy” nodes) have a higher chance of receiving network rewards.
More details on the system are provided in the OT-RFC-21.
For more information on the DKG Edge Node visit this page, or see this talk at DKGCon2024.
With the protocol tokenomics upgrade, the DKG publishing fee system is also being optimized. With learnings from V6 and removing neighborhood pricing complexities, the new system of “sectors” relies on a simpler and more robust implementation of pricing on the network. The new implementation is expected to solve the reward “cliff effect” (where only nodes with close to the maximum stake were receiving rewards), as well as solve some recurring problems with obtaining the right market price.
To determine the price of network service in V8 in a specific sector (V8.0 is starting with 1 sector), a sector-wide DKG publishing fee is computed based on individual node asks. To achieve a price equilibrium, the V8 pricing system utilizes the statistics-based sigma approach, determining the publishing fee as a stake-weighted average ask of all Core Node asks within 1 standard deviation from the current price equilibrium. That is a mouthful :), so here’s the formula:
for all nodes which satisfy the
That means the fee will also be available on-chain and can be easily read, which will eliminate the previously encountered “bid suggestion” issues that would occur in V6.
You can manage your Core node ask in the new Staking Dashboard.
As a key implementation component, requiring real economic conditions for proper testing, the pricing system will be improved during the Tuning period.
The OT-RFC-21 has laid out the V8 updated tokenomics with additional factors incentivizing positive behavior on the network. It sets 4 key factors for Core Node performance: the amount of TRAC stake, the amount of publishing performed through the node, the node ask, and total uptime. We encourage exploring the RFC further until the detailed documentation for the implementation is published.
where is the standard deviation across the population of all node asks. If a Core Node ask is outside of the bounds, it will therefore not influence the DKG service fee.
