DKG persistance mechanisms
All DH nodes participating in the ODN store the DKG data in two ways:
- by taking responsibility for storing a published data graph through the process of replication, after the process of negotiation and selection by the smart contract randomness beacon called the "offer task". For the service of storing and providing datasets, DH nodes are compensated in TRAC tokens locked at publishing
- by opportunistically storing datasets for which there is a relatively high likelihood of becoming compensated in the future (by replacing a designated DH node). As DH nodes act in the interest of maximizing their service utilization, as long as there is enough free space on their disk, DH nodes will opportunistically store non-designated datasets in their graph databases
Having the above two modes of storing in mind, all DH nodes can perform data pruning (dataset removal) once every 24h in the following situations:
- when a dataset expires, according to the data holding dataset longevity parameter (meaning there is no longer any designated compensation available)
- when the disk space reaches a set threshold, after which the node will prune "low estimated value" datasets. Low estimated value datasets are data graphs for which a node is not a designated DH node and are closest to expiry date (so potential compensation achievable by designation is lowest)
These features have been introduced in v5 and are editable in the node configuration file. Below is an example of config parameters which enable pruning, including pruning of "Low estimated value datasets", which will activate at 50% of hard disk usage.
You will know that pruning command has been started when you see this log:
trace - Dataset pruning command started. This command will work in background and will try to remove expired and low estimated value datasets.