> For the complete documentation index, see [llms.txt](https://docs.origintrail.io/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://docs.origintrail.io/use-dkg/storage-sparql-http.md). # Storage SPARQL HTTP The DKG node can use any **SPARQL 1.1 Protocol**–compliant store you run yourself, instead of its default daemon-managed local Oxigraph server (or the embedded `oxigraph-worker` fallback). That gives you: * **Real on-disk persistence** (e.g. Oxigraph server with RocksDB) * **Larger graphs** without holding everything in the Node process * **Existing infrastructure** (GraphDB, Blazegraph, Jena Fuseki, Neptune, Stardog) ## Backend: `sparql-http` Configure the node to use the **`sparql-http`** backend with a **query endpoint**. `updateEndpoint` is optional and defaults to `queryEndpoint`, which covers stores that use one URL for both query and update. ### Config (CLI / config.json) In `~/.dkg/config.json` (or your `DKG_HOME` config): ```json { "name": "my-node", "apiPort": 9200, "listenPort": 9001, "store": { "backend": "sparql-http", "options": { "queryEndpoint": "http://127.0.0.1:7878/query", "updateEndpoint": "http://127.0.0.1:7878/update" } } } ``` Optional: * **`updateEndpoint`** — SPARQL update endpoint. Defaults to `queryEndpoint` when omitted. * **`timeout`** — request timeout in ms (default `30000`). * **`auth`** — `Authorization` header value, e.g. `"Bearer "` or `"Basic "`. * **`consistencyProfile`** — the transactional guarantee your endpoint provides. Defaults to `"best-effort"`. See below. ### Transactional guarantees (`consistencyProfile`) The node cannot infer what runs behind a SPARQL URL, so an explicitly configured `sparql-http` endpoint is assumed to give no transactional guarantees. Declare what your server actually provides: | Value | Meaning | Unlocks | | ------------------------- | ----------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------- | | `"best-effort"` (default) | No guarantee is claimed. | — | | `"atomic-update"` | A whole multi-operation SPARQL Update applies as one transaction. | Atomic data/metadata replacement, required by RFC-64 graph-scoped materialization and durable sync. | | `"atomic-readback"` | Adds that a query issued after a completed update observes that update. | Everything above, plus receipt-bearing author-commit CAS. | ```json { "store": { "backend": "sparql-http", "options": { "queryEndpoint": "http://127.0.0.1:7878/query", "updateEndpoint": "http://127.0.0.1:7878/update", "consistencyProfile": "atomic-readback" } } } ``` A single-process Oxigraph server (the `oxigraph serve` command below, or the same binary in Docker) provides `atomic-readback`; the daemon-managed `oxigraph-server` backend declares it for you. Declare it only when your deployment really provides it — the setting states a guarantee, it does not relax a check. A federating proxy, a read replica or a multi-writer cluster in front of the endpoint may not qualify. {% hint style="warning" %} Leaving this at the default on a node that syncs RFC-64 Context Graphs means every graph-scoped materialization is refused with `VM_ATOMIC_REPLACE_UNSUPPORTED`, and the node converges nothing while its peers look healthy. {% endhint %} ### Oxigraph server 1. **Install and run Oxigraph** (Rust binary with RocksDB): * Download from [oxigraph/oxigraph releases](https://github.com/oxigraph/oxigraph/releases) or build from source. * Run the server, e.g.: ```bash oxigraph serve --bind 127.0.0.1:7878 --location /path/to/oxigraph-data ``` * Default paths are often `/query` and `/update` (check the server’s docs). 2. **Point the DKG at it** using the `sparql-http` config above with your host/port. 3. Start the DKG node as usual; it will use the remote store for all triples. {% hint style="info" %} For a local Oxigraph server you do **not** have to run it yourself: set `"store": { "backend": "oxigraph-server" }` (the `dkg init` default) and the daemon fetches the pinned `oxigraph` binary, spawns it on `127.0.0.1`, and supervises it. Use the manual `sparql-http` steps above only when you run Oxigraph (or another SPARQL store) yourself or off-host. {% endhint %} Managed Oxigraph accepts optional launch settings under `store.options`: ```json { "store": { "backend": "oxigraph-server", "options": { "port": 7878, "location": "/var/lib/dkg/oxigraph-data", "cacheDir": "/var/lib/dkg/oxigraph-bin", "clientTimeoutMs": 30000, "memoryHighMiB": 2048, "memoryMaxMiB": 3072 } } } ``` The daemon sizes the startup readiness wait automatically from the RocksDB write-ahead log retained in `location`. Oxigraph is not closed cleanly on shutdown (it installs no SIGTERM handler), so every start replays the previous session's log, and that replay is what a fixed deadline used to cut short. The derived wait assumes a pessimistic 4 MB/s replay floor and is not truncated below that work allowance; when a replay is pending the daemon logs the size and the allowance, then reports progress every 10 seconds. `readyTimeoutMs` **overrides** that automatic sizing with an explicit maximum, used verbatim and never extended. It must be a positive integer; invalid values are ignored. You normally should not set it — if you configured it as a workaround for a node that would not start, remove it. When an explicit value is lower than the automatic estimate the daemon logs a warning naming both numbers, because that combination is what keeps a recovering node down. `clientTimeoutMs` is the SPARQL HTTP client deadline in milliseconds. Managed Oxigraph defaults to 30 seconds. Without an explicit native deadline, Oxigraph 0.5.x client deadlines are capped at 55 seconds, ahead of the server's implicit 60-second HTTP cutoff. A client query deadline triggers a supervised database restart, since ending an HTTP request does not prove that its database evaluation stopped. If a caller cancels an in-flight managed read earlier, the caller still receives its cancellation immediately. The adapter retains the original deadline for possible abandoned database work and requests recovery then, unless that database generation has already restarted or the store has closed. This is conservative: an abandoned query may have finished without the client receiving confirmation. Recovery restarts the whole managed database and can interrupt concurrent writes; use the existing indeterminate-outcome/idempotency handling when retrying them. Repeated recovery indicates that the query workload still needs investigation. `queryTimeoutS` opts into the native Oxigraph query deadline (`oxigraph serve --timeout-s`); it is not derived or enabled by default. Oxigraph 0.5.x creates timer threads per query, so enabling long native deadlines under load can exhaust resources. On macOS with 0.5.x the daemon omits this unsafe native option even when configured. On supported combinations an explicit value extends `clientTimeoutMs`, if needed, so the native deadline has at least five seconds to fire first. The client deadline also bounds updates and response decoding, but update timeouts alone do not request a database restart. Restart the node after changing launch settings. Blazegraph uses the same bounded/retryable DKG contract with backend-specific cancellation semantics. Its 30-second client deadline sends a wider 120-second `X-BIGDATA-MAX-QUERY-MILLIS` server bound: the wider server limit caps abandoned work without allowing Blazegraph's mid-stream timeout behavior to be mistaken for a complete CONSTRUCT result. Store admission pressure returns `503 STORE_SCHEDULER_BUSY` with `outcome: "not_started"`; an adapter deadline returns `503 STORE_OPERATION_TIMEOUT`, `retryable: true`, and `outcome: "indeterminate"`. For an indeterminate write, retry the same Knowledge Asset name/idempotency key rather than creating a new asset. On Linux hosts using systemd, `memoryMaxMiB` runs managed Oxigraph in a dedicated transient user scope with a finite hard limit and disables swap for that scope, so database pressure cannot spill into host swap and stall the node. Optional `memoryHighMiB` sets its soft reclaim threshold and must not exceed `memoryMaxMiB`. This keeps Oxigraph outside the DKG daemon service's cgroup, so a finite `MemoryMax` on the daemon cannot kill the store or throttle the Node.js control plane as one combined process group. Enable lingering once for the node service account (`sudo loginctl enable-linger `) so its user manager and bus exist before the system service starts at boot; verify with `systemctl --user is-system-running` as that account. The daemon supplies the required user-bus environment automatically. Startup fails rather than silently dropping configured limits when the scope cannot be created. The scoped watchdog requires util-linux `setpriv` with `--pdeathsig` support on Linux. It installs a kernel parent-death signal so abrupt watchdog death also terminates Oxigraph and releases its database lock. After daemon loss or a forwarded shutdown signal, a child that ignores graceful termination is killed after five seconds. Missing `setpriv` fails startup instead of launching an unprotected database. Without memory limits, managed Oxigraph on Linux and macOS runs under the same parent watchdog, in the daemon's own cgroup and without the `setpriv` requirement. When the supervisor kills an unresponsive daemon worker, which cannot stop its children itself, the watchdog stops Oxigraph within about a second and the respawned worker can open the store. The watchdog notices because it is reparented, so a recycled daemon PID cannot hide the daemon's death. On Windows, Oxigraph is still launched directly. The daemon records who owns the store in `dkg-oxigraph-owner.json` in the store directory. It writes the record right after starting Oxigraph (the daemon, the process it launched, and the binary), then adds Oxigraph itself once the store is ready. Each process is identified by PID and process start time, within the boot the record was written in: after a reboot the record is ignored and only the command and parent rules below apply. Before each Oxigraph start, the daemon stops a process that holds `/LOCK` only in these cases: * The recorded daemon or launcher has exited, and the process is either the Oxigraph named in the record (its PID and start time, still running this node's Oxigraph for this store: a start time alone can repeat within its one-second resolution on macOS), whichever process adopted it afterwards (PID 1, a subreaper such as a `systemctl --user` manager, or a watchdog that cannot act), or, for a launch killed before its store was ready, a child of the recorded launcher that runs this node's Oxigraph for this store. * It runs this node's Oxigraph as `serve --location `, with that binary as the program itself rather than an argument to another program (the recorded binary, the current binary or the `oxigraph` on PATH, or a pinned `oxigraph-vX.Y.Z` download in the managed binary cache), no live recorded owner exists, and its parent is gone: it was reparented to PID 1 or its parent has exited. This covers orphans from releases before the owner record. Command lines are compared argument by argument where the platform exposes them (`/proc` on Linux). macOS shows only the joined command text, so there a store or binary path that contains whitespace is never matched. While the recorded daemon and launcher both run, no holder is touched. Only a confirmed exit counts: if the record exists but cannot be read, or the daemon cannot read a recorded owner, the holder or its parent (a `ps` timeout, a `/proc` read error), it leaves the holder running and logs why. A record with unrecognised content is ignored, as if there were none. Any other process holding the lock is logged with its parent's command and left running. An orphan from an earlier release that a subreaper adopted cannot be told apart from a process with a live owner, so stop it by hand. The daemon never removes or rewrites the LOCK file, and never adopts an older Oxigraph process. Size the daemon service and Oxigraph scope independently, while keeping their combined maxima within host capacity. For example, an 8 GiB node can reserve 3 GiB for managed Oxigraph and place a separate finite 4 GiB cap on the DKG service, leaving roughly 1 GiB for the OS. These options are deliberately opt-in and are not supported on non-systemd platforms. ### Other stores * **Blazegraph:** One URL for both query and update. Set only `queryEndpoint` or set both options to the same URL (e.g. `http://127.0.0.1:9999/blazegraph/namespace/kb/sparql`). * **Apache Jena Fuseki:** Typically `http://host:3030/dataset/query` and `http://host:3030/dataset/update`. * **GraphDB, Neptune, Stardog:** Use the vendor’s SPARQL query and update URLs; add `auth` if required. ### Blazegraph provisioned by `dkg init` (Docker) When you pick the `blazegraph` backend in `dkg init` and leave the URL blank, the CLI offers to provision a container itself. For new containers, the provisioner uses the pinned multi-architecture image and data path declared in `blazegraph-image.json`, stores the journal in a named Docker volume, and uses `--restart unless-stopped`. Blazegraph output uses Docker's compressed `local` log driver with a 4 GB rotation budget (`200m` × 20 files), so an unattended container cannot fill the host disk. The provisioner auto-bumps the host port if 9999 is taken and is idempotent — re-running `dkg init` against an already-provisioned namespace reuses the running container. A reused legacy container is never recreated automatically, because doing so could discard its journal. If its volume or log policy does not match the current configuration, the CLI prints explicit backup/migration and unbounded-log warnings instead. ## Programmatic (DKGAgent) When creating an agent in code, pass `storeConfig`: ```ts import { DKGAgent } from '@origintrail-official/dkg-agent'; const agent = await DKGAgent.create({ name: 'MyAgent', storeConfig: { backend: 'sparql-http', options: { queryEndpoint: 'http://127.0.0.1:7878/query', updateEndpoint: 'http://127.0.0.1:7878/update', }, }, }); await agent.start(); ``` ## Store defaults New installs default to a **daemon-managed local Oxigraph server** (`store.backend: "oxigraph-server"`): `dkg init`, `dkg openclaw/hermes/mcp setup`, or accepting the wizard default writes this block. The daemon fetches the pinned `oxigraph` binary on first boot and runs it on loopback, giving MVCC concurrent reads and incremental RocksDB persistence. If a config has **no** `store` block at all, the runtime falls back to the embedded in-process **`oxigraph-worker`** (a single-writer store that rewrites its on-disk N-Quads dump under `dataDir` on every flush) — fine for development and small nodes. For very large graphs or existing infrastructure, use `sparql-http` with an external store. --- # Agent Instructions This documentation is published with GitBook. GitBook is the documentation platform designed so that both humans and AI agents can read, navigate, and reason over technical content effectively. Learn more at gitbook.com. ## Querying This Documentation If you need additional information that is not directly available in this page, you can query the documentation by asking a question. Perform an HTTP GET request on the following URL with the `ask` and `goal` query parameters: ``` GET https://docs.origintrail.io/use-dkg/storage-sparql-http.md?ask=&goal= ``` `ask` is the immediate question: it should be specific, self-contained, and written in natural language. `goal` is what the user is ultimately trying to achieve, the reason they need the answer. Sharing it helps GitBook give you a better, more relevant answer. A goal is most helpful when it describes the outcome the user wants rather than restating the question. For example, with `ask=how do I create an API token`, a goal like `automate deployments from our CI pipeline` lets GitBook tailor the answer to that use case. The response will contain a direct answer to the question and relevant excerpts and sources from the documentation. Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.