Run MDK on a private HyperDHT

Overview

This page sets up the local and air-gapped profile: every MDK process joins a HyperDHT network you run, binds its DHT node to an interface you choose, and never contacts the public node1/2/3.hyperdht.org bootstrap nodes. It follows hyperdht's own isolated-DHT pattern: one bootstrap node plus persistent DHT nodes, and every component started with that bootstrap list. The profile is HyperDHT configuration only. There is no proxy, relay or second transport in front of HRPC.

Discovery and transport are separate

Discovery is how the Kernel learns a Worker's public key. Transport is how it then reaches that key. Every discovery mode, including the same-process case, uses HRPC over HyperDHT, so every mode needs a reachable bootstrap. Moving to a private network changes the bootstrap, not the discovery mode.

A same-process Kernel and Worker still open a HyperDHT connection to each other. Without a reachable bootstrap, public or private, that connection never forms and the Worker never reaches READY. The deployment topology concept covers the three shapes in depth.

Choose a bind interface

dhtHost is the interface each DHT node binds (hyperdht's host option). Without it a node binds every interface.

Every process in the stack needs the same bootstrap list. The Kernel, Workers, Gateway, app runtimes, the MCP server and CLI clients each start a DHT node, and one process left on the defaults joins the public network.

Pass the profile to each component

startGateway({ bootstrap, dhtHost }) covers all three DHT nodes the Gateway starts: its net facility (the key the Kernel and Workers admit), its Kernel client and its app runtime client. Bootstrap lists that name a public *.hyperdht.org node are refused with ERR_MDK_DHT_BOOTSTRAP_PUBLIC.

Run the profile

Each subsection is an alternative; pick the one that matches how the stack runs.

A. Programmatic, one machine

Start the bootstrap first and hand its list to every component:

const { startDhtBootstrap, getKernel, startGateway } = require('@tetherto/mdk-core')
const { startAvalonWorker } = require('@tetherto/mdk-worker-avalon')

const storeDir = '/var/lib/mdk/avalon-1'
const net = await startDhtBootstrap({ host: '127.0.0.1', port: 49737 })
const dht = { bootstrap: net.bootstrap, dhtHost: '127.0.0.1' }

const kernel = await getKernel({ hrpc: dht, discovery: { mode: 'local' } })
const worker = await startAvalonWorker({ workerId: 'avalon-1', model: 'a1346', storeDir, allowedKeys: [kernel.getClientPublicKey()], ...dht })
await kernel.registerWorker(worker.runtime.getPublicKey())
const gateway = await startGateway({ kernel, ...dht })

Close the bootstrap with net.close() after every component has stopped.

B. mdk run

Add spec.network.dht to mdk.yaml:

spec:
  network:
    dht:
      host: 127.0.0.1   # bind interface, and where the generated bootstrap is reached (default 127.0.0.1)
      port: 49737       # UDP port of the generated bootstrap

The process that runs the Kernel (mdk run, or mdk run kernel in a split stack) also runs the bootstrap and stops it after the Kernel. Every other target (gateway, mcp, worker <name>, app <name>) and every CLI command that calls the Kernel joins host:port. To join a bootstrap you already run elsewhere, list it and no bootstrap is started. Set host to this machine's LAN address, or leave it unset to bind every interface. A loopback host with a non-loopback bootstrap is refused, because a socket bound to loopback cannot reach it:

spec:
  network:
    dht:
      host: 10.20.0.7
      bootstrap:
        - 10.20.0.5:49737

C. PM2 with startServices

await startServices({ runtime: 'pm2', repoRoot, dht: {}, services })

The generated ecosystem.config.js starts mdk-dht-bootstrap first, on 127.0.0.1:49737, and sets MDK_DHT_BOOTSTRAP and MDK_DHT_HOST=127.0.0.1 on every service. For an isolated LAN, pass dht: { host: '<this machine>', bootstrap: ['<lan ip>:49737'] } and no bootstrap app is generated. Without host, MDK_DHT_HOST is not set and every service binds all interfaces.

D. Docker Compose with startServices

await startServices({ runtime: 'docker', repoRoot, image, dht: { internal: true }, services })

A container cannot reach another container's 127.0.0.1, so the Docker profile uses a dedicated bridge network instead of loopback:

• The network is mdk-dht, subnet 172.31.249.0/24 unless dht.subnet says otherwise
• The generated dht-bootstrap service sits at .2, and services take .10 onward in the order they are listed
• Every service gets MDK_DHT_BOOTSTRAP=<subnet>.2:49737 and MDK_DHT_HOST=<its own address>, so its DHT node binds that network only
• With internal: true the network has no route out; each Gateway also joins the default network so its published port works

The Kernel is not part of the generated file. Run it on the same Docker network, at an address outside the generated ones, with getKernel({ hrpc: { bootstrap: ['172.31.249.2:49737'], dhtHost: '<its address>' } }).

Verify the stack is private

1. List the UDP sockets the MDK processes hold, and check each is bound to the chosen interface rather than 0.0.0.0 or *:

   lsof -nP -iUDP | grep node

2. Start one process with DEBUG=mdk:dht-bootstrap set to log the bootstrap address it serves.

3. On an air-gapped host, block outbound UDP 49737 at the firewall and confirm Workers still reach READY.

The private-dht integration test in backend/core/mdk/tests/integration/ checks the same things for each discovery mode and for the Gateway: sockets bound to 127.0.0.1, and routing tables that hold loopback nodes only.

Next steps

• Deployment topologies: choose between single-process, local and microservices shapes
• Supervised services: run a multi-Worker site under PM2 or Docker
• Hosted-site reference configuration: the full settings table for a managed site
• @tetherto/mdk-core reference: startDhtBootstrap, startServices options and the DHT error codes