remotes.md

Remotes

For operators who need to reach another machine. A remote is a bundle that combines transport, file movement, and command execution: it can move a job or workspace tree, invoke httk there, and report status. A remote never schedules managers. Scheduling belongs to the launcher selected by the destination workspace; see {doc}launchers for that side of the workflow.

A remote bundle contains remote.json and one executable named adapter, with optional credentials.json for values that should not enter the shareable metadata. Project remotes live at httk_project/remotes/NAME; global remotes live at ~/.config/httk/remotes/NAME. Project definitions take precedence over global definitions with the same name.

Setting one up

Create an SSH remote, configure its connection, and verify that a compatible httk answers on the other machine:

$ httk workflow remote add --template ssh kappa
$ httk workflow remote configure \
      --set host=login.example.org \
      --set username=me \
      --set check_connectivity=yes kappa
$ httk workflow remote check kappa

remote check invokes the adapter's historical install operation; despite that protocol name, the maintained adapters verify the remote and do not install anything. Settings that are credentials are stored in credentials.json, which is excluded from signed project manifests. Use remote show [--json] to inspect a definition without printing credential values; remote list, remote remove, and remote import-v1 cover the other common management tasks.

Making httk available over SSH

ssh runs its command in a non-interactive shell, so the environment your login files set up interactively — module load lines, a virtualenv — is not applied, and httk is often not even on PATH. Put that setup in the remote's prelude setting rather than in ~/.bashrc, so it applies to httk's ssh commands only and does not disturb every other tool that logs in over ssh:

$ httk workflow remote configure --set prelude='module load Python/3.13.5-bundle
source ~/venv/bin/activate' kappa

The prelude runs (under set -e, so a failing line aborts before anything else) ahead of every command the adapter sends over ssh — including the httk workspace status that remote check uses to find httk in the first place. When only the httk program lives somewhere non-standard but the environment is otherwise ready, the narrower httk_command=/path/to/httk setting is enough.

This adapter prelude is distinct from a workspace's environment.prelude (below): the adapter prelude bootstraps the shell so httk can run at all, while environment.prelude is applied later by the manager once it is already running on the remote.

Initialize a named workspace on the remote by putting the remote name before the path:

$ httk workspace init --name runs kappa:/scratch/me/httk/runs
$ httk workspace status kappa:runs

The NAME:WORKSPACE spelling is a binding, not a filesystem path. Transfer a job into that workspace and run its manager there:

$ httk workflow transfer --job JOB default kappa:runs
$ httk workflow run --workspace kappa:runs --count 4

The remote invocation asks the owning machine to run httk workflow manager run --workspace runs --detach …. The target workspace then applies its own manager.launch, manager.workers, scheduler settings, and environment.prelude, exactly as if the command had been run on the login node. Fetch finished jobs back with the reverse transfer:

$ httk workflow transfer kappa:runs default

Names listed in machine_names are self-addressing: login:runs is treated as a local workspace binding when login is configured as one of this machine's names, so it does not invoke a remote adapter. To use a second tree on the same host through the adapter contract, create a distinct remote with the local template:

$ httk workflow remote add --template local local-tree
$ httk workspace init --name scratch local-tree:/tmp/me/httk/scratch

A mounted filesystem with a separate executor

Use the mount template when the remote filesystem is available locally as a mount (sshfs, NFS, any shared mount) but commands must run on the remote through a separate execution channel — for example a login node reachable only through a tunnelled wrapper, with neither ssh nor rsync available for transfers. Files move as bytes over the mount; every httk command about the workspace runs on the remote through the executor.

Three settings describe it: mount_root is the local path where the remote tree is mounted, remote_root is the same tree as the remote machine spells it (keep it to the project subtree you transfer into — remote_root=/ maps every absolute remote path onto the mount and is not recommended), and exec_command is an executor prefix (parsed once with shlex.split) that runs one shell command line on the remote and relays its stdout, stderr and exit status.

$ httk workflow remote add --template mount sigma
$ httk workflow remote configure \
      --set mount_root=/home/me/work/mounts/sigma \
      --set remote_root=/proj/x/users/me/httk \
      --set exec_command="/home/me/bin/hpc run" sigma

configure refuses unless mount_root is an existing directory (set check_mount=no to configure the remote before the filesystem is mounted) and unless the executor can run a cheap true on the remote (set check_connectivity=no to configure it anyway). The prelude and httk_command settings mean the same as for ssh.

A transfer also refuses when mount_root does not exist, but an empty, unmounted mount point cannot be told apart from a mounted-but-empty one, so run httk workflow remote check sigma before transfers as the operator's safeguard that the filesystem is actually mounted and httk answers on the far side.

The mount is for transfers only. Never httk workspace init on the mount as a local workspace, and never run status, collect, fsck or any analysis against the mounted tree: the remote machine owns the workspace, so every httk command about it goes through the executor (sigma:runs), exactly as with an ssh remote:

$ httk workspace init --name runs sigma:/proj/x/users/me/httk/runs
$ httk workflow transfer --job JOB default sigma:runs
$ httk workflow run --workspace sigma:runs --count 4
$ httk workspace status sigma:runs

From Python

The low-level adapter API is in httk.workflow.adapters. This example uses the local template so it can be exercised without an SSH server; use the CLI to configure an SSH remote's persisted settings, because there is no single high-level Python equivalent of remote configure --set:

from pathlib import Path

from httk.workflow.adapters import (
    add_remote,
    probe_remote_workspace,
    resolve_remote,
    run_adapter,
)
from httk.workflow.registry import resolve_workspace

project = Path(".").resolve()
add_remote("local-tree", template="local", project=project)
target = resolve_remote("local-tree", project=project)

# The workspace named runs must already exist in the local registry.
result = run_adapter(
    target.bundle,
    "status",
    {"argv": ["httk", "workspace", "status", "--json", "runs"]},
    timeout=None,
)
workspace_id, root = probe_remote_workspace(target, "runs", timeout=None)
binding = resolve_workspace("local-tree:runs", project=project)
print(result, workspace_id, root, binding)

add_remote creates a maintained adapter bundle, resolve_remote applies project-before-global resolution, and run_adapter executes one of the six adapter operations. probe_remote_workspace validates the remote status document and returns the remote workspace UUID and root. The registry's resolve_workspace keeps the NAME:WORKSPACE binding in one place; a remote workspace has no local path until the adapter reports it.

Writing a remote adapter

A custom remote is a versioned bundle with remote.json, one executable named adapter, and optional credentials.json. The dispatcher answers six operations — configure, install (the operation behind remote check), invoke, push, pull, and status — with one JSON result per invocation. It must implement transport, file movement, and remote command execution while leaving manager scheduling to the destination workspace's launcher. The engine refuses malformed metadata, a missing or non-executable dispatcher, unavailable required binaries, unsupported operations, non-zero dispatcher exits, and malformed or unsuccessful result documents.

The complete bundle layout, operation request and result documents, settings and credential handling, and refusal rules are in {doc}details/adapter_authoring.

For the transfer completion protocol, crash recovery, and metadata bounds on quota-limited filesystems, see {doc}transfer_reclamation.