Build an Auditable Research Agent: Evidence Ledgers, Sandboxes, and Replay

TL;DR: A trustworthy research agent needs two separate contracts. The research contract says how claims, sources, conflicts, and confidence are recorded. The execution contract says where tools run, which credentials they can use, and how every action can be inspected later. This tutorial combines the open-source SandBase Skills evidence-ledger workflow with SandBase Harness for persistent sessions, sandboxed tools, audit, and replay.

Why a Good Answer Is Not Enough

A polished answer can still be wrong. The common failure is a missing chain of custody:

1. Which source supports each claim?
2. Are two citations genuinely independent?
3. Did any credible source disagree?
4. Was confidence reduced when evidence conflicted?
5. Which tool calls produced the artifacts?
6. Can another person replay the run?

Prompting can improve the first four, but it cannot enforce filesystem boundaries, credential policy, persistent event logs, or replay. Runtime controls alone cannot make a weak research method rigorous.

The useful architecture is layered:

Layer	Responsibility	Open-source component
Research method	Source diversity, claim mapping, conflicts, confidence	multi-source-search Skill
Output validation	Reject inconsistent evidence ledgers	Bundled offline validator
Agent runtime	Sessions, tools, permissions, credentials, artifacts	SandBase Harness
Isolation	Local, Docker, Kubernetes, or worker sandboxes	Harness sandbox backends
Inspection	Persistent events, audit, and replay	Harness session runtime

Step 1: Install the Research Contract

The flagship Skill uses search and page-reading tools supplied by the host agent. It does not require a SandBase account for this workflow.

Preview or install it with GitHub CLI:

gh skill preview sandbaseai/sandbase-skills research/multi-source-search

gh skill install sandbaseai/sandbase-skills research/multi-source-search \
  --agent codex --scope user

Or install it into a DeepSeek Harness project:

npx --yes github:sandbaseai/sandbase-skills add multi-source-search

This creates .dsh/skills/multi-source-search. The Skill tells the agent to distinguish primary from secondary evidence, map claims to source IDs, record conflicts explicitly, and avoid high confidence when unresolved evidence disagrees.

Use a falsifiable test prompt:

Compare the branch-protection capabilities of GitHub, GitLab, and Bitbucket.
Use primary documentation where possible. Separate observed facts from
inference, record conflicting evidence, and return a machine-checkable
evidence ledger.

Step 2: Make the Ledger a Build Artifact

The repository includes a complete example and an offline validator:

git clone https://github.com/sandbaseai/sandbase-skills.git
cd sandbase-skills

python3 research/multi-source-search/scripts/validate_report.py \
  examples/verifiable-research-report.json

A valid report prints:

VALID: 3 source(s), 1 claim(s), 2 provider(s)

The validator checks internal consistency. It rejects unknown or duplicate source IDs, unused evidence, confidence inconsistent with support, and high-confidence claims with unresolved conflicts.

The boundary matters: it does not prove that a webpage is true. It proves that the report follows its declared evidence model. Important decisions still need human sampling or a second verification pass.

Treat the JSON report as an artifact rather than disposable model output. Commit it for reproducible internal research, attach it to a session, or pass it to a review step.

Step 3: Run in a Governed Runtime

SandBase Harness is a local-first agent runtime. It adds persistent sessions, credentials, permission policies, artifacts, and sandbox backends around the model loop.

Run the tagged release from source:

git clone --branch v0.3.4 --depth 1 \
  https://github.com/sandbaseai/sandbase-harness.git
cd sandbase-harness
npm ci
npm run build

mkdir ../my-research-agent
cd ../my-research-agent
node ../sandbase-harness/dist/index.js init
node ../sandbase-harness/dist/index.js start

Open http://127.0.0.1:3000/dashboard, configure a model provider, and choose the sandbox backend for the risk:

• local process for trusted development;
• Docker for per-session container boundaries;
• Kubernetes for an existing cluster control plane;
• self-hosted workers for separately managed machines.

Never put a credential value in the prompt. Store credentials in the runtime vault, attach only what a tool requires, and gate mutations or network access with permission policy.

Step 4: Connect Skill and Runtime

For a native DSH workflow, install the SandBase Skills bundle:

dsh plugin --profile web add github:sandbaseai/sandbase-skills
dsh web

For MCP-compatible clients, the Harness bridge is published as a pinned multi-architecture OCI image:

docker pull ghcr.io/sandbaseai/sandbase-harness-mcp:0.3.4

docker run --rm -i \
  -e MANAGED_AGENTS_URL=http://host.docker.internal:3000 \
  ghcr.io/sandbaseai/sandbase-harness-mcp:0.3.4

If authentication is enabled, pass MANAGED_AGENTS_API_KEY through the process environment. Do not put it in plugin manifests, prompts, or committed configuration.

The components solve different problems:

• the Skill teaches the agent how to research;
• the bridge provides governed runtime operations;
• the Harness event stream records what actually happened.

Step 5: Define Acceptance Criteria First

A research task is easier to review when “done” is mechanical:

Acceptance criteria:
- At least three sources, including two primary sources.
- Every material claim references one or more source IDs.
- Conflicts are listed rather than silently reconciled.
- The evidence ledger passes validate_report.py.
- Raw notes and final JSON are attached as session artifacts.
- The session contains no credential values.
- A reviewer can replay the session and locate every used tool result.

This shifts evaluation from “the prose sounds convincing” to verifiable properties.

Step 6: Audit the Run

Inspect three levels after completion.

Research integrity

Run the validator and manually sample the most important citations. Confirm URLs resolve, paraphrases match, and “independent” sources are not copies of one announcement.

Execution integrity

Review tool calls, permission decisions, artifact writes, and sandbox selection. A correct answer produced by an over-privileged agent is still an operational failure.

Replay integrity

Resume or replay the session and confirm that the result can be reconstructed from recorded events and artifacts. Persistent sessions and resumable event streams make review independent of a transient terminal buffer.

A Minimal Threat Model

Threat	Control
Search result injects instructions	Treat retrieved text as data; preserve system and Skill precedence
Model invents a citation	Require source IDs and validator checks
Mirrors appear independent	Canonicalize sources and inspect provenance
Tool writes outside the project	Use sandbox boundaries and permission policy
Secret leaks to a prompt or artifact	Keep secrets in the vault; review artifacts
Final prose hides disagreement	Require conflicts and calibrated confidence
Run cannot be reproduced	Persist events and attach the ledger

No single control is sufficient. The goal is defense in depth with artifacts that make failures observable.

What to Measure

For repeated research workflows, track process quality:

• material claims linked to evidence;
• claims supported by a primary source;
• unresolved-conflict rate;
• validator failure rate;
• denied or escalated tool calls;
• artifact completeness;
• successful replay rate;
• review time per report.

These metrics make improvements testable. A model may improve synthesis, a Skill may improve evidence coverage, and runtime policy may reduce execution risk.

Reproduce, Inspect, Improve

Both components are Apache-2.0 open source:

• sandbaseai/sandbase-skills — 88 portable Agent Skills, including the evidence-ledger workflow and validator.
• sandbaseai/sandbase-harness — local-first runtime with sessions, sandboxed tools, credentials, audit, replay, and MCP integration.

Start with the worked evidence-ledger example. Then run the same task inside a sandboxed Harness session and review it as if the answer came from an untrusted colleague: verify the claims, inspect the actions, and demand a replayable trail. That is the difference between an agent that produces text and an agent whose work can be accepted.