All research systems
WitnessExperimental

Does an agent’s account of its actions agree with the execution record?

Question → hypothesis/design → architecture → evaluation → results → limitations → next questions

Motivation

Narrating a tool call does not mean the harness dispatched it.

Hypothesis

Comparing claims with structured execution records can expose discrepancies that narrative-only review misses.

Architecture

  • A reconciliation engine over structured tool-execution records
  • An Agent Guard audit adapter as one integration
  • Post-execution evidence separate from pre-dispatch authorization

Evaluation methodology

The repository describes an observed narrative/execution mismatch and the reconciliation implementation. General detection accuracy is not established here.

Results

Evaluation in progress

Limitations

  • Incomplete audit records limit what can be established
  • Agreement with a record does not prove that the external action succeeded

Current status

An Agent Rails evidence component for reconciling reported actions with recorded execution.

Roadmap

  • Evaluate false positives and missed discrepancies across more harnesses

Next questions

  • What constitutes sufficient evidence of a completed external action?