Back to notes
The Tesseract MemoryTechnical note

Testing plugin activation without mistaking it for model behavior

Package assertions can catch an automatic hook returning, but execution, handoff and audit routing need tests of what the assistant actually does.

Published 2 min read
Topics

Remove the automatic entry point

HumanToAction's version 0.2 changelog records the removal of UserPromptSubmit and its Python runtime. The current manifest has no hooks field, the hooks directory is absent and allow_implicit_invocation is false. The skill also requires explicit selection, so the activation change exists in both packaging and instructions.

Match the claim to what the test observes

The unit tests check those files and settings, then look for required activation and handoff text. These assertions can detect an accidental packaging change. The evaluation fixtures receive a narrower check: positive prompts must contain an invocation marker, negative prompts must omit it, and required case IDs must exist.

No model runs those cases in this suite. The assertions therefore establish neither a false-activation rate nor correct routing through a conversation. They test that the evaluation inputs and package structure are present, not that an assistant behaves as intended.

Evaluate the action, not just the wording

After explicit activation, the assistant may refine internally and continue, produce a handoff, or report audit findings. A useful model evaluation needs to distinguish those outputs and catch scope changes: an audit must not quietly become a rewrite, and a same-task request needn't become a handoff.

Ambiguous destinations belong in that evaluation alongside clear requests. The package and fixture checks were inspected without execution for this write-up, and no conversation-level evaluation was run.

Explicit activation metadata and the static-test boundary

The release removes a hook and disables implicit selection. Tests inspect those facts; they do not execute the model-facing routing contract.

Direct callBlocked / denied
Recorded release change
Current plugin package
Static verification

Scroll or drag the background to move. Use the zoom buttons to resize.Arrow keys move between components. Enter selects.

Choose a component to explore

Select a numbered component on the map or use the component menu. Its details and connections will appear here.

No component selected.

All connections (9)
  1. 01 · v0.1 automatic hook → 02 · v0.2 hook removalHistorical v0.2 change · Direct call
  2. 02 · v0.2 hook removal → 03 · Root plugin manifestNo hook registration · Direct call
  3. 03 · Root plugin manifest → 04 · Implicit invocation disabledRegistered skill metadata · Direct call
  4. 04 · Implicit invocation disabled → 06 · Explicit activation and routing rulesExplicit selection only · Direct call
  5. 05 · Python structural checks → 03 · Root plugin manifestAssert hooks absent · Direct call
  6. 05 · Python structural checks → 04 · Implicit invocation disabledAssert implicit flag false · Direct call
  7. 07 · Positive and negative cases → 05 · Python structural checksCheck IDs and invocation markers · Direct call
  8. 06 · Explicit activation and routing rules → 08 · Model behaviourModel-facing contract · Direct call
  9. 05 · Python structural checks → 08 · Model behaviourNo semantic execution or pass rate · Blocked / denied

Build context