stepanov1975/hermes-local-knowledge

Hermes Local Knowledge

Goal

Hermes Local Knowledge is a reusable Hermes Agent plugin that routes a local question to the whole artifact the agent should inspect first: a skill, support document, script, runbook, cron job, MCP server, or generated tool routing note.

It is an artifact router, not chunk RAG. Search results are routing hints; the agent should still read the selected source before acting.

Unquoted searches recognize narrow artifact-type nouns only when a query ends in one unambiguous type noun and includes at least two high-signal terms: runbook, skill, doc/document/documentation/reference, and memory (including their exact plural forms) join the existing script, cron/job, and MCP/wrapper intent. At most one fully matching artifact family is promoted. The exact target must carry a complete configured entity label in its own ID, title, or path and must match a distinct operation/topic term; body-only mentions and evidence from support siblings do not qualify. Eligibility is resolved across the complete current index, then one stable move keeps an owning skill adjacent to a promoted support document without reordering unrelated results. Topic-only, mixed, underspecified, ambiguous, or conversational uses of those nouns retain baseline ranking exactly. Quoted searches, explicit type filters, script-reference protection, skill-parent lifting, and support-document diversity retain their normal behavior.

Sources

source_root is the curated local tree to index. $HERMES_HOME remains a separate runtime source for skills, cron jobs, and MCP configuration.

Built-in excluded directory names are .archive, .git, .mypy_cache, .pytest_cache, .venv, .worktrees, __pycache__, htmlcov, logs, node_modules, venv, and worktrees. exclude_dir_names adds deployment-specific exclusions.

Five tools

The plugin registers these native tools in the local_knowledge toolset:

Native tool responses are intentionally model-facing and concise. Search, get, and neighbor calls return only routing metadata needed to choose or inspect the next artifact, plus actionable warnings or rebuild state when present. Internal trigger vocabularies, index/configuration diagnostics, hashes, counts, and local database paths remain available to the service, CLI doctor, and local telemetry but are not serialized into routine agent context. The usage-report tool similarly returns a compact improvement-oriented projection rather than the complete telemetry schema. Its observer diagnostics are privacy-safe aggregates for the current root/version/window, explicitly limited to best-effort observed callbacks; overlapping skip reasons are not a complete learning-capture denominator.

The plugin registers pre_llm_call, on_session_end, and on_session_finalize hooks. On official Hermes v2026.9.14 and newer, public tool_execution middleware captures tool completions into a bounded plugin-owned serial observer while tools remain parallel. Older hosts retain a warned, best-effort post_tool_call fallback; the two transports are never registered together. See observer transport, lifecycle fences, privacy and loss limits. When the configured source is available, Hermes hosts with cache-safe plugin prompt sections add one concise system-prompt hint for each new session: use knowledge_search before broad file search for local Hermes/homelab artifacts, then verify live state directly. Older hosts use the same hint through a deduplicated pre_llm_call compatibility fallback. In either case, pre_llm_call continues to bind optional same-turn implicit search-result feedback; the remaining lifecycle hooks support exact successful consumption through knowledge_get, skill_view, or read_file plus tool-OKF capture/generation while keeping correlation inside the plugin when Hermes dispatches local-knowledge tools through its deferred-tool bridge.

Implicit feedback is enabled by default. It learns a local routing hint when the same artifact is consumed after distinct matching searches, with each supported consumer tied to that search's unassisted baseline in the same Hermes session, task, and turn:

local_knowledge:
  implicit_feedback:
    enabled: true
    min_confirmations: 2
    max_generic_queries: 5

Only a recent search from the same Hermes session, task, and turn is eligible. skill_view and read_file must run in a later model request, after the search result is available, and count only when their successful result resolves by canonical source path to exactly one baseline artifact in the same current index snapshot. Repeated consumption from one search is deduplicated, and overly generic artifacts stop receiving implicit promotion. Explicit feedback remains authoritative, and implicit evidence is not used as evaluation ground truth.

Install and model-token notice

Install and enable the directory plugin:

hermes plugins install https://github.com/stepanov1975/hermes-local-knowledge.git --enable

An SSH URL is also supported on hosts with GitHub SSH access:

hermes plugins install git@github.com:stepanov1975/hermes-local-knowledge.git --enable

The Python package exposes the local_knowledge = hermes_local_knowledge.plugin entry point for environments that install plugins into Hermes' Python environment.

If updating to v0.4.14 caused Hermes' plugin scan to disable the plugin, update to v0.4.15 and then re-enable the reviewed plugin once:

hermes plugins update local_knowledge
hermes plugins enable local_knowledge

Automatic tool-OKF generation and token use

Existing-artifact lookup works without model calls. local_knowledge.okf.auto_generate defaults to true because generated routing notes significantly improve future tool discovery.

Installers and agents should tell users that automatic generation:

• a detached worker invokes the active Hermes model and consumes additional model tokens;
• one worker claims at most max_candidates_per_session candidates (default 2) and makes at most one structured batch call when new content must be authored;
• max_generation_seconds is passed as the provider-request timeout, while provider retry/fallback policy can extend total elapsed time and token use behind that host call;
• session finalization does not wait for generation.

Users who do not want the additional model calls can disable automatic generation while retaining existing lookup and manual OKF management:

hermes config set local_knowledge.okf.auto_generate false

Generated tool notes use Open Knowledge Format v0.2 concept metadata (type plus structured generated.by/generated.at provenance). When automatic generation is enabled, the same bounded worker opportunistically claims completed generator-v3 notes and converts their deterministic frontmatter to v0.2 without a model call, preserving the existing routing content and generation timestamp. A legacy note that cannot be converted losslessly falls back to normal model-backed regeneration.

Install the proactive router skill

The per-turn hook makes the primary search tool discoverable without loading a full routing skill. The bundled skill adds the detailed search/fetch/inspect workflow, freshness guidance, and feedback procedure:

hermes local-knowledge install-router-skill --json

installed and current are successful statuses. A different existing skill produces conflict; review it before choosing --force.

To use an intentionally customized proactive skill instead, configure its deployed runtime path before running doctor or installer commands:

local_knowledge:
  router_skill_path: skills/note-taking/local-knowledge-router/SKILL.md

Relative paths resolve from $HERMES_HOME. The path must name an active SKILL.md under $HERMES_HOME/skills; that runtime path may be a symlink to a separately managed custom-skill repository. Doctor validates the skill's frontmatter name but does not require custom content to match the bundled copy. Invalid or missing configured custom skills appear as explicit doctor warnings while doctor retains its diagnostic-success exit status; the installer fails without modifying the configured target. install-router-skill, including --force, returns current with router_skill_mode: custom and does not overwrite a valid configured custom skill.

From a source checkout before plugin CLI registration is available:

python -m hermes_local_knowledge.cli install-router-skill --json

The plugin also exposes the same file as local_knowledge:local-knowledge-router for explicit skill_view(...) loads. That namespaced copy is not a substitute for installing the normal proactive skill. After installing or changing the skill, run /reload-skills and start a new/reset session, or start a fresh Hermes session.

Configuration

Put non-secret settings in $HERMES_HOME/config.yaml:

local_knowledge:
  source_root: ~/repos/local-operations
  state_dir: ~/.hermes/local_knowledge
  # Optional: use a deployed custom proactive skill instead of the bundled copy.
  # router_skill_path: skills/note-taking/local-knowledge-router/SKILL.md
  custom_skill_dirs: [custom_skills]
  script_dirs: [scripts, hermes_home/scripts]
  memory_dirs: [memory]
  runbook_dirs: [docs]
  include_markdown_docs: true
  exclude_dir_names: [build, dist]
  known_entities: [Hermes, GitHub, MCP, Cron]
  okf:
    enabled: true
    auto_generate: true  # uses additional model tokens in a bounded background worker
    max_candidates_per_session: 2
    max_generation_seconds: 120
    min_use_count: 1

Canonical settings, aliases, and defaults:

All nested okf keys also accept their flat okf_* form. YAML lists are preferred in config.yaml; comma-separated or bracket-list strings written by hermes config set are normalized.

Each invocation follows Hermes' context-local active profile, including hosts that keep multiple profile managers in one process. By default, each profile therefore uses its own config, runtime artifacts, index, telemetry, and learned feedback under $HERMES_HOME/local_knowledge. A shared source_root is safe when profiles should search the same curated tree, but keep state_dir profile-specific unless combined telemetry and feedback are explicitly intended. Process-level LOCAL_KNOWLEDGE_ROOT and LOCAL_KNOWLEDGE_STATE_DIR overrides still apply to every profile in that process.

When source_root is omitted, runtime skills, cron jobs, and MCP configuration are still indexed from $HERMES_HOME, but arbitrary root-level Markdown is not included by default. Generated state belongs outside a source repository and must not be committed.

Verified-routing shadow mode (off by default)

An optional shadow lane captures task-linked searches with separate immediate lookup intent, investigates selected current Markdown/script sources, and verifies useful evidence coverage against the complete baseline. Candidates are leads, not a mandatory reading list; source availability is selection advice. Exact bounded excerpts carry whole-file hashes and line provenance but do not claim complete content inspection. Search results never change. Exact recurrence has a cheap freshness/contract gate; a bounded detached-worker applicability check can label paraphrases or changed answer quantities as asynchronous would-reuse when the same sources cover them. Lexical shortlisting is not the semantic verdict, and neither path demonstrates savings.

Eligible turn-end hooks wake a finite detached supervisor (teardown remains a fallback), so ready shadow work can progress across batches without another message. Each supervisor is limited to 15 minutes and 16 child batches; this can spend more tokens than one batch. Child interruptions are recovered only while the supervisor survives, using existing lease expiry and ambiguous-call closure. No permanent scheduler is installed.

Native knowledge_search accepts optional bounded lookup fields (intent, target, operation, context), explicitly assistant-supplied rather than authority or execution permission. No transcript window is captured. Enabling shadow mode retains bounded private task/lookup text and sends task/source evidence to the configured model provider; it incurs additional model calls. It is independent of OKF and implicit feedback. See configuration, lifecycle, cost, limitations and operator commands before opting in. Neither installation nor upgrade enables it.

Cached applicability veto (candidate, off by default)

local_knowledge.verified_routing.mode: veto opts into source-only background review of actual explicit feedback promotions. A fresh, exactly bound, cited inapplicable receipt cancels only that promotion and restores the complete unassisted page; ordinary retrieval remains available. No positive AI routing is enabled, no model runs in search, and off/shadow behavior is unchanged. Missing host scope or unusable evidence retains incumbent behavior. This adds private source/model disclosure and preparation cost, not demonstrated ranking gains. See enablement, evidence binding and rollback.

CLI

The primary standalone entry point is:

python -m hermes_local_knowledge.cli build --from-hermes-config
python -m hermes_local_knowledge.cli search 'backup runbook' --from-hermes-config --limit 8
python -m hermes_local_knowledge.cli get skill:backup-runbook --from-hermes-config --json
python -m hermes_local_knowledge.cli neighbors skill:backup-runbook --from-hermes-config --json
python -m hermes_local_knowledge.cli evaluate --from-hermes-config --json --details
python -m hermes_local_knowledge.cli okf status --from-hermes-config --json
python -m hermes_local_knowledge.cli doctor --hermes-home "${HERMES_HOME:-$HOME/.hermes}" --rebuild --query 'backup runbook'

Use python -m hermes_local_knowledge.cli <command> --help for manual OKF claim, validate, complete, fail, and retry operations.

python -m hermes_local_knowledge.indexer ... is preserved as a compatibility entry point to the same CLI. New documentation and automation should use hermes_local_knowledge.cli.

The installed hermes local-knowledge surface is intentionally smaller:

hermes local-knowledge install-router-skill
hermes local-knowledge doctor --rebuild --query 'backup runbook'

The host also uses an internal bounded worker subcommand for automatic OKF generation.

Index freshness

Managed native lookups, and CLI search/get/neighbors with --from-hermes-config and no explicit --db, rebuild when the index is missing, corrupt, older than format 4, or marked dirty by completed OKF publication. A newer index format is rejected rather than overwritten.

Ordinary source-file, cron-registry and MCP-config changes are picked up by opportunistic age refresh, enabled by default (index_max_age_seconds: 3600; 0 disables it; values clamp to 0–604800 seconds). Pre-LLM activity and managed lookups cheaply read successful-build metadata and schedule background maintenance when stale. No source scan or model call runs on the trigger path, and idle installations do no work. The initiating lookup can return the old valid index. This is not a fixed freshness deadline or a filesystem watcher.

A daemon thread captures the resolved profile configuration, independently of the tool observer queue. One thread per process/index is admitted; both existing build locks serialize builders across processes, and freshness is checked again inside those locks. Healthy managed reads do not wait for build locks. During the JSONL-before-SQLite publication window, a busy compatibility gate plus full validation against the existing hash-matched rollback companion permits old-index reads; a busy gate alone never bypasses validation. Missing/corrupt/older-format and OKF-dirty repair remains synchronous. Failed builds retain the prior valid pair and write an error-class-only index_refresh.json receipt with a five-minute retry cooldown, shared across processes; doctor JSON exposes the receipt, and failures are logged. If launching or recording a receipt fails, logging and a process-local cooldown remain best-effort fallbacks. Existing atomic publication/recovery rules remain unchanged.

Short-lived CLI processes or host shutdown may interrupt daemon maintenance: there is no exit drain or completion guarantee. Older hosts use the supported pre-LLM hook on a best-effort basis; managed lookups are an independent trigger. For immediate freshness, either:

• pass rebuild=true to knowledge_search, knowledge_get, or knowledge_neighbors; or
• run python -m hermes_local_knowledge.cli build --from-hermes-config (or doctor --rebuild).

No rebuild cron is required. Operators who need a fixed freshness interval may optionally schedule the explicit build command. An explicit CLI --db is caller-owned: it is read as supplied and is not rebuilt automatically or used to consume shared OKF-dirty tokens.

Generated state

The plugin writes local/generated state under state_dir:

index.sqlite
index.jsonl
usage.sqlite
okf_queue.sqlite
okfs/tools/*.md
okf_worker.log
index_build.lock
index_build.sqlite
index_refresh.json
okf_index_dirty/

index_build.lock is the regular-file compatibility gate shared with v0.3.12 builders. New builders hold it and the SQLite transaction lock in index_build.sqlite for collection, validation, and publication. SQLite metadata binds index.sqlite to the exact index.jsonl bytes: a caught publication failure restores the prior pair, and a crash-split pair is classified as corrupt and rebuilt by the next managed lookup. okf_index_dirty/ contains tokens that make the next managed lookup rebuild; a successful build removes only the tokens it covered.

None of these files belong in source control.

OKF lifecycle

Tool OKFs are compact routing notes for tools Hermes has actually used. They are hints, not authoritative tool documentation.

1. When okf.enabled is true, the observer (or legacy post_tool_call fallback) records only a bounded structural projection: tool identity/toolset, a sanitized schema shape, argument shape, counters, and a redacted error class. It does not persist raw argument values, tool output, transcripts, emails, OCR text, private documents, schema descriptions/examples/defaults, or secret values.
2. When okf.auto_generate is true, accepted on_session_finalize work follows already-admitted observations, performs a read-only, tightly bounded durable-queue check and launches a detached worker only when work is available. Legacy hosts run this check inline without an observer fence.
3. The worker acquires one fixed lease of max(300, 2 * max_generation_seconds + 120) seconds and claims at most max_candidates_per_session rows. Lease and claim ownership are checked again before publication.
4. Claimed legacy notes that can be converted losslessly are updated without a model call. If any claimed rows still need model-authored content, the worker makes at most one ctx.llm.complete_structured batch call with the privacy-safe routing projection and a bounded same-toolset related-tool allowlist. The model receives no terminal or file tools.
5. Each result is identity-checked, rendered to a worker-unique temporary file, prevalidated, and published inside a short token/lease-fenced transaction. A stale worker cannot publish. Successful publication marks the managed index dirty.

Version 0.4.0 reads the current v0.3.12 queue shape by normalizing a selected claim's stored schema into the bounded routing projection. It does not promise a general migration ladder for arbitrary older private schemas.

Feedback and evaluation

Lookup telemetry and feedback stay in <state_dir>/usage.sqlite. Tool handlers fail open for telemetry-only errors; explicit knowledge_feedback writes remain strict so callers know whether feedback was recorded. Do not put secrets or private document text in queries or feedback notes.

Managed searches may use one deterministic feedback prior when the index was built for the configured source root. Only the latest significant explicit rating for a query/artifact pair is eligible. Among ratings accepted by the current tool, only useful is positive; legacy persisted great rows remain positive compatibility input. A newer rejection for that route or matching current query suppresses an older overlap route. A matching artifact already present in current results may move to rank one. If it is absent, the plugin performs at most one retry with an accepted query no longer than the current query and the mapped artifact type, and promotes only the exact artifact when that live retry rediscovers it. Searches against an explicit caller-owned --db remain unassisted.

When implicit feedback is enabled and no explicit route matches, mature same-turn evidence may supply the lower-priority route. Consumption through skill_view and read_file is accepted only from a later model request when a successful call resolves to exactly one caller-visible baseline artifact in the same current index snapshot; same-request parallel calls and route-assisted-only results remain ineligible. It uses the same current-index promotion or one verified typed-retry path, and a matching explicit rejection can veto it.

knowledge_usage_report summarizes recent activity before changing ranking, triggers, source coverage, or graph edges. Its model-facing search_quality section isolates live native searches from the current plugin version while excluding known probe queries. When observed consumption exists, the nested implicit_consumed_rank_lower_bound aggregates where it fell in each search's unassisted baseline_top_ids_json; it is a lower-bound operational diagnostic, never evaluation ground truth. event_cohorts separates current searches, probes, hourly doctor runs, other CLI/native activity, and historical-version searches, while the nested feedback summary distinguishes learned knowledge_get, skill_view, and read_file signals. Empty sections and duplicated aggregate/configuration diagnostics are omitted from the native response; the complete local telemetry report remains available to service, CLI, and evaluation code.

evaluate is read-only and intentionally measures the unassisted index ranking to avoid training/evaluation leakage. It replays positive local feedback against the current index and reports exact Hit@k/MRR plus parent-equivalent metrics. Parent equivalence is deliberately limited to a skill_support_doc and its owning skill; generic graph neighbors are not treated as successful equivalents.

For a bounded blinded benchmark that requires explicit labels for every reviewed case and item, see docs/lean-human-benchmark.md. The finalizer creates private benchmark data only; it does not implement or approve a ranking candidate.

For historical comparisons from a source checkout:

python scripts/compare_historical_query_versions.py \
  --usage-db ~/.hermes/local_knowledge/usage.sqlite \
  v0.4.2 WORKTREE

The comparator opens live telemetry read-only, freezes one private source/runtime/OKF corpus, and builds each ref with that ref's own code. Nonnegative recorded explicit and enabled implicit feedback boundaries plus a matching index-corpus hash prove that the recorded event inputs are available. Implicit rows are accepted only when their exact successful search event has matching root, query, session, task, non-empty turn, and baseline-result membership. Searches recorded with implicit feedback disabled do not require unused implicit state. The report calls a ref replay event-time exact only when that ref's plugin version and index format also match the recorded event and the replay reproduces its recorded baseline/final pages plus route provenance; a changed candidate otherwise remains a counterfactual replay over exact inputs, not historical output. Migrated legacy rows without complete boundaries and same-turn provenance remain explicitly non-exact best-effort evidence. The machine report counts these evidence classes without exposing raw queries.

Correction-route acceptance uses explicit_resolution Hit@1 as the primary outcome, then verified_event Hit@1/MRR, while requiring no increase in negative-artifact exposure, no unaccepted production ordering changes, and no replay errors. The comparison assessment distinguishes rejected, accepted_improved, and accepted_unchanged_or_insufficient_evidence. Direct/legacy aggregate metrics remain useful coverage and trend signals, but are not sufficient by themselves to prove a correction. Raw query/artifact report details require --details. Independently, evaluator failure diagnostics can contain raw query context, exception messages, tracebacks, and captured child output; these remain inside the owner-only evaluation directory and must not be published.

The comparator emits stage/count progress and a five-second elapsed heartbeat on stderr while child evaluation runs; --json stdout remains one JSON document. Each child writes an atomic private requests/*.progress.json status file, owner-only stdout/stderr captures, an invocation receipt, and a failure JSONL sidecar. With private diagnostics configured, API printed output streams immediately to an owner-only *.failures.stdout.log sidecar (the receipt's captured_stdout_file), so it survives a stalled callback killed on parent interruption; normal completion reports only its byte count and hash in the single JSON stdout payload. Nonfatal failure diagnostics point to the same private capture. Without diagnostics, the direct evaluator retains its in-memory capture contract. This exposes ongoing work without forwarding arbitrary child stderr. Fatal errors and failed acceptance retain the private evaluation directory, even without --keep-work-dir, and print its location on stderr. Inspect failure.json for orchestration errors and requests/*.failures.jsonl for lookup exceptions (including errors converted into replay results). Successful temporary runs are still removed unless --keep-work-dir is selected; remove retained failed directories after investigation.

Verification

Operator checks:

hermes local-knowledge doctor
hermes local-knowledge doctor --json
hermes local-knowledge doctor --rebuild --query 'backup runbook'

doctor reports missing/outdated router skill state and disabled automatic OKF generation as nonfatal warnings. Resolve them or report the deliberate choice.

Development gates:

PYTHONDONTWRITEBYTECODE=1 python -m pytest tests/test_public_contract.py tests/test_plugin.py -q -p no:cacheprovider
PYTHONDONTWRITEBYTECODE=1 python -m pytest -q -p no:cacheprovider
python -m ruff check .
python -m mypy
python scripts/check_version_policy.py --base-ref origin/main
git diff --check

See CONTRIBUTING.md, SECURITY.md, and docs/github-security.md for contribution, vulnerability-reporting, and repository-security guidance.

Acknowledgments

This project benefits from people who contribute code, share ideas, or inspire features:

• @xXLODXx — contributed the initial opt-in implicit usage-feedback implementation and its confirmation and generic-artifact gates in PR #27, adapting ideas from hermes-skill-router.

Owner map

• config.py — configuration models, aliases, defaults, and the single resolver.
• implicit.py — default-enabled same-turn search-result consumption attribution.
• artifacts.py — whole-artifact models, source collection, privacy-safe metadata extraction, and graph edges.
• index.py — format-4 SQLite/JSONL publication, cross-version and SQLite build locking, managed rebuild classification, and deterministic search/get/neighbors.
• telemetry.py — local usage and feedback persistence/reporting.
• routing.py — bounded live-root explicit/implicit feedback matching, promotion, and one verified typed retry.
• evaluation.py — read-only feedback-label replay and exact/parent-equivalent metrics.
• service.py — one resolved configuration's managed index and telemetry lifecycle.
• okf.py — privacy-safe OKF queue, hooks, detached worker, validation, and fenced publication.
• plugin.py — Hermes registration for five tools, three lifecycle hooks plus tool-execution middleware (a fourth, best-effort post-tool hook on older hosts instead of middleware), the bundled skill, and installed CLI adapter; register is its public export.
• cli.py — primary standalone command surface and the smaller Hermes CLI adapter.
• indexer.py — thin compatibility facade exporting exactly Artifact, Edge, IndexSettings, build_index, search_index, get_artifact, get_neighbors, and main.
• __init__.py — package version.