Make an unanswered sentence the next tree, and frame the core around all forms of I/O
Claude · 2026-10-25 · 90a991d2ea4bf8da5c5c843fe3e3376dbcaf0dfd · parents 1ac4ef3
A sentence no tree answers used to be a dead end ("No service in the swarm answers ..."). It is now a path: the outcome carries a proposal for the tree that would answer it (a name from the words, a trigger with slots where the sentence had numbers or hosts, the inputs those slots need, and the `pyrtan new ... --from` command that writes it), and the sentence is recorded, counted, at GET /unanswered on the service or hub.
New module grow.py holds propose(), the scaffold renderer and the Unanswered log; the scaffold replaces the template that lived in cli.py and passes `pyrtan check`. `pyrtan new` takes --from SENTENCE (the name becomes optional); `pyrtan unanswered --at URL [--scaffold DIR]` lists what nobody answered and writes a tree file for each. The hub collects its peers' unanswered sentences the way it collects gaps; a service with a hub leaves the record to the hub. The demo page shows the proposal when nothing answers and lists the unanswered sentences beside what the swarm can do. No model is involved: the proposal is derived by rules.
Prose across README, the design record, the tree format reference, CONTRIBUTING, the deploy guide and module docstrings now describes the core as a set of expandable services over all forms of I/O, with HTTP as the first adapter and the host allow-list as the rule every further adapter must arrive with, instead of as a tool for the web alone.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_019zkT2UtnaKNvrfoegCnJDg
Changes
CONTRIBUTING.md | 7 +- README.md | 83 +++++++++++++++------ _meta/design.md | 101 ++++++++++++++++++-------- _meta/tree-format.md | 5 ++ deploy/README.md | 3 + src/pyrtan/__init__.py | 2 + src/pyrtan/cli.py | 106 +++++++++++++++------------ src/pyrtan/grow.py | 262 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ src/pyrtan/hub.py | 28 ++++++- src/pyrtan/page.py | 31 +++++++- src/pyrtan/runtime.py | 7 +- src/pyrtan/service.py | 20 ++++- src/pyrtan/transport.py | 7 +- tests/test_cli.py | 44 +++++++++++ tests/test_grow.py | 92 +++++++++++++++++++++++ tests/test_service_hub.py | 18 +++++ 16 files changed, 700 insertions(+), 116 deletions(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index ffac9e0..fc547c9 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md@@ -5,8 +5,11 @@ does not exist, write it; if one gives a wrong answer, the trace shows where.## Adding or fixing a tree-1. `pyrtan new my-tree --dir examples/my-tree` and edit the file. Keep the- `hosts` list to exactly the hosts it needs.+1. `pyrtan new --from "the sentence it answers" --dir examples/my-tree`+ (or `pyrtan new my-tree`) and edit the file. If a running swarm has+ already been asked the sentence, `pyrtan unanswered --at URL --scaffold+ DIR` writes the same file. Keep the `hosts` list to exactly the hosts it+ needs.2. `pyrtan check examples/my-tree` until it is clean. Warnings are worth reading; an `info` about a `decide` without `else` is a choice, not a fault. diff --git a/README.md b/README.md index 937a4a9..583963d 100644 --- a/README.md +++ b/README.md@@ -4,23 +4,29 @@through hardcoded, inspectable decision trees.** Pyrtan looks like an assistant from the outside. You give it a sentence; it-does something on the web and tells you what happened. Inside there is no-model. Every service is a set of **trees**: declarative files that say which-sentences they answer to, which hosts they may touch, what they observe, and-exactly how they decide what to do next. Every run produces a **trace** that-names each observation and the condition that chose each branch.+reads or changes something through whatever form of I/O it has an adapter+for, and tells you what happened. Today that form is HTTP: APIs and pages.+The core is built so that files, shells, serial lines, queues and devices+arrive as further adapters under the same rules. Inside there is no model.+Every service is a set of **trees**: declarative files that say which+sentences they answer to, which endpoints they may touch, what they observe,+and exactly how they decide what to do next. Every run produces a **trace**+that names each observation and the condition that chose each branch.That is the trade. A model generalises to sentences nobody wrote a tree for;-Pyrtan does not, and says so. In return, Pyrtan is deterministic, costs-nothing per run, works offline against recorded responses, cannot reach a-host its author did not list, and can always explain itself.+Pyrtan does not, says so, and hands back the tree that would, ready to fill+in. In return, Pyrtan is deterministic, costs nothing per run, works offline+against recorded responses, cannot reach an endpoint its author did not+list, and can always explain itself.Trees are close in spirit to Claude's skills, with two differences. They are-much narrower: a tree does one thing against one web property. And they are+much narrower: a tree does one thing against one property, whether that is+an API, a site, a device or any other source or sink of I/O. And they arelive: a service reloads a tree the moment its file changes, a tree can hand-part of its work to any other tree in the swarm, and every decision that-meets a value no branch covers is recorded as a **gap**, which is how a tree-tells its author where to grow next.+part of its work to any other tree in the swarm, every decision that meets a+value no branch covers is recorded as a **gap**, and every sentence nobody+answers is recorded as **unanswered** with a proposed tree. That is how the+swarm tells its authors where to grow next.``` $ pyrtan run "what's the weather in Oslo" --trees examples@@ -116,11 +122,12 @@ whole format is in [`_meta/tree-format.md`](_meta/tree-format.md).| `pyrtan list DIR` | What is loaded and the sentences it answers to. | | `pyrtan run "…" --trees DIR` | Answers a sentence in this process, printing the trace. `--replay FIXTURE` answers HTTP from a recording; `--record FIXTURE` makes one. | | `pyrtan run --tree NAME --input k=v` | Runs a named tree. |-| `pyrtan serve DIR --port 8701` | Hosts the trees as a service: `/health`, `/capabilities`, `/intent`, `/run`, `/resume`, `/gaps`. Reloads trees when their files change. |+| `pyrtan serve DIR --port 8701` | Hosts the trees as a service: `/health`, `/capabilities`, `/intent`, `/run`, `/resume`, `/gaps`, `/unanswered`. Reloads trees when their files change. || `pyrtan hub --port 8700` | Routes between services. Services register with `--hub URL`; peers carry a `link` of `up`, `stale` or `down`. | | `pyrtan ask "…" --at URL` | Answers a sentence through a running service or hub. |-| `pyrtan gaps --at URL` | Every decision that met a value with no branch: the list of what to write next. |-| `pyrtan new NAME` | Scaffolds a tree file. |+| `pyrtan gaps --at URL` | Every decision that met a value with no branch: the branches to write next. |+| `pyrtan unanswered --at URL` | Every sentence no tree answered, with the tree that would: the trees to write next. `--scaffold DIR` writes them. |+| `pyrtan new --from "…"` | Scaffolds a tree file from a sentence: numbers and hosts in it become slots, and the file passes `pyrtan check`. `pyrtan new NAME` scaffolds from a name. |## A swarm@@ -142,14 +149,41 @@ tree and splits into many without the callers changing. A tree thatroutes it to whichever peer can; the peer's steps come back inside the caller's trace.+## When no service in the swarm answers++A sentence nobody answers is not a dead end; it is the next tree. The hub+(or a service on its own) records it at `/unanswered` and answers with a+**proposal**: a tree name drawn from the words, a trigger pattern with slots+where the sentence had numbers or hosts, the inputs those slots need, and+the one command that writes the file.++```+$ pyrtan ask "top 3 stories on lobsters"+error (no-match): No service in the swarm answers 'top 3 stories on lobsters'. To write the tree that would: pyrtan new top-stories-lobsters --from "top 3 stories on lobsters"++$ pyrtan unanswered --at http://127.0.0.1:8700 --scaffold trees/+'top 3 stories on lobsters' x1+ would be: top-stories-lobsters answering top {count:int} stories on lobsters+ wrote trees/top-stories-lobsters.toml+now: pyrtan check trees/+```++The scaffold is a complete tree that checks clean: a trigger, its inputs, an+`observe`, a `decide` on the status and a `done` and `fail`. The author+fills in what it observes and how it decides; a service serving that+directory picks the file up the moment it is saved. No model drafts+anything: the proposal is derived from the words by rules, and the demo page+shows the same proposal and the same command whenever nothing answers.+## What Pyrtan refuses - **No model in the loop.** Nothing in this repository calls an LLM. A model may one day sit *beside* the swarm, proposing branches for recorded gaps; it will not sit inside a run deciding anything.-- **No host a tree did not name.** Every HTTP node's host is checked against- the tree's `hosts` at load time where it is literal and at run time where- it is templated.+- **No endpoint a tree did not name.** Every HTTP node's host is checked+ against the tree's `hosts` at load time where it is literal and at run+ time where it is templated. Every further form of I/O arrives with an+ allow-list of the same kind, or does not arrive.- **No code in trees.** Templates are paths and filters; conditions are comparisons. Branching is only ever a `decide` node, visible in the trace. - **No loops.** A path through a tree is finite by construction.@@ -159,7 +193,8 @@ caller's trace.``` src/pyrtan/ the package: tree, intent, template, conditions, extract,- html, transport, runtime, trace, registry, service, hub, cli+ html, transport, runtime, trace, registry, grow, service,+ hub, cliexamples/ trees that run against real public APIs, and offline against tests/fixtures/examples.json tests/ pytest; a swarm of two services behind a hub runs on localhost@@ -170,10 +205,12 @@ deploy/ Dockerfile at the root; a compose stack for OpenMediaVault## Status-0.1.0. The core, the service, the hub and the command are built and tested.-Not yet built: a browser adapter (for pages that need a script to render),-authentication between services, persistent gap storage, and a tree-marketplace. See [`_meta/design.md`](_meta/design.md) for the sequencing.+0.1.0. The core, the service, the hub and the command are built and tested,+and HTTP is the one form of I/O the core speaks. Not yet built: further+adapters (a browser for pages that need a script to render; files, shells,+serial lines, message queues), authentication between services, persistent+storage for gaps and unanswered sentences, and a tree marketplace. See+[`_meta/design.md`](_meta/design.md) for the sequencing.Pyrtan is a product of [Hillvon.io](https://hillvon.io). diff --git a/_meta/design.md b/_meta/design.md index f42792c..1a142bb 100644 --- a/_meta/design.md +++ b/_meta/design.md@@ -2,7 +2,7 @@| | | |---|---|-| **Status** | **Built, v0.1.** The core, the service, the hub and the command exist and are tested (111 tests, including a two-service swarm behind a hub on localhost). No deployment, no users, no browser adapter yet. |+| **Status** | **Built, v0.1.** The core, the service, the hub and the command exist and are tested (133 tests, including a two-service swarm behind a hub on localhost). No deployment, no users; HTTP is the only form of I/O the core speaks yet. || **Date** | 2026-10-24 | | **Decision** | Pyrtan is an open-source automation tool: a core set of expandable services, each holding declarative decision trees that read and act on all forms of I/O, routed by a hub, with no model in the loop and a complete trace of every run. | | **Where** | This repository: `src/pyrtan`, `examples/`, `tests/`. AGPL-3.0-or-later. |@@ -21,17 +21,25 @@ output, and an explanation. Inside there is no model. The explanation isnot a model's account of itself; it is the literal path through the tree. **"Hardcoded decision trees of the web."** A tree is a file an author-writes, naming the hosts it may touch, the requests it makes, the values it-extracts, and the comparisons that choose the next step. "Of the web" means-the tree encodes how one web property behaves: what a 404 means here, which-JSON path holds the title, which weather code means rain. The knowledge a-model would hold implicitly, a tree holds explicitly, for one site.--**"Swarm of services."** A tree is held by a service; a service is a small-process that answers `/intent` and `/run`; a hub routes between services-and reports which are up. The swarm is the unit of deployment and of-ownership: one team can own the trees for one property, run them on their-own box, and register with a shared hub.+writes, naming the endpoints it may touch, the requests it makes, the+values it extracts, and the comparisons that choose the next step. "Of the+web" means the tree encodes how one property behaves: what a 404 means+here, which JSON path holds the title, which weather code means rain. The+knowledge a model would hold implicitly, a tree holds explicitly, for one+property. The web was the brief's example and is the first form of I/O the+core speaks; the shape is the same for any other. An adapter names what it+may reach, a node observes or acts through it, and a `decide` reads what+came back. A file, a shell, a serial line or a queue is the same tree with+a different adapter under it.++**"Swarm of services."** Read as a core set of expandable services. A tree+is held by a service; a service is a small process that answers `/intent`+and `/run`; a hub routes between services and reports which are up. The+core is the runtime, the service and the hub; what expands is the set of+trees, the set of services holding them, and the forms of I/O they reach.+The swarm is the unit of deployment and of ownership: one team can own the+trees for one property, run them on their own box, and register with a+shared hub.**"Similar to Claude skills but much more focused and dynamic."** A skill is a prose instruction package a model reads and interprets. A tree is@@ -41,8 +49,9 @@ delegate to any other tree in the swarm by sentence or by name, resolved atrun time; and a decision that meets a value no branch covers is recorded as a *gap*, so the set of trees grows where reality disagreed with its author.-**"Total automation."** Read as the ambition, not as a claim. v0.1 reads-APIs and pages and can POST. What it does not yet do is in Section 7.+**"Total automation."** Read as the ambition, not as a claim. v0.1 speaks+HTTP: it reads APIs and pages and can POST. Other forms of I/O are adapters+still to come (Section 6); what is open is in Section 7.## 1. What this design refuses@@ -60,9 +69,12 @@ Four things a reader might expect, and will not find.shows. The cost is verbosity (the weather tree spends seven `set` nodes turning a code into a word); the benefit is that a tree can be audited by someone who does not program.-- **Reaching a host the author did not list.** `hosts` is checked at load- time for literal URLs and at run time for templated ones. A tree that- takes a host from its input must say `hosts = ["*"]` out loud.+- **Reaching an endpoint the author did not list.** `hosts` is checked at+ load time for literal URLs and at run time for templated ones. A tree+ that takes a host from its input must say `hosts = ["*"]` out loud. The+ same rule binds every later form of I/O: an adapter arrives with an+ allow-list of the same kind (paths, devices, queues), or it does not+ arrive.- **Loops.** A tree is a directed acyclic graph; `pyrtan check` names the cycle and refuses. Repetition is `each` over a list with a limit, so every run is finite and every trace is complete.@@ -78,8 +90,9 @@ traded away by a convenient feature.2. **The trace is complete.** Every node visited appears, with what it observed and, for a decision, which condition held. Delegated runs appear inside the caller's trace one level deeper.-3. **Hosts are closed.** No request leaves to a host outside the tree's- list, whether the URL was literal or built at run time.+3. **Endpoints are closed.** No request leaves to a host outside the+ tree's list, whether the URL was literal or built at run time, and no+ future adapter is admitted without an allow-list of the same kind.4. **Reads retry; writes do not.** An `observe` may be retried on transport failure; an `act` is attempted once. The checker refuses `retries` on an act and refuses a write method on an observe.@@ -101,9 +114,11 @@ sentence ──► intent.py ──► registry.py ──► runtime.py ──│ └─ html.py parser + selector subset └─ transport.py urllib | recorded | recording-service.py one process, one registry, HTTP endpoints, hot reload, gaps+grow.py an unanswered sentence -> a proposed tree, a scaffold, a log+service.py one process, one registry, HTTP endpoints, hot reload, gaps,+ unansweredhub.py peers, link up/stale/down, routing by capability, forwarding-cli.py check · list · run · ask · serve · hub · gaps · new+cli.py check · list · run · ask · serve · hub · gaps · unanswered · new``` **The tree** (`tree.py`) is TOML, loaded into dataclasses and checked on@@ -132,7 +147,22 @@ carry.the standard library's client, a recorded fixture that raises on any request it did not foresee, and a recorder that writes such fixtures. Every example tree is tested offline through the second, and-`pyrtan run --record` produces fixtures through the third.+`pyrtan run --record` produces fixtures through the third. The transport+is also the seam where forms of I/O plug in. A node names a source (`http`+today), the runtime enforces that source's allow-list, and the transport+moves the bytes. A file, shell, serial or queue adapter is a new node+source with the same three parts, and the same recorded and recording+wrappers around it.++**Growth** (`grow.py`) is where a dead end becomes the next tree. A+sentence no tree answers is recorded as *unanswered*, counted, and+answered with a proposal: a name drawn from the words, a trigger with slots+where the sentence had numbers or hosts, the inputs those slots need, and+the `pyrtan new … --from` command that writes a scaffold which passes+`check`. The proposal is derived by rules, not by a model, and it is a+first draft for an author, not an answer for a caller. A service without a+hub keeps its own unanswered log; one with a hub leaves the record to the+hub, which also collects its peers' at `/unanswered`.**The service** (`service.py`) wraps a registry in the standard library's threading HTTP server. It reloads trees from disk on every request that@@ -174,8 +204,10 @@ to end.**A model fallback for unmatched sentences.** The obvious feature, and the one that would quietly turn the product into the thing it is defined-against. Not built. A sentence no tree answers returns `no-match`, and the-honest response to a frequent `no-match` is a new tree.+against. Not built. A sentence no tree answers returns `no-match`, is+recorded as unanswered, and comes back with the tree that would answer it.+The honest response to a frequent `no-match` is a new tree, and the swarm+now hands the author its first draft rather than an excuse.**A message bus between services.** NATS or similar would give the swarm events and fan-out. v0.1 has `act.emit`, which records an event on the@@ -183,8 +215,9 @@ service; nothing consumes it yet. Request/response over HTTP covers everycase the examples needed, and a bus would be a dependency and an operational component before there is a swarm large enough to need one.-**Persisting gaps.** Gaps live in a service's memory and are lost on-restart. Enough to prove the loop (run, see gap, add branch, reload), not+**Persisting gaps and unanswered sentences.** Both live in a service's or+hub's memory and are lost on restart. Enough to prove the two loops (run,+see gap, add branch, reload; ask, see unanswered, scaffold, reload), notenough for a fleet. Deferred to Section 7. ## 5. What the examples show@@ -208,9 +241,13 @@ Five trees, all against real public APIs, all tested offline:file per service and collected by the hub. A `pyrtan watch` that re-runs a set of sentences on a schedule and reports when a path changes (the way to learn a site has changed before a person does).-3. **v0.3: browser adapter.** `observe` with `browser = { url, wait_for,- actions }` on Playwright as an optional extra, for pages that need a- script to render, with the same host rule and the same extraction.+3. **v0.3: more forms of I/O.** First the browser: `observe` with+ `browser = { url, wait_for, actions }` on Playwright as an optional+ extra, for pages that need a script to render, with the same host rule+ and the same extraction. Then file, shell, serial and queue adapters,+ each a node source with its own allow-list (paths, commands, devices,+ topics), so a tree reads a log, runs a named command or polls a device+ by the same rules it reads a page, and the trace reads the same.4. **v0.4: trees as a shared thing.** A directory of trees people can pull by name, with the checker run on contribution, and `version` meaning something.@@ -239,8 +276,10 @@ Five trees, all against real public APIs, all tested offline:## 8. What this design does not claim-It does not claim to answer sentences nobody wrote a tree for. It does not-claim trees are quick to write: the weather tree is 15 nodes for a two-call+It does not claim to answer sentences nobody wrote a tree for; it claims+to say which tree would, and to write that tree's first draft. It does not+claim to reach anything beyond HTTP yet. It does not claim trees are quick+to write: the weather tree is 15 nodes for a two-callAPI, and that is the price of leaving nothing implicit. It does not claim the swarm is secure across a network boundary (Section 7). It does not claim the examples will keep working as the sites they read change; it diff --git a/_meta/tree-format.md b/_meta/tree-format.md index 2705f56..ea344b6 100644 --- a/_meta/tree-format.md +++ b/_meta/tree-format.md@@ -228,3 +228,8 @@ empty string into a URL.Every outcome carries `trace`: `steps` (node, kind, summary, observed, chosen, next, elapsed_ms, depth) and `gaps`.++A `no-match` error also carries `output.proposal`: the tree that would+answer the sentence, as `name`, `trigger` (slots where the sentence had+numbers or hosts), `inputs`, and `scaffold`, the `pyrtan new … --from`+command that writes it. The sentence is recorded at `/unanswered`.diff --git a/deploy/README.md b/deploy/README.md index 53307b1..a1a16fb 100644 --- a/deploy/README.md +++ b/deploy/README.md@@ -112,4 +112,7 @@ same `uv tool install` line with `--force`, then`hacker-news-item` three times. - Hot reload: edit a file in `trees/` from the share and ask again. - Gaps: a decision that met a value no branch covers, at `/gaps`.+- Unanswered: ask something no tree answers and the page shows the tree+ that would, with the command that writes it; the list is at+ `/unanswered`.- Health as one field: `/peers` on the hub shows each service's `link`. diff --git a/src/pyrtan/__init__.py b/src/pyrtan/__init__.py index 0e37601..038ac85 100644 --- a/src/pyrtan/__init__.py +++ b/src/pyrtan/__init__.py@@ -7,6 +7,8 @@ The public surface is small on purpose:- :class:`pyrtan.Registry` holds the trees a service knows about. - :class:`pyrtan.Runner` executes a tree against a transport and returns an :class:`pyrtan.Outcome` with a full :class:`pyrtan.Trace`.+- :func:`pyrtan.grow.propose` turns a sentence nobody answers into the tree+ that would.""" from .tree import Tree, load_tree, load_tree_text, TreeError diff --git a/src/pyrtan/cli.py b/src/pyrtan/cli.py index 8f4710e..8a0e655 100644 --- a/src/pyrtan/cli.py +++ b/src/pyrtan/cli.py@@ -7,8 +7,9 @@pyrtan ask TEXT --at URL answer a sentence through a service or hub pyrtan serve [PATH ...] [--port N] host trees as a service pyrtan hub [--peer URL ...] route between services- pyrtan gaps [PATH ... | --at URL] branches waiting to be written- pyrtan new NAME [--dir PATH] scaffold a tree file+ pyrtan gaps --at URL branches waiting to be written+ pyrtan unanswered --at URL [--scaffold DIR] trees waiting to be written+ pyrtan new [NAME] [--from TEXT] [--dir PATH] scaffold a tree file""" from __future__ import annotations@@ -21,49 +22,13 @@ from pathlib import Pathfrom typing import Any from . import __version__+from .grow import Proposal, proposefrom .registry import Registry, default_paths from .runtime import Outcome, Runner from .service import RemoteResolver, http_json, outcome_from_dict, serve from .transport import RecordedTransport, RecordingTransport, UrllibTransport from .tree import check, load_tree-TEMPLATE = '''# {name}: say what this tree does, in one line, below.--name = "{name}"-version = 1-description = ""-tags = []-hosts = ["example.com"]-start = "fetch"--[[triggers]]-pattern = "{words}"--[nodes.fetch]-kind = "observe"-http = {{ method = "GET", url = "https://example.com/" }}-into = "resp"-extract = [{{ into = "title", css = "title" }}]-next = "status"--[nodes.status]-kind = "decide"-on = "resp.status"-branches = [- {{ when = {{ eq = 200 }}, then = "reply" }},- {{ else = "unexpected" }},-]--[nodes.reply]-kind = "done"-say = "{{{{ state.title }}}}"-output = {{ title = "{{{{ state.title }}}}" }}--[nodes.unexpected]-kind = "fail"-reason = "unexpected-response"-say = "example.com answered {{{{ state.resp.status }}}}."-'''def main(argv: list[str] | None = None) -> int:@@ -118,12 +83,18 @@ def main(argv: list[str] | None = None) -> int:p.add_argument("--at", help="a running service or hub") p.add_argument("--json", action="store_true")+ p = sub.add_parser("unanswered", help="sentences nobody answered, each with the tree that would")+ p.add_argument("--at", help="a running service or hub")+ p.add_argument("--json", action="store_true")+ p.add_argument("--scaffold", metavar="DIR", help="write a tree file for each sentence into DIR (existing files are kept)")+p = sub.add_parser("new", help="scaffold a tree file")- p.add_argument("name")+ p.add_argument("name", nargs="?", help="the tree's name (derived from --from when omitted)")+ p.add_argument("--from", dest="sentence", metavar="TEXT", help="the sentence the tree should answer; numbers and hosts in it become slots")p.add_argument("--dir", default=".") args = parser.parse_args(argv)- return {"check": cmd_check, "list": cmd_list, "run": cmd_run, "ask": cmd_ask, "serve": cmd_serve, "hub": cmd_hub, "gaps": cmd_gaps, "new": cmd_new}[args.command](args)+ return {"check": cmd_check, "list": cmd_list, "run": cmd_run, "ask": cmd_ask, "serve": cmd_serve, "hub": cmd_hub, "gaps": cmd_gaps, "unanswered": cmd_unanswered, "new": cmd_new}[args.command](args)# ---------------------------------------------------------------- commands@@ -321,15 +292,58 @@ def cmd_gaps(args: argparse.Namespace) -> int:return 0+def cmd_unanswered(args: argparse.Namespace) -> int:+ if not args.at:+ print("pyrtan unanswered: give --at URL of a running service or hub", file=sys.stderr)+ return 2+ try:+ status, body = http_json("GET", f"{args.at.rstrip('/')}/unanswered")+ except Exception as e:+ print(f"pyrtan: {args.at}: {e}", file=sys.stderr)+ return 3+ entries = (body or {}).get("unanswered", []) if isinstance(body, dict) else []+ if args.json:+ print(json.dumps({"unanswered": entries}, indent=2, ensure_ascii=False))+ return 0+ if not entries:+ print("nothing unanswered: every sentence so far found a tree")+ return 0+ for e in entries:+ prop = e.get("proposal") or propose(e["text"]).to_dict()+ peer = f" ({e['peer']})" if e.get("peer") else ""+ print(f"{e['text']!r} x{e.get('count', 1)}{peer}")+ print(f" would be: {prop['name']} answering {prop['trigger']}")+ if args.scaffold:+ path = _write_scaffold(Proposal(prop["text"], prop["name"], prop["trigger"], prop.get("inputs") or {}), Path(args.scaffold))+ print(f" {'wrote' if path else 'kept'} {Path(args.scaffold) / (prop['name'] + '.toml')}")+ else:+ print(f" {prop['scaffold']}")+ if args.scaffold:+ print(f"now: pyrtan check {args.scaffold}")+ return 0+++def _write_scaffold(proposal: Proposal, directory: Path) -> Path | None:+ """Write the proposal's tree file unless one is already there."""+ path = directory / f"{proposal.name}.toml"+ if path.exists():+ return None+ path.parent.mkdir(parents=True, exist_ok=True)+ path.write_text(proposal.scaffold(), encoding="utf-8")+ return path++def cmd_new(args: argparse.Namespace) -> int:- name = args.name- path = Path(args.dir) / f"{name}.toml"+ if not args.name and not args.sentence:+ print("pyrtan new: give a NAME, or --from \"the sentence it should answer\"", file=sys.stderr)+ return 2+ proposal = propose(args.sentence or args.name.replace("-", " "), name=args.name)+ path = Path(args.dir) / f"{proposal.name}.toml"if path.exists(): print(f"pyrtan: {path} exists", file=sys.stderr) return 1- path.parent.mkdir(parents=True, exist_ok=True)- path.write_text(TEMPLATE.format(name=name, words=name.replace("-", " ")), encoding="utf-8")- print(f"wrote {path}; edit it, then: pyrtan check {path}")+ _write_scaffold(proposal, Path(args.dir))+ print(f"wrote {path}: {proposal.name!r} answering {proposal.trigger!r}; edit it, then: pyrtan check {path}")return 0 diff --git a/src/pyrtan/grow.py b/src/pyrtan/grow.py new file mode 100644 index 0000000..f659210 --- /dev/null +++ b/src/pyrtan/grow.py@@ -0,0 +1,262 @@+"""Where the swarm grows: sentences nobody answered, and the tree that would.++A ``decide`` that meets a value no branch covers records a *gap*: a branch+waiting to be written. A sentence no tree answers is the same thing one+level up: a tree waiting to be written. This module gives that second case+the same treatment the first already has.++- :func:`propose` turns a sentence into a :class:`Proposal`: a tree name+ drawn from its words, a trigger pattern with slots where the sentence had+ numbers or hosts, and the inputs those slots need.+- :meth:`Proposal.scaffold` renders the proposal as a tree file an author+ fills in, the same skeleton ``pyrtan new`` writes.+- :class:`Unanswered` is the log a service or hub keeps of sentences it+ could not answer, counted, each with its proposal. ``GET /unanswered``+ serves it; ``pyrtan unanswered --at URL --scaffold DIR`` turns it into+ files.++No model is involved. The proposal is derived from the words by rules, and+it is a starting point, not an answer: the author decides what the tree+observes and how it decides.+"""++from __future__ import annotations++import re+import shlex+import threading+import time+from dataclasses import dataclass, field+from typing import Any++from .intent import Trigger, TriggerError, normalise++# Words that carry no meaning for a tree's name: dropped from the name,+# kept in the trigger (where "what's the" is worth matching literally).+FILLER = {"a", "an", "the", "what", "what's", "whats", "how", "please", "me", "my", "tell", "show", "give", "is", "are", "of", "to", "for", "in", "on", "at", "do", "does", "it", "and", "with"}+INT_RE = re.compile(r"^-?\d+$")+FLOAT_RE = re.compile(r"^-?\d+\.\d+$")+HOST_RE = re.compile(r"^[a-z0-9-]+(\.[a-z0-9-]+)+(:\d+)?$", re.IGNORECASE)+PATH_RE = re.compile(r"^[\w.\-]+/[\w.\-/]+$")+NAME_MAX = 40+UNANSWERED_LIMIT = 200++SCAFFOLD = '''# {name}: say what this tree does, in one line, below.+# Proposed from the sentence: {text}++name = "{name}"+version = 1+description = ""+tags = []+hosts = {hosts}+start = "fetch"++[[triggers]]+pattern = "{trigger}"+{inputs}+[nodes.fetch]+kind = "observe"+http = {{ method = "GET", url = "{url}" }}+into = "resp"+extract = [{{ into = "title", css = "title" }}]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ {{ when = {{ eq = 200 }}, then = "reply" }},+ {{ else = "unexpected" }},+]++[nodes.reply]+kind = "done"+say = "{{{{ state.title }}}}"+output = {{ title = "{{{{ state.title }}}}" }}++[nodes.unexpected]+kind = "fail"+reason = "unexpected-response"+say = "{host_word} answered {{{{ state.resp.status }}}}."+'''++INPUT_BLOCK = '''+[inputs.{slot}]+type = "{type}"+description = "{description}"+'''+++@dataclass+class Proposal:+ """A tree that would answer ``text``, before anyone has written it."""++ text: str+ name: str+ trigger: str+ inputs: dict[str, dict[str, Any]] = field(default_factory=dict)++ def to_dict(self) -> dict[str, Any]:+ return {"text": self.text, "name": self.name, "trigger": self.trigger, "inputs": self.inputs, "scaffold": self.command()}++ def command(self, directory: str | None = None) -> str:+ """The ``pyrtan new`` invocation that writes this tree's file."""+ parts = ["pyrtan", "new", self.name, "--from", _quote(self.text)]+ if directory:+ parts += ["--dir", _quote(directory)]+ return " ".join(parts)++ def scaffold(self) -> str:+ """The tree file, ready for ``pyrtan check`` and for an author."""+ host_slot = next((s for s, spec in self.inputs.items() if spec.get("role") == "host"), None)+ if host_slot:+ hosts, url, host_word = '["*"]', f"https://{{{{ inputs.{host_slot} | lower }}}}/", f"{{{{ inputs.{host_slot} }}}}"+ else:+ hosts, url, host_word = '["example.com"]', "https://example.com/", "example.com"+ inputs = "".join(INPUT_BLOCK.format(slot=slot, type=spec["type"], description=spec.get("description", "")) for slot, spec in self.inputs.items())+ return SCAFFOLD.format(name=self.name, text=_toml_safe(self.text), hosts=hosts, trigger=_toml_safe(self.trigger), inputs=inputs, url=url, host_word=host_word)+++def propose(text: str, name: str | None = None) -> Proposal:+ """A proposal for the sentence ``text``.++ Numbers in the sentence become ``{count:int}`` / ``{value:float}``+ slots, host names ``{host:word}``, ``owner/repo``-shaped words+ ``{target:word}``; every other word is literal. The name is the literal+ words with filler dropped, or ``name`` if given.+ """+ clean = normalise(text) or text.strip() or "new tree"+ tokens = clean.split(" ")+ pieces: list[str] = []+ inputs: dict[str, dict[str, Any]] = {}+ literal_words: list[str] = []+ for tok in tokens:+ slot = _slot_for(tok, inputs)+ if slot is not None:+ pieces.append(slot)+ else:+ pieces.append(_literal(tok))+ literal_words.append(tok)+ trigger = " ".join(p for p in pieces if p)+ try:+ Trigger(trigger)+ except TriggerError:+ # Everything became a slot, or the words left nothing matchable:+ # fall back to the sentence itself, literally.+ trigger = " ".join(_literal(t) for t in tokens if _literal(t)) or "new tree"+ inputs = {}+ literal_words = tokens+ return Proposal(text=clean, name=_name_for(name, literal_words, tokens, list(inputs)), trigger=trigger, inputs=inputs)+++def _slot_for(tok: str, inputs: dict[str, dict[str, Any]]) -> str | None:+ bare = tok.strip("\"'(),;:!?")+ if INT_RE.match(bare):+ slot = _unique("count", inputs)+ inputs[slot] = {"type": "int", "description": f"as in: {bare}"}+ return f"{{{slot}:int}}"+ if FLOAT_RE.match(bare):+ slot = _unique("value", inputs)+ inputs[slot] = {"type": "float", "description": f"as in: {bare}"}+ return f"{{{slot}:float}}"+ if HOST_RE.match(bare) and not bare.endswith(("...", ".")):+ slot = _unique("host", inputs)+ inputs[slot] = {"type": "str", "description": f"a host name, as in: {bare}", "role": "host"}+ return f"{{{slot}:word}}"+ if PATH_RE.match(bare):+ slot = _unique("target", inputs)+ inputs[slot] = {"type": "str", "description": f"as in: {bare}"}+ return f"{{{slot}:word}}"+ return None+++def _unique(base: str, taken: dict[str, Any]) -> str:+ if base not in taken:+ return base+ n = 2+ while f"{base}{n}" in taken:+ n += 1+ return f"{base}{n}"+++def _literal(tok: str) -> str:+ # Trigger syntax characters would be read as slots or groups; a literal+ # word must not contain them.+ return re.sub(r"[{}\[\]()|]", "", tok).strip()+++def _name_for(given: str | None, literal_words: list[str], tokens: list[str], slots: list[str] = ()) -> str:+ if given:+ return given+ words = [re.sub(r"[^a-z0-9]+", "-", w.lower()).strip("-") for w in literal_words if w.lower() not in FILLER]+ words = [w for w in words if w]+ if len(words) < 2 and slots:+ # "is example.com up" would be called "up"; "host-up" says more.+ words = [re.sub(r"\d+$", "", s) for s in slots] + words+ if not words:+ words = [re.sub(r"[^a-z0-9]+", "-", w.lower()).strip("-") for w in tokens]+ words = [w for w in words if w]+ name = "-".join(words)[:NAME_MAX].strip("-")+ name = re.sub(r"-{2,}", "-", name)+ if not name or not name[0].isalnum():+ name = "new-tree"+ return name+++def _quote(s: str) -> str:+ """Shell-quote ``s``, preferring double quotes so an apostrophe in a+ sentence does not turn into ``'what'"'"'s'``."""+ if re.fullmatch(r"[\w./:@%+=,-]+", s):+ return s+ if not any(c in s for c in '"$`\\'):+ return f'"{s}"'+ return shlex.quote(s)+++def _toml_safe(s: str) -> str:+ return s.replace("\\", "\\\\").replace('"', '\\"')+++def no_match_outcome(text: str, where: str) -> dict[str, Any]:+ """The ``say`` and ``output`` an unanswered sentence gets back: what+ happened, and the one command that would change it."""+ p = propose(text)+ say = f"{where} answers {text!r}. To write the tree that would: {p.command()}"+ return {"say": say, "output": {"unanswered": text, "proposal": p.to_dict()}}+++class Unanswered:+ """Sentences nobody answered, counted, with a proposal for each.++ Bounded: beyond ``limit`` distinct sentences the least recently seen is+ dropped. Lives in memory, like gaps; a restart forgets it.+ """++ def __init__(self, limit: int = UNANSWERED_LIMIT) -> None:+ self.limit = limit+ self._entries: dict[str, dict[str, Any]] = {}+ self._lock = threading.Lock()++ def record(self, text: str, peer: str | None = None) -> dict[str, Any]:+ key = normalise(text).lower() or text+ now = time.time()+ with self._lock:+ entry = self._entries.get(key)+ if entry is None:+ entry = {"text": normalise(text) or text, "count": 0, "first": now, "last": now, "proposal": propose(text).to_dict()}+ if peer:+ entry["peer"] = peer+ self._entries[key] = entry+ entry["count"] += 1+ entry["last"] = now+ while len(self._entries) > self.limit:+ oldest = min(self._entries, key=lambda k: self._entries[k]["last"])+ del self._entries[oldest]+ return dict(entry)++ def to_list(self) -> list[dict[str, Any]]:+ with self._lock:+ return sorted((dict(e) for e in self._entries.values()), key=lambda e: (-e["count"], -e["last"]))++ def __len__(self) -> int:+ return len(self._entries)diff --git a/src/pyrtan/hub.py b/src/pyrtan/hub.py index 8480497..75ed25f 100644 --- a/src/pyrtan/hub.py +++ b/src/pyrtan/hub.py@@ -16,6 +16,7 @@ Endpoints:``POST /run`` forwarded to the peer holding ``tree`` ``POST /resume`` forwarded to the peer named in the snapshot ``GET /gaps`` every peer's gaps, with the peer that saw them+``GET /unanswered`` sentences no peer answered, each with a proposed tree``GET /`` the demo page; ``GET /api`` is the JSON index A service that holds several trees and a hub that routes between services@@ -31,6 +32,7 @@ import timefrom dataclasses import dataclass, field from typing import Any+from .grow import Unanswered, no_match_outcomefrom .intent import Trigger, best_matches from .page import PAGE from .service import Server, http_json@@ -68,6 +70,7 @@ class Hub:self.name = name self.poll_every = poll_every self.peers: dict[str, Peer] = {}+ self.unanswered = Unanswered()self.started = time.time() self._lock = threading.RLock() self._stop = threading.Event()@@ -155,6 +158,7 @@ class Hub:"stale": sum(1 for p in peers if p.link == "stale"), "down": sum(1 for p in peers if p.link == "down"), "trees": sum(len(p.trees) for p in peers),+ "unanswered": len(self.unanswered),"uptime_s": int(time.time() - self.started), }@@ -172,7 +176,10 @@ class Hub:return 200, {"status": "error", "reason": "bad-request", "say": "Send {\"text\": \"...\"}."} matches = best_matches(text, self.candidates()) # type: ignore[arg-type] if not matches:- return 200, {"status": "error", "reason": "no-match", "say": f"No service in the swarm answers {text!r}."}+ entry = self.unanswered.record(text)+ answer = no_match_outcome(text, "No service in the swarm")+ answer["output"]["seen"] = entry["count"]+ return 200, {"status": "error", "reason": "no-match", **answer}tree_names = {m.tree[1] for m in matches} # type: ignore[index] if len(tree_names) > 1: return 200, {"status": "error", "reason": "ambiguous", "say": f"{text!r} matches more than one tree: {sorted(tree_names)}", "output": {"candidates": sorted(tree_names)}}@@ -221,6 +228,21 @@ class Hub:out.extend({**g, "peer": peer.url} for g in body.get("gaps", [])) return {"gaps": out}+ def unanswered_all(self) -> dict[str, Any]:+ """This hub's unanswered sentences, then each peer's own (a peer+ without a hub keeps its own record; one with a hub leaves it here)."""+ out = self.unanswered.to_list()+ for peer in list(self.peers.values()):+ if peer.link == "down":+ continue+ try:+ status, body = http_json("GET", f"{peer.url}/unanswered", timeout=5.0)+ except Exception:+ continue+ if status == 200 and isinstance(body, dict):+ out.extend({**u, "peer": peer.url} for u in body.get("unanswered", []))+ return {"unanswered": out}+def route(self, method: str, path: str, body: dict[str, Any] | None) -> tuple[int, Any]: body = body or {} if method == "GET" and path == "/health":@@ -232,10 +254,12 @@ class Hub:return 200, self.capabilities() if method == "GET" and path == "/gaps": return 200, self.gaps()+ if method == "GET" and path == "/unanswered":+ return 200, self.unanswered_all()if method == "GET" and path in ("/", "/index.html"): return 200, PAGE if method == "GET" and path == "/api":- return 200, {"pyrtan": "hub", "name": self.name, "endpoints": ["/health", "/peers", "/capabilities", "/intent", "/run", "/resume", "/gaps", "/register"]}+ return 200, {"pyrtan": "hub", "name": self.name, "endpoints": ["/health", "/peers", "/capabilities", "/intent", "/run", "/resume", "/gaps", "/unanswered", "/register"]}if method == "POST" and path == "/register": url = body.get("url") if not isinstance(url, str) or not url.startswith(("http://", "https://")): diff --git a/src/pyrtan/page.py b/src/pyrtan/page.py index 7972a88..9298fbd 100644 --- a/src/pyrtan/page.py +++ b/src/pyrtan/page.py@@ -2,8 +2,10 @@It lists what the swarm can answer, takes a sentence, posts it to ``/intent`` on the same origin, and shows the answer, the output and the-trace. No external assets, no build step; it exists so that someone can-point a browser at a running Pyrtan and understand it in a minute.+trace. When nothing answers, it shows the tree that would, and the command+that scaffolds it; the sentences nobody answered are listed beside what the+swarm can do. No external assets, no build step; it exists so that someone+can point a browser at a running Pyrtan and understand it in a minute.""" from __future__ import annotations@@ -42,7 +44,7 @@ PAGE = r"""<!doctype html><body> <main> <h1>Pyrtan <small id="who">…</small></h1>- <p class="lede">A core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees. Type a sentence a tree answers to; the answer comes back with the exact path that produced it. No model is involved.</p>+ <p class="lede">A core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees. Type a sentence a tree answers to; the answer comes back with the exact path that produced it. If nothing answers, you get the tree that would. No model is involved.</p><form id="ask"> <input type="text" id="text" placeholder="what's the weather in Oslo" autofocus autocomplete="off">@@ -58,7 +60,10 @@ PAGE = r"""<!doctype html><h2 style="font-size:17px;margin:32px 0 8px">What this swarm can answer</h2> <div class="card"><table id="caps"><thead><tr><th>Tree</th><th>Answers to</th><th>From</th></tr></thead><tbody></tbody></table></div>- <footer>The same endpoints the page uses: <code>POST /intent {"text": "…"}</code>, <code>POST /run {"tree": "…", "inputs": {…}}</code>, <code>GET /capabilities</code>, <code>GET /health</code>, <code>GET /gaps</code>. <a href="https://github.com/b-i-t-8/pyrtan">Source</a>.</footer>+ <h2 style="font-size:17px;margin:32px 0 8px">Waiting to be written</h2>+ <div class="card"><table id="unanswered"><thead><tr><th>Nobody answered</th><th>Would be</th><th>To write it</th></tr></thead><tbody><tr><td colspan="3" class="mute">Nothing yet: every sentence so far found a tree.</td></tr></tbody></table></div>++ <footer>The same endpoints the page uses: <code>POST /intent {"text": "…"}</code>, <code>POST /run {"tree": "…", "inputs": {…}}</code>, <code>GET /capabilities</code>, <code>GET /health</code>, <code>GET /gaps</code>, <code>GET /unanswered</code>. <a href="https://github.com/b-i-t-8/pyrtan">Source</a>.</footer></main> <script> const $ = (s) => document.querySelector(s);@@ -89,6 +94,19 @@ async function load() {const s = sample(t); if (s && !s.includes("…")) { const li = document.createElement("li"); li.innerHTML = `<a href="#" data-q="${esc(s)}">${esc(s)}</a>`; tries.appendChild(li); } } tries.addEventListener("click", (e) => { const a = e.target.closest("a[data-q]"); if (!a) return; e.preventDefault(); $("#text").value = a.dataset.q; ask(); });+ loadUnanswered();+}+async function loadUnanswered() {+ let u; try { u = await (await fetch("unanswered")).json(); } catch (e) { return; }+ const rows = (u.unanswered || []);+ if (!rows.length) return;+ const body = $("#unanswered tbody"); body.innerHTML = "";+ for (const e of rows) {+ const p = e.proposal || {};+ const tr = document.createElement("tr");+ tr.innerHTML = `<td>${esc(e.text)} <span class="mute">×${esc(e.count||1)}${e.peer ? " · " + esc(e.peer) : ""}</span></td><td><code>${esc(p.name||"")}</code><div class="mute"><code>${esc(p.trigger||"")}</code></div></td><td><code>${esc(p.scaffold||"")}</code></td>`;+ body.appendChild(tr);+ }} function renderTrace(tr) { if (!tr || !tr.steps) return "";@@ -111,6 +129,10 @@ async function ask(snapshot, answer) {const card = document.createElement("div"); card.className = "card"; let html = `<div><span class="status ${esc(r.status)}">${esc(r.status)}</span>${r.reason ? ` <span class="mute">(${esc(r.reason)})</span>` : ""} <span class="mute">· ${esc(r.tree||"")} · ${ms} ms${r.peer ? " · " + esc(r.peer) : ""}</span></div>`; if (r.say) html += `<div class="say">${esc(r.say)}</div>`;+ if (r.reason === "no-match" && r.output && r.output.proposal) {+ const p = r.output.proposal;+ html += `<div class="gap">No tree answers this yet. The one that would: <code>${esc(p.name)}</code>, answering <code>${esc(p.trigger)}</code>.</div><pre>${esc(p.scaffold)}</pre>`;+ }if (r.status === "asked") html += `<form class="resume"><input type="text" placeholder="${esc(r.question||"")}" autofocus><button>Answer</button></form>`; if (r.output != null) html += `<details open><summary>output</summary><pre>${esc(JSON.stringify(r.output, null, 2))}</pre></details>`; if (r.trace) html += `<details open><summary>trace</summary><pre>${esc(renderTrace(r.trace))}</pre></details>`;@@ -118,6 +140,7 @@ async function ask(snapshot, answer) {out.prepend(card); const rf = card.querySelector("form.resume"); if (rf) rf.addEventListener("submit", (e) => { e.preventDefault(); ask(r.snapshot, rf.querySelector("input").value); });+ if (r.reason === "no-match") loadUnanswered();$("#go").disabled = false; } $("#ask").addEventListener("submit", (e) => { e.preventDefault(); ask(); }); diff --git a/src/pyrtan/runtime.py b/src/pyrtan/runtime.py index 8d480a8..5932203 100644 --- a/src/pyrtan/runtime.py +++ b/src/pyrtan/runtime.py@@ -16,7 +16,9 @@ An outcome's ``status`` is one of:``error`` The run could not proceed: a transport failure, an unhandled observation (a ``decide`` with no matching branch, recorded as a gap), a host outside- the allow-list, a missing template value, or a limit reached.+ the allow-list, a missing template value, or a limit reached. A+ ``no-match`` error carries, in ``output.proposal``, the tree that would+ answer the sentence, so a dead end always points at the next tree.Nothing here calls a model. The runner is the whole of Pyrtan's "thinking"; the trace is the whole of its explanation.@@ -31,6 +33,7 @@ from typing import Any, Protocolfrom . import conditions from .extract import extract+from .grow import no_match_outcomefrom .registry import Ambiguous, NoMatch, Registry from .template import MISSING, TemplateError, render, resolve from .trace import Step, Trace@@ -107,7 +110,7 @@ class Runner:except NoMatch: if self.resolver is not None: return self.resolver.run_intent(text, depth)- return Outcome("error", "", reason="no-match", say=f"Nothing here answers {text!r}.")+ return Outcome("error", "", reason="no-match", **no_match_outcome(text, "Nothing here"))except Ambiguous as e: return Outcome("error", "", reason="ambiguous", say=str(e), output={"candidates": [m.tree for m in e.matches]}) return self.run_tree(match.tree, match.inputs, depth) diff --git a/src/pyrtan/service.py b/src/pyrtan/service.py index f641f1d..023cfce 100644 --- a/src/pyrtan/service.py +++ b/src/pyrtan/service.py@@ -15,6 +15,10 @@ Endpoints, all JSON:``GET /gaps`` Decisions that met a value no branch covered, since the service started. Each is a branch waiting to be written.+``GET /unanswered``+ Sentences no tree answered, counted, each with a proposed tree. Each is+ a tree waiting to be written; ``pyrtan unanswered --at URL --scaffold+ DIR`` writes the files.``GET /`` The demo page (:mod:`pyrtan.page`); ``GET /api`` is the JSON index.@@ -22,7 +26,7 @@ A service may be given a hub (``--hub``). Then it registers itself on start,and a ``delegate`` to a tree it does not hold is forwarded to the hub. The server is the standard library's threading HTTP server. It is meant for-a swarm of small processes on a trusted network, behind whatever ingress+a set of small processes on a trusted network, behind whatever ingressthe operator already runs; it does not do TLS or authentication itself. """@@ -38,6 +42,7 @@ from pathlib import Pathfrom typing import Any, Callable from .page import PAGE+from .grow import Unansweredfrom .registry import Registry from .runtime import Outcome, Runner from .trace import Trace@@ -104,6 +109,7 @@ class Service:self.started = time.time() self.runner = Runner(registry, transport or UrllibTransport(), resolver=RemoteResolver(self.hub) if self.hub else None, env=env or {}) self.gaps: list[dict[str, Any]] = []+ self.unanswered = Unanswered()self.runs = 0 self._lock = threading.Lock()@@ -111,7 +117,7 @@ class Service:def health(self) -> dict[str, Any]: self.registry.reload()- return {"status": "ok", "name": self.name, "trees": self.registry.names(), "errors": self.registry.errors, "runs": self.runs, "gaps": len(self.gaps), "uptime_s": int(time.time() - self.started)}+ return {"status": "ok", "name": self.name, "trees": self.registry.names(), "errors": self.registry.errors, "runs": self.runs, "gaps": len(self.gaps), "unanswered": len(self.unanswered), "uptime_s": int(time.time() - self.started)}def capabilities(self) -> dict[str, Any]: self.registry.reload()@@ -122,7 +128,11 @@ class Service:if not isinstance(text, str) or not text.strip(): return {"status": "error", "reason": "bad-request", "say": "Send {\"text\": \"...\"}."} self.registry.reload()- return self._finish(self.runner.run_intent(text, int(body.get("depth", 0))), body)+ outcome = self.runner.run_intent(text, int(body.get("depth", 0)))+ if outcome.reason == "no-match" and self.hub is None:+ # With a hub, the hub saw the sentence and keeps the record.+ self.unanswered.record(text)+ return self._finish(outcome, body)def run(self, body: dict[str, Any]) -> dict[str, Any]: tree = body.get("tree")@@ -165,10 +175,12 @@ class Service:return 200, self.capabilities() if method == "GET" and path == "/gaps": return 200, {"gaps": self.gaps}+ if method == "GET" and path == "/unanswered":+ return 200, {"unanswered": self.unanswered.to_list()}if method == "GET" and path in ("/", "/index.html"): return 200, PAGE if method == "GET" and path == "/api":- return 200, {"pyrtan": "service", "name": self.name, "endpoints": ["/health", "/capabilities", "/intent", "/run", "/resume", "/gaps"]}+ return 200, {"pyrtan": "service", "name": self.name, "endpoints": ["/health", "/capabilities", "/intent", "/run", "/resume", "/gaps", "/unanswered"]}if method == "POST" and path == "/intent": return 200, self.intent(body) if method == "POST" and path == "/run": diff --git a/src/pyrtan/transport.py b/src/pyrtan/transport.py index a530357..aa9fa49 100644 --- a/src/pyrtan/transport.py +++ b/src/pyrtan/transport.py@@ -1,6 +1,9 @@-"""How a tree reaches the web.+"""How a tree reaches the outside: the HTTP form of I/O.-A :class:`Transport` turns a request into a :class:`Response`. The real one is+A :class:`Transport` turns a request into a :class:`Response`. Further forms+of I/O (files, shells, serial lines, queues) arrive as further transports+behind the same seam, each with a recorded and a recording wrapper so trees+over them are testable offline the same way. The real one is:class:`UrllibTransport`, on the standard library. :class:`RecordedTransport` replays responses from a fixture, so a tree can be tested end to end with no network, and :class:`RecordingTransport` wraps a real transport to write such diff --git a/tests/test_cli.py b/tests/test_cli.py index 63b91ed..f990965 100644 --- a/tests/test_cli.py +++ b/tests/test_cli.py@@ -62,5 +62,49 @@ def test_new_scaffolds_a_tree_that_checks(tmp_path, capsys):assert run(capsys, "new", "my-thing", "--dir", str(tmp_path))[0] == 1+def test_new_from_a_sentence(tmp_path, capsys):+ code, out, _ = run(capsys, "new", "--from", "top 3 stories on hacker news", "--dir", str(tmp_path))+ assert code == 0 and "top {count:int} stories on hacker news" in out+ path = tmp_path / "top-stories-hacker-news.toml"+ assert path.exists() and "[inputs.count]" in path.read_text()+ assert run(capsys, "check", str(path))[0] == 0+ code, out, _ = run(capsys, "new", "hn", "--from", "top 3 stories on hacker news", "--dir", str(tmp_path))+ assert code == 0 and (tmp_path / "hn.toml").exists()+ assert run(capsys, "new", "--dir", str(tmp_path))[0] == 2+++def test_run_no_match_proposes_a_tree(capsys):+ code, out, _ = run(capsys, "run", "tell me a joke", "--trees", str(EXAMPLES), "--replay", str(FIXTURES / "examples.json"), "--json")+ assert code == 3+ data = json.loads(out)+ assert data["reason"] == "no-match" and data["output"]["proposal"]["name"] == "joke"++def test_gaps_needs_a_target(capsys): assert run(capsys, "gaps")[0] == 2+ assert run(capsys, "unanswered")[0] == 2+++def test_unanswered_lists_and_scaffolds(tmp_path, capsys):+ from pyrtan import Registry+ from pyrtan.service import Server, Service, http_json++ svc = Service(Registry.from_paths([EXAMPLES]), name="x")+ server = Server(svc.route, quiet=True).start()+ try:+ code, out, _ = run(capsys, "unanswered", "--at", server.url)+ assert code == 0 and "nothing unanswered" in out+ http_json("POST", f"{server.url}/intent", {"text": "ping example.org"})+ http_json("POST", f"{server.url}/intent", {"text": "ping example.org"})+ http_json("POST", f"{server.url}/intent", {"text": "tell me a joke"})+ code, out, _ = run(capsys, "unanswered", "--at", server.url)+ assert code == 0 and out.startswith("'ping example.org' x2") and "would be: host-ping answering ping {host:word}" in out and "pyrtan new joke --from" in out+ code, out, _ = run(capsys, "unanswered", "--at", server.url, "--json")+ assert [e["text"] for e in json.loads(out)["unanswered"]] == ["ping example.org", "tell me a joke"]+ code, out, _ = run(capsys, "unanswered", "--at", server.url, "--scaffold", str(tmp_path))+ assert code == 0 and "wrote" in out and (tmp_path / "host-ping.toml").exists() and (tmp_path / "joke.toml").exists()+ assert run(capsys, "check", str(tmp_path))[0] == 0+ code, out, _ = run(capsys, "unanswered", "--at", server.url, "--scaffold", str(tmp_path))+ assert "kept" in out and "wrote" not in out+ finally:+ server.stop()diff --git a/tests/test_grow.py b/tests/test_grow.py new file mode 100644 index 0000000..098da22 --- /dev/null +++ b/tests/test_grow.py@@ -0,0 +1,92 @@+"""An unanswered sentence becomes a proposed tree, a scaffold that checks,+and an entry in the unanswered log."""++import pytest++from pyrtan.grow import Proposal, Unanswered, no_match_outcome, propose+from pyrtan.intent import Trigger+from pyrtan.tree import check, load_tree_text+++@pytest.mark.parametrize(+ "text, name, trigger, inputs",+ [+ ("top 3 stories on hacker news", "top-stories-hacker-news", "top {count:int} stories on hacker news", {"count": 3}),+ ("is example.com up", "host-up", "is {host:word} up", {"host": "example.com"}),+ ("what's the latest release of astral-sh/uv", "latest-release", "what's the latest release of {target:word}", {"target": "astral-sh/uv"}),+ ("convert 12.5 km to miles", "convert-km-miles", "convert {value:float} km to miles", {"value": 12.5}),+ ("turn on the lights", "turn-lights", "turn on the lights", {}),+ ("Weather in Oslo?", "weather-oslo", "Weather in Oslo", {}),+ ("ping 1 then 2", "ping-then", "ping {count:int} then {count2:int}", {"count": 1, "count2": 2}),+ ],+)+def test_propose_names_slots_and_matches_the_sentence(text, name, trigger, inputs):+ p = propose(text)+ assert (p.name, p.trigger) == (name, trigger)+ assert Trigger(p.trigger).match(text) == inputs+ assert set(p.inputs) == set(inputs)+++def test_propose_survives_trigger_syntax_and_bare_numbers():+ p = propose("{weird} [tokens] | here")+ Trigger(p.trigger) # compiles+ assert "{" not in p.trigger.replace("{count", "")+ p = propose("42")+ assert p.trigger == "42" and p.inputs == {}+ assert propose("").name == "new-tree"+++def test_given_name_wins():+ assert propose("my thing", name="custom").name == "custom"+++@pytest.mark.parametrize("text", ["top 3 stories on hacker news", "is example.com up", "what's the latest release of astral-sh/uv", 'say "hi" to {everyone}', "turn on the lights"])+def test_scaffold_loads_and_checks_clean(text):+ p = propose(text)+ tree = load_tree_text(p.scaffold())+ assert tree.name == p.name+ assert [t.pattern for t in tree.triggers] == [p.trigger]+ assert [x for x in check(tree) if x.level == "error"] == []+ assert set(tree.inputs) == set(p.inputs)+++def test_host_slot_opens_hosts_and_builds_the_url():+ tree = load_tree_text(propose("is example.com up").scaffold())+ assert tree.hosts == ["*"]+ assert "inputs.host" in tree.node("fetch").get("http")["url"]+++def test_command_quotes_for_a_shell():+ assert propose("top 3 stories on hacker news").command() == 'pyrtan new top-stories-hacker-news --from "top 3 stories on hacker news"'+ assert propose("what's up").command("trees") == 'pyrtan new up --from "what\'s up" --dir trees'+ assert propose("costs $5").command().endswith("--from 'costs $5'")+++def test_no_match_outcome_points_at_the_next_tree():+ out = no_match_outcome("tell me a joke", "Nothing here")+ assert out["say"].startswith("Nothing here answers 'tell me a joke'. To write the tree that would: pyrtan new joke --from")+ assert out["output"]["unanswered"] == "tell me a joke"+ assert out["output"]["proposal"]["name"] == "joke"+++def test_unanswered_counts_and_bounds():+ log = Unanswered(limit=2)+ first = log.record("Tell me a joke")+ assert first["count"] == 1 and first["proposal"]["name"] == "joke"+ assert log.record("tell me a joke!")["count"] == 2 # same sentence, normalised+ log.record("second")+ log.record("third")+ texts = [e["text"] for e in log.to_list()]+ assert texts == ["third", "second"] # the least recently seen went+ log = Unanswered()+ log.record("rare")+ log.record("common")+ log.record("common")+ assert [e["count"] for e in log.to_list()] == [2, 1] # most seen first+++def test_proposal_round_trips_through_its_dict():+ p = propose("is example.com up")+ d = p.to_dict()+ again = Proposal(d["text"], d["name"], d["trigger"], d["inputs"])+ assert again.scaffold() == p.scaffold()diff --git a/tests/test_service_hub.py b/tests/test_service_hub.py index ad07a2e..6493b03 100644 --- a/tests/test_service_hub.py +++ b/tests/test_service_hub.py@@ -94,6 +94,12 @@ def test_service_endpoints(example_service):assert out["status"] == "done" and "trace" not in out _, out = http_json("POST", f"{server.url}/intent", {"text": "nothing matches this"}) assert out["reason"] == "no-match"+ assert out["output"]["proposal"]["trigger"] == "nothing matches this"+ assert "pyrtan new nothing-matches-this --from" in out["say"]+ _, un = http_json("GET", f"{server.url}/unanswered")+ assert [(u["text"], u["count"]) for u in un["unanswered"]] == [("nothing matches this", 1)]+ _, health = http_json("GET", f"{server.url}/health")+ assert health["unanswered"] == 1status, out = http_json("GET", f"{server.url}/nowhere") assert status == 404 _, out = http_json("POST", f"{server.url}/run", {"tree": "site-status", "inputs": {"host": "teapot.example"}})@@ -170,6 +176,18 @@ def test_swarm_of_two_behind_a_hub(tmp_path):_, out = http_json("POST", f"{hub_srv.url}/run", {"tree": "nope"}) assert out["reason"] == "no-such-tree"+ # A sentence nobody answers: the hub says so, proposes the tree, and+ # remembers. B, which has a hub, leaves the record to the hub.+ _, out = http_json("POST", f"{hub_srv.url}/intent", {"text": "trace route to 10.0.0.1"})+ assert out["reason"] == "no-match" and out["output"]["seen"] == 1+ assert out["say"].startswith("No service in the swarm answers 'trace route to 10.0.0.1'. To write the tree that would: pyrtan new trace-route --from")+ _, out = http_json("POST", f"{srv_b.url}/intent", {"text": "trace route to 10.0.0.1"})+ assert out["reason"] == "no-match"+ assert http_json("GET", f"{srv_b.url}/unanswered")[1]["unanswered"] == []+ _, un = http_json("GET", f"{hub_srv.url}/unanswered")+ assert [(u["text"], u["count"]) for u in un["unanswered"]] == [("trace route to 10.0.0.1", 2)]+ assert un["unanswered"][0]["proposal"]["inputs"]["host"]["type"] == "str"+# A peer that dies is marked and skipped. srv_a.stop() hub.refresh_all()