Merge pull request #10 from b-i-t-8/claude/ecstatic-euler-rki9t4
b-i-t-8 · 2026-10-25 · 0be7ddebeeb11c62e6d584e5d1de66afa4079329 · parents f8b9801 a97167a
Trees in the browser: pyrtan.js, conformance vectors, trusted members that run trees (0.6.0)
Changes
README.md | 23 +- _meta/design.md | 9 +- _meta/members.md | 75 +- deploy/README.md | 7 + deploy/openmediavault/docker-compose.yml | 2 + pyproject.toml | 2 +- site/README.md | 2 +- src/pyrtan/__init__.py | 2 +- src/pyrtan/catalog.py | 2 +- src/pyrtan/cli.py | 10 +- src/pyrtan/federation.py | 104 ++- src/pyrtan/hub.py | 56 +- src/pyrtan/js/pyrtan.js | 654 ++++++++++++++++ src/pyrtan/page.py | 88 ++- src/pyrtan/service.py | 6 +- src/pyrtan/transport.py | 2 +- tests/conformance/make_vectors.py | 92 +++ tests/conformance/run.cjs | 37 + tests/conformance/vectors.json | 2088 ++++++++++++++++++++++++++++++++++++++++++++++++++ tests/test_conformance.py | 33 + tests/test_federation.py | 111 +++ uv.lock | 2 +- 22 files changed, 3344 insertions(+), 63 deletions(-)
diff --git a/README.md b/README.md index 94647bd..6dd1235 100644 --- a/README.md +++ b/README.md@@ -212,9 +212,19 @@ hub's box dies and a new one comes up with the same name, its members'next announce says what they kept and the hub takes it back; `pyrtan kept` shows who holds what, `pyrtan fetch KEY` gets a blob back from whoever has it. Install the console to the home screen and it is the-phone app, offline-capable. Nothing secret is ever a blob. The design is-in [`_meta/members.md`](_meta/members.md). Members do not run trees yet;-that is the JavaScript runtime of v0.6. The answer to a federated+phone app, offline-capable. Nothing secret is ever a blob.++A member that joined **with the hub's token** is trusted, and **runs+trees**: the hub hands it its own trees and any `--handout DIR` (trees+for the devices, not the box) as blobs, the device announces them, and a+sentence for one of them goes into the device's mailbox as an `Ask` and+comes back as its `Answer`, with `peer: <member>` and `via: ["member"]`.+The runtime that does it is `pyrtan.js`, the same tree runtime in+JavaScript, served at `/pyrtan.js` by every hub and written beside the+console; `tests/conformance/` holds the vectors both runtimes must agree+on, generated from the examples by the Python runtime and checked+against the JavaScript one in the test suite. The design is in+[`_meta/members.md`](_meta/members.md). The answer to a federatedsentence carries `via: [the hub it crossed]`, and the console's Peers tab shows who follows whom. The design is in [`_meta/federation.md`](_meta/federation.md).@@ -359,10 +369,11 @@ site/ pyrtan.com: the landing page, the seed list, the catalog## Status-0.5.0. The core, the service, the hub, the command, the console, the+0.6.0. The core, the service, the hub, the command, the console, thecatalog client, the language rules, federation between hubs, the-gossiped directory, members (a browser or a phone as part of the swarm)-and the blobs they keep are built and tested; HTTP is the one form of I/O the core speaks; a swarm can+gossiped directory, members (a browser or a phone as part of the swarm),+the blobs they keep and the trees they run (`pyrtan.js`, conformant with+the Python runtime) are built and tested; HTTP is the one form of I/O the core speaks; a swarm canrequire a token, keep its gaps, refuse private addresses, and follow other swarms. Not yet built: further adapters (a browser for pages that need a script to render; files, shells, serial lines, message queues), diff --git a/_meta/design.md b/_meta/design.md index 17aca24..2107c4a 100644 --- a/_meta/design.md +++ b/_meta/design.md@@ -290,9 +290,12 @@ Five trees, all against real public APIs, all tested offline:joins a hub through a mailbox, carries the directory, and keeps the hub's blobs so a dead box loses nothing; the console installs as the phone app. Design in [`members.md`](members.md).-6. **v0.6: trees in the browser.** `pyrtan.js`, a second runtime for the- same tree files, with the Python fixtures as shared conformance- vectors, so a member answers sentences through its hub's mailbox.+6. **v0.6, built: trees in the browser.** `pyrtan.js`, a second runtime+ for the same tree files, with the Python fixtures as shared+ conformance vectors (`tests/conformance/`), so a trusted member+ answers sentences through its hub's mailbox (`Ask`/`Answer`); a hub+ hands its own trees and `--handout` ones to the devices that joined+ with its token. Design in [`members.md`](members.md) §7.7. **v0.7: more forms of I/O.** First the browser: `observe` with `browser = { url, wait_for, actions }` on Playwright as an optional extra, for pages that need a script to render, with the same host rule diff --git a/_meta/members.md b/_meta/members.md index e8dbae7..dbd3fa2 100644 --- a/_meta/members.md +++ b/_meta/members.md@@ -2,10 +2,10 @@| | | |---|---|-| **Status** | **Built, v0.5.** Members, mailboxes, `Keep`/`Kept`/`Fetch`, the blob store, the console as an installable member; tests with Python members and a browser driven end to end. Members do not run trees; that is v0.6. |+| **Status** | **Built, v0.5; members run trees since v0.6.** Members, mailboxes, `Keep`/`Kept`/`Fetch`, the blob store, the console as an installable member; tests with Python members and a browser driven end to end. Since v0.6 a *trusted* member (one that joined with the hub's token) holds the hub's trees and runs them on `Ask`; see §7. || **Date** | 2026-10-25 | | **Decision** | A device that cannot be reached joins a hub by following it and polling a mailbox the hub keeps for it. It carries the directory like a hub, and it keeps the hub's blobs, so that a swarm's state lives on the devices of the people who run it and a dead box loses nothing. Nothing secret is ever a blob. |-| **Where** | `src/pyrtan/federation.py` (members, mailboxes, keeping), `src/pyrtan/store.py` (blobs), `src/pyrtan/page.py` (the member in the console, the manifest, the service worker). |+| **Where** | `src/pyrtan/federation.py` (members, mailboxes, keeping, hand-out, `Ask`/`Answer`), `src/pyrtan/store.py` (blobs), `src/pyrtan/page.py` (the member in the console, the manifest, the service worker), `src/pyrtan/js/pyrtan.js` (the runtime the member runs trees with), `tests/conformance/` (what both runtimes must say). |## 0. The brief@@ -101,10 +101,8 @@ it. The operator paths are behind the token; the keeper pathsHMAC-CTR of my own is not something to hand people's data to. If private blobs matter, that is where a dependency would be justified, and the design note should say so when it is taken.-- **Members running trees** in this version. A member announces- `trees: []`. Routing to a member through its mailbox (`Ask` in,- `Answer` out, the hub holding the request open) is the next design,- and needs the JavaScript runtime first.+- **Any member running trees.** Only a member that joined with the+ hub's token runs them (§7); a keeper announces `trees: []`.- **A quorum.** One hub writes each of its blobs; the holders are copies, not voters. Last version wins, by the hub's own say.@@ -127,10 +125,65 @@ it. The operator paths are behind the token; the keeper paths## 6. Sequencing-- **v0.5, this:** members, mailboxes, keeping, the console as the member- and the phone app.-- **v0.6:** `pyrtan.js`, the tree runtime in JavaScript, with the Python- fixtures as shared conformance vectors; then `Ask`/`Answer` through- the mailbox so a phone answers sentences for its hub.+- **v0.5:** members, mailboxes, keeping, the console as the member and+ the phone app.+- **v0.6, built:** `pyrtan.js`, the tree runtime in JavaScript, with the+ Python fixtures as shared conformance vectors; `Ask`/`Answer` through+ the mailbox so a phone answers sentences for its hub (§7).- **Later:** encrypted private blobs, when a dependency for it is justified; members that keep for each other through a hub.++## 7. Members that run trees (v0.6)++**The runtime.** `src/pyrtan/js/pyrtan.js` is the tree runtime again,+in JavaScript, served by every hub and service at `/pyrtan.js` and+written beside the console by `pyrtan console --out`. It reads the same+tree (the TOML's own table, as JSON), walks the same nodes with the same+templates, conditions, extraction and summaries, and returns the same+outcome and trace. It is not allowed to drift: `tests/conformance/+make_vectors.py` runs every example through the Python runtime against+the recorded responses with a fixed clock and writes `vectors.json`;+`tests/conformance/run.cjs` runs the same trees and inputs through+`pyrtan.js` and compares status, say, reason, output, path and gaps;+`tests/test_conformance.py` runs both and fails when the vectors are+stale. Three differences are documented in the file's header: `float`+accepts any number, CSS selection is the browser's `DOMParser` plus a+`:contains` shim, and a member never reads sentences (the hub does).++**Trust.** A member that follows with the hub's bearer token on the+`Follow` is recorded `trusted`; one without is a keeper. Trust is what+decides whether the hub puts its trees, and the sentences they answer,+on a device: a keeper holds copies, a trusted device acts. The console+sends the token it was connected with, so joining from the hub's own+console with the token entered is enough.++**Hand-out.** On a trusted follow, and on every refresh after, the hub+gives each trusted member every tree it should run as a `Keep` of+`trees/<name>` carrying the raw tree: its own `--trees`, and any+`--handout DIR` (`$PYRTAN_HANDOUT`), which are trees the hub hands out+and does not run itself. The member's `Kept` confirms; a changed tree is+sent again. The member holds them in its store like any blob, so they+survive a reload, and announces their names as its capabilities. The hub+takes the names it handed out and ignores the rest, and the triggers it+routes by are its own copy's, never the member's word.++**Routing.** A trusted member with trees is a holder like a service: its+trees show in `/capabilities` with `via: member`, a sentence or a `/run`+for one of them reaches it, and the hub announces them to the hubs that+follow it. Order is unchanged, our own trees, then a service of ours,+then a trusted member, then a hub we follow, so a tree the hub runs+itself stays on the hub and the member is the fallback for it; a+`--handout` tree runs only on members. `Ask` goes into the member's+mailbox with a run id, the tree and the inputs; the hub holds the+request open until the member's `Answer` names that run id (thirty+seconds; a trusted member polls every four seconds), then returns the+outcome with `peer: <member id>` and `via: ["member"]`. An `Answer` for+a run nobody waits for, or from a member other than the one asked, is+refused. A member that does not answer in time gets a `member-timeout`+error, and the `Ask` still in its mailbox is answered to nobody.++**Why a phone would run a tree at all.** It has the network the box+does not: the internet when the NAS is egress-restricted, the LAN the+person is on, the geolocation and the camera of v0.7. And it is the+person's own device, acting by the same inspectable tree, with the same+trace.diff --git a/deploy/README.md b/deploy/README.md index 03184dc..871b347 100644 --- a/deploy/README.md +++ b/deploy/README.md@@ -74,6 +74,13 @@ the directory every hub gossips; `pyrtan follow URL --at <your hub>hub routes to the hubs you follow, and hubs that follow yours may ask it. See [`../_meta/federation.md`](../_meta/federation.md).+Your phone is part of it too: open the hub's console on the phone, enter+the token, and in the Member tab join; the phone then keeps the hub's+blobs and runs the hub's trees when asked. Trees meant for the phones+rather than the NAS (they need the internet the NAS is kept from, or the+network the person is on) go in a `handout` directory next to `trees`,+with `PYRTAN_HANDOUT: /handout` in the compose file.+### Without the plugin Over SSH, from the checkout: diff --git a/deploy/openmediavault/docker-compose.yml b/deploy/openmediavault/docker-compose.yml index d960924..dca671c 100644 --- a/deploy/openmediavault/docker-compose.yml +++ b/deploy/openmediavault/docker-compose.yml@@ -42,6 +42,7 @@ x-pyrtan: &pyrtanPYRTAN_STATE: /state/federation.json PYRTAN_STORE: /state/store # blobs: the hub's own and the ones it keeps for others PYRTAN_REPLICAS: "2" # how many members or hubs should keep each of its blobs+ # PYRTAN_HANDOUT: /handout # trees for the phones that joined with the token, not for this box (mount a directory at /handout)services: hub:@@ -55,6 +56,7 @@ services:- "8700:8700" volumes: - hub-state:/state+ # - /srv/dev-disk-by-uuid-CHANGE-ME/pyrtan/handout:/handout # with PYRTAN_HANDOUT above: trees the phones runnews: <<: *pyrtan diff --git a/pyproject.toml b/pyproject.toml index e0662ee..15bafee 100644 --- a/pyproject.toml +++ b/pyproject.toml@@ -1,6 +1,6 @@[project] name = "pyrtan"-version = "0.5.0"+version = "0.6.0"description = "Open-source total automation: a core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees." readme = "README.md" license = "AGPL-3.0-or-later" diff --git a/site/README.md b/site/README.md index a4c237e..3ee41b6 100644 --- a/site/README.md +++ b/site/README.md@@ -7,7 +7,7 @@ static directory and publishes it.| Path | What | Built from | | :-- | :-- | :-- | | `/` | the landing page, with the live directory read from the seeds in the visitor's browser | `site/index.html` |-| `/console/` | the console (installable: manifest and service worker beside it); point it at any hub started with `--cors https://pyrtan.com`; its Member tab joins that hub | `pyrtan console --out` |+| `/console/` | the console (installable: manifest, service worker and `pyrtan.js`, the tree runtime, beside it); point it at any hub started with `--cors https://pyrtan.com`; its Member tab joins that hub, and runs its trees when joined with the token | `pyrtan console --out` || `/catalog/` | the trees people can `pyrtan pull`, and `index.json` the command reads | `pyrtan index examples`, and the tree files | | `/rendezvous/seeds.json` | hubs to follow first | `site/rendezvous/seeds.json` | | `/rendezvous/directory.json` | a snapshot of the directory as the seeds know it, refreshed hourly | `site/snapshot.py` | diff --git a/src/pyrtan/__init__.py b/src/pyrtan/__init__.py index 939c739..f0a53a2 100644 --- a/src/pyrtan/__init__.py +++ b/src/pyrtan/__init__.py@@ -33,4 +33,4 @@ __all__ = ["Response", ]-__version__ = "0.5.0"+__version__ = "0.6.0"diff --git a/src/pyrtan/catalog.py b/src/pyrtan/catalog.py index aea85b9..966aa77 100644 --- a/src/pyrtan/catalog.py +++ b/src/pyrtan/catalog.py@@ -148,7 +148,7 @@ def _fetch(url: str, timeout: float = 20.0) -> bytes:raise CatalogError(f"{url}: {e}") from e if parts.scheme not in ("http", "https"): raise CatalogError(f"{url}: a catalog is reached over https (or a file: path)")- req = urllib.request.Request(url, headers={"User-Agent": "pyrtan-catalog/0.5", "Accept": "application/json, application/toml, text/plain, */*"})+ req = urllib.request.Request(url, headers={"User-Agent": "pyrtan-catalog/0.6", "Accept": "application/json, application/toml, text/plain, */*"})try: with urllib.request.urlopen(req, timeout=timeout) as resp: return resp.read() diff --git a/src/pyrtan/cli.py b/src/pyrtan/cli.py index a3efed5..1a870dd 100644 --- a/src/pyrtan/cli.py +++ b/src/pyrtan/cli.py@@ -44,7 +44,7 @@ from . import __version__from .catalog import Catalog, CatalogError, build_index, catalog_url from .federation import bootstrap_candidates, first_live_seed from .grow import Proposal, propose-from .page import MANIFEST, PAGE, SERVICE_WORKER+from .page import MANIFEST, PAGE, PYRTAN_JS, SERVICE_WORKERfrom .registry import Registry, default_paths from .runtime import Outcome, Runner from .service import RemoteResolver, http_json, outcome_from_dict, serve@@ -111,6 +111,7 @@ def main(argv: list[str] | None = None) -> int:p.add_argument("--port", type=int, default=8700) p.add_argument("--name", default="hub", help="what this hub calls itself, to the console and to hubs that follow it") p.add_argument("--trees", action="append", default=[], metavar="PATH", help="trees this hub holds itself, in-process (repeatable); a swarm in one process")+ p.add_argument("--handout", action="append", default=[t for t in os.environ.get("PYRTAN_HANDOUT", "").split(os.pathsep) if t], metavar="PATH", help="trees for this hub's trusted members (a phone or a browser joined with the token) to run, not this hub (repeatable; default $PYRTAN_HANDOUT); its own --trees are handed out too")p.add_argument("--env", action="append", default=[], metavar="K=V", help="a value the hub's own trees may read as {{ env.K }}") p.add_argument("--gaps", metavar="FILE", help="keep the hub's own trees' gaps in this file") p.add_argument("--egress", choices=EGRESS_POLICIES, default=os.environ.get("PYRTAN_EGRESS", "any"), help="for the hub's own trees: 'public' refuses private addresses")@@ -442,11 +443,13 @@ def cmd_hub(args: argparse.Namespace) -> int:args.host, args.port, name=args.name, peers=args.peer, poll_every=args.poll, quiet=args.quiet, token=args.token, cors=args.cors, federate=args.federate or bool(args.advertise) or bool(follow) or args.listed, advertise=args.advertise, federation_secret=args.federation_secret, state=args.state, open_follows=not args.closed, follow=follow, listed=args.listed,- trees=[Path(t) for t in args.trees], env=_kv(args.env), gaps=args.gaps, egress=args.egress,+ trees=[Path(t) for t in args.trees], handout=[Path(t) for t in args.handout], env=_kv(args.env), gaps=args.gaps, egress=args.egress,store=args.store, replicas=args.replicas, ) fed = f", federated as {hub.federation.id}" + (f" following {len(hub.federation.following)}" if hub.federation.following else "") + (", listed" if hub.federation.listed else "") if hub.federation else "" local = f", holding {len(hub.local.trees)} tree(s)" if hub.local else ""+ if hub.handout_registry is not None:+ local += f", handing out {len(hub.handout_registry.trees())} tree(s) to trusted members"print(f"pyrtan hub at {server.url}: {len(hub.peers)} peer(s){local}{fed}" + _access_note(args), flush=True) try: server.wait()@@ -739,7 +742,8 @@ def cmd_console(args: argparse.Namespace) -> int:out.write_text(PAGE, encoding="utf-8") (out.parent / "manifest.webmanifest").write_text(MANIFEST, encoding="utf-8") (out.parent / "sw.js").write_text(SERVICE_WORKER, encoding="utf-8")- print(f"wrote {out}, manifest.webmanifest and sw.js beside it; serve the folder from any static host, and point it at a hub with --cors set for that host")+ (out.parent / "pyrtan.js").write_text(PYRTAN_JS, encoding="utf-8")+ print(f"wrote {out}, manifest.webmanifest, sw.js and pyrtan.js beside it; serve the folder from any static host, and point it at a hub with --cors set for that host")return 0 diff --git a/src/pyrtan/federation.py b/src/pyrtan/federation.py index 77d1d85..6c53346 100644 --- a/src/pyrtan/federation.py +++ b/src/pyrtan/federation.py@@ -84,7 +84,8 @@ DIRECTORY_TTL = 7 * 86400 # an entry unheard of for this long is droppedDIRECTORY_MAX = 500 HEARD_UP = 600.0 # a directory entry seen this recently reads "up" HEARD_STALE = 86400.0-ACTIVITY_TYPES = ("Follow", "Accept", "Reject", "Undo", "Announce", "Keep", "Kept", "Fetch")+ACTIVITY_TYPES = ("Follow", "Accept", "Reject", "Undo", "Announce", "Keep", "Kept", "Fetch", "Ask", "Answer")+ASK_TIMEOUT = 30.0 # seconds a hub waits for a member to answer an AskMEMBER_PREFIX = "urn:pyrtan:member:" # an actor with no inbox: a browser or a phone, which polls a mailbox MAILBOX_MAX = 200 # activities a hub holds for a member that has not polled HOLDER_FRESH = 86400.0 # a holder confirmed within this counts toward the replicas@@ -212,6 +213,7 @@ class Remote:last_error: str | None = None since: float = field(default_factory=time.time) kind: str = "hub" # hub | member+ trusted: bool = False # a member that joined with the operator's token: it may run the hub's trees@property def link(self) -> str:@@ -227,14 +229,14 @@ class Remote:return "down" def to_dict(self) -> dict[str, Any]:- return {"url": self.url, "name": self.name, "kind": self.kind, "state": self.state, "link": self.link, "trees": [t["name"] for t in self.trees], "last_ok": self.last_ok, "last_error": self.last_error}+ return {"url": self.url, "name": self.name, "kind": self.kind, "state": self.state, "link": self.link, "trees": [t["name"] for t in self.trees], "last_ok": self.last_ok, "last_error": self.last_error, "trusted": self.trusted}def to_state(self) -> dict[str, Any]:- return {"url": self.url, "inbox": self.inbox, "secret": self.secret, "name": self.name, "state": self.state, "since": self.since, "kind": self.kind}+ return {"url": self.url, "inbox": self.inbox, "secret": self.secret, "name": self.name, "state": self.state, "since": self.since, "kind": self.kind, "trusted": self.trusted}@classmethod def from_state(cls, d: dict[str, Any]) -> "Remote":- return cls(d["url"], d["inbox"], d["secret"], d.get("name", ""), d.get("state", "pending"), since=float(d.get("since", time.time())), kind=d.get("kind", "hub"))+ return cls(d["url"], d["inbox"], d["secret"], d.get("name", ""), d.get("state", "pending"), since=float(d.get("since", time.time())), kind=d.get("kind", "hub"), trusted=bool(d.get("trusted", False)))def heard(seen: float, now: float | None = None) -> str:@@ -258,6 +260,10 @@ class Federation:self._auto_digest: dict[str, str] = {} self.keeping: dict[str, dict[str, Any]] = {} # our blobs: key -> {sha256, size, updated, holders: {actor: {at, sha256}}} self.mailboxes: dict[str, deque[dict[str, Any]]] = {} # activities waiting for members+ self.handout: Callable[[], dict[str, Any]] = lambda: {} # the trees trusted members run: name -> the tree as JSON+ self._handed: dict[str, dict[str, dict[str, Any]]] = {} # member -> key -> {sha256, at, confirmed}+ self._asks: dict[str, dict[str, Any]] = {} # run id -> {member, event, answer}+ self.ask_timeout = ASK_TIMEOUTself.following: dict[str, Remote] = {} self.followers: dict[str, Remote] = {} self.directory_entries: dict[str, dict[str, Any]] = {}@@ -544,6 +550,7 @@ class Federation:self.announce_up() self.announce() self.replicate()+ self.hand_out()# ------------------------------------------------------------ keeping@@ -637,6 +644,52 @@ class Federation:except StoreError: continue+ def hand_out(self, limit: int = 20) -> int:+ """Give every trusted member the trees it should run (``trees/<name>``+ blobs), each once per version until the member confirms. Returns+ how many were sent."""+ trees = self.handout()+ if not trees:+ return 0+ now = time.time()+ sent = 0+ for remote in list(self.followers.values()):+ if remote.kind != "member" or not remote.trusted or remote.state != "accepted":+ continue+ handed = self._handed.setdefault(remote.url, {})+ for name, tree in trees.items():+ key = f"trees/{name}"+ raw = tree.raw+ sha, size = blob_digest(raw)+ h = handed.get(key)+ if h and h["sha256"] == sha and (h["confirmed"] or now - h["at"] < ASK_WINDOW):+ continue+ if sent >= limit:+ return sent+ ok, _, _ = self.deliver(remote, self._activity("Keep", {"key": key, "sha256": sha, "size": size, "updated": now, "data": raw}))+ if ok:+ handed[key] = {"sha256": sha, "at": now, "confirmed": False}+ sent += 1+ return sent++ def ask(self, remote: Remote, tree: str, inputs: dict[str, Any], depth: int = 0, timeout: float | None = None) -> dict[str, Any]:+ """Have a trusted member run a tree: an Ask in its mailbox, then wait+ for its Answer (at most ``timeout`` seconds, default ``ask_timeout``;+ a member polls)."""+ timeout = self.ask_timeout if timeout is None else timeout+ run = uuid.uuid4().hex+ pending = {"member": remote.url, "event": threading.Event(), "answer": None}+ self._asks[run] = pending+ try:+ self.deliver(remote, self._activity("Ask", {"run": run, "tree": tree, "inputs": inputs, "depth": depth}))+ if not pending["event"].wait(timeout):+ return {"status": "error", "reason": "member-timeout", "tree": tree, "say": f"{remote.name or remote.url} did not answer within {int(timeout)} s; a member answers when it next polls."}+ answer = dict(pending["answer"] or {})+ answer.setdefault("tree", tree)+ return answer+ finally:+ self._asks.pop(run, None)+def kept_for(self, actor: str, key: str) -> dict[str, Any] | None: """A blob we keep for ``actor``, for it to take back.""" return self.store.get_kept(actor, key)@@ -720,7 +773,9 @@ class Federation:return None return actor if verify(secret, method, path, body, h) is None else None- def handle_inbox(self, activity: Any, headers: dict[str, str] | None) -> tuple[int, dict[str, Any]]:+ def handle_inbox(self, activity: Any, headers: dict[str, str] | None, trusted: bool = False) -> tuple[int, dict[str, Any]]:+ """``trusted`` says the request also carried the operator's token: a+ member that follows that way may run the hub's trees."""h = _lower(headers) if not isinstance(activity, dict) or activity.get("type") not in ACTIVITY_TYPES or not isinstance(activity.get("actor"), str): return 400, {"status": "error", "reason": "bad-activity", "say": f"Send an activity of type {', '.join(ACTIVITY_TYPES)} with an actor."}@@ -730,10 +785,10 @@ class Federation:kind = activity["type"] self.inbox_log.append({**activity, "received": time.time()}) if kind == "Follow":- return self._on_follow(actor, activity, h)+ return self._on_follow(actor, activity, h, trusted)followed = self.following.get(actor) follower = self.followers.get(actor)- if kind in ("Announce", "Keep", "Kept", "Fetch"):+ if kind in ("Announce", "Keep", "Kept", "Fetch", "Answer"):# Announce from a hub we follow: its trees are ours to route to; # from a follower: only the directory it carries. Keeping goes # both ways between any two that share a secret.@@ -743,7 +798,7 @@ class Federation:if remote is None: return 403, {"status": "error", "reason": "unknown-actor", "say": f"{actor} is not a hub this one follows or is followed by."} problem = verify(remote.secret, "POST", "/inbox", activity, h)- if problem and kind in ("Announce", "Keep", "Kept", "Fetch") and followed and follower and followed is not follower:+ if problem and kind in ("Announce", "Keep", "Kept", "Fetch", "Answer") and followed and follower and followed is not follower:# Mutual follow: the announce may be signed with the other pair. remote = follower if remote is followed else followed problem = verify(remote.secret, "POST", "/inbox", activity, h)@@ -773,6 +828,12 @@ class Federation:remote.name = obj.get("name") or remote.name remote.last_ok = time.time() remote.last_error = None+ if remote.kind == "member" and remote.trusted:+ # A trusted member runs the trees we handed it, and only those;+ # what each tree answers to is our copy's word, not the member's.+ handed = self.handout()+ names = {t.get("name") for t in obj.get("trees", []) if isinstance(t, dict)}+ remote.trees = [handed[n].capability() for n in sorted(names & set(handed))]taken = self.merge_directory(obj.get("directory", []), via=actor) # What it keeps for us that we no longer have: ask for it back. wanted = [k for k in obj.get("kept", []) if isinstance(k, str) and not self.store.get_own(k)]@@ -793,12 +854,27 @@ class Federation:obj = activity.get("object") if not isinstance(obj, dict) or not isinstance(obj.get("key"), str): return 400, {"status": "error", "reason": "bad-activity", "say": "A Kept names a key."}- if "data" in obj:+ if obj["key"].startswith("trees/"):+ with self._lock:+ handed = self._handed.setdefault(actor, {}).get(obj["key"])+ if handed and handed.get("sha256") == obj.get("sha256"):+ handed["confirmed"] = True+ elif "data" in obj:self._adopt(obj["key"], obj, actor) elif isinstance(obj.get("sha256"), str): self.confirm(actor, obj["key"], obj["sha256"]) remote.last_ok = time.time() return 200, {"status": "ok"}+ if kind == "Answer":+ obj = activity.get("object")+ run = obj.get("run") if isinstance(obj, dict) else None+ pending = self._asks.get(run) if isinstance(run, str) else None+ if pending is None or pending["member"] != actor:+ return 404, {"status": "error", "reason": "no-such-ask", "say": "No run waits for this answer from this member."}+ pending["answer"] = obj.get("outcome") if isinstance(obj.get("outcome"), dict) else {"status": "error", "reason": "bad-answer", "say": "The member's answer was not an outcome."}+ pending["event"].set()+ remote.last_ok = time.time()+ return 200, {"status": "ok"}if kind == "Fetch": obj = activity.get("object") key = obj.get("key") if isinstance(obj, dict) else None@@ -810,12 +886,13 @@ class Federation:with self._lock: self.followers.pop(actor, None) self.mailboxes.pop(actor, None)+ self._handed.pop(actor, None)self.store.drop_kept(actor) self._save() return 200, {"status": "ok"} return 400, {"status": "error", "reason": "bad-activity"}- def _on_follow(self, actor: str, activity: dict[str, Any], h: dict[str, str]) -> tuple[int, dict[str, Any]]:+ def _on_follow(self, actor: str, activity: dict[str, Any], h: dict[str, str], trusted: bool = False) -> tuple[int, dict[str, Any]]:pair = activity.get("secret") inbox = activity.get("inbox") if not isinstance(pair, str) or len(pair) < 16 or not isinstance(inbox, str):@@ -848,12 +925,15 @@ class Federation:name = str(activity.get("name") or doc.get("name") or "") kind = "hub" with self._lock:- remote = Remote(actor, inbox, pair, name, state="accepted", last_ok=time.time(), kind=kind)+ remote = Remote(actor, inbox, pair, name, state="accepted", last_ok=time.time(), kind=kind, trusted=bool(trusted and kind == "member"))self.followers[actor] = remote+ self._handed.pop(actor, None)self._save() self.deliver(remote, self._activity("Accept", {"type": "Follow", "actor": actor, "object": self.id})) self.deliver(remote, self._activity("Announce", self._announcement(remote)))- return 202, {"status": "ok", "say": f"{self.id} accepts {actor}.", "mailbox": f"{self.id}/mailbox" if kind == "member" else None}+ if remote.trusted:+ self.hand_out()+ return 202, {"status": "ok", "say": f"{self.id} accepts {actor}.", "mailbox": f"{self.id}/mailbox" if kind == "member" else None, "trusted": remote.trusted}# ------------------------------------------------------------ views diff --git a/src/pyrtan/hub.py b/src/pyrtan/hub.py index ae91a69..c7cd453 100644 --- a/src/pyrtan/hub.py +++ b/src/pyrtan/hub.py@@ -44,9 +44,10 @@ from . import __version__from .federation import MAX_HOPS, Federation, Remote, sign from .grow import Unanswered, no_match_outcome from .intent import Trigger, read-from .page import MANIFEST, PAGE, SERVICE_WORKER+from .page import MANIFEST, PAGE, PYRTAN_JS, SERVICE_WORKERfrom .registry import Registry from .runtime import Outcome+from .tree import Treefrom .service import Server, Service, bearer_ok, http_json, outcome_from_dict, signed_path from .store import StoreError, check_key@@ -117,8 +118,9 @@ class _HubResolver:class Hub:- def __init__(self, *, name: str = "hub", poll_every: float = 10.0, token: str | None = None, local: Service | None = None) -> None:+ def __init__(self, *, name: str = "hub", poll_every: float = 10.0, token: str | None = None, local: Service | None = None, handout: Registry | None = None) -> None:self.name = name+ self.handout_registry = handout # trees for trusted members only: this hub hands them out and does not run themself.poll_every = poll_every self.token = token or None self.peers: dict[str, Peer] = {}@@ -143,9 +145,32 @@ class Hub:"gaps": lambda: self.gaps()["gaps"], "unanswered": lambda: self.unanswered_all()["unanswered"], }- self.federation = Federation(advertise, name=self.name, secret=secret, state=state, open_follows=open_follows, capabilities=self.local_capabilities, listed=listed, store=store, replicas=replicas, auto_keys=auto)+ self.federation = Federation(advertise, name=self.name, secret=secret, state=state, open_follows=open_follows, capabilities=self.announced_capabilities, listed=listed, store=store, replicas=replicas, auto_keys=auto)+ self.federation.handout = self.handout_treesreturn self.federation+ def handout_trees(self) -> dict[str, Tree]:+ """The trees a trusted member may run: the ones this hub holds itself,+ and the ones given it to hand out (``--handout``), by name."""+ out: dict[str, Tree] = {}+ for reg in (self.local.service.registry if self.local is not None else None, self.handout_registry):+ if reg is None:+ continue+ reg.reload()+ for t in reg.trees():+ out.setdefault(t.name, t)+ return out++ def member_capabilities(self) -> list[dict[str, Any]]:+ """The trees this hub's trusted members have announced they run."""+ if self.federation is None:+ return []+ return [{**t, "peer": r.url, "link": r.link, "via": "member"} for r in list(self.federation.followers.values()) if r.kind == "member" and r.trusted for t in r.trees]++ def announced_capabilities(self) -> list[dict[str, Any]]:+ """What this hub answers for: its own trees, its services', and its members'."""+ return self.local_capabilities() + self.member_capabilities()+def hold(self, service: Service) -> Local: """Hold trees in this process. The service's delegations reach the hub directly, so a local tree can call anything the hub routes to."""@@ -220,6 +245,8 @@ class Hub:out: list[Local | Peer | Remote] = [self.local] if self.local is not None else [] out.extend(self.peers.values()) if self.federation is not None:+ # Trusted members run our own trees; a hub we follow answers for its.+ out.extend(r for r in self.federation.followers.values() if r.kind == "member" and r.trusted and r.trees)out.extend(self.federation.following.values()) return out@@ -265,7 +292,7 @@ class Hub:} def capabilities(self) -> dict[str, Any]:- trees = self.local_capabilities()+ trees = self.announced_capabilities()if self.federation is not None: for remote in list(self.federation.following.values()): for t in remote.trees:@@ -323,6 +350,17 @@ class Hub:answer.setdefault("peer", "local") return 200, answer hops = list(body.get("hops") or [])+ if isinstance(peer, Remote) and peer.kind == "member":+ # One of our own devices runs it: an Ask in its mailbox, its Answer back.+ if path != "/run":+ return 200, {"status": "error", "reason": "not-on-member", "say": "A member runs trees; it cannot be asked a sentence or resumed."}+ assert self.federation is not None+ answer = self.federation.ask(peer, str(body.get("tree")), dict(body.get("inputs") or {}), int(body.get("depth", 0)))+ answer.setdefault("peer", peer.url)+ answer["via"] = ["member"]+ if not body.get("trace", True):+ answer.pop("trace", None)+ return 200, answerif isinstance(peer, Remote): # Over federation: one hop, signed, and the request says where it has been. if len(hops) >= MAX_HOPS or (self.federation and self.federation.id in hops):@@ -408,7 +446,7 @@ class Hub:if method == "GET" and path == "/directory": return 200, fed.directory() if method == "POST" and path == "/inbox":- return fed.handle_inbox(body, headers)+ return fed.handle_inbox(body, headers, trusted=bool(self.token) and bearer_ok(headers.get("Authorization") or headers.get("authorization"), self.token))if method == "GET" and path in ("/following", "/followers"): return 200, {path[1:]: fed.to_dict()[path[1:]]} if method == "GET" and path == "/outbox":@@ -462,6 +500,8 @@ class Hub:return 200, ("application/manifest+json", MANIFEST) if method == "GET" and path == "/sw.js": return 200, ("text/javascript; charset=utf-8", SERVICE_WORKER)+ if method == "GET" and path == "/pyrtan.js":+ return 200, ("text/javascript; charset=utf-8", PYRTAN_JS)if method == "GET" and path == "/api": endpoints = ["/health", "/peers", "/capabilities", "/intent", "/run", "/resume", "/gaps", "/unanswered", "/register"] if fed is not None:@@ -484,7 +524,8 @@ class Hub:def _rank(peer: Local | Peer | Remote) -> int: # Our own trees, then a service of ours, then a hub we follow; the healthier of each.- return {"up": 0, "pending": 1, "stale": 2, "down": 3}[peer.link] + (10 if isinstance(peer, Remote) else 0) - (1 if isinstance(peer, Local) else 0)+ # Our own trees, then a service of ours, then our trusted members, then a hub we follow; the healthier of each.+ return {"up": 0, "pending": 1, "stale": 2, "down": 3}[peer.link] + (5 if isinstance(peer, Remote) and peer.kind == "member" else 10 if isinstance(peer, Remote) else 0) - (1 if isinstance(peer, Local) else 0)def serve_hub(@@ -505,6 +546,7 @@ def serve_hub(follow: list[str] | None = None, listed: bool = False, trees: list[str | Path] | None = None,+ handout: list[str | Path] | None = None,env: dict[str, Any] | None = None, gaps: str | None = None, egress: str = "any",@@ -512,7 +554,7 @@ def serve_hub(replicas: int = 2, ) -> tuple[Hub, Server]: local = Service(Registry.from_paths(trees), name=f"{name}", env=env, gaps=gaps, egress=egress) if trees else None- hub = Hub(name=name, poll_every=poll_every, token=token, local=local)+ hub = Hub(name=name, poll_every=poll_every, token=token, local=local, handout=Registry.from_paths(handout) if handout else None)server = Server(hub.route, host, port, quiet, token=token, cors=cors, federated=federate or bool(advertise)).start() if federate or advertise: hub.federate(advertise or server.url, secret=federation_secret, state=state, open_follows=open_follows, listed=listed, store=store or (str(Path(state).with_name("store")) if state else None), replicas=replicas) diff --git a/src/pyrtan/js/pyrtan.js b/src/pyrtan/js/pyrtan.js new file mode 100644 index 0000000..d9486e8 --- /dev/null +++ b/src/pyrtan/js/pyrtan.js@@ -0,0 +1,654 @@+/* pyrtan.js: the tree runtime, in JavaScript.+ *+ * The same tree files the Python runtime reads (as JSON, the TOML's own+ * table), walked the same way: observe, act, decide, set, each, delegate,+ * ask, done, fail; the same templates, conditions and extraction; the same+ * trace. tests/conformance holds the vectors both runtimes must agree on.+ *+ * What differs, and is written down here rather than hidden:+ * - `float` in a condition's `type` means any number here; JSON has no+ * distinct float, so 4.0 and 4 are the same value.+ * - CSS extraction uses the host's selector engine (DOMParser) plus+ * `:contains(text)`; in a host without a DOM, a css extract is MISSING.+ * - Regular expressions are the host's; the subset trees use is common.+ * - There is no intent matching here: a member runs the tree its hub+ * already chose. `delegate` goes to the resolver (the hub).+ *+ * No model. No dependencies. Runs in a browser as `Pyrtan` and in node.+ */+(function (root, factory) {+ if (typeof module === "object" && module.exports) module.exports = factory();+ else root.Pyrtan = factory();+})(typeof globalThis !== "undefined" ? globalThis : this, function () {+ "use strict";++ const MAX_STEPS = 500, MAX_DEPTH = 8, DEFAULT_TIMEOUT = 10.0, DEFAULT_RETRIES = 1;+ const MISSING = Object.freeze({ toString: () => "<missing>", missing: true });+ const isMissing = (v) => v === MISSING;+ const isObj = (v) => v !== null && typeof v === "object" && !Array.isArray(v);+ const isBool = (v) => typeof v === "boolean";+ const isNum = (v) => typeof v === "number" && !Number.isNaN(v);++ class TemplateError extends Error {}+ class TreeError extends Error {}+ class TransportError extends Error {}+ class ConditionError extends Error {}++ // ------------------------------------------------------------ python-ish text+ // Summaries and the chosen branch in a trace read like the Python runtime's,+ // so both runtimes' traces compare as text.+ function pyrepr(v) {+ if (v === null || v === undefined || isMissing(v)) return "None";+ if (isBool(v)) return v ? "True" : "False";+ if (isNum(v)) return Number.isInteger(v) ? String(v) : String(v);+ if (typeof v === "string") {+ const q = v.includes("'") && !v.includes('"') ? '"' : "'";+ const body = v.replace(/\\/g, "\\\\").replace(new RegExp(q, "g"), "\\" + q).replace(/\n/g, "\\n").replace(/\r/g, "\\r").replace(/\t/g, "\\t");+ return q + body + q;+ }+ if (Array.isArray(v)) return "[" + v.map(pyrepr).join(", ") + "]";+ if (isObj(v)) return "{" + Object.keys(v).map((k) => pyrepr(k) + ": " + pyrepr(v[k])).join(", ") + "}";+ return String(v);+ }+ function pyJson(v) { // json.dumps(v, ensure_ascii=False): ", " and ": " separators+ if (v === null || v === undefined || isMissing(v)) return "null";+ if (isBool(v)) return v ? "true" : "false";+ if (isNum(v)) return Number.isInteger(v) ? String(v) : String(v);+ if (typeof v === "string") return JSON.stringify(v);+ if (Array.isArray(v)) return "[" + v.map(pyJson).join(", ") + "]";+ if (isObj(v)) return "{" + Object.keys(v).map((k) => JSON.stringify(k) + ": " + pyJson(v[k])).join(", ") + "}";+ return JSON.stringify(v);+ }+ function brief(v, limit = 80) {+ if (isMissing(v)) return "<missing>";+ const text = typeof v === "string" ? v : pyrepr(v);+ return text.length <= limit ? text : text.slice(0, limit - 1) + "…";+ }+ function short(v, limit = 200) {+ if (typeof v === "string" && v.length > limit) return v.slice(0, limit) + "…";+ if (Array.isArray(v) || isObj(v)) { const t = pyJson(v); if (t.length > limit) return t.slice(0, limit) + "…"; }+ return v;+ }+ const condText = (cond) => Object.keys(cond).map((k) => `${k} ${pyrepr(cond[k])}`).join(" and ");+ // A tree's regular expressions are written for Python; the inline flags+ // (?i) (?s) (?m) at the start become JavaScript flags.+ function pyRegex(pattern, flags = "") {+ let src = String(pattern), fl = flags;+ const m = /^\(\?([imsx]+)\)/.exec(src);+ if (m) { src = src.slice(m[0].length); for (const f of m[1]) if ("ims".includes(f) && !fl.includes(f)) fl += f; }+ return new RegExp(src, fl);+ }++ // ------------------------------------------------------------ templates+ const PLACEHOLDER = /\{\{\s*([^{}]*?)\s*\}\}/g;+ const PATH_TOKEN = /([A-Za-z_][A-Za-z0-9_\-]*)|\[(-?\d+)\]|\["([^"]*)"\]|\['([^']*)'\]/y;++ function resolve(path, scope) {+ path = String(path).trim();+ if (!path) return MISSING;+ let pos = 0, cur = scope;+ while (pos < path.length) {+ if (path[pos] === ".") { pos++; continue; }+ PATH_TOKEN.lastIndex = pos;+ const m = PATH_TOKEN.exec(path);+ if (!m) throw new TemplateError(`bad path ${pyrepr(path)} at offset ${pos}`);+ pos = PATH_TOKEN.lastIndex;+ const [, name, index, dq, sq] = m;+ if (name !== undefined) cur = getKey(cur, name);+ else if (index !== undefined) cur = getIndex(cur, parseInt(index, 10));+ else cur = getKey(cur, dq !== undefined ? dq : sq);+ if (isMissing(cur)) return MISSING;+ }+ return cur;+ }+ function getKey(cur, key) {+ if (isObj(cur)) return Object.prototype.hasOwnProperty.call(cur, key) ? cur[key] : MISSING;+ if (Array.isArray(cur) && /^\d+$/.test(key)) return getIndex(cur, parseInt(key, 10));+ return MISSING;+ }+ function getIndex(cur, index) {+ if (Array.isArray(cur) || typeof cur === "string") {+ const i = index < 0 ? cur.length + index : index;+ if (i < 0 || i >= cur.length) return MISSING;+ return typeof cur === "string" ? cur[i] : cur[i];+ }+ return MISSING;+ }+ const pyQuote = (s) => encodeURIComponent(String(s)).replace(/[!'()*]/g, (c) => "%" + c.charCodeAt(0).toString(16).toUpperCase());+ const pyStr = (v) => (typeof v === "string" ? v : isBool(v) ? (v ? "True" : "False") : v === null || v === undefined ? "None" : Array.isArray(v) || isObj(v) ? pyrepr(v) : String(v));+ const FILTERS = {+ url: (v) => pyQuote(pyStr(v)),+ json: (v) => pyJson(v),+ int: (v) => { const n = Number(typeof v === "string" ? v.trim() : v); if (Number.isNaN(n)) throw new TemplateError(`int: ${pyrepr(v)}`); return Math.trunc(n); },+ float: (v) => { const n = Number(typeof v === "string" ? v.trim() : v); if (Number.isNaN(n)) throw new TemplateError(`float: ${pyrepr(v)}`); return n; },+ str: (v) => pyStr(v),+ lower: (v) => pyStr(v).toLowerCase(),+ upper: (v) => pyStr(v).toUpperCase(),+ strip: (v) => pyStr(v).trim(),+ first: (v) => ((Array.isArray(v) || typeof v === "string") && v.length ? v[0] : MISSING),+ last: (v) => ((Array.isArray(v) || typeof v === "string") && v.length ? v[v.length - 1] : MISSING),+ length: (v) => (Array.isArray(v) || typeof v === "string" ? v.length : isObj(v) ? Object.keys(v).length : MISSING),+ title: (v) => pyStr(v).toLowerCase().replace(/(^|[^a-z0-9'])([a-z])/gi, (m, pre, c) => pre + c.toUpperCase()).replace(/'([a-z])/gi, (m, c) => "'" + c.toUpperCase()),+ };+ function renderValue(expr, scope) {+ const parts = expr.split("|").map((p) => p.trim());+ let value = resolve(parts[0], scope);+ for (const name of parts.slice(1)) {+ if (!(name in FILTERS)) throw new TemplateError(`unknown filter ${pyrepr(name)} in {{ ${expr} }}`);+ if (isMissing(value)) continue;+ value = FILTERS[name](value);+ }+ return value;+ }+ function render(template, scope, strict = true) {+ if (typeof template === "string") {+ const trimmed = template.trim();+ const whole = /^\{\{\s*([^{}]*?)\s*\}\}$/.exec(trimmed);+ if (whole && trimmed === template) {+ const value = renderValue(whole[1], scope);+ if (isMissing(value)) { if (strict) throw new TemplateError(`${template} resolves to nothing`); return null; }+ return value;+ }+ return template.replace(PLACEHOLDER, (m, expr) => {+ const value = renderValue(expr, scope);+ if (isMissing(value)) { if (strict) throw new TemplateError(`${m} resolves to nothing`); return ""; }+ if (Array.isArray(value) || isObj(value)) return pyJson(value);+ return pyStr(value);+ });+ }+ if (Array.isArray(template)) return template.map((t) => render(t, scope, strict));+ if (isObj(template)) { const out = {}; for (const k of Object.keys(template)) out[k] = render(template[k], scope, strict); return out; }+ return template;+ }++ // ------------------------------------------------------------ conditions+ function num(v) {+ if (isBool(v)) return null;+ if (isNum(v)) return v;+ if (typeof v === "string") { const t = v.trim(); if (t === "" || Number.isNaN(Number(t))) return null; return Number(t); }+ return null;+ }+ function looseEq(a, b) {+ if (isBool(a) || isBool(b)) return isBool(a) && isBool(b) && a === b;+ if (a === b) return true;+ if (Array.isArray(a) && Array.isArray(b)) return pyJson(a) === pyJson(b);+ if (isObj(a) && isObj(b)) return pyJson(sortKeys(a)) === pyJson(sortKeys(b));+ if ((a === null || a === undefined) && (b === null || b === undefined)) return true;+ const na = num(a), nb = num(b);+ if (na !== null && nb !== null && (typeof a === "string" || typeof b === "string")) return na === nb;+ return false;+ }+ function sortKeys(o) { const out = {}; for (const k of Object.keys(o).sort()) out[k] = isObj(o[k]) ? sortKeys(o[k]) : o[k]; return out; }+ const isEmpty = (v) => isMissing(v) || v === null || v === undefined || ((typeof v === "string" || Array.isArray(v)) && v.length === 0) || (isObj(v) && Object.keys(v).length === 0);+ function typeIs(name, v) {+ switch (name) {+ case "str": case "string": return typeof v === "string";+ case "int": return isNum(v) && Number.isInteger(v);+ case "float": case "number": return isNum(v);+ case "bool": return isBool(v);+ case "list": return Array.isArray(v);+ case "dict": case "map": return isObj(v);+ case "null": case "none": return v === null || v === undefined;+ default: throw new ConditionError(`unknown type ${pyrepr(name)}`);+ }+ }+ function test(key, expected, value) {+ if (key === "exists") return (!isMissing(value) && value !== null && value !== undefined) === Boolean(expected);+ if (key === "empty") return isEmpty(value) === Boolean(expected);+ if (isMissing(value)) return false;+ switch (key) {+ case "eq": return looseEq(value, expected);+ case "ne": return !looseEq(value, expected);+ case "lt": case "lte": case "gt": case "gte": {+ const a = num(value), b = num(expected);+ if (a === null || b === null) return false;+ return key === "lt" ? a < b : key === "lte" ? a <= b : key === "gt" ? a > b : a >= b;+ }+ case "between": { const a = num(value), lo = num(expected[0]), hi = num(expected[1]); if (a === null || lo === null || hi === null) return false; return lo <= a && a <= hi; }+ case "matches": return value !== null && value !== undefined && pyRegex(expected).test(pyStr(value));+ case "contains":+ if (Array.isArray(value)) return value.some((item) => looseEq(item, expected));+ if (isObj(value)) return Object.prototype.hasOwnProperty.call(value, expected);+ return pyStr(value).includes(pyStr(expected));+ case "startswith": return pyStr(value).startsWith(pyStr(expected));+ case "endswith": return pyStr(value).endsWith(pyStr(expected));+ case "in": return expected.some((option) => looseEq(value, option));+ case "type": return typeIs(expected, value);+ default: throw new ConditionError(`unknown condition key ${pyrepr(key)}`);+ }+ }+ const holds = (cond, value) => Object.keys(cond).every((k) => test(k, cond[k], value));++ // ------------------------------------------------------------ html (host engine)+ const INLINE = new Set(["a", "abbr", "b", "bdi", "bdo", "cite", "code", "data", "dfn", "em", "i", "kbd", "label", "mark", "q", "s", "samp", "small", "span", "strong", "sub", "sup", "time", "u", "var", "del", "ins"]);+ function textOf(el) {+ const parts = [];+ for (const child of el.childNodes) {+ if (child.nodeType === 3) parts.push(child.nodeValue);+ else if (child.nodeType === 1) parts.push(INLINE.has(child.tagName.toLowerCase()) ? textOf(child) : " " + textOf(child) + " ");+ }+ return parts.join("").replace(/\s+/g, " ").trim();+ }+ function selectAll(rootEl, selector) {+ // The host's engine, plus :contains(text) and a group of them.+ const out = [];+ for (const part of splitGroups(selector)) {+ const m = /^(.*?)\s*:contains\((?:"([^"]*)"|'([^']*)'|([^)]*))\)\s*$/.exec(part);+ if (m) {+ const base = m[1].trim() || "*", needle = m[2] !== undefined ? m[2] : m[3] !== undefined ? m[3] : m[4];+ for (const el of rootEl.querySelectorAll(base)) if (textOf(el).includes(needle) && !out.includes(el)) out.push(el);+ } else {+ for (const el of rootEl.querySelectorAll(part)) if (!out.includes(el)) out.push(el);+ }+ }+ return out;+ }+ function splitGroups(selector) {+ const parts = []; let depth = 0, cur = "";+ for (const ch of selector) { if (ch === "(") depth++; if (ch === ")") depth--; if (ch === "," && depth === 0) { parts.push(cur.trim()); cur = ""; } else cur += ch; }+ if (cur.trim()) parts.push(cur.trim());+ return parts;+ }+ const html = {+ parse(text) {+ if (typeof DOMParser === "undefined") return null;+ const doc = new DOMParser().parseFromString(text, "text/html");+ return doc.documentElement;+ },+ };++ // ------------------------------------------------------------ extract+ function extract(entry, source, cache) {+ if ("json" in entry || "path" in entry) return resolve(entry.json !== undefined ? entry.json : entry.path, source);+ if ("regex" in entry) {+ const text = textSource(entry, source);+ if (isMissing(text)) return MISSING;+ const re = pyRegex(entry.regex, (entry.dotall ? "s" : "") + "g");+ const groups = pyRegex(entry.regex + "|").exec("").length - 1;+ const group = entry.group !== undefined ? entry.group : groups ? 1 : 0;+ if (entry.all) return Array.from(text.matchAll(re), (m) => m[group]);+ const m = re.exec(text);+ return m ? (m[group] === undefined ? null : m[group]) : MISSING;+ }+ if ("css" in entry) {+ const text = textSource(entry, source);+ if (isMissing(text)) return MISSING;+ let doc = cache && cache.get(text);+ if (!doc) { doc = html.parse(text); if (!doc) return MISSING; if (cache) cache.set(text, doc); }+ const found = selectAll(doc, entry.css);+ if (entry.all) return found.map((el) => elementValue(entry, el));+ if (!found.length) return MISSING;+ return elementValue(entry, found[0]);+ }+ throw new Error(`extract entry ${pyrepr(entry.into)} has no way`);+ }+ function textSource(entry, source) {+ const frm = entry.from !== undefined ? entry.from : "body";+ const value = typeof frm === "string" ? resolve(frm, source) : MISSING;+ if (isMissing(value) || value === null || value === undefined) return MISSING;+ return typeof value === "string" ? value : pyStr(value);+ }+ function elementValue(entry, el) {+ if (entry.fields) {+ const out = {};+ for (const name of Object.keys(entry.fields)) {+ const spec = entry.fields[name];+ const sub = selectAll(el, typeof spec === "string" ? spec : spec.css)[0] || null;+ if (!sub) out[name] = null;+ else if (typeof spec !== "string" && "attr" in spec) out[name] = sub.getAttribute(spec.attr);+ else out[name] = textOf(sub);+ }+ return out;+ }+ if ("attr" in entry) return el.getAttribute(entry.attr);+ return textOf(el);+ }++ // ------------------------------------------------------------ trees+ const INPUT_TYPES = ["str", "int", "float", "bool", "list", "dict"];+ function inferType(v) { return isBool(v) ? "bool" : isNum(v) ? (Number.isInteger(v) ? "int" : "float") : Array.isArray(v) ? "list" : isObj(v) ? "dict" : "str"; }+ function coerce(spec, value) {+ const bad = () => { throw new TreeError(`input ${pyrepr(spec.name)} must be ${spec.type}, got ${pyrepr(value)}`); };+ if (spec.type === "int") { if (isBool(value)) bad(); const n = Number(typeof value === "string" ? value.trim() : value); if (Number.isNaN(n) || (typeof value === "string" && !/^-?\d+$/.test(value.trim()))) bad(); value = Math.trunc(n); }+ else if (spec.type === "float") { const n = Number(typeof value === "string" ? value.trim() : value); if (Number.isNaN(n) || isBool(value)) bad(); value = n; }+ else if (spec.type === "bool") {+ if (typeof value === "string") { const low = value.trim().toLowerCase(); if (["true", "yes", "1", "on"].includes(low)) value = true; else if (["false", "no", "0", "off"].includes(low)) value = false; else bad(); }+ else value = Boolean(value);+ }+ else if (spec.type === "str") value = pyStr(value);+ else if (spec.type === "list" && !Array.isArray(value)) bad();+ else if (spec.type === "dict" && !isObj(value)) bad();+ if (spec.min !== undefined && spec.min !== null && isNum(value) && value < spec.min) throw new TreeError(`input ${pyrepr(spec.name)} must be at least ${spec.min}, got ${value}`);+ if (spec.max !== undefined && spec.max !== null && isNum(value) && value > spec.max) throw new TreeError(`input ${pyrepr(spec.name)} must be at most ${spec.max}, got ${value}`);+ if (spec.one_of && !spec.one_of.some((o) => looseEq(o, value) && typeof o === typeof value)) throw new TreeError(`input ${pyrepr(spec.name)} must be one of ${pyrepr(spec.one_of)}, got ${pyrepr(value)}`);+ if (spec.pattern && typeof value === "string" && !pyRegex("^(?:" + String(spec.pattern).replace(/^\(\?[imsx]+\)/, (f) => { spec._flags = f; return ""; }) + ")$", (spec._flags || "").replace(/[()?x]/g, "")).test(value)) throw new TreeError(`input ${pyrepr(spec.name)} must match ${pyrepr(spec.pattern)}, got ${pyrepr(value)}`);+ return value;+ }+ class Tree {+ constructor(raw) {+ this.raw = raw;+ this.name = raw.name; this.version = raw.version || 1; this.description = raw.description || "";+ this.hosts = raw.hosts || []; this.start = raw.start; this.tags = raw.tags || []; this.license = raw.license || "";+ this.language = raw.language || {};+ this.triggers = (raw.triggers || []).map((t) => (typeof t === "string" ? { pattern: t } : t));+ this.inputs = {};+ for (const [name, s] of Object.entries(raw.inputs || {})) {+ const spec = isObj(s) ? s : { default: s };+ this.inputs[name] = { name, type: spec.type || inferType(spec.default), default: spec.default === undefined ? null : spec.default, required: spec.required !== undefined ? Boolean(spec.required) : spec.default === undefined, description: spec.description || "", min: spec.min, max: spec.max, one_of: spec.one_of, pattern: spec.pattern };+ }+ this.nodes = {};+ for (const [id, spec] of Object.entries(raw.nodes || {})) this.nodes[id] = { id, kind: spec.kind, spec, get: (k, d) => (spec[k] !== undefined ? spec[k] : d), next: spec.next };+ }+ node(id) { const n = this.nodes[id]; if (!n) throw new TreeError(`${this.name}: no node ${pyrepr(id)}`); return n; }+ hostAllowed(url) {+ let host = "";+ try { host = new URL(url).hostname.toLowerCase(); } catch (e) { return false; }+ if (!host) return false;+ for (let allowed of this.hosts) {+ allowed = allowed.toLowerCase();+ if (allowed === "*") return true;+ if (allowed.startsWith("*.")) { const suffix = allowed.slice(1); if (host.endsWith(suffix) || host === allowed.slice(2)) return true; }+ else if (host === allowed) return true;+ }+ return false;+ }+ bindInputs(given) {+ given = given || {};+ const unknown = Object.keys(given).filter((k) => !(k in this.inputs)).sort();+ if (unknown.length) throw new TreeError(`${this.name}: unknown inputs ${pyrepr(unknown)}`);+ const bound = {};+ for (const [name, spec] of Object.entries(this.inputs)) {+ if (name in given && given[name] !== null && given[name] !== undefined) bound[name] = coerce(spec, given[name]);+ else if (spec.default !== null && spec.default !== undefined) bound[name] = spec.default;+ else if (spec.required) throw new TreeError(`${this.name}: input ${pyrepr(name)} is required`);+ else bound[name] = null;+ }+ return bound;+ }+ capability() {+ const inputs = {};+ for (const [n, s] of Object.entries(this.inputs)) {+ const d = {};+ if (s.type) d.type = s.type;+ if (s.default !== null && s.default !== undefined && s.default !== "" && s.default !== false) d.default = s.default;+ if (s.required && (s.default === null || s.default === undefined)) d.required = true;+ if (s.description) d.description = s.description;+ inputs[n] = d;+ }+ return { name: this.name, version: this.version, description: this.description, tags: this.tags, hosts: this.hosts, license: this.license, language: this.language, triggers: this.triggers.map((t) => t.pattern), inputs };+ }+ }++ // ------------------------------------------------------------ transports+ class Response {+ constructor(status, headers, body, url, elapsed_ms) { this.status = status; this.headers = headers; this.body = body; this.url = url; this.elapsed_ms = elapsed_ms || 0; }+ get json() { try { return JSON.parse(this.body); } catch (e) { return null; } }+ toState() { return { status: this.status, ok: this.status >= 200 && this.status < 300, headers: this.headers, body: this.body, json: this.json, url: this.url, elapsed_ms: this.elapsed_ms }; }+ }+ class FetchTransport {+ constructor(opts) { this.userAgent = (opts && opts.userAgent) || "pyrtan.js"; }+ async request(method, url, headers, body, timeout) {+ const ctl = typeof AbortController !== "undefined" ? new AbortController() : null;+ const timer = ctl ? setTimeout(() => ctl.abort(), (timeout || DEFAULT_TIMEOUT) * 1000) : null;+ const started = Date.now();+ try {+ const r = await fetch(url, { method: method.toUpperCase(), headers: Object.assign({ Accept: "application/json, text/html;q=0.9, */*;q=0.8" }, headers || {}), body: body === null || body === undefined ? undefined : body, signal: ctl ? ctl.signal : undefined, redirect: "follow" });+ const text = await r.text();+ const h = {}; r.headers.forEach((v, k) => { h[k.toLowerCase()] = v; });+ return new Response(r.status, h, text, r.url || url, Date.now() - started);+ } catch (e) {+ throw new TransportError(`${method} ${url}: ${e && e.message ? e.message : e}`);+ } finally { if (timer) clearTimeout(timer); }+ }+ }+ class RecordedTransport {+ constructor(responses) { this.responses = JSON.parse(JSON.stringify(responses || {})); this.calls = []; }+ async request(method, url) {+ const key = `${method.toUpperCase()} ${url}`;+ this.calls.push([method.toUpperCase(), url]);+ let rec = this.responses[key];+ if (rec === undefined) throw new TransportError(`no recorded response for ${key}`);+ if (Array.isArray(rec)) rec = rec.length > 1 ? rec.shift() : rec[0];+ if ("error" in rec) throw new TransportError(String(rec.error));+ let text = rec.body === undefined ? "" : rec.body;+ if (typeof text !== "string") text = pyJson(text);+ const headers = {}; for (const [k, v] of Object.entries(rec.headers || {})) headers[k.toLowerCase()] = v;+ return new Response(rec.status === undefined ? 200 : Number(rec.status), headers, text, url, 0);+ }+ }++ // ------------------------------------------------------------ runtime+ class Halt { constructor(outcome) { this.outcome = outcome; } }+ const statePath = (path) => { path = String(path).trim(); const head = path.split(".")[0].split("[")[0]; return ["inputs", "state", "item", "index", "tree", "now", "env"].includes(head) ? path : "state." + path; };+ function put(state, path, value) {+ if (path.startsWith("state.")) path = path.slice(6);+ const parts = path.split("."); let cur = state;+ for (const part of parts.slice(0, -1)) { let nxt = cur[part]; if (!isObj(nxt)) { nxt = {}; cur[part] = nxt; } cur = nxt; }+ cur[parts[parts.length - 1]] = value;+ }+ const plain = (v) => (isMissing(v) ? null : v);+ function nowScope(date) {+ const d = date || new Date();+ const iso = d.toISOString().replace(/\.\d{3}Z$/, "+00:00");+ return { iso, date: iso.slice(0, 10), epoch: Math.floor(d.getTime() / 1000), year: d.getUTCFullYear() };+ }+ const sleep = (ms) => new Promise((r) => setTimeout(r, ms));++ class Trace {+ constructor(tree) { this.tree = tree; this.steps = []; this.started = Date.now() / 1000; this.gaps = []; }+ add(step) { this.steps.push(step); return step; }+ gap(node, on, observed) { this.gaps.push({ tree: this.tree, node, on, observed: short(observed) }); }+ toDict() { return { tree: this.tree, started: this.started, steps: this.steps.map(stepDict), gaps: this.gaps }; }+ render() {+ const lines = [`trace: ${this.tree}`];+ for (const s of this.steps) lines.push(`${" ".repeat((s.depth || 0) + 1)}${s.node} (${s.kind})${s.chosen ? " [" + s.chosen + "]" : ""}: ${s.summary}${s.next ? " -> " + s.next : ""}`);+ for (const g of this.gaps) lines.push(` gap: ${g.node} saw ${g.on} = ${pyJson(g.observed)} and had no branch for it`);+ return lines.join("\n");+ }+ }+ function step(node, kind, summary, extra) { return Object.assign({ node, kind, summary, observed: null, chosen: null, next: null, elapsed_ms: 0, depth: 0 }, extra || {}); }+ function stepDict(s) { const d = { node: s.node, kind: s.kind, summary: s.summary, next: s.next, elapsed_ms: s.elapsed_ms, depth: s.depth }; if (s.observed !== null && s.observed !== undefined) d.observed = s.observed; if (s.chosen !== null && s.chosen !== undefined) d.chosen = s.chosen; return d; }+ function outcome(status, tree, fields) { return Object.assign({ status, tree, say: null, output: null, reason: null, question: null, snapshot: null, trace: null, inputs: {} }, fields || {}); }+ function outcomeDict(o, withTrace = true) {+ const d = { status: o.status, tree: o.tree, inputs: o.inputs || {} };+ for (const k of ["say", "output", "reason", "question", "snapshot"]) if (o[k] !== null && o[k] !== undefined) d[k] = o[k];+ if (withTrace && o.trace) d.trace = o.trace.toDict();+ return d;+ }++ class Runner {+ constructor(opts) {+ opts = opts || {};+ this.trees = {};+ for (const t of opts.trees || []) this.add(t);+ this.transport = opts.transport || new FetchTransport();+ this.resolver = opts.resolver || null; // {runTree(name, inputs, depth), runIntent(text, depth)} -> outcome dicts+ this.env = opts.env || {};+ this.clock = opts.clock || (() => new Date());+ this.maxSteps = opts.maxSteps || MAX_STEPS; this.maxDepth = opts.maxDepth || MAX_DEPTH;+ this.emitted = [];+ }+ add(treeOrRaw) { const t = treeOrRaw instanceof Tree ? treeOrRaw : new Tree(treeOrRaw); this.trees[t.name] = t; return t; }+ get(name) { return this.trees[name] || null; }++ async runTree(name, inputs, depth = 0) {+ const tree = this.get(name);+ if (!tree) {+ if (this.resolver) return fromDict(await this.resolver.runTree(name, inputs || {}, depth), name);+ return outcome("error", name, { reason: "no-such-tree", say: `No tree named ${pyrepr(name)}.` });+ }+ return this.run(tree, inputs || {}, { depth });+ }+ async runIntent(text, depth = 0) {+ if (this.resolver) return fromDict(await this.resolver.runIntent(text, depth), "");+ return outcome("error", "", { reason: "no-match", say: `Nothing here answers ${pyrepr(text)}.` });+ }+ async run(tree, inputs, opts) {+ opts = opts || {};+ const depth = opts.depth || 0;+ const trace = opts.trace || new Trace(tree.name);+ if (depth > this.maxDepth) return outcome("error", tree.name, { reason: "too-deep", say: `Delegation deeper than ${this.maxDepth} levels.`, trace });+ let bound;+ try { bound = tree.bindInputs(inputs); } catch (e) { if (e instanceof TreeError) return outcome("error", tree.name, { reason: "bad-input", say: e.message, trace }); throw e; }+ const state = opts.state ? opts.state : {};+ const scope = Object.assign({ inputs: bound, state, tree: { name: tree.name, version: tree.version }, now: nowScope(this.clock()), env: this.env }, opts.scopeExtra || {});+ try {+ return await this.walk(tree, opts.start || tree.start, scope, trace, depth);+ } catch (h) {+ if (!(h instanceof Halt)) throw h;+ h.outcome.inputs = bound;+ h.outcome.trace = h.outcome.trace || trace;+ return h.outcome;+ }+ }+ async walk(tree, start, scope, trace, depth) {+ let nodeId = start, steps = 0;+ for (;;) {+ steps++;+ if (steps > this.maxSteps) throw new Halt(outcome("error", tree.name, { reason: "too-many-steps", say: `More than ${this.maxSteps} steps.`, trace }));+ const node = tree.node(nodeId);+ const started = Date.now();+ let result;+ try {+ result = await this["do_" + node.kind](tree, node, scope, trace, depth);+ } catch (e) {+ if (e instanceof TemplateError) { trace.add(step(node.id, node.kind, `template: ${e.message}`, { depth })); throw new Halt(outcome("error", tree.name, { reason: "template", say: `${node.id}: ${e.message}`, trace })); }+ throw e;+ }+ if (result && result.status) { result.trace = trace; throw new Halt(result); }+ const [st, nextId] = result;+ st.elapsed_ms = Date.now() - started; st.depth = depth; st.next = nextId === undefined ? null : nextId;+ trace.add(st);+ if (nextId === undefined || nextId === null) throw new Halt(outcome("error", tree.name, { reason: "dead-end", say: `${node.id} has nowhere to go.`, trace }));+ nodeId = nextId;+ }+ }+ do_observe(tree, node, scope, trace, depth) { return this.observeOrAct(tree, node, scope, trace, true); }+ async do_act(tree, node, scope, trace, depth) {+ const emit = node.get("emit");+ if (emit !== undefined && emit !== null) {+ const event = { event: render(emit.event, scope), data: render(emit.data !== undefined ? emit.data : {}, scope), tree: tree.name };+ this.emitted.push(event);+ put(scope.state, node.get("into", "last"), event);+ return [step(node.id, "act", `emitted ${event.event}`, { observed: event.data }), node.next];+ }+ return this.observeOrAct(tree, node, scope, trace, false);+ }+ async observeOrAct(tree, node, scope, trace, retry) {+ const state = scope.state, into = node.get("into", "last"), http = node.get("http");+ let source, summary;+ if (http !== undefined && http !== null) {+ const req = render(http, scope);+ const method = String(req.method || "GET").toUpperCase(), url = String(req.url);+ if (!tree.hostAllowed(url)) return outcome("error", tree.name, { reason: "host-not-allowed", say: `${node.id}: ${url} is outside this tree's hosts ${pyrepr(tree.hosts)}.` });+ const headers = {}; for (const [k, v] of Object.entries(req.headers || {})) headers[String(k)] = String(v);+ let body = req.body;+ if (body !== undefined && body !== null && typeof body !== "string") { body = JSON.stringify(body); if (!("Content-Type" in headers)) headers["Content-Type"] = "application/json"; }+ const timeout = Number(node.get("timeout", DEFAULT_TIMEOUT));+ const attempts = retry ? 1 + Number(node.get("retries", DEFAULT_RETRIES)) : 1;+ let lastError = null, resp = null;+ for (let attempt = 0; attempt < attempts; attempt++) {+ try { resp = await this.transport.request(method, url, headers, body === undefined ? null : body, timeout); break; }+ catch (e) { if (!(e instanceof TransportError)) throw e; lastError = e.message; if (attempt + 1 < attempts) await sleep(Math.min(200 * (attempt + 1), 1000)); }+ }+ if (!resp) { trace.add(step(node.id, node.kind, `${method} ${url} failed: ${lastError}`)); return outcome("error", tree.name, { reason: "transport", say: `${node.id}: ${lastError}` }); }+ const observed = resp.toState();+ put(state, into, observed);+ source = observed;+ summary = `${method} ${url} -> ${resp.status}`;+ } else {+ const frm = node.get("from");+ const sourceVal = resolve(statePath(frm), scope);+ if (isMissing(sourceVal)) source = {};+ else if (isObj(sourceVal)) source = sourceVal;+ else source = { value: sourceVal, body: typeof sourceVal === "string" ? sourceVal : null };+ summary = `from ${frm}`;+ }+ const extracted = {}; const cache = new Map();+ for (const entry of node.get("extract", []) || []) {+ const value = extract(entry, source, cache);+ if (isMissing(value)) extracted[entry.into] = null;+ else { put(state, entry.into, value); extracted[entry.into] = value; }+ }+ const keys = Object.keys(extracted);+ if (keys.length) summary += "; " + keys.map((k) => `${k}=${brief(extracted[k])}`).join(", ");+ return [step(node.id, node.kind, summary, { observed: keys.length ? extracted : null }), node.next];+ }+ do_decide(tree, node, scope, trace, depth) {+ const on = statePath(node.get("on"));+ const value = resolve(on, scope);+ for (const br of node.get("branches")) {+ if ("else" in br) return [step(node.id, "decide", `${on} = ${brief(value)}`, { observed: plain(value), chosen: "else" }), br.else];+ if (holds(br.when, value)) return [step(node.id, "decide", `${on} = ${brief(value)}`, { observed: plain(value), chosen: condText(br.when) }), br.then];+ }+ trace.gap(node.id, on, plain(value));+ trace.add(step(node.id, "decide", `${on} = ${brief(value)}: no branch`, { observed: plain(value), depth }));+ return outcome("error", tree.name, { reason: "gap", say: `${node.id}: no branch for ${on} = ${brief(value)}.`, output: { gap: trace.gaps[trace.gaps.length - 1] } });+ }+ do_set(tree, node, scope) {+ const values = {};+ for (const [key, tmpl] of Object.entries(node.get("set"))) { const value = render(tmpl, scope); put(scope.state, key, value); values[key] = value; }+ return [step(node.id, "set", Object.keys(values).map((k) => `${k}=${brief(values[k])}`).join(", "), { observed: values }), node.next];+ }+ async do_each(tree, node, scope, trace, depth) {+ const state = scope.state;+ let items = resolve(statePath(node.get("over")), scope);+ if (isMissing(items) || items === null || items === undefined) items = [];+ if (!Array.isArray(items)) return outcome("error", tree.name, { reason: "not-a-list", say: `${node.id}: ${node.get("over")} is not a list.` });+ let limit = node.get("limit");+ if (limit !== undefined && limit !== null) { limit = Math.trunc(Number(render(limit, scope))); items = items.slice(0, limit); }+ const results = []; let failures = 0;+ for (let index = 0; index < items.length; index++) {+ const item = items[index];+ const withInputs = render(node.get("with", {}), Object.assign({}, scope, { item, index }));+ let sub;+ if ("tree" in node.spec) sub = await this.runTree(node.get("tree"), withInputs, depth + 1);+ else sub = await this.run(tree, scope.inputs, { depth: depth + 1, start: node.get("start"), state, scopeExtra: { item, index }, trace });+ if (sub.status === "done") results.push(sub.output !== null && sub.output !== undefined ? sub.output : sub.say);+ else { failures++; results.push(null); if (node.get("stop_on_fail", false)) return outcome("error", tree.name, { reason: "each-failed", say: `${node.id}: item ${index} failed: ${sub.say || sub.reason}` }); }+ if (sub.trace && sub.trace !== trace) { for (const s of sub.trace.steps) { s.depth = depth + 1; trace.steps.push(s); } trace.gaps.push(...sub.trace.gaps); }+ }+ put(state, node.get("into"), results);+ return [step(node.id, "each", `${items.length} item(s)` + (failures ? `, ${failures} failed` : "")), node.next];+ }+ async do_delegate(tree, node, scope, trace, depth) {+ const withInputs = render(node.get("with", {}), scope);+ let sub, summary;+ if ("intent" in node.spec) { const text = render(node.get("intent"), scope); sub = await this.runIntent(pyStr(text), depth + 1); summary = `intent ${pyrepr(pyStr(text))} -> ${sub.tree || "?"}: ${sub.status}`; }+ else { sub = await this.runTree(node.get("tree"), withInputs, depth + 1); summary = `tree ${node.get("tree")}: ${sub.status}`; }+ if (sub.trace && sub.trace !== trace) { for (const s of sub.trace.steps) { s.depth = depth + 1; trace.steps.push(s); } trace.gaps.push(...sub.trace.gaps); }+ put(scope.state, node.get("into"), { status: sub.status, say: sub.say, output: sub.output, reason: sub.reason, tree: sub.tree });+ if (sub.status === "asked") return outcome("error", tree.name, { reason: "ask-in-delegate", say: `${node.id}: ${sub.tree} asked a question; delegated trees cannot pause.` });+ return [step(node.id, "delegate", summary), node.next];+ }+ do_ask(tree, node, scope, trace, depth) {+ if (depth > 0) return outcome("error", tree.name, { reason: "ask-in-subflow", say: `${node.id}: only a top-level run can ask.` });+ const prompt = render(node.get("prompt"), scope);+ trace.add(step(node.id, "ask", `asked: ${prompt}`, { depth }));+ return outcome("asked", tree.name, { question: pyStr(prompt), snapshot: { tree: tree.name, node: node.id, inputs: scope.inputs, state: scope.state, trace: trace.steps.map(stepDict) } });+ }+ do_done(tree, node, scope, trace, depth) {+ let say = node.get("say"), output = node.get("output");+ say = say !== undefined && say !== null ? render(say, scope, false) : null;+ output = output !== undefined && output !== null ? render(output, scope, false) : null;+ trace.add(step(node.id, "done", say ? pyStr(say) : "done", { depth }));+ return outcome("done", tree.name, { say: say !== null ? pyStr(say) : null, output });+ }+ do_fail(tree, node, scope, trace, depth) {+ let say = node.get("say");+ say = say !== undefined && say !== null ? render(say, scope, false) : null;+ const reason = pyStr(node.get("reason"));+ trace.add(step(node.id, "fail", `${reason}: ${say || ""}`.replace(/: $/, ""), { depth }));+ const out = node.get("output");+ return outcome("failed", tree.name, { reason, say: say !== null ? pyStr(say) : null, output: out !== undefined && out !== null ? render(out, scope, false) : null });+ }+ }+ function fromDict(d, fallbackTree) {+ if (!isObj(d)) return outcome("error", fallbackTree, { reason: "remote", say: "no answer" });+ let trace = null;+ if (isObj(d.trace)) { trace = new Trace(d.trace.tree || fallbackTree); trace.steps = (d.trace.steps || []).map((s) => step(s.node, s.kind, s.summary, { observed: s.observed === undefined ? null : s.observed, chosen: s.chosen === undefined ? null : s.chosen, next: s.next === undefined ? null : s.next, elapsed_ms: s.elapsed_ms || 0, depth: s.depth || 0 })); trace.gaps = d.trace.gaps || []; }+ return outcome(d.status || "error", d.tree || fallbackTree, { say: d.say === undefined ? null : d.say, output: d.output === undefined ? null : d.output, reason: d.reason === undefined ? null : d.reason, question: d.question === undefined ? null : d.question, snapshot: d.snapshot === undefined ? null : d.snapshot, trace, inputs: d.inputs || {} });+ }++ return { MISSING, Tree, Runner, Trace, FetchTransport, RecordedTransport, Response, render, resolve, holds, extract, outcomeDict, fromDict, pyrepr, pyJson, html, TemplateError, TreeError, TransportError, ConditionError, version: "0.6.0" };+});diff --git a/src/pyrtan/page.py b/src/pyrtan/page.py index 0fdca9a..51e9a43 100644 --- a/src/pyrtan/page.py +++ b/src/pyrtan/page.py@@ -20,9 +20,13 @@ from __future__ import annotationsimport json import urllib.parse+from pathlib import Pathfrom . import __version__+# The tree runtime in JavaScript, served beside the console so a member can run trees.+PYRTAN_JS = (Path(__file__).parent / "js" / "pyrtan.js").read_text(encoding="utf-8")+PAGE = r"""<!doctype html> <html lang="en"> <head>@@ -31,6 +35,7 @@ PAGE = r"""<!doctype html><title>Pyrtan</title> <meta name="description" content="Pyrtan console: ask the swarm, read the trace, see what it can answer, find the gaps, and join a hub as a member that keeps its blobs."> <link rel="manifest" href="manifest.webmanifest">+<script src="pyrtan.js"></script><meta name="theme-color" content="#2b5db8"> <style> :root { --bg:#fbfaf7; --fg:#1d1d1b; --mute:#6b6a64; --line:#e3e1da; --card:#ffffff; --ok:#1f7a4d; --bad:#a3382b; --ask:#8a6400; --accent:#2b5db8; --accent-fg:#fff; --soft:#f1efe9 }@@ -161,17 +166,25 @@ PAGE = r"""<!doctype html><input type="password" id="member-fedsecret" placeholder="federation secret, if the hub takes follows only with one" autocomplete="off"> <button type="submit">Join this hub</button> </form>- <div class="mute" style="font-size:14px">Makes an identity for this browser (kept in local storage), follows the hub with it, and starts polling. Nothing leaves this device but signed activities to the hub.</div>+ <div class="mute" style="font-size:14px">Makes an identity for this browser (kept in local storage), follows the hub with it, and starts polling. Nothing leaves this device but signed activities to the hub. Joined with the hub's token (entered above), this device is <em>trusted</em>: the hub hands it its trees and routes sentences to it, and the trees run here.</div></div> <div id="member-on" hidden> <div><span class="status done">member</span> <code id="member-id"></code> <span class="mute" id="member-state"></span></div>- <div class="mute" style="font-size:14px;margin:6px 0">Name <code id="member-name-shown"></code> · mailbox polled <span id="member-polled">never</span> · <label><input type="checkbox" id="member-listed"> listed in the directory</label></div>+ <div class="mute" style="font-size:14px;margin:6px 0">Name <code id="member-name-shown"></code> · <span id="member-trust"></span> · mailbox polled <span id="member-polled">never</span> · <label><input type="checkbox" id="member-listed"> listed in the directory</label></div><div style="display:flex;gap:8px;flex-wrap:wrap;margin:8px 0"> <button type="button" class="quiet" id="member-poll">Poll now</button> <button type="button" class="quiet" id="member-leave">Leave</button> </div> <h2 style="font-size:15px;margin:14px 0 6px">Kept here for this hub</h2> <table id="kept"><thead><tr><th>Key</th><th>Size</th><th>Updated</th><th></th></tr></thead><tbody></tbody></table>+ <h2 style="font-size:15px;margin:14px 0 6px">Trees this device runs</h2>+ <table id="member-trees"><thead><tr><th>Tree</th><th>Answers to</th><th>Reaches</th></tr></thead><tbody></tbody></table>+ <form class="row" id="member-run" style="margin-top:8px">+ <select id="member-run-tree" style="font:inherit;padding:10px 12px;border:1px solid var(--line);border-radius:8px;background:var(--card);color:var(--fg)"></select>+ <input type="text" id="member-run-inputs" placeholder='inputs as JSON, e.g. {"place": "Oslo"}' autocomplete="off">+ <button type="submit">Run here</button>+ </form>+ <div id="member-out"></div><h2 style="font-size:15px;margin:14px 0 6px">Activity</h2> <pre id="member-log" style="max-height:220px"></pre> </div>@@ -466,7 +479,7 @@ function signHeaders(secret, actor, method, path, body) {const randomHex = (n) => hex(crypto.getRandomValues(new Uint8Array(n))); const member = {- hub: null, me: null, timer: null, log: [], directory: new Map(), db: null,+ hub: null, me: null, timer: null, log: [], directory: new Map(), db: null, runner: null, runs: 0,key() { return "pyrtan.member:" + (this.hub || ""); }, load() { try { this.me = JSON.parse(localStorage.getItem(this.key()) || "null"); } catch (e) { this.me = null; } }, save() { try { if (this.me) localStorage.setItem(this.key(), JSON.stringify(this.me)); else localStorage.removeItem(this.key()); } catch (e) {} },@@ -490,14 +503,16 @@ const member = {}, activity(type, object, extra) { return Object.assign({type, id: this.me.id + "/activities/" + randomHex(8), actor: this.me.id, published: new Date().toISOString().replace(/\.\d+Z$/, "Z"), object}, extra || {}); }, async join(name, fedSecret) {- this.me = {id: "urn:pyrtan:member:" + randomHex(16), secret: randomHex(32), name: name || "browser", state: "pending", listed: true, joined: Date.now()};+ this.me = {id: "urn:pyrtan:member:" + randomHex(16), secret: randomHex(32), name: name || "browser", state: "pending", listed: true, joined: Date.now(), trusted: false};this.save(); const act = this.activity("Follow", this.hub, {secret: this.me.secret, inbox: "mailbox", name: this.me.name}); const headers = Object.assign({"Accept": "application/json", "Content-Type": "application/json"}, signHeaders(fedSecret || this.me.secret, this.me.id, "POST", "/inbox", act));+ if (token) headers["Authorization"] = "Bearer " + token; // the operator's token makes this device trusted: it runs the hub's treesconst r = await fetch((base ? base.replace(/\/+$/, "") : "") + "/inbox", {method: "POST", headers, body: canonical(act)}); const data = await r.json().catch(() => ({})); if (r.status !== 202) { this.me = null; this.save(); this.note(`follow refused: ${data.say || r.status}`); problem(`The hub did not take the follow: ${data.say || r.status}`, true); this.render(); return; }- this.note(`followed ${this.hub}`);+ this.me.trusted = !!data.trusted; this.save();+ this.note(`followed ${this.hub}` + (this.me.trusted ? " as a trusted device" : ""));await this.poll(); this.start(); this.render();@@ -515,28 +530,55 @@ const member = {for (const a of (r.data && r.data.activities) || []) { if (a.type === "Accept") { this.me.state = "accepted"; this.note("accepted"); } else if (a.type === "Announce") { for (const e of a.object.directory || []) this.directory.set(e.id, e); this.note(`announce: ${(a.object.trees||[]).length} trees, ${(a.object.directory||[]).length} directory entries`); }- else if (a.type === "Keep") { await this.keptPut(a.object); await this.send("POST", "/inbox", this.activity("Kept", {key: a.object.key, sha256: a.object.sha256})); this.note(`keeping ${a.object.key} (${a.object.size} B)`); }+ else if (a.type === "Keep") { await this.keptPut(a.object); if (a.object.key.startsWith("trees/")) this.hold(a.object.data); await this.send("POST", "/inbox", this.activity("Kept", {key: a.object.key, sha256: a.object.sha256})); this.note(`keeping ${a.object.key} (${a.object.size} B)`); }+ else if (a.type === "Ask") { await this.answer(a.object); }else if (a.type === "Fetch") { const b = await this.keptGet(a.object.key); if (b) { await this.send("POST", "/inbox", this.activity("Kept", {key: b.key, sha256: b.sha256, data: b.data})); this.note(`handed back ${b.key}`); } else this.note(`asked for ${a.object.key}, not kept here`); } else if (a.type === "Reject") { this.me.state = "rejected"; this.note("rejected: " + (a.summary || "")); } } this.save(); this.render(); },+ // ---- running trees: the hub hands them over as trees/<name> blobs; Ask runs one here, Answer goes back.+ ensureRunner() {+ if (this.runner) return this.runner;+ const self = this;+ const resolver = { // a member may ask the hub it follows: delegate goes there, signed+ async runTree(name, inputs, depth) { const r = await self.send("POST", "/run", {tree: name, inputs, depth}); return r.data; },+ async runIntent(text, depth) { const r = await self.send("POST", "/intent", {text, depth}); return r.data; },+ };+ this.runner = new Pyrtan.Runner({transport: new Pyrtan.FetchTransport(), resolver});+ return this.runner;+ },+ hold(raw) { try { this.ensureRunner().add(raw); } catch (e) { this.note(`tree refused: ${e.message}`); } },+ async restoreTrees() { for (const b of await this.keptAll()) if (b.key.startsWith("trees/")) this.hold(b.data); },+ capabilities() { return this.runner ? Object.values(this.runner.trees).map(t => t.capability()) : []; },+ async answer(ask) {+ const runner = this.ensureRunner();+ let outcome;+ try { outcome = Pyrtan.outcomeDict(await runner.runTree(ask.tree, ask.inputs || {}, ask.depth || 0)); }+ catch (e) { outcome = {status: "error", tree: ask.tree, reason: "member-error", say: String(e && e.message || e)}; }+ this.runs++;+ await this.send("POST", "/inbox", this.activity("Answer", {run: ask.run, outcome}));+ this.note(`ran ${ask.tree} for the hub: ${outcome.status}` + (outcome.say ? ` · ${String(outcome.say).slice(0, 80)}` : ""));+ this.render();+ },async announce() { if (!this.me || this.me.state !== "accepted") return; const kept = (await this.keptAll()).map(b => b.key).sort();- const own = this.me.listed ? [{id: this.me.id, name: this.me.name, kind: "member", trees: [], seen: Date.now() / 1000}] : [];+ const trees = this.me.trusted ? this.capabilities() : [];+ const own = this.me.listed ? [{id: this.me.id, name: this.me.name, kind: "member", trees: trees.map(t => t.name), seen: Date.now() / 1000}] : [];const heard = [...this.directory.values()].filter(e => e.id !== this.me.id); try {- const r = await this.send("POST", "/inbox", this.activity("Announce", {type: "Capabilities", name: this.me.name, trees: [], directory: own.concat(heard), kept}));+ const r = await this.send("POST", "/inbox", this.activity("Announce", {type: "Capabilities", name: this.me.name, trees, directory: own.concat(heard), kept}));for (const e of (r.data && r.data.directory) || []) this.directory.set(e.id, e); } catch (e) { this.note("announce failed: " + e.message); } }, start() { this.load(); if (!this.me || this.timer) { this.render(); return; }- this.timer = setInterval(() => this.poll(), 20000);+ // A trusted device answers Asks, so it polls faster: a sentence waits one poll.+ this.timer = setInterval(() => this.poll(), this.me.trusted ? 4000 : 20000);this.announceTimer = setInterval(() => this.announce(), 60000);- this.poll().then(() => this.announce());+ this.restoreTrees().then(() => this.poll()).then(() => this.announce());}, stop() { clearInterval(this.timer); clearInterval(this.announceTimer); this.timer = null; }, async render() {@@ -546,14 +588,34 @@ const member = {$("#member-id").textContent = this.me.id; $("#member-state").textContent = this.me.state; $("#member-name-shown").textContent = this.me.name; $("#member-polled").textContent = this.me.polled ? new Date(this.me.polled).toLocaleTimeString() : "never"; $("#member-listed").checked = !!this.me.listed;- const body = $("#kept tbody"); body.innerHTML = "";- for (const b of await this.keptAll()) { const tr = document.createElement("tr"); tr.innerHTML = `<td><code>${esc(b.key)}</code></td><td>${esc(b.size)} B</td><td class="mute">${b.updated ? new Date(b.updated * 1000).toLocaleString() : ""}</td><td><a href="#" data-show="${esc(b.key)}">show</a></td>`; body.appendChild(tr); }+ $("#member-trust").textContent = this.me.trusted ? "trusted: runs the hub's trees" : "keeper only (join with the hub's token to run trees)";+ const tb = $("#member-trees tbody"); tb.innerHTML = "";+ const sel = $("#member-run-tree"); const chosen = sel.value; sel.innerHTML = "";+ for (const t of this.capabilities()) {+ const tr = document.createElement("tr"); tr.innerHTML = `<td><code>${esc(t.name)}</code> <span class="mute">v${esc(t.version)}</span></td><td>${(t.triggers||[]).map(p => `<div><code>${esc(p)}</code></div>`).join("")}</td><td class="mute">${esc((t.hosts||[]).join(", "))}</td>`; tb.appendChild(tr);+ const o = document.createElement("option"); o.value = t.name; o.textContent = t.name; sel.appendChild(o);+ }+ if (chosen) sel.value = chosen;+ if (!tb.children.length) tb.innerHTML = `<tr><td colspan="3" class="mute">${this.me.trusted ? "None handed over yet; the hub sends what this device should run, and this device picks it up on its next poll." : "A keeper runs nothing."}</td></tr>`;+ const blobs = await this.keptAll();+ const body = $("#kept tbody"); body.innerHTML = ""; // cleared after the await, so two renders at once do not double the rows+ for (const b of blobs) { const tr = document.createElement("tr"); tr.innerHTML = `<td><code>${esc(b.key)}</code></td><td>${esc(b.size)} B</td><td class="mute">${b.updated ? new Date(b.updated * 1000).toLocaleString() : ""}</td><td><a href="#" data-show="${esc(b.key)}">show</a></td>`; body.appendChild(tr); }if (!body.children.length) body.innerHTML = `<tr><td colspan="4" class="mute">Nothing yet. The hub sends its blobs as it needs keepers; "Poll now" asks sooner.</td></tr>`; }, }; $("#join-form").addEventListener("submit", (e) => { e.preventDefault(); member.join($("#member-name").value.trim(), $("#member-fedsecret").value); }); $("#member-poll").addEventListener("click", () => member.poll().then(() => member.announce())); $("#member-leave").addEventListener("click", () => member.leave());+$("#member-run").addEventListener("submit", async (e) => {+ e.preventDefault();+ const name = $("#member-run-tree").value; if (!name) return;+ let inputs = {}; try { inputs = JSON.parse($("#member-run-inputs").value || "{}"); } catch (err) { problem("inputs must be JSON", true); return; }+ const t0 = performance.now();+ const r = Pyrtan.outcomeDict(await member.ensureRunner().runTree(name, inputs));+ const card = document.createElement("div"); card.className = "card";+ card.innerHTML = `<div><span class="status ${esc(r.status)}">${esc(r.status)}</span>${r.reason ? ` <span class="mute">(${esc(r.reason)})</span>` : ""} <span class="mute">· ${esc(r.tree||"")} · ${Math.round(performance.now() - t0)} ms · ran in this browser</span></div>${r.say ? `<div class="say">${esc(r.say)}</div>` : ""}${r.output != null ? `<details open><summary>output</summary><pre>${esc(JSON.stringify(r.output, null, 2))}</pre></details>` : ""}${r.trace ? `<details open><summary>trace</summary><pre>${esc(renderTrace(r.trace))}</pre></details>` : ""}`;+ $("#member-out").prepend(card);+});$("#member-listed").addEventListener("change", (e) => { member.me.listed = e.target.checked; member.save(); member.announce(); }); $("#kept").addEventListener("click", async (e) => { const a = e.target.closest("a[data-show]"); if (!a) return; e.preventDefault(); const b = await member.keptGet(a.dataset.show); alert(b ? JSON.stringify(b.data, null, 2).slice(0, 4000) : "not kept"); }); if ("serviceWorker" in navigator && (location.protocol === "https:" || ["localhost", "127.0.0.1"].includes(location.hostname))) { navigator.serviceWorker.register("sw.js").catch(() => {}); }@@ -581,7 +643,7 @@ MANIFEST = json.dumps({# phone in a tunnel, a hub that is down): the shell comes from the cache, # everything the page asks the hub for goes to the network and nowhere else. SERVICE_WORKER = r"""const SHELL = "pyrtan-shell-__VERSION__";-const FILES = ["./", "manifest.webmanifest"];+const FILES = ["./", "manifest.webmanifest", "pyrtan.js"];self.addEventListener("install", (e) => { e.waitUntil(caches.open(SHELL).then(c => c.addAll(FILES)).then(() => self.skipWaiting())); }); self.addEventListener("activate", (e) => { e.waitUntil(caches.keys().then(keys => Promise.all(keys.filter(k => k !== SHELL).map(k => caches.delete(k)))).then(() => self.clients.claim())); }); self.addEventListener("fetch", (e) => { diff --git a/src/pyrtan/service.py b/src/pyrtan/service.py index 67d6e10..ac71024 100644 --- a/src/pyrtan/service.py +++ b/src/pyrtan/service.py@@ -53,7 +53,7 @@ from typing import Any, Callablefrom . import __version__ from .gaps import GapStore-from .page import MANIFEST, PAGE, SERVICE_WORKER+from .page import MANIFEST, PAGE, PYRTAN_JS, SERVICE_WORKERfrom .grow import Unanswered from .registry import Registry from .runtime import Outcome, Runner@@ -63,7 +63,7 @@ from .transport import Transport, UrllibTransportMAX_BODY = 1 << 20 # What answers without a token: the console, its index, and liveness.-PUBLIC_PATHS = frozenset({"/", "/index.html", "/console", "/api", "/health", "/actor", "/directory", "/manifest.webmanifest", "/sw.js"})+PUBLIC_PATHS = frozenset({"/", "/index.html", "/console", "/api", "/health", "/actor", "/directory", "/manifest.webmanifest", "/sw.js", "/pyrtan.js"})# On a federated hub, a request carrying a signature reaches the route, # which verifies it (see :mod:`pyrtan.federation`). SIGNED_PATHS = frozenset({"/inbox", "/run", "/intent", "/resume", "/capabilities", "/mailbox"})@@ -232,6 +232,8 @@ class Service:return 200, ("application/manifest+json", MANIFEST) if method == "GET" and path == "/sw.js": return 200, ("text/javascript; charset=utf-8", SERVICE_WORKER)+ if method == "GET" and path == "/pyrtan.js":+ return 200, ("text/javascript; charset=utf-8", PYRTAN_JS)if method == "GET" and path == "/api": return 200, {"pyrtan": "service", "name": self.name, "version": __version__, "auth": "bearer" if self.token else "none", "endpoints": ["/health", "/capabilities", "/intent", "/run", "/resume", "/gaps", "/unanswered"]} if method == "POST" and path == "/intent": diff --git a/src/pyrtan/transport.py b/src/pyrtan/transport.py index 1943948..dc36905 100644 --- a/src/pyrtan/transport.py +++ b/src/pyrtan/transport.py@@ -129,7 +129,7 @@ class UrllibTransport:``public`` (refuse private, loopback, link-local and metadata addresses, on the first URL and on every redirect)."""- def __init__(self, user_agent: str = "pyrtan/0.5 (+https://github.com/b-i-t-8/pyrtan)", egress: str = "any") -> None:+ def __init__(self, user_agent: str = "pyrtan/0.6 (+https://github.com/b-i-t-8/pyrtan)", egress: str = "any") -> None:if egress not in EGRESS_POLICIES: raise ValueError(f"egress policy must be one of {EGRESS_POLICIES}, got {egress!r}") self.user_agent = user_agent diff --git a/tests/conformance/make_vectors.py b/tests/conformance/make_vectors.py new file mode 100644 index 0000000..db87505 --- /dev/null +++ b/tests/conformance/make_vectors.py@@ -0,0 +1,92 @@+"""Write tests/conformance/vectors.json: what both runtimes must say.++Every example tree as JSON (the TOML's own table), and for a fixed set of+runs (a sentence read by the Python matcher, or a tree and inputs) the+outcome the Python runtime produced against the recorded responses in+tests/fixtures/examples.json, with a fixed clock. The JavaScript runtime+(src/pyrtan/js/pyrtan.js) must produce the same status, say, output,+reason, path and gaps; tests/conformance/run.cjs checks it, and+tests/test_conformance.py keeps this file from drifting.++ uv run python tests/conformance/make_vectors.py+"""++from __future__ import annotations++import datetime as _dt+import json+import sys+from pathlib import Path++ROOT = Path(__file__).resolve().parents[2]+sys.path.insert(0, str(ROOT / "src"))++from pyrtan import Registry, RecordedTransport, Runner # noqa: E402++CLOCK = _dt.datetime(2026, 10, 25, 12, 0, tzinfo=_dt.timezone.utc)+CASES: list[dict] = [+ {"intent": "What's the top story on Hacker News?"},+ {"intent": "top 3 stories on hacker news"},+ {"intent": "top 99 stories on hacker news"},+ {"intent": "hacker news item 999"},+ {"intent": "weather in Oslo"},+ {"tree": "weather", "inputs": {"place": "Oslo", "units": "fahrenheit"}},+ {"tree": "weather", "inputs": {"place": "Oslo", "units": "kelvin"}},+ {"intent": "how cold is it in Nowhereville"},+ {"intent": "is Example.com up"},+ {"intent": "is broken.example up"},+ {"intent": "check status of teapot.example"},+ {"tree": "site-status", "inputs": {"host": "http://x"}},+ {"intent": "latest release of b-i-t-8/pyrtan"},+ {"intent": "latest b-i-t-8/nothing release"},+ {"intent": "latest release of b-i-t-8/busy"},+ {"intent": "weather in Paris"},+ {"tree": "hacker-news-item", "inputs": {"id": 101}},+ {"tree": "hacker-news", "inputs": {"count": 2}},+]+++def _stable(d: dict) -> dict:+ """Drop the wall-clock fields (when the trace started, how long each step took)."""+ trace = d.get("trace")+ if isinstance(trace, dict):+ trace.pop("started", None)+ for step in trace.get("steps", []):+ step.pop("elapsed_ms", None)+ return d+++def build() -> dict:+ reg = Registry.from_paths([ROOT / "examples"])+ assert not reg.errors, reg.errors+ fixture = ROOT / "tests" / "fixtures" / "examples.json"+ runs = []+ for case in CASES:+ runner = Runner(reg, RecordedTransport.from_file(fixture), clock=lambda: CLOCK)+ if "intent" in case:+ reading = reg.read(case["intent"], CLOCK)+ match = reading.winner+ assert match is not None, case+ outcome = runner.run_tree(match.tree, match.inputs)+ runs.append({"intent": case["intent"], "tree": match.tree, "inputs": match.inputs, "expect": _stable(outcome.to_dict())})+ else:+ outcome = runner.run_tree(case["tree"], case["inputs"])+ runs.append({"tree": case["tree"], "inputs": case["inputs"], "expect": _stable(outcome.to_dict())})+ return {+ "pyrtan-conformance": 1,+ "clock": CLOCK.isoformat(),+ "fixture": "tests/fixtures/examples.json",+ "trees": {t.name: t.raw for t in reg.trees()},+ "runs": runs,+ }+++def main() -> int:+ out = ROOT / "tests" / "conformance" / "vectors.json"+ out.write_text(json.dumps(build(), indent=1, ensure_ascii=False, sort_keys=True) + "\n", encoding="utf-8")+ print(f"wrote {out}: {len(CASES)} runs")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/tests/conformance/run.cjs b/tests/conformance/run.cjs new file mode 100644 index 0000000..fe49d92 --- /dev/null +++ b/tests/conformance/run.cjs@@ -0,0 +1,37 @@+// Run every vector through pyrtan.js and compare with what Python said.+// node tests/conformance/run.cjs (exit 1 on any difference)+const fs = require("fs");+const path = require("path");+const ROOT = path.resolve(__dirname, "..", "..");+const Pyrtan = require(path.join(ROOT, "src", "pyrtan", "js", "pyrtan.js"));+const vectors = JSON.parse(fs.readFileSync(path.join(__dirname, "vectors.json"), "utf8"));+const fixture = JSON.parse(fs.readFileSync(path.join(ROOT, vectors.fixture), "utf8"));+const clock = () => new Date(vectors.clock);++// What must agree: the outcome and the path, not the timings or the summaries' wording.+function shape(d) {+ return {+ status: d.status, tree: d.tree, say: d.say === undefined ? null : d.say, reason: d.reason === undefined ? null : d.reason,+ output: d.output === undefined ? null : d.output, inputs: d.inputs || {},+ steps: (d.trace ? d.trace.steps : []).map((s) => ({ node: s.node, kind: s.kind, chosen: s.chosen === undefined ? null : s.chosen, next: s.next === undefined ? null : s.next, depth: s.depth || 0 })),+ gaps: (d.trace ? d.trace.gaps : []).map((g) => ({ node: g.node, on: g.on, observed: g.observed })),+ };+}+const canon = (v) => JSON.stringify(v, (k, x) => (x && typeof x === "object" && !Array.isArray(x) ? Object.keys(x).sort().reduce((o, kk) => ((o[kk] = x[kk]), o), {}) : x));++(async () => {+ let failed = 0;+ for (const run of vectors.runs) {+ const runner = new Pyrtan.Runner({ trees: Object.values(vectors.trees), transport: new Pyrtan.RecordedTransport(fixture), clock });+ const got = Pyrtan.outcomeDict(await runner.runTree(run.tree, run.inputs));+ const a = canon(shape(run.expect)), b = canon(shape(got));+ const label = run.intent ? `"${run.intent}"` : `${run.tree} ${JSON.stringify(run.inputs)}`;+ if (a === b) { console.log(`ok ${label}`); continue; }+ failed++;+ console.log(`DIFF ${label}`);+ const ea = shape(run.expect), eb = shape(got);+ for (const k of Object.keys(ea)) if (canon(ea[k]) !== canon(eb[k])) console.log(` ${k}:\n python: ${canon(ea[k]).slice(0, 400)}\n js: ${canon(eb[k]).slice(0, 400)}`);+ }+ console.log(`${vectors.runs.length - failed}/${vectors.runs.length} agree`);+ process.exit(failed ? 1 : 0);+})();diff --git a/tests/conformance/vectors.json b/tests/conformance/vectors.json new file mode 100644 index 0000000..7962818 --- /dev/null +++ b/tests/conformance/vectors.json@@ -0,0 +1,2088 @@+{+ "clock": "2026-10-25T12:00:00+00:00",+ "fixture": "tests/fixtures/examples.json",+ "pyrtan-conformance": 1,+ "runs": [+ {+ "expect": {+ "inputs": {+ "count": 1+ },+ "output": {+ "stories": [+ {+ "by": "alice",+ "id": 101,+ "score": 42,+ "title": "Pyrtan 0.1",+ "url": "https://example.org/p"+ }+ ]+ },+ "say": "Top of Hacker News: Pyrtan 0.1 — https://example.org/p",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "check",+ "node": "fetch-ids",+ "observed": {+ "ids": [+ 101,+ 102,+ 103+ ]+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/topstories.json -> 200; ids=[101, 102, 103]"+ },+ {+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "any",+ "node": "check",+ "observed": 200,+ "summary": "state.ids_resp.status = 200"+ },+ {+ "chosen": "type 'list'",+ "depth": 0,+ "kind": "decide",+ "next": "stories",+ "node": "any",+ "observed": [+ 101,+ 102,+ 103+ ],+ "summary": "state.ids = [101, 102, 103]"+ },+ {+ "depth": 1,+ "kind": "observe",+ "next": "kind",+ "node": "fetch",+ "observed": {+ "by": "alice",+ "score": 42,+ "title": "Pyrtan 0.1",+ "type": "story",+ "url": "https://example.org/p"+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/item/101.json -> 200; title=Pyrtan 0.1, url=https://example.org/p, score=42, by=alice, type=story"+ },+ {+ "chosen": "eq 'story'",+ "depth": 1,+ "kind": "decide",+ "next": "has-url",+ "node": "kind",+ "observed": "story",+ "summary": "state.type = story"+ },+ {+ "chosen": "exists True",+ "depth": 1,+ "kind": "decide",+ "next": "reply",+ "node": "has-url",+ "observed": "https://example.org/p",+ "summary": "state.url = https://example.org/p"+ },+ {+ "depth": 1,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Pyrtan 0.1 (42 points, by alice) https://example.org/p"+ },+ {+ "depth": 0,+ "kind": "each",+ "next": "reply",+ "node": "stories",+ "summary": "1 item(s)"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Top of Hacker News: Pyrtan 0.1 — https://example.org/p"+ }+ ],+ "tree": "hacker-news"+ },+ "tree": "hacker-news"+ },+ "inputs": {+ "count": 1+ },+ "intent": "What's the top story on Hacker News?",+ "tree": "hacker-news"+ },+ {+ "expect": {+ "inputs": {+ "count": 3+ },+ "output": {+ "stories": [+ {+ "by": "alice",+ "id": 101,+ "score": 42,+ "title": "Pyrtan 0.1",+ "url": "https://example.org/p"+ },+ {+ "by": "x",+ "id": 102,+ "score": 7,+ "title": "Ask HN: self post",+ "url": "https://news.ycombinator.com/item?id=102"+ },+ null+ ]+ },+ "say": "Top of Hacker News: Pyrtan 0.1 — https://example.org/p",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "check",+ "node": "fetch-ids",+ "observed": {+ "ids": [+ 101,+ 102,+ 103+ ]+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/topstories.json -> 200; ids=[101, 102, 103]"+ },+ {+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "any",+ "node": "check",+ "observed": 200,+ "summary": "state.ids_resp.status = 200"+ },+ {+ "chosen": "type 'list'",+ "depth": 0,+ "kind": "decide",+ "next": "stories",+ "node": "any",+ "observed": [+ 101,+ 102,+ 103+ ],+ "summary": "state.ids = [101, 102, 103]"+ },+ {+ "depth": 1,+ "kind": "observe",+ "next": "kind",+ "node": "fetch",+ "observed": {+ "by": "alice",+ "score": 42,+ "title": "Pyrtan 0.1",+ "type": "story",+ "url": "https://example.org/p"+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/item/101.json -> 200; title=Pyrtan 0.1, url=https://example.org/p, score=42, by=alice, type=story"+ },+ {+ "chosen": "eq 'story'",+ "depth": 1,+ "kind": "decide",+ "next": "has-url",+ "node": "kind",+ "observed": "story",+ "summary": "state.type = story"+ },+ {+ "chosen": "exists True",+ "depth": 1,+ "kind": "decide",+ "next": "reply",+ "node": "has-url",+ "observed": "https://example.org/p",+ "summary": "state.url = https://example.org/p"+ },+ {+ "depth": 1,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Pyrtan 0.1 (42 points, by alice) https://example.org/p"+ },+ {+ "depth": 1,+ "kind": "observe",+ "next": "kind",+ "node": "fetch",+ "observed": {+ "by": "x",+ "score": 7,+ "title": "Ask HN: self post",+ "type": "story",+ "url": null+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/item/102.json -> 200; title=Ask HN: self post, url=None, score=7, by=x, type=story"+ },+ {+ "chosen": "eq 'story'",+ "depth": 1,+ "kind": "decide",+ "next": "has-url",+ "node": "kind",+ "observed": "story",+ "summary": "state.type = story"+ },+ {+ "chosen": "else",+ "depth": 1,+ "kind": "decide",+ "next": "self-post",+ "node": "has-url",+ "summary": "state.url = <missing>"+ },+ {+ "depth": 1,+ "kind": "set",+ "next": "reply",+ "node": "self-post",+ "observed": {+ "url": "https://news.ycombinator.com/item?id=102"+ },+ "summary": "url=https://news.ycombinator.com/item?id=102"+ },+ {+ "depth": 1,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Ask HN: self post (7 points, by x) https://news.ycombinator.com/item?id=102"+ },+ {+ "depth": 1,+ "kind": "observe",+ "next": "kind",+ "node": "fetch",+ "observed": {+ "by": "y",+ "score": null,+ "title": null,+ "type": "comment",+ "url": null+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/item/103.json -> 200; title=None, url=None, score=None, by=y, type=comment"+ },+ {+ "chosen": "else",+ "depth": 1,+ "kind": "decide",+ "next": "not-a-story",+ "node": "kind",+ "observed": "comment",+ "summary": "state.type = comment"+ },+ {+ "depth": 1,+ "kind": "fail",+ "next": null,+ "node": "not-a-story",+ "summary": "not-a-story: Item 103 is a comment, not a story."+ },+ {+ "depth": 0,+ "kind": "each",+ "next": "reply",+ "node": "stories",+ "summary": "3 item(s), 1 failed"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Top of Hacker News: Pyrtan 0.1 — https://example.org/p"+ }+ ],+ "tree": "hacker-news"+ },+ "tree": "hacker-news"+ },+ "inputs": {+ "count": 3+ },+ "intent": "top 3 stories on hacker news",+ "tree": "hacker-news"+ },+ {+ "expect": {+ "inputs": {},+ "reason": "bad-input",+ "say": "input 'count' must be at most 30, got 99",+ "status": "error",+ "trace": {+ "gaps": [],+ "steps": [],+ "tree": "hacker-news"+ },+ "tree": "hacker-news"+ },+ "inputs": {+ "count": 99+ },+ "intent": "top 99 stories on hacker news",+ "tree": "hacker-news"+ },+ {+ "expect": {+ "inputs": {+ "id": 999+ },+ "reason": "no-such-item",+ "say": "There is no Hacker News item 999.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "kind",+ "node": "fetch",+ "observed": {+ "by": null,+ "score": null,+ "title": null,+ "type": null,+ "url": null+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/item/999.json -> 200; title=None, url=None, score=None, by=None, type=None"+ },+ {+ "chosen": "exists False",+ "depth": 0,+ "kind": "decide",+ "next": "missing",+ "node": "kind",+ "summary": "state.type = <missing>"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "missing",+ "summary": "no-such-item: There is no Hacker News item 999."+ }+ ],+ "tree": "hacker-news-item"+ },+ "tree": "hacker-news-item"+ },+ "inputs": {+ "id": 999+ },+ "intent": "hacker news item 999",+ "tree": "hacker-news-item"+ },+ {+ "expect": {+ "inputs": {+ "place": "Oslo",+ "units": "celsius"+ },+ "output": {+ "country": "Norway",+ "humidity": 80,+ "place": "Oslo",+ "sky": "raining",+ "temperature": 4.2,+ "unit": "°C",+ "wind_kmh": 12.3+ },+ "say": "Oslo, Norway: 4.2°C, raining, humidity 80%, wind 12.3 km/h.",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "found",+ "node": "geocode",+ "observed": {+ "country": "Norway",+ "hit": {+ "country": "Norway",+ "latitude": 59.91,+ "longitude": 10.75,+ "name": "Oslo"+ },+ "lat": 59.91,+ "lon": 10.75,+ "name": "Oslo"+ },+ "summary": "GET https://geocoding-api.open-meteo.com/v1/search?name=Oslo&count=1&language=en&format=json -> 200; hit={'name': 'Oslo', 'country': 'Norway', 'latitude': 59.91, 'longitude': 10.75}, lat=59.91, lon=10.75, name=Oslo, country=Norway"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "forecast",+ "node": "found",+ "observed": {+ "country": "Norway",+ "latitude": 59.91,+ "longitude": 10.75,+ "name": "Oslo"+ },+ "summary": "state.hit = {'name': 'Oslo', 'country': 'Norway', 'latitude': 59.91, 'longitude': 10.75}"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "have-temp",+ "node": "forecast",+ "observed": {+ "code": 61,+ "humidity": 80,+ "temp": 4.2,+ "unit": "°C",+ "wind": 12.3+ },+ "summary": "GET https://api.open-meteo.com/v1/forecast?latitude=59.91&longitude=10.75¤t=temperature_2m,relative_humidity_2m,wind_speed_10m,weather_code&temperature_unit=celsius -> 200; temp=4.2, humidity=80, wind=12.3, code=61, unit=°C"+ },+ {+ "chosen": "type 'number'",+ "depth": 0,+ "kind": "decide",+ "next": "sky",+ "node": "have-temp",+ "observed": 4.2,+ "summary": "state.temp = 4.2"+ },+ {+ "chosen": "between [51, 67]",+ "depth": 0,+ "kind": "decide",+ "next": "rain",+ "node": "sky",+ "observed": 61,+ "summary": "state.code = 61"+ },+ {+ "depth": 0,+ "kind": "set",+ "next": "reply",+ "node": "rain",+ "observed": {+ "sky": "raining"+ },+ "summary": "sky=raining"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Oslo, Norway: 4.2°C, raining, humidity 80%, wind 12.3 km/h."+ }+ ],+ "tree": "weather"+ },+ "tree": "weather"+ },+ "inputs": {+ "place": "Oslo"+ },+ "intent": "weather in Oslo",+ "tree": "weather"+ },+ {+ "expect": {+ "inputs": {+ "place": "Oslo",+ "units": "fahrenheit"+ },+ "output": {+ "country": "Norway",+ "humidity": 80,+ "place": "Oslo",+ "sky": "clear",+ "temperature": 39.6,+ "unit": "°F",+ "wind_kmh": 12.3+ },+ "say": "Oslo, Norway: 39.6°F, clear, humidity 80%, wind 12.3 km/h.",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "found",+ "node": "geocode",+ "observed": {+ "country": "Norway",+ "hit": {+ "country": "Norway",+ "latitude": 59.91,+ "longitude": 10.75,+ "name": "Oslo"+ },+ "lat": 59.91,+ "lon": 10.75,+ "name": "Oslo"+ },+ "summary": "GET https://geocoding-api.open-meteo.com/v1/search?name=Oslo&count=1&language=en&format=json -> 200; hit={'name': 'Oslo', 'country': 'Norway', 'latitude': 59.91, 'longitude': 10.75}, lat=59.91, lon=10.75, name=Oslo, country=Norway"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "forecast",+ "node": "found",+ "observed": {+ "country": "Norway",+ "latitude": 59.91,+ "longitude": 10.75,+ "name": "Oslo"+ },+ "summary": "state.hit = {'name': 'Oslo', 'country': 'Norway', 'latitude': 59.91, 'longitude': 10.75}"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "have-temp",+ "node": "forecast",+ "observed": {+ "code": 0,+ "humidity": 80,+ "temp": 39.6,+ "unit": "°F",+ "wind": 12.3+ },+ "summary": "GET https://api.open-meteo.com/v1/forecast?latitude=59.91&longitude=10.75¤t=temperature_2m,relative_humidity_2m,wind_speed_10m,weather_code&temperature_unit=fahrenheit -> 200; temp=39.6, humidity=80, wind=12.3, code=0, unit=°F"+ },+ {+ "chosen": "type 'number'",+ "depth": 0,+ "kind": "decide",+ "next": "sky",+ "node": "have-temp",+ "observed": 39.6,+ "summary": "state.temp = 39.6"+ },+ {+ "chosen": "eq 0",+ "depth": 0,+ "kind": "decide",+ "next": "clear",+ "node": "sky",+ "observed": 0,+ "summary": "state.code = 0"+ },+ {+ "depth": 0,+ "kind": "set",+ "next": "reply",+ "node": "clear",+ "observed": {+ "sky": "clear"+ },+ "summary": "sky=clear"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Oslo, Norway: 39.6°F, clear, humidity 80%, wind 12.3 km/h."+ }+ ],+ "tree": "weather"+ },+ "tree": "weather"+ },+ "inputs": {+ "place": "Oslo",+ "units": "fahrenheit"+ },+ "tree": "weather"+ },+ {+ "expect": {+ "inputs": {},+ "reason": "bad-input",+ "say": "input 'units' must be one of ['celsius', 'fahrenheit'], got 'kelvin'",+ "status": "error",+ "trace": {+ "gaps": [],+ "steps": [],+ "tree": "weather"+ },+ "tree": "weather"+ },+ "inputs": {+ "place": "Oslo",+ "units": "kelvin"+ },+ "tree": "weather"+ },+ {+ "expect": {+ "inputs": {+ "place": "Nowhereville",+ "units": "celsius"+ },+ "reason": "unknown-place",+ "say": "I could not find a place called Nowhereville.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "found",+ "node": "geocode",+ "observed": {+ "country": null,+ "hit": null,+ "lat": null,+ "lon": null,+ "name": null+ },+ "summary": "GET https://geocoding-api.open-meteo.com/v1/search?name=Nowhereville&count=1&language=en&format=json -> 200; hit=None, lat=None, lon=None, name=None, country=None"+ },+ {+ "chosen": "else",+ "depth": 0,+ "kind": "decide",+ "next": "unknown-place",+ "node": "found",+ "summary": "state.hit = <missing>"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "unknown-place",+ "summary": "unknown-place: I could not find a place called Nowhereville."+ }+ ],+ "tree": "weather"+ },+ "tree": "weather"+ },+ "inputs": {+ "place": "Nowhereville"+ },+ "intent": "how cold is it in Nowhereville",+ "tree": "weather"+ },+ {+ "expect": {+ "inputs": {+ "host": "example.com"+ },+ "output": {+ "elapsed_ms": 0,+ "host": "example.com",+ "status": 200,+ "up": true+ },+ "say": "example.com is up: 200 in 0 ms.",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "probe",+ "summary": "GET https://example.com/ -> 200"+ },+ {+ "chosen": "between [200, 399]",+ "depth": 0,+ "kind": "decide",+ "next": "up",+ "node": "status",+ "observed": 200,+ "summary": "state.resp.status = 200"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "up",+ "summary": "example.com is up: 200 in 0 ms."+ }+ ],+ "tree": "site-status"+ },+ "tree": "site-status"+ },+ "inputs": {+ "host": "example.com"+ },+ "intent": "is Example.com up",+ "tree": "site-status"+ },+ {+ "expect": {+ "inputs": {+ "host": "broken.example"+ },+ "output": {+ "host": "broken.example",+ "status": 503,+ "up": false+ },+ "reason": "server-error",+ "say": "broken.example is answering 503: up, but broken.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "probe",+ "summary": "GET https://broken.example/ -> 503"+ },+ {+ "chosen": "gte 500",+ "depth": 0,+ "kind": "decide",+ "next": "erroring",+ "node": "status",+ "observed": 503,+ "summary": "state.resp.status = 503"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "erroring",+ "summary": "server-error: broken.example is answering 503: up, but broken."+ }+ ],+ "tree": "site-status"+ },+ "tree": "site-status"+ },+ "inputs": {+ "host": "broken.example"+ },+ "intent": "is broken.example up",+ "tree": "site-status"+ },+ {+ "expect": {+ "inputs": {+ "host": "teapot.example"+ },+ "output": {+ "host": "teapot.example",+ "note": "client error on /",+ "status": 418,+ "up": true+ },+ "say": "teapot.example answers but refuses the front page (418); the server is up.",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "probe",+ "summary": "GET https://teapot.example/ -> 418"+ },+ {+ "chosen": "between [400, 499]",+ "depth": 0,+ "kind": "decide",+ "next": "refusing",+ "node": "status",+ "observed": 418,+ "summary": "state.resp.status = 418"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "refusing",+ "summary": "teapot.example answers but refuses the front page (418); the server is up."+ }+ ],+ "tree": "site-status"+ },+ "tree": "site-status"+ },+ "inputs": {+ "host": "teapot.example"+ },+ "intent": "check status of teapot.example",+ "tree": "site-status"+ },+ {+ "expect": {+ "inputs": {},+ "reason": "bad-input",+ "say": "input 'host' must match '(?i)^[a-z0-9.\\\\-]+(:[0-9]+)?$', got 'http://x'",+ "status": "error",+ "trace": {+ "gaps": [],+ "steps": [],+ "tree": "site-status"+ },+ "tree": "site-status"+ },+ "inputs": {+ "host": "http://x"+ },+ "tree": "site-status"+ },+ {+ "expect": {+ "inputs": {+ "repo": "b-i-t-8/pyrtan"+ },+ "output": {+ "name": "First",+ "prerelease": false,+ "published": "2026-10-24T00:00:00Z",+ "repo": "b-i-t-8/pyrtan",+ "tag": "v0.1.0",+ "url": "https://github.com/b-i-t-8/pyrtan/releases/tag/v0.1.0"+ },+ "say": "b-i-t-8/pyrtan: v0.1.0 (2026-10-24T00:00:00Z) https://github.com/b-i-t-8/pyrtan/releases/tag/v0.1.0",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "name": "First",+ "prerelease": false,+ "published": "2026-10-24T00:00:00Z",+ "tag": "v0.1.0",+ "url": "https://github.com/b-i-t-8/pyrtan/releases/tag/v0.1.0"+ },+ "summary": "GET https://api.github.com/repos/b-i-t-8/pyrtan/releases/latest -> 200; tag=v0.1.0, name=First, published=2026-10-24T00:00:00Z, url=https://github.com/b-i-t-8/pyrtan/releases/tag/v0.1.0, prerelease=False"+ },+ {+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "reply",+ "node": "status",+ "observed": 200,+ "summary": "state.resp.status = 200"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "b-i-t-8/pyrtan: v0.1.0 (2026-10-24T00:00:00Z) https://github.com/b-i-t-8/pyrtan/releases/tag/v0.1.0"+ }+ ],+ "tree": "github-release"+ },+ "tree": "github-release"+ },+ "inputs": {+ "repo": "b-i-t-8/pyrtan"+ },+ "intent": "latest release of b-i-t-8/pyrtan",+ "tree": "github-release"+ },+ {+ "expect": {+ "inputs": {+ "repo": "b-i-t-8/nothing"+ },+ "reason": "no-release",+ "say": "b-i-t-8/nothing has no releases, or does not exist.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "name": null,+ "prerelease": null,+ "published": null,+ "tag": null,+ "url": null+ },+ "summary": "GET https://api.github.com/repos/b-i-t-8/nothing/releases/latest -> 404; tag=None, name=None, published=None, url=None, prerelease=None"+ },+ {+ "chosen": "eq 404",+ "depth": 0,+ "kind": "decide",+ "next": "none",+ "node": "status",+ "observed": 404,+ "summary": "state.resp.status = 404"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "none",+ "summary": "no-release: b-i-t-8/nothing has no releases, or does not exist."+ }+ ],+ "tree": "github-release"+ },+ "tree": "github-release"+ },+ "inputs": {+ "repo": "b-i-t-8/nothing"+ },+ "intent": "latest b-i-t-8/nothing release",+ "tree": "github-release"+ },+ {+ "expect": {+ "inputs": {+ "repo": "b-i-t-8/busy"+ },+ "reason": "rate-limited",+ "say": "GitHub is rate-limiting unauthenticated requests (403).",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "name": null,+ "prerelease": null,+ "published": null,+ "tag": null,+ "url": null+ },+ "summary": "GET https://api.github.com/repos/b-i-t-8/busy/releases/latest -> 403; tag=None, name=None, published=None, url=None, prerelease=None"+ },+ {+ "chosen": "in [403, 429]",+ "depth": 0,+ "kind": "decide",+ "next": "rate-limited",+ "node": "status",+ "observed": 403,+ "summary": "state.resp.status = 403"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "rate-limited",+ "summary": "rate-limited: GitHub is rate-limiting unauthenticated requests (403)."+ }+ ],+ "tree": "github-release"+ },+ "tree": "github-release"+ },+ "inputs": {+ "repo": "b-i-t-8/busy"+ },+ "intent": "latest release of b-i-t-8/busy",+ "tree": "github-release"+ },+ {+ "expect": {+ "inputs": {+ "place": "Paris",+ "units": "celsius"+ },+ "reason": "transport",+ "say": "geocode: no recorded response for GET https://geocoding-api.open-meteo.com/v1/search?name=Paris&count=1&language=en&format=json",+ "status": "error",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": null,+ "node": "geocode",+ "summary": "GET https://geocoding-api.open-meteo.com/v1/search?name=Paris&count=1&language=en&format=json failed: no recorded response for GET https://geocoding-api.open-meteo.com/v1/search?name=Paris&count=1&language=en&format=json"+ }+ ],+ "tree": "weather"+ },+ "tree": "weather"+ },+ "inputs": {+ "place": "Paris"+ },+ "intent": "weather in Paris",+ "tree": "weather"+ },+ {+ "expect": {+ "inputs": {+ "id": 101+ },+ "output": {+ "by": "alice",+ "id": 101,+ "score": 42,+ "title": "Pyrtan 0.1",+ "url": "https://example.org/p"+ },+ "say": "Pyrtan 0.1 (42 points, by alice) https://example.org/p",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "kind",+ "node": "fetch",+ "observed": {+ "by": "alice",+ "score": 42,+ "title": "Pyrtan 0.1",+ "type": "story",+ "url": "https://example.org/p"+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/item/101.json -> 200; title=Pyrtan 0.1, url=https://example.org/p, score=42, by=alice, type=story"+ },+ {+ "chosen": "eq 'story'",+ "depth": 0,+ "kind": "decide",+ "next": "has-url",+ "node": "kind",+ "observed": "story",+ "summary": "state.type = story"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "reply",+ "node": "has-url",+ "observed": "https://example.org/p",+ "summary": "state.url = https://example.org/p"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Pyrtan 0.1 (42 points, by alice) https://example.org/p"+ }+ ],+ "tree": "hacker-news-item"+ },+ "tree": "hacker-news-item"+ },+ "inputs": {+ "id": 101+ },+ "tree": "hacker-news-item"+ },+ {+ "expect": {+ "inputs": {+ "count": 2+ },+ "output": {+ "stories": [+ {+ "by": "alice",+ "id": 101,+ "score": 42,+ "title": "Pyrtan 0.1",+ "url": "https://example.org/p"+ },+ {+ "by": "x",+ "id": 102,+ "score": 7,+ "title": "Ask HN: self post",+ "url": "https://news.ycombinator.com/item?id=102"+ }+ ]+ },+ "say": "Top of Hacker News: Pyrtan 0.1 — https://example.org/p",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "observe",+ "next": "check",+ "node": "fetch-ids",+ "observed": {+ "ids": [+ 101,+ 102,+ 103+ ]+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/topstories.json -> 200; ids=[101, 102, 103]"+ },+ {+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "any",+ "node": "check",+ "observed": 200,+ "summary": "state.ids_resp.status = 200"+ },+ {+ "chosen": "type 'list'",+ "depth": 0,+ "kind": "decide",+ "next": "stories",+ "node": "any",+ "observed": [+ 101,+ 102,+ 103+ ],+ "summary": "state.ids = [101, 102, 103]"+ },+ {+ "depth": 1,+ "kind": "observe",+ "next": "kind",+ "node": "fetch",+ "observed": {+ "by": "alice",+ "score": 42,+ "title": "Pyrtan 0.1",+ "type": "story",+ "url": "https://example.org/p"+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/item/101.json -> 200; title=Pyrtan 0.1, url=https://example.org/p, score=42, by=alice, type=story"+ },+ {+ "chosen": "eq 'story'",+ "depth": 1,+ "kind": "decide",+ "next": "has-url",+ "node": "kind",+ "observed": "story",+ "summary": "state.type = story"+ },+ {+ "chosen": "exists True",+ "depth": 1,+ "kind": "decide",+ "next": "reply",+ "node": "has-url",+ "observed": "https://example.org/p",+ "summary": "state.url = https://example.org/p"+ },+ {+ "depth": 1,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Pyrtan 0.1 (42 points, by alice) https://example.org/p"+ },+ {+ "depth": 1,+ "kind": "observe",+ "next": "kind",+ "node": "fetch",+ "observed": {+ "by": "x",+ "score": 7,+ "title": "Ask HN: self post",+ "type": "story",+ "url": null+ },+ "summary": "GET https://hacker-news.firebaseio.com/v0/item/102.json -> 200; title=Ask HN: self post, url=None, score=7, by=x, type=story"+ },+ {+ "chosen": "eq 'story'",+ "depth": 1,+ "kind": "decide",+ "next": "has-url",+ "node": "kind",+ "observed": "story",+ "summary": "state.type = story"+ },+ {+ "chosen": "else",+ "depth": 1,+ "kind": "decide",+ "next": "self-post",+ "node": "has-url",+ "summary": "state.url = <missing>"+ },+ {+ "depth": 1,+ "kind": "set",+ "next": "reply",+ "node": "self-post",+ "observed": {+ "url": "https://news.ycombinator.com/item?id=102"+ },+ "summary": "url=https://news.ycombinator.com/item?id=102"+ },+ {+ "depth": 1,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Ask HN: self post (7 points, by x) https://news.ycombinator.com/item?id=102"+ },+ {+ "depth": 0,+ "kind": "each",+ "next": "reply",+ "node": "stories",+ "summary": "2 item(s)"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Top of Hacker News: Pyrtan 0.1 — https://example.org/p"+ }+ ],+ "tree": "hacker-news"+ },+ "tree": "hacker-news"+ },+ "inputs": {+ "count": 2+ },+ "tree": "hacker-news"+ }+ ],+ "trees": {+ "github-release": {+ "description": "The latest release of a GitHub repository.",+ "hosts": [+ "api.github.com"+ ],+ "inputs": {+ "repo": {+ "description": "owner/name",+ "pattern": "^[A-Za-z0-9_.\\-]+/[A-Za-z0-9_.\\-]+$",+ "type": "str"+ }+ },+ "license": "MIT",+ "name": "github-release",+ "nodes": {+ "fetch": {+ "extract": [+ {+ "into": "tag",+ "json": "json.tag_name"+ },+ {+ "into": "name",+ "json": "json.name"+ },+ {+ "into": "published",+ "json": "json.published_at"+ },+ {+ "into": "url",+ "json": "json.html_url"+ },+ {+ "into": "prerelease",+ "json": "json.prerelease"+ }+ ],+ "http": {+ "headers": {+ "Accept": "application/vnd.github+json"+ },+ "method": "GET",+ "url": "https://api.github.com/repos/{{ inputs.repo }}/releases/latest"+ },+ "into": "resp",+ "kind": "observe",+ "next": "status"+ },+ "none": {+ "kind": "fail",+ "reason": "no-release",+ "say": "{{ inputs.repo }} has no releases, or does not exist."+ },+ "rate-limited": {+ "kind": "fail",+ "reason": "rate-limited",+ "say": "GitHub is rate-limiting unauthenticated requests ({{ state.resp.status }})."+ },+ "reply": {+ "kind": "done",+ "output": {+ "name": "{{ state.name }}",+ "prerelease": "{{ state.prerelease }}",+ "published": "{{ state.published }}",+ "repo": "{{ inputs.repo }}",+ "tag": "{{ state.tag }}",+ "url": "{{ state.url }}"+ },+ "say": "{{ inputs.repo }}: {{ state.tag }} ({{ state.published }}) {{ state.url }}"+ },+ "status": {+ "branches": [+ {+ "then": "reply",+ "when": {+ "eq": 200+ }+ },+ {+ "then": "none",+ "when": {+ "eq": 404+ }+ },+ {+ "then": "rate-limited",+ "when": {+ "in": [+ 403,+ 429+ ]+ }+ },+ {+ "else": "unexpected"+ }+ ],+ "kind": "decide",+ "on": "resp.status"+ },+ "unexpected": {+ "kind": "fail",+ "reason": "unexpected-response",+ "say": "GitHub answered {{ state.resp.status }}."+ }+ },+ "start": "fetch",+ "tags": [+ "github",+ "read-only"+ ],+ "triggers": [+ {+ "pattern": "[what's the] latest release of {repo:word}"+ },+ {+ "pattern": "latest {repo:word} release"+ }+ ],+ "version": 1+ },+ "hacker-news": {+ "description": "Top stories on Hacker News, from its public API.",+ "hosts": [+ "hacker-news.firebaseio.com"+ ],+ "inputs": {+ "count": {+ "default": 1,+ "description": "How many stories to return",+ "max": 30,+ "min": 1,+ "type": "int"+ }+ },+ "license": "MIT",+ "name": "hacker-news",+ "nodes": {+ "any": {+ "branches": [+ {+ "then": "nothing",+ "when": {+ "empty": true+ }+ },+ {+ "then": "stories",+ "when": {+ "type": "list"+ }+ },+ {+ "else": "unexpected"+ }+ ],+ "kind": "decide",+ "on": "ids"+ },+ "check": {+ "branches": [+ {+ "then": "any",+ "when": {+ "eq": 200+ }+ },+ {+ "then": "hn-down",+ "when": {+ "gte": 500+ }+ },+ {+ "else": "unexpected"+ }+ ],+ "kind": "decide",+ "on": "ids_resp.status"+ },+ "fetch-ids": {+ "extract": [+ {+ "into": "ids",+ "json": "json"+ }+ ],+ "http": {+ "method": "GET",+ "url": "https://hacker-news.firebaseio.com/v0/topstories.json"+ },+ "into": "ids_resp",+ "kind": "observe",+ "next": "check"+ },+ "hn-down": {+ "kind": "fail",+ "reason": "upstream-down",+ "say": "Hacker News answered {{ state.ids_resp.status }}; try again later."+ },+ "nothing": {+ "kind": "fail",+ "reason": "no-stories",+ "say": "Hacker News returned an empty front page."+ },+ "reply": {+ "kind": "done",+ "output": {+ "stories": "{{ state.stories }}"+ },+ "say": "Top of Hacker News: {{ state.stories[0].title }} — {{ state.stories[0].url }}"+ },+ "stories": {+ "into": "stories",+ "kind": "each",+ "limit": "{{ inputs.count }}",+ "next": "reply",+ "over": "ids",+ "tree": "hacker-news-item",+ "with": {+ "id": "{{ item }}"+ }+ },+ "unexpected": {+ "kind": "fail",+ "reason": "unexpected-response",+ "say": "Hacker News answered {{ state.ids_resp.status }} with something this tree does not recognise."+ }+ },+ "start": "fetch-ids",+ "tags": [+ "news",+ "read-only"+ ],+ "triggers": [+ {+ "inputs": {+ "count": 1+ },+ "pattern": "[what's the] top story on hacker news"+ },+ {+ "pattern": "top {count:int} stories on hacker news"+ }+ ],+ "version": 1+ },+ "hacker-news-item": {+ "description": "One Hacker News item by id: title, url, score, author.",+ "hosts": [+ "hacker-news.firebaseio.com"+ ],+ "inputs": {+ "id": {+ "description": "The item id",+ "min": 1,+ "type": "int"+ }+ },+ "license": "MIT",+ "name": "hacker-news-item",+ "nodes": {+ "fetch": {+ "extract": [+ {+ "into": "title",+ "json": "json.title"+ },+ {+ "into": "url",+ "json": "json.url"+ },+ {+ "into": "score",+ "json": "json.score"+ },+ {+ "into": "by",+ "json": "json.by"+ },+ {+ "into": "type",+ "json": "json.type"+ }+ ],+ "http": {+ "method": "GET",+ "url": "https://hacker-news.firebaseio.com/v0/item/{{ inputs.id }}.json"+ },+ "into": "resp",+ "kind": "observe",+ "next": "kind"+ },+ "has-url": {+ "branches": [+ {+ "then": "reply",+ "when": {+ "exists": true+ }+ },+ {+ "else": "self-post"+ }+ ],+ "kind": "decide",+ "on": "url"+ },+ "kind": {+ "branches": [+ {+ "then": "has-url",+ "when": {+ "eq": "story"+ }+ },+ {+ "then": "has-url",+ "when": {+ "eq": "job"+ }+ },+ {+ "then": "missing",+ "when": {+ "exists": false+ }+ },+ {+ "else": "not-a-story"+ }+ ],+ "kind": "decide",+ "on": "type"+ },+ "missing": {+ "kind": "fail",+ "reason": "no-such-item",+ "say": "There is no Hacker News item {{ inputs.id }}."+ },+ "not-a-story": {+ "kind": "fail",+ "reason": "not-a-story",+ "say": "Item {{ inputs.id }} is a {{ state.type }}, not a story."+ },+ "reply": {+ "kind": "done",+ "output": {+ "by": "{{ state.by }}",+ "id": "{{ inputs.id }}",+ "score": "{{ state.score }}",+ "title": "{{ state.title }}",+ "url": "{{ state.url }}"+ },+ "say": "{{ state.title }} ({{ state.score }} points, by {{ state.by }}) {{ state.url }}"+ },+ "self-post": {+ "kind": "set",+ "next": "reply",+ "set": {+ "url": "https://news.ycombinator.com/item?id={{ inputs.id }}"+ }+ }+ },+ "start": "fetch",+ "tags": [+ "news",+ "read-only"+ ],+ "triggers": [+ {+ "pattern": "hacker news item {id:int}"+ }+ ],+ "version": 1+ },+ "site-status": {+ "description": "Whether a web site answers, and how quickly.",+ "hosts": [+ "*"+ ],+ "inputs": {+ "host": {+ "description": "A host name, with no scheme",+ "pattern": "(?i)^[a-z0-9.\\-]+(:[0-9]+)?$",+ "type": "str"+ }+ },+ "license": "MIT",+ "name": "site-status",+ "nodes": {+ "erroring": {+ "kind": "fail",+ "output": {+ "host": "{{ inputs.host }}",+ "status": "{{ state.resp.status }}",+ "up": false+ },+ "reason": "server-error",+ "say": "{{ inputs.host }} is answering {{ state.resp.status }}: up, but broken."+ },+ "odd": {+ "kind": "fail",+ "reason": "unexpected-status",+ "say": "{{ inputs.host }} answered {{ state.resp.status }}, which this tree has no branch for."+ },+ "probe": {+ "http": {+ "method": "GET",+ "url": "https://{{ inputs.host | lower }}/"+ },+ "into": "resp",+ "kind": "observe",+ "next": "status",+ "retries": 1,+ "timeout": 8+ },+ "refusing": {+ "kind": "done",+ "output": {+ "host": "{{ inputs.host }}",+ "note": "client error on /",+ "status": "{{ state.resp.status }}",+ "up": true+ },+ "say": "{{ inputs.host }} answers but refuses the front page ({{ state.resp.status }}); the server is up."+ },+ "status": {+ "branches": [+ {+ "then": "up",+ "when": {+ "between": [+ 200,+ 399+ ]+ }+ },+ {+ "then": "refusing",+ "when": {+ "between": [+ 400,+ 499+ ]+ }+ },+ {+ "then": "erroring",+ "when": {+ "gte": 500+ }+ },+ {+ "else": "odd"+ }+ ],+ "kind": "decide",+ "on": "resp.status"+ },+ "up": {+ "kind": "done",+ "output": {+ "elapsed_ms": "{{ state.resp.elapsed_ms }}",+ "host": "{{ inputs.host }}",+ "status": "{{ state.resp.status }}",+ "up": true+ },+ "say": "{{ inputs.host }} is up: {{ state.resp.status }} in {{ state.resp.elapsed_ms }} ms."+ }+ },+ "start": "probe",+ "tags": [+ "ops",+ "read-only"+ ],+ "triggers": [+ {+ "pattern": "is {host:host} up"+ },+ {+ "pattern": "is {host:host} down"+ },+ {+ "pattern": "[check] status of {host:host}"+ }+ ],+ "version": 1+ },+ "weather": {+ "description": "Current weather for a place, from Open-Meteo.",+ "hosts": [+ "geocoding-api.open-meteo.com",+ "api.open-meteo.com"+ ],+ "inputs": {+ "place": {+ "description": "A town or city name",+ "type": "str"+ },+ "units": {+ "default": "celsius",+ "one_of": [+ "celsius",+ "fahrenheit"+ ],+ "type": "str"+ }+ },+ "language": {+ "synonyms": {+ "weather": [+ "forecast",+ "temperature"+ ]+ }+ },+ "license": "MIT",+ "name": "weather",+ "nodes": {+ "clear": {+ "kind": "set",+ "next": "reply",+ "set": {+ "sky": "clear"+ }+ },+ "cloudy": {+ "kind": "set",+ "next": "reply",+ "set": {+ "sky": "cloudy"+ }+ },+ "fog": {+ "kind": "set",+ "next": "reply",+ "set": {+ "sky": "foggy"+ }+ },+ "forecast": {+ "extract": [+ {+ "into": "temp",+ "json": "json.current.temperature_2m"+ },+ {+ "into": "humidity",+ "json": "json.current.relative_humidity_2m"+ },+ {+ "into": "wind",+ "json": "json.current.wind_speed_10m"+ },+ {+ "into": "code",+ "json": "json.current.weather_code"+ },+ {+ "into": "unit",+ "json": "json.current_units.temperature_2m"+ }+ ],+ "http": {+ "method": "GET",+ "url": "https://api.open-meteo.com/v1/forecast?latitude={{ state.lat }}&longitude={{ state.lon }}¤t=temperature_2m,relative_humidity_2m,wind_speed_10m,weather_code&temperature_unit={{ inputs.units }}"+ },+ "into": "fc",+ "kind": "observe",+ "next": "have-temp"+ },+ "found": {+ "branches": [+ {+ "then": "forecast",+ "when": {+ "exists": true+ }+ },+ {+ "else": "unknown-place"+ }+ ],+ "kind": "decide",+ "on": "hit"+ },+ "geocode": {+ "extract": [+ {+ "into": "hit",+ "json": "json.results[0]"+ },+ {+ "into": "lat",+ "json": "json.results[0].latitude"+ },+ {+ "into": "lon",+ "json": "json.results[0].longitude"+ },+ {+ "into": "name",+ "json": "json.results[0].name"+ },+ {+ "into": "country",+ "json": "json.results[0].country"+ }+ ],+ "http": {+ "method": "GET",+ "url": "https://geocoding-api.open-meteo.com/v1/search?name={{ inputs.place | url }}&count=1&language=en&format=json"+ },+ "into": "geo",+ "kind": "observe",+ "next": "found"+ },+ "have-temp": {+ "branches": [+ {+ "then": "sky",+ "when": {+ "type": "number"+ }+ },+ {+ "else": "no-forecast"+ }+ ],+ "kind": "decide",+ "on": "temp"+ },+ "no-forecast": {+ "kind": "fail",+ "reason": "no-forecast",+ "say": "Open-Meteo found {{ state.name }} but returned no current temperature (status {{ state.fc.status }})."+ },+ "rain": {+ "kind": "set",+ "next": "reply",+ "set": {+ "sky": "raining"+ }+ },+ "reply": {+ "kind": "done",+ "output": {+ "country": "{{ state.country }}",+ "humidity": "{{ state.humidity }}",+ "place": "{{ state.name }}",+ "sky": "{{ state.sky }}",+ "temperature": "{{ state.temp }}",+ "unit": "{{ state.unit }}",+ "wind_kmh": "{{ state.wind }}"+ },+ "say": "{{ state.name }}, {{ state.country }}: {{ state.temp }}{{ state.unit }}, {{ state.sky }}, humidity {{ state.humidity }}%, wind {{ state.wind }} km/h."+ },+ "sky": {+ "branches": [+ {+ "then": "clear",+ "when": {+ "eq": 0+ }+ },+ {+ "then": "cloudy",+ "when": {+ "between": [+ 1,+ 3+ ]+ }+ },+ {+ "then": "fog",+ "when": {+ "between": [+ 45,+ 48+ ]+ }+ },+ {+ "then": "rain",+ "when": {+ "between": [+ 51,+ 67+ ]+ }+ },+ {+ "then": "snow",+ "when": {+ "between": [+ 71,+ 77+ ]+ }+ },+ {+ "then": "rain",+ "when": {+ "between": [+ 80,+ 82+ ]+ }+ },+ {+ "then": "snow",+ "when": {+ "between": [+ 85,+ 86+ ]+ }+ },+ {+ "then": "storm",+ "when": {+ "between": [+ 95,+ 99+ ]+ }+ },+ {+ "else": "unknown-sky"+ }+ ],+ "kind": "decide",+ "on": "code"+ },+ "snow": {+ "kind": "set",+ "next": "reply",+ "set": {+ "sky": "snowing"+ }+ },+ "storm": {+ "kind": "set",+ "next": "reply",+ "set": {+ "sky": "stormy"+ }+ },+ "unknown-place": {+ "kind": "fail",+ "reason": "unknown-place",+ "say": "I could not find a place called {{ inputs.place }}."+ },+ "unknown-sky": {+ "kind": "set",+ "next": "reply",+ "set": {+ "sky": "weather code {{ state.code }}"+ }+ }+ },+ "start": "geocode",+ "tags": [+ "weather",+ "read-only"+ ],+ "triggers": [+ {+ "pattern": "[what's the] weather in {place}"+ },+ {+ "pattern": "[what's the] weather for {place}"+ },+ {+ "pattern": "how [hot|cold|warm] is it in {place}"+ }+ ],+ "version": 1+ }+ }+}diff --git a/tests/test_conformance.py b/tests/test_conformance.py new file mode 100644 index 0000000..db27650 --- /dev/null +++ b/tests/test_conformance.py@@ -0,0 +1,33 @@+"""Both runtimes read the same tree and say the same thing.++The vectors are what Python says; node runs pyrtan.js over them. The+vectors file is regenerated here and compared, so it cannot drift from+the examples or the runtime without this test saying so.+"""++import json+import shutil+import subprocess+import sys+from pathlib import Path++import pytest++ROOT = Path(__file__).resolve().parent.parent+sys.path.insert(0, str(ROOT / "tests" / "conformance"))++from make_vectors import build # noqa: E402++VECTORS = ROOT / "tests" / "conformance" / "vectors.json"+++def test_vectors_are_current():+ fresh = json.dumps(build(), indent=1, ensure_ascii=False, sort_keys=True) + "\n"+ assert VECTORS.read_text(encoding="utf-8") == fresh, "tests/conformance/vectors.json is stale: run `uv run python tests/conformance/make_vectors.py`"+++@pytest.mark.skipif(shutil.which("node") is None, reason="node is not installed")+def test_javascript_runtime_agrees_with_python():+ run = subprocess.run(["node", str(ROOT / "tests" / "conformance" / "run.cjs")], capture_output=True, text=True, cwd=str(ROOT))+ assert run.returncode == 0, run.stdout + run.stderr+ assert f"{len(json.loads(VECTORS.read_text())['runs'])}/" in run.stdoutdiff --git a/tests/test_federation.py b/tests/test_federation.py index bcdfa21..b764380 100644 --- a/tests/test_federation.py +++ b/tests/test_federation.py@@ -2,6 +2,7 @@sockets: who may ask whom, what is announced, and what is refused.""" import json+import threadingimport time import pytest@@ -634,3 +635,113 @@ def test_cli_keep_kept_fetch(two, tmp_path, capsys):assert code == 0 and f"from {b.url}" in out and json.loads((tmp_path / "back.json").read_text()) == {"hello": "world"} assert run("fetch", "nothing", "--at", a.url, "--token", "token-a")[0] == 1 assert run("keep", "bad key", str(f), "--at", a.url, "--token", "token-a")[0] == 1+++class TrustedMember(Member):+ """A member that runs trees: it holds what the hub hands it and answers Asks."""++ def __init__(self, hub_url: str, name: str = "phone", answers: dict | None = None):+ super().__init__(hub_url, name)+ self.trees: dict[str, dict] = {}+ self.asks: list[dict] = []+ self.answers = answers or {}++ def follow(self, secret: str | None = None, token: str | None = None):+ activity = {"id": f"{self.id}/activities/follow", "actor": self.id, "published": "now", "type": "Follow", "object": self.hub, "secret": self.secret, "inbox": "mailbox", "name": self.name}+ headers = sign(secret or self.secret, self.id, "POST", "/inbox", activity)+ if token:+ headers["Authorization"] = f"Bearer {token}"+ return http_json("POST", f"{self.hub}/inbox", activity, headers=headers)++ def poll(self) -> list[dict]:+ acts = super().poll()+ for a in acts:+ if a["type"] == "Keep" and a["object"]["key"].startswith("trees/"):+ self.trees[a["object"]["data"]["name"]] = a["object"]["data"]+ elif a["type"] == "Ask":+ self.asks.append(a["object"])+ outcome = self.answers.get(a["object"]["tree"])+ if outcome is not None:+ self.post({"type": "Answer", "object": {"run": a["object"]["run"], "outcome": {**outcome, "output": {"inputs": a["object"]["inputs"]}}}})+ return acts++ def announce(self, listed: bool = True, trees: list | None = None):+ caps = trees if trees is not None else [{"name": n, "triggers": ["whatever the member says"]} for n in self.trees]+ return self.post({"type": "Announce", "object": {"type": "Capabilities", "name": self.name, "trees": caps, "directory": [], "kept": sorted(self.kept)}})+++def test_trusted_member_runs_the_trees_its_hub_hands_out(tmp_path):+ (tmp_path / "t").mkdir()+ (tmp_path / "h").mkdir()+ (tmp_path / "t" / "alpha.toml").write_text(ECHO.format(name="alpha", word="alpha"))+ (tmp_path / "h" / "gamma.toml").write_text(ECHO.format(name="gamma", word="gamma"))+ from pyrtan.hub import serve_hub++ hub, srv = serve_hub(port=0, quiet=True, name="solo", trees=[tmp_path / "t"], handout=[tmp_path / "h"], federate=True, token="tok", poll_every=60)+ fed = hub.federation+ try:+ # Joining with the hub's token makes a member trusted; without it, not.+ stranger = TrustedMember(srv.url, "stranger")+ assert stranger.follow()[1]["trusted"] is False+ assert [a["type"] for a in stranger.poll()] == ["Accept", "Announce"]+ stranger.announce(trees=[{"name": "alpha", "triggers": ["alpha {text}"]}])+ assert fed.followers[stranger.id].trees == [] and "alpha" not in stranger.trees+ phone = TrustedMember(srv.url, "phone", answers={"gamma": {"status": "done", "say": "gamma, from the phone", "trace": {"tree": "gamma", "steps": [{"node": "say", "kind": "done", "depth": 0}], "gaps": []}}})+ status, out = phone.follow(token="tok")+ assert status == 202 and out["trusted"] is True and fed.followers[phone.id].trusted+ # The hub hands out its own trees and the --handout ones at once, in the mailbox after the Accept and the Announce.+ acts = phone.poll()+ assert [a["type"] for a in acts] == ["Accept", "Announce", "Keep", "Keep"]+ assert sorted(phone.trees) == ["alpha", "gamma"] and phone.trees["gamma"]["name"] == "gamma"+ assert all(h["confirmed"] for h in fed._handed[phone.id].values())+ assert fed.hand_out() == 0 # nothing new to give+ # What the member says it runs is filtered to what it was handed, with the hub's own triggers.+ status, out = phone.announce(trees=[{"name": "gamma", "triggers": ["nonsense"]}, {"name": "alpha", "triggers": ["alpha {text}"]}, {"name": "zeta", "triggers": ["zeta"]}])+ assert status == 200+ caps = {t["name"]: t for t in fed.followers[phone.id].trees}+ assert sorted(caps) == ["alpha", "gamma"] and caps["gamma"]["triggers"] == ["gamma {text}"]+ _, out = http_json("GET", f"{srv.url}/capabilities", token="tok")+ assert {(t["name"], t["peer"], t.get("via")) for t in out["trees"]} == {("alpha", "local", None), ("alpha", phone.id, "member"), ("gamma", phone.id, "member")}+ # A sentence for gamma goes to the phone: an Ask in its mailbox, its Answer back to the caller.+ got: dict = {}++ def call():+ got["out"] = http_json("POST", f"{srv.url}/intent", {"text": "gamma hi there"}, token="tok")[1]++ t = threading.Thread(target=call)+ t.start()+ deadline = time.time() + 5+ while not phone.asks and time.time() < deadline:+ phone.poll()+ time.sleep(0.05)+ t.join(5)+ assert phone.asks and phone.asks[0]["tree"] == "gamma" and phone.asks[0]["inputs"] == {"text": "hi there"}+ out = got["out"]+ assert out["status"] == "done" and out["say"] == "gamma, from the phone"+ assert out["peer"] == phone.id and out["via"] == ["member"] and out["output"] == {"inputs": {"text": "hi there"}}+ assert [s["kind"] for s in out["trace"]["steps"]] == ["intent", "done"] and out["trace"]["steps"][0]["next"] == "say"+ # A tree the hub runs itself stays on the hub, even though the phone holds it too.+ _, out = http_json("POST", f"{srv.url}/intent", {"text": "alpha hi"}, token="tok")+ assert out["status"] == "done" and out["peer"] == "local"+ # An answer nobody waits for, or from the wrong member, lands nowhere.+ assert phone.post({"type": "Answer", "object": {"run": "nope", "outcome": {"status": "done"}}})[0] == 404+ fed.ask_timeout = 0.2+ pending = fed.ask(fed.followers[phone.id], "gamma", {"text": "x"}, timeout=0.5) # the phone is not polling+ assert pending["status"] == "error" and pending["reason"] == "member-timeout" and pending["tree"] == "gamma"+ phone.answers = {}+ _, out = http_json("POST", f"{srv.url}/run", {"tree": "gamma", "inputs": {"text": "x"}}, token="tok")+ assert out["reason"] == "member-timeout" and out["peer"] == phone.id+ # The Ask is still in the mailbox; an Answer arriving late is refused, not misfiled.+ acts = phone.poll()+ asks = [a for a in acts if a["type"] == "Ask"]+ assert len(asks) == 2+ assert phone.post({"type": "Answer", "object": {"run": asks[-1]["object"]["run"], "outcome": {"status": "done"}}})[0] == 404+ # Leaving forgets what was handed out; joining again hands it out again.+ assert phone.post({"type": "Undo", "object": {"type": "Follow", "actor": phone.id, "object": srv.url}})[0] == 200+ assert phone.id not in fed._handed+ assert phone.follow(token="tok")[0] == 202 and [a["type"] for a in phone.poll()].count("Keep") == 2+ # Trust survives a restart of the hub's state.+ assert fed.followers[phone.id].to_dict()["trusted"] is True+ finally:+ hub.stop()+ srv.stop()diff --git a/uv.lock b/uv.lock index ee71beb..be65163 100644 --- a/uv.lock +++ b/uv.lock@@ -147,7 +147,7 @@ wheels = [[[package]] name = "pyrtan"-version = "0.5.0"+version = "0.6.0"source = { editable = "." } [package.optional-dependencies]