Merge main (LAN discovery 0.16.0, exchange rates and synonyms, chat links 0.17.0) into agents; agents are release 0.18.0
Claude · 2026-10-28 · 396520b8eee47c9d9a447f2879d49437d6652280 · parents 1fd5f76 4f666fb
Main shipped 0.16.0 and 0.17.0 while agents were built, so agents are 0.18.0, and the docs and messages that said 0.15 say 0.18. The hub takes both chat_reach and suggestions; the console's renderCaps keeps main's placeholder suggestion and leaves choosing a tab to load(), since accepting a suggestion re-renders the trees without moving the page.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rD7jeHL6LXCUuEzUg6jPV
Changes
CONTRIBUTING.md | 8 + README.md | 54 +- _meta/agents.md | 2 +- _meta/chat.md | 28 + _meta/design.md | 4 +- _meta/federation.md | 43 +- _meta/tree-format.md | 3 +- deploy/README.md | 61 +- deploy/openmediavault/docker-compose.yml | 6 +- examples/air-quality/air-quality.toml | 108 + examples/books/books.toml | 66 + examples/dad-joke/dad-joke.toml | 54 + examples/dictionary/define.toml | 66 + examples/earthquakes/earthquakes.toml | 63 + examples/exchange-rate/exchange-rate.toml | 87 + examples/hn-search/hn-search.toml | 77 + examples/holidays/holidays.toml | 65 + examples/iss/iss.toml | 82 + examples/packages/npm-package.toml | 57 + examples/packages/pypi-package.toml | 58 + examples/sun/sun.toml | 80 + examples/synonyms/synonyms.toml | 62 + examples/wikipedia/wikipedia.toml | 79 + pyproject.toml | 2 +- site/README.md | 8 +- site/rendezvous/network.json | 2 +- src/pyrtan/__init__.py | 2 +- src/pyrtan/chat.py | 127 +- src/pyrtan/cli.py | 15 +- src/pyrtan/federation.py | 173 +- src/pyrtan/hub.py | 66 +- src/pyrtan/js/pyrtan.js | 8 +- src/pyrtan/lan.py | 210 ++ src/pyrtan/nlp.py | 12 +- src/pyrtan/page.py | 144 +- src/pyrtan/template.py | 13 +- tests/conformance/make_vectors.py | 33 +- tests/conformance/vectors.json | 5627 +++++++++++++++++++++++++++++++++++++++++++++---- tests/conftest.py | 5 +- tests/fixtures/examples.json | 362 ++++ tests/test_catalog.py | 15 +- tests/test_chat.py | 103 +- tests/test_cli.py | 12 +- tests/test_examples.py | 108 + tests/test_federation.py | 50 + tests/test_lan.py | 143 ++ tests/test_nlp.py | 6 + tests/test_service_hub.py | 5 +- tests/test_template.py | 4 + uv.lock | 2 +- 50 files changed, 7862 insertions(+), 608 deletions(-)
diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 8ece119..a0b9970 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md@@ -24,6 +24,14 @@ does not exist, write it; if one gives a wrong answer, the trace shows where.A tree that reads a page should prefer an API or a feed when the site offers one, and should decide on the status code before it decides on the body.+A tree whose every node is `http` against an API that lets other origins+read it (it answers with `Access-Control-Allow-Origin`) also runs in a+browser: the console at pyrtan.com runs it in the page, with no hub.+`curl -sI -H "Origin: https://pyrtan.com" URL` shows whether it does.+Keep to headers a page may send as they are (`Accept` is fine; a+custom `User-Agent` is not): anything else makes the browser send a+preflight request first, which many APIs do not answer.+A tree meant for the catalog at pyrtan.com carries a `license` line (an SPDX identifier; `MIT`, `Apache-2.0` or `CC0-1.0` are the usual choices) and a `namespace` line (who publishes it; the catalog id is diff --git a/README.md b/README.md index 941ce83..e787ebd 100644 --- a/README.md +++ b/README.md@@ -165,10 +165,10 @@ hosted box is not lent out. The design is in [`_meta/io.md`](_meta/io.md).| `pyrtan run --tree NAME --input k=v` | Runs a named tree. | | `pyrtan match "…" --trees DIR` | How a sentence reads, without running anything: the rewrites, every trigger's verdict, the winner. The tool for writing triggers. | | `pyrtan serve DIR --port 8701` | Hosts the trees as a service: `/health`, `/capabilities`, `/intent`, `/run`, `/resume`, `/gaps`, `/unanswered`, `/suggestions`. Reloads trees when their files change. `--gaps FILE` keeps gaps across restarts; `--egress public` refuses private addresses. |-| `pyrtan hub --port 8700` | Routes between services. Services register with `--hub URL`; peers carry a `link` of `up`, `stale` or `down`. `--trees DIR` holds trees in the hub itself; `--federate` lets other hubs follow it and it follow them; `--listed` puts it in the directory; `--chat` keeps a lobby where people meet strangers and bots (`--no-chat-typing` turns its typing indicator off, `--chat-public` lets callers without the token in, to meet people only). |+| `pyrtan hub --port 8700` | Routes between services. Services register with `--hub URL`; peers carry a `link` of `up`, `stale` or `down`. `--trees DIR` holds trees in the hub itself; `--federate` lets other hubs follow it and it follow them; `--listed` puts it in the directory; `--lan` finds hubs on the local network by multicast, with nobody to ask; `--chat` keeps a lobby where people meet strangers and bots (`--no-chat-typing` turns its typing indicator off, `--chat-public` lets callers without the token in, to meet people only; `--chat-reach N` links an open lobby with up to N others, for chat only). || `pyrtan keep KEY [FILE] --at HUB` | Gives your hub a blob to keep; its keepers get copies. `kept` shows who holds what; `fetch KEY` gets a blob back from your hub or whoever holds it. | | `pyrtan follow [URL] --at HUB` | Makes your hub follow another: it then routes sentences to that hub's trees too. With no URL, `pyrtan join`. `--network NAME` follows within a network of yours; `--listed` also puts your hub in the directory. `unfollow` undoes it; `peers` lists services, hubs followed and followers; `directory` lists every hub that asked to be found, as the swarm knows it. |-| `pyrtan join [INVITE] --at HUB` | Puts your hub in a network and follows a hub in it: the default network pyrtan.com publishes, or the one an invite (`NAME:SECRET@HUB`) names. |+| `pyrtan join [INVITE] --at HUB` | Puts your hub in a network and follows a hub in it: the default network (built in), or the one an invite (`NAME:SECRET@HUB`) names. || `pyrtan network new NAME --at HUB` | Starts a network of your own, a network of one, and prints its invite. `network list` shows each network with its invite; `network leave NAME` cuts the links made in it. | | `pyrtan ask "…" --at URL` | Answers a sentence through a running service or hub. | | `pyrtan chat --at HUB` | Talks to a random stranger or a bot through a hub started `--chat`. `--bot` for a bot, `--people` for people only, `--interests` to meet someone who shares one, `--trace` to see the path behind each bot line, `--no-typing` to hide "Stranger is typing…". |@@ -205,7 +205,11 @@ its `link`; and the gaps. The same console lives atjoin one, the browser is the hub. It holds the catalog's trees, reads the sentence and runs the tree in the page (`pyrtan.js`, the same runtime and the same reading as Python's), and keeps gaps and unanswered sentences-while the page is open. Trees that need a box (files, programs, serial+while the page is open. Nineteen of the examples run there: weather,+sunrise and sunset, air quality, Wikipedia, a dictionary, synonyms,+exchange rates, Hacker News (top stories, an item, a search), books,+npm and PyPI versions, GitHub releases, earthquakes, where the ISS is,+public holidays, a dad joke, and small talk. Trees that need a box (files, programs, seriallines, brokers, a browser to drive) or a site that does not let a page read it say so; for those, join a hub started with `--cors https://pyrtan.com` by giving its URL and token. Forget it and the@@ -275,15 +279,25 @@ keeper. Leaving a network (`pyrtan network leave`) cuts the links madein it and forgets what was heard there. A follow made outside any network, and a member's, works as it always has.-The **default network** is the one pyrtan.com publishes in-`rendezvous/network.json`: `pyrtan`, with a secret that is public on-purpose. It names the network rather than guarding it: anyone may join,-and what keeps a stranger's hub harmless is that following is still the-one act of trust. `pyrtan join` (or `pyrtan hub --join default`, or-`$PYRTAN_JOIN=default`) puts a hub in it and follows the first hub in-`rendezvous/hubs.json` that answers, then the first from the snapshot;-when none does, the hub is the network so far, and others join through-it. pyrtan.com runs no hub: it is static files on GitHub Pages. Every+The **default network** is built into pyrtan: `pyrtan`, with a secret+that is public on purpose. It names the network rather than guarding+it: anyone may join, and what keeps a stranger's hub harmless is that+following is still the one act of trust. `pyrtan join` (or `pyrtan hub+--join default`, or `$PYRTAN_JOIN=default`) puts a hub in it and follows+the first hub that accepts, trying what the hub already knows before+asking anyone: the hubs it follows in the network, then the ones its+directory has heard in it, then the hub list, then the snapshot, each+fetched only if the ones before it did not do. So a hub that has been in+the network once never needs pyrtan.com again; when nobody accepts, the+hub is the network so far, and others join through it. `$PYRTAN_HUB_LIST`+takes any number of sources (a hub list, any hub's `/directory`, a+snapshot or a mirror of one), so pyrtan.com is one starting point among+any others. On one LAN not even that is needed: hubs started `--lan`+(with `--advertise http://<LAN address>:PORT`) announce themselves over+UDP multicast that never passes a router, and `pyrtan join` tries the+hubs heard there right after the ones it follows, so two boxes in one+house find each other with the internet unplugged. pyrtan.com runs no hub: it is static files on GitHub Pages,+and publishes the default network's name and secret only as a mirror. Everyhour its build asks every hub it knows of for its directory (the hub list, then everyone the last snapshot or a hub names), keeps the default network's part, and publishes it as `rendezvous/directory.json`, so the@@ -515,7 +529,13 @@ seconds without news.its hub follows with a signed `Meet`; the lines travel as `Say` and the end as `Part`, typing as `Typing`, hub to hub, signed with the pair's secret; the outbox logs a `Say` without its text and leaves `Typing` out. A hub answers a `Meet` only from a-hub that follows it, and never passes one on.+hub that follows it, and never passes one on. Following is trust (it+routes sentences to the other hub's trees), so to reach more strangers+without it, start an open lobby with `--chat-reach N`: the hub keeps up to+N *chat links* with other open lobbies it hears of (on the LAN with+`--lan`, or in its directory), for chat and nothing else; nothing is+routed, announced or kept over a link, and a hub takes one only when its+operator also gave `--chat-reach`.The lobby follows the hub's token: with one, only callers who have it chat, or a key the hub grants `chat` (a bot runs trees, so a lobby must@@ -700,18 +720,18 @@ site/ pyrtan.com: the landing page, the hub list, the catalog## Status-0.15.0. The core, the service, the hub, the command, the console, the+0.18.0. The core, the service, the hub, the command, the console, thecatalog (namespaces, publishing by pull request through the gate, gap reports gossiped back to authors), the language rules, federation-between hubs, networks (any hub starts one; pyrtan.com publishes the-default), the gossiped directory, members (a browser or a phone as+between hubs, networks (any hub starts one; the default one is built+in; hubs on one LAN find each other by multicast), the gossiped directory, members (a browser or a phone aspart of the swarm), the blobs they keep and the trees they run (`pyrtan.js`, conformant with the Python runtime), six forms of I/O (HTTP, a browser, files, programs, serial lines, MQTT queues), each behind an allow-list, watches (a tree that runs when a clock, a file, a topic or a serial line speaks), `pyrtan grow` (the branch a gap asks for, drafted by rules), and the chat lobby (strangers on the hub and the-hubs it follows, bots made of trees, a typing indicator, and lobbies open to the public) are built and tested; a swarm can require a+hubs it follows, bots made of trees, a typing indicator, lobbies open to the public, and chat links between open lobbies) are built and tested; a swarm can require atoken, keep its gaps and unanswered sentences, refuse private addresses and its own box, and follow other swarms. Identities (keys, device notes, name claims, names at pyrtan.com and pins, grants on hubs, the console's diff --git a/_meta/agents.md b/_meta/agents.md index 799f1f3..f24838b 100644 --- a/_meta/agents.md +++ b/_meta/agents.md@@ -2,7 +2,7 @@| | | |---|---|-| **Status** | **Built, v0.15.** Agents in the console (`agents.js`): WebLLM in the browser or a server of the person's own, probing, meanings for gaps, triggers for missed sentences, trees for unanswered ones, chat, memory and skills. `/suggestions` on services and hubs, `pyrtan suggestions`, `pyrtan grow --suggestions`. Tested with a scripted model in node and in Chromium. |+| **Status** | **Built, v0.18.** Agents in the console (`agents.js`): WebLLM in the browser or a server of the person's own, probing, meanings for gaps, triggers for missed sentences, trees for unanswered ones, chat, memory and skills. `/suggestions` on services and hubs, `pyrtan suggestions`, `pyrtan grow --suggestions`. Tested with a scripted model in node and in Chromium. || **Date** | 2026-10-28 | | **Decision** | A model may sit **beside** the swarm, never inside it. A person starts it in the console; it is a client of the public API like any caller; everything it proposes is data, checked by rules where it is made and again where it is written, and only a person accepts it. The trees stay deterministic; the model helps them grow. | | **Where** | `src/pyrtan/js/agents.js` (the agent), `src/pyrtan/js/pyrtan.js` (`grow`: the rule drafts and the checks), `src/pyrtan/suggestions.py` (the checks, the store, writing a suggestion into a file), `src/pyrtan/page.py` (the Agents tab), `tests/agents.cjs`, `tests/test_agents.py`, `tests/browser/console_agents.cjs`. | diff --git a/_meta/chat.md b/_meta/chat.md index a1ec497..3e0f709 100644 --- a/_meta/chat.md +++ b/_meta/chat.md@@ -124,6 +124,32 @@ While a seat is asking other hubs it is *busy*, and nobody can pair itmeanwhile; two seats that ask each other's hubs at once both get `met: false` and try again on their next round.+**Chat links (v0.17).** A hub follows few hubs, because following routes+sentences to their trees; so the strangers a lobby could reach were+few. A *chat link* is the relationship chat needs and nothing more. A+hub started `--chat-reach N` (`$PYRTAN_CHAT_REACH`) keeps up to N of+them: on every poll it tries a hub or two it heard of on the LAN+(`--lan`) or in its directory (public networks only), that it neither+follows nor is followed by, with a `ChatLink` carrying a fresh pair+secret and signed with it, as an open `Follow` is. The other hub checks+the signature and that the inbox is the one the sender's `/actor`+names, and takes the link only when its operator also gave+`--chat-reach` and its lobby is open to strangers anyway (`--chat-public`,+or no token); a closed federation (`--closed`, a federation secret)+takes none, and a hub takes at most twice its own reach. Then either+end sends `Meet`, `Say`, `Typing` and `Part` over it, exactly as over a+follow, and the two lobbies share their strangers.++A link is kept apart from following on purpose: it lives in the lobby,+not in `following` or `followers`, so nothing is routed over it, no+trees or directory are announced on it, and no blob is kept; the inbox+looks a link up only for the four chat activities. It lives in memory+and is made again after a restart. A hub that refused is not asked again+for a day (`GET /peers` shows the links and each refusal's reason), and+a link whose hub fails three `Meet`s in a row is dropped and replaced.+With `--lan`, two open lobbies in one house link with each other and+nothing else needed.+## 4. Typing (v0.12) While a person types, the client says so (`POST /chat/typing {seat,@@ -206,5 +232,7 @@ for that conversation.- **v0.12, built:** the typing indicator, per person and per hub (§4). - **v0.13, built:** public lobbies: `--chat-public`, guests meet people only (§1).+- **v0.17, built:** chat links, `--chat-reach N`: open lobbies share+ their strangers without following each other (§3).- **Later:** a member's lobby that works offline between two phones of the same hub. diff --git a/_meta/design.md b/_meta/design.md index be9fa80..e312516 100644 --- a/_meta/design.md +++ b/_meta/design.md@@ -62,7 +62,7 @@ Four things a reader might expect, and will not find.path, and that the path can be read. A model anywhere in the path makes the trace a story rather than a record. A model may later sit *beside* the swarm, reading gaps and proposing branches for a person to accept;- that is an author's tool, not a runtime. Since v0.15 it does: an agent+ that is an author's tool, not a runtime. Since v0.18 it does: an agentin the console, checked by rules, accepted by a person ([`agents.md`](agents.md)). - **Code in trees.** Templates are paths and filters. Conditions are@@ -325,7 +325,7 @@ Five trees, all against real public APIs, all tested offline:(`grow.py`: a missing value, the hole between ranges, else the value), `--apply` writes it into the file, and a person says what it means. Unanswered sentences persist with `--unanswered FILE`.-10. **v0.15, built: agents beside the swarm.** Open-weights models a+10. **v0.18, built: agents beside the swarm.** Open-weights models aperson starts in the console (WebLLM in the browser, or a server of their own) probe the trees with sentences of their own, say what a gap's value means, find the patterns trees missed and the trees diff --git a/_meta/federation.md b/_meta/federation.md index 8f5aef4..4cad130 100644 --- a/_meta/federation.md +++ b/_meta/federation.md@@ -215,11 +215,16 @@ the visitor's browser.request, the role of a torrent's bootstrap nodes, and it is the only way pyrtan.com learns that a hub exists. `pyrtan join` (and `pyrtan follow` with no URL) puts a hub in the default network (§7a) and follows the-first listed hub whose `/actor` answers; `pyrtan hub --join default`-does the same on start. A listed hub is an ordinary federated hub in-the default network, up when its operator's box is; nothing needs it to-be up always (below). `$PYRTAN_HUB_LIST` points at another list (a URL-or a file), `$PYRTAN_RENDEZVOUS` at one hub. (Until v0.13 this was+first hub that accepts, in this order, each fetched only if the ones+before it did not do: the hubs it already follows in the network, the+hubs its own directory heard in it (most recent first), the hub list,+the snapshot. `pyrtan hub --join default` does the same on start, so a+restart asks nobody. A listed hub is an ordinary federated hub in the+default network, up when its operator's box is; nothing needs it to be+up always (below). `$PYRTAN_HUB_LIST` names any number of sources,+separated by spaces or commas, each a URL or a file: a hub list, any+hub's `/directory`, a snapshot, or a mirror of one; `$PYRTAN_RENDEZVOUS`+names one hub. pyrtan.com is one starting point, not the only one. (Until v0.13 this was`seeds.json`, and pyrtan.com briefly ran a hub of its own on Fly.io; both are gone.)@@ -240,6 +245,26 @@ after the listed ones (`$PYRTAN_SNAPSHOT` overrides). A new hub cantherefore bootstrap from any hub the snapshot knows of that is up, and the only way the directory is empty is if nobody has been up for a week. Nothing of ours polls: GitHub does.+**On the LAN (v0.16).** Two boxes in one house should not need the+internet to find each other. A hub started `--lan` says it is here on a+UDP multicast group (`239.255.70.84:8770`, administratively scoped, TTL+1, so it never passes a router): its id, its name and the public+networks it is in. It says so on start, with a question every other hub+on the LAN answers at once, and every thirty seconds after; what it+hears it keeps for three minutes (`GET /lan`, operator). `pyrtan join`+then tries the hubs heard on the LAN in that network right after the+ones it already follows, before the directory, the hub list and the+snapshot (`known_hubs`), so a hub started `--lan --join default` on a+LAN with another such hub is in the network with it, with nothing on+the internet asked. A beacon is a hint like a directory entry: anyone on+the LAN can send one, nothing is routed because of it, and the follow it+leads to is checked and signed like any other. It carries no secret and+names only public networks. A hub announces only an id others can reach+(`--advertise http://<LAN address>:PORT`; never `127.0.0.1` or+`0.0.0.0`); without one it still asks and listens, and can join a hub+it hears. In Docker, multicast needs `network_mode: host`. Standard+library only: `src/pyrtan/lan.py`.+**What an entry is not.** A hub two follows away cannot verify who made an entry: there are no public keys in the standard library, and HMAC proves things only between two hubs that share a secret. So an entry is@@ -286,8 +311,12 @@ naming its networks. The `directory` blob a hub's keepers hold is thepublic part only. `pyrtan network leave` unfollows the hubs followed in the network, drops those that followed in it and forgets its entries.-**The default network.** pyrtan.com publishes `rendezvous/network.json`:-the name `pyrtan` and a secret that is public on purpose. A secret on a+**The default network** is built in (v0.15): the name `pyrtan` and a+secret that is public on purpose, so joining it reads nothing from+anyone. pyrtan.com publishes the same two in `rendezvous/network.json`+as a mirror, which a test holds equal to the built-in pair;+`$PYRTAN_NETWORK` names another (a URL or a file of that shape), which+is how the tests keep away from it. A secret on apublic page guards nothing, and is not meant to: it names the network, so the public directory is one network among others and a private one never mixes into it. What makes a stranger's hub harmless is what always diff --git a/_meta/tree-format.md b/_meta/tree-format.md index 35b714a..fbf9aa3 100644 --- a/_meta/tree-format.md +++ b/_meta/tree-format.md@@ -327,7 +327,8 @@ service was started with, for tokens). Paths take `.key`, `[0]`, `[-1]`,`["key"]`. Filters: `url`, `json`, `int`, `float`, `str`, `lower`, `upper`, `strip`,-`title`, `first`, `last`, `length`.+`title`, `first`, `last`, `length`, `join` (a list as `a, b, c`, for a+sentence).A string that is exactly one placeholder keeps the value's type (a list stays a list). A placeholder that resolves to nothing fails the node, except diff --git a/deploy/README.md b/deploy/README.md index 73b5956..5f17da0 100644 --- a/deploy/README.md +++ b/deploy/README.md@@ -70,10 +70,16 @@ file's environment `PYRTAN_ADVERTISE` (that address), `PYRTAN_JOIN:default` and `PYRTAN_LISTED: "true"`, with `PYRTAN_TOKEN` set and `PYRTAN_EGRESS: public` (hubs that follow yours may ask it to run trees, and `any` would let them point a tree at your LAN). The hub then joins-the default network pyrtan.com publishes, follows the first hub in it-that answers (from pyrtan.com's hub list, then its snapshot),+the default network (built in), follows the first hub in it that+accepts (one it already knows, else from pyrtan.com's hub list, then+its snapshot),routes to it, and is listed in the directory every hub gossips; `pyrtan-follow URL --at <your hub> --token …` follows any other. For a network+follow URL --at <your hub> --token …` follows any other. With other hubs+in the same house (a second NAS, a laptop running `pyrtan hub`), set+`PYRTAN_LAN: "true"` too, with `network_mode: host` on the hub service+and `PYRTAN_ADVERTISE` at the box's LAN address: the hubs find each+other by multicast, and join through each other before asking+pyrtan.com anything. For a networkof your own, `pyrtan network new home --at <your hub> --token …` prints an invite; a friend's hub joins with `pyrtan join <invite>`, or by adding it to its `PYRTAN_JOIN`. Following is one-directional trust: your@@ -95,6 +101,55 @@ NAS: mount the paths and pass the devices through in the compose filedrive a browser need Playwright, which the image does not carry; run those from a checkout with `uv sync --extra browser`.+### A family intranet: a private network++For a hub that serves a family (or a team) and nobody else, keep it off+everything public: no `PYRTAN_LISTED`, no `PYRTAN_JOIN: default`, nothing+added to pyrtan.com's `hubs.json`, no `PYRTAN_CHAT_PUBLIC` and no+`PYRTAN_CHAT_REACH`. Make a private network instead, and share its+invite only with the family. In the hub's environment:++```yaml+PYRTAN_TOKEN: "<long random string>" # openssl rand -hex 24; the family's way in+PYRTAN_EGRESS: public # hubs that follow this one may ask it to run trees+PYRTAN_ADVERTISE: "http://<nas-ip>:8700" # an address every family hub can reach (below)+PYRTAN_CHAT: "true" # the lobby, behind the token: family only+```++Then, once, from any machine that reaches the hub:++```+pyrtan network new family --at http://<nas-ip>:8700 --token <token>+```++It prints an invite, `family:SECRET@http://<nas-ip>:8700`. Send it the+way you would send a house key. Another family hub (a second NAS, a+laptop running `pyrtan hub`) joins with `pyrtan join <invite> --at <that+hub> --token <its token>`, or with the invite in its `PYRTAN_JOIN`. A+private network's hubs never reach the public directory or the snapshot,+and its secret signs every follow made in it. Following is one-way:+the hub that joined follows the NAS, so its people can meet the NAS's+in the lobby, and for the NAS's people to meet theirs, the NAS follows+it back: `pyrtan follow http://<that hub>:8700 --network family --at+http://<nas-ip>:8700 --token <token>`.++Phones and browsers do not need a hub of their own. Open the NAS's+console at `http://<nas-ip>:8700/`, enter the token, and chat in the+Chat tab; join in the Member tab to keep the hub's blobs and run its+trees.++`PYRTAN_ADVERTISE` is the address the other hubs reach this one at. If+every family hub is in one house, that is the NAS's LAN address. If+they are in different houses, keep it off the public internet and put+every hub and phone on one private overlay network instead, such as+[Tailscale](https://tailscale.com) (`tailscale up` on each machine, not+`tailscale funnel`, which is public), and advertise the NAS's address+there (`http://<nas-name>.<tailnet>.ts.net:8700`). An internal hostname+under a domain you own works too, if every family network resolves it.+`PYRTAN_LAN` adds nothing here: hubs on the LAN only announce public+networks, and the invite already carries the NAS's address, so leave+it off and the hub needs no `network_mode: host`.+### Without the plugin Over SSH, from the checkout: diff --git a/deploy/openmediavault/docker-compose.yml b/deploy/openmediavault/docker-compose.yml index 6e502cc..958624c 100644 --- a/deploy/openmediavault/docker-compose.yml +++ b/deploy/openmediavault/docker-compose.yml@@ -35,9 +35,12 @@ x-pyrtan: &pyrtanPYRTAN_EGRESS: any # Federation: once the hub is reachable from outside (a tunnel or a # port-forward with TLS), give it its public URL and it joins the- # default network pyrtan.com publishes, follows a hub in it, and lists+ # default network (built into pyrtan), follows a hub in it, and lists# itself in the directory. Needs the token, and PYRTAN_EGRESS: public # above: hubs that follow this one may ask it to run trees.+ # PYRTAN_LAN: "true" finds the other hubs in the house by multicast,+ # with nobody to ask; it needs `network_mode: host` on the hub and+ # PYRTAN_ADVERTISE set to the box's LAN address (http://<nas-ip>:8700).# PYRTAN_JOIN may also hold invites to networks of your own # (NAME:SECRET@HUB, from `pyrtan network new`), space-separated. # PYRTAN_ADVERTISE: "https://hub.example.net"@@ -47,6 +50,7 @@ x-pyrtan: &pyrtan# PYRTAN_WATCH: "true" # run the trees' [[watches]]: a clock, a file, a topic or a serial line starts a run # PYRTAN_CHAT: "true" # a lobby at /chat: people meet a random stranger here or on a hub this one follows, or a bot made of its trees # PYRTAN_CHAT_PUBLIC: "true" # open the lobby to people without the token (they meet people, never a bot); everything else keeps the token+ # PYRTAN_CHAT_REACH: "8" # an open lobby links with up to 8 other open lobbies it hears of (LAN, directory), for chat only, never a follow# PYRTAN_CHAT_TYPING: "0" # turn the lobby's "Stranger is typing…" off for everyone (each person can also turn it off for themselves) PYRTAN_UNANSWERED: /state/unanswered.jsonl # sentences nobody answered, kept across restarts (the hub's; services with a hub leave the record to it) PYRTAN_STATE: /state/federation.json diff --git a/examples/air-quality/air-quality.toml b/examples/air-quality/air-quality.toml new file mode 100644 index 0000000..571f3f3 --- /dev/null +++ b/examples/air-quality/air-quality.toml@@ -0,0 +1,108 @@+# The air where a place is, now: the European Air Quality Index and the+# fine particles behind it, from Open-Meteo. The index's own bands name it.++name = "air-quality"+version = 1+namespace = "pyrtan"+description = "Current air quality for a place: the European AQI, its band, and PM2.5 and PM10, from Open-Meteo."+tags = ["weather", "health", "read-only"]+license = "MIT"+hosts = ["geocoding-api.open-meteo.com", "air-quality-api.open-meteo.com"]+start = "geocode"++[[triggers]]+pattern = "[what's the] air quality (in|for) {place}"+[[triggers]]+pattern = "how is the air in {place}"+[[triggers]]+pattern = "(aqi|pollution) in {place}"++[inputs.place]+type = "str"+description = "A town or city name"++[nodes.geocode]+kind = "observe"+http = { method = "GET", url = "https://geocoding-api.open-meteo.com/v1/search?name={{ inputs.place | url }}&count=1&language=en&format=json" }+into = "geo"+extract = [+ { into = "hit", json = "json.results[0]" },+ { into = "lat", json = "json.results[0].latitude" },+ { into = "lon", json = "json.results[0].longitude" },+ { into = "name", json = "json.results[0].name" },+ { into = "country", json = "json.results[0].country" },+]+next = "found"++[nodes.found]+kind = "decide"+on = "hit"+branches = [+ { when = { exists = true }, then = "air" },+ { else = "unknown-place" },+]++[nodes.air]+kind = "observe"+http = { method = "GET", url = "https://air-quality-api.open-meteo.com/v1/air-quality?latitude={{ state.lat }}&longitude={{ state.lon }}¤t=european_aqi,pm2_5,pm10" }+into = "aq"+extract = [+ { into = "aqi", json = "json.current.european_aqi" },+ { into = "pm25", json = "json.current.pm2_5" },+ { into = "pm10", json = "json.current.pm10" },+]+next = "band"++# The bands of the European Air Quality Index.+[nodes.band]+kind = "decide"+on = "aqi"+branches = [+ { when = { type = "number", lte = 20 }, then = "good" },+ { when = { lte = 40 }, then = "fair" },+ { when = { lte = 60 }, then = "moderate" },+ { when = { lte = 80 }, then = "poor" },+ { when = { lte = 100 }, then = "very-poor" },+ { when = { gt = 100 }, then = "extremely-poor" },+ { else = "no-reading" },+]++[nodes.good]+kind = "set"+set = { band = "good" }+next = "reply"+[nodes.fair]+kind = "set"+set = { band = "fair" }+next = "reply"+[nodes.moderate]+kind = "set"+set = { band = "moderate" }+next = "reply"+[nodes.poor]+kind = "set"+set = { band = "poor" }+next = "reply"+[nodes.very-poor]+kind = "set"+set = { band = "very poor" }+next = "reply"+[nodes.extremely-poor]+kind = "set"+set = { band = "extremely poor" }+next = "reply"++[nodes.reply]+kind = "done"+say = "{{ state.name }}, {{ state.country }}: air quality {{ state.band }} (European AQI {{ state.aqi }}), PM2.5 {{ state.pm25 }} µg/m³, PM10 {{ state.pm10 }} µg/m³."+output = { place = "{{ state.name }}", country = "{{ state.country }}", aqi = "{{ state.aqi }}", band = "{{ state.band }}", pm2_5 = "{{ state.pm25 }}", pm10 = "{{ state.pm10 }}" }++[nodes.unknown-place]+kind = "fail"+reason = "unknown-place"+say = "I could not find a place called {{ inputs.place }}."++[nodes.no-reading]+kind = "fail"+reason = "no-reading"+say = "Open-Meteo found {{ state.name }} but gave no air quality reading (status {{ state.aq.status }})."diff --git a/examples/books/books.toml b/examples/books/books.toml new file mode 100644 index 0000000..93a1f6f --- /dev/null +++ b/examples/books/books.toml@@ -0,0 +1,66 @@+# A book by its title or its author, from Open Library's search: the best+# match, who wrote it and when it first came out.++name = "books"+version = 1+namespace = "pyrtan"+description = "A book from Open Library: the best match for a title or an author, who wrote it, and when it first came out."+tags = ["reference", "books", "read-only"]+license = "MIT"+hosts = ["openlibrary.org"]+start = "search"++[[triggers]]+pattern = "who wrote {title}"+[[triggers]]+pattern = "find the book {title}"+[[triggers]]+pattern = "books by {title}"++[inputs.title]+type = "str"+description = "A title or an author, as in: The Left Hand of Darkness"++[nodes.search]+kind = "observe"+http = { method = "GET", url = "https://openlibrary.org/search.json?q={{ inputs.title | url }}&limit=1&fields=title,author_name,first_publish_year,key" }+into = "resp"+extract = [+ { into = "found", json = "json.numFound" },+ { into = "book", json = "json.docs[0].title" },+ { into = "author", json = "json.docs[0].author_name[0]" },+ { into = "year", json = "json.docs[0].first_publish_year" },+ { into = "key", json = "json.docs[0].key" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "any" },+ { else = "unexpected" },+]++[nodes.any]+kind = "decide"+on = "book"+branches = [+ { when = { exists = true }, then = "reply" },+ { else = "no-book" },+]++[nodes.reply]+kind = "done"+say = "{{ state.book }}, by {{ state.author }} ({{ state.year }}). https://openlibrary.org{{ state.key }}"+output = { title = "{{ state.book }}", author = "{{ state.author }}", first_published = "{{ state.year }}", url = "https://openlibrary.org{{ state.key }}", matches = "{{ state.found }}" }++[nodes.no-book]+kind = "fail"+reason = "no-book"+say = "Open Library has nothing matching {{ inputs.title }}."++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "Open Library answered {{ state.resp.status }}."diff --git a/examples/dad-joke/dad-joke.toml b/examples/dad-joke/dad-joke.toml new file mode 100644 index 0000000..64c6910 --- /dev/null +++ b/examples/dad-joke/dad-joke.toml@@ -0,0 +1,54 @@+# A dad joke, from icanhazdadjoke.com. It answers JSON only when asked for+# JSON alone, so this tree says so in its Accept header.++name = "dad-joke"+version = 1+namespace = "pyrtan"+description = "A random dad joke, from icanhazdadjoke.com."+tags = ["fun", "read-only"]+license = "MIT"+hosts = ["icanhazdadjoke.com"]+start = "fetch"++[[triggers]]+pattern = "tell me a [dad] joke"+[[triggers]]+pattern = "[a] dad joke"+[[triggers]]+pattern = "make me laugh"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://icanhazdadjoke.com/", headers = { Accept = "application/json" } }+into = "resp"+extract = [+ { into = "joke", json = "json.joke" },+ { into = "id", json = "json.id" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "has-joke" },+ { else = "unexpected" },+]++[nodes.has-joke]+kind = "decide"+on = "joke"+branches = [+ { when = { exists = true }, then = "reply" },+ { else = "unexpected" },+]++[nodes.reply]+kind = "done"+say = "{{ state.joke }}"+output = { joke = "{{ state.joke }}", url = "https://icanhazdadjoke.com/j/{{ state.id }}" }++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "icanhazdadjoke.com answered {{ state.resp.status }}."diff --git a/examples/dictionary/define.toml b/examples/dictionary/define.toml new file mode 100644 index 0000000..2ea6f99 --- /dev/null +++ b/examples/dictionary/define.toml@@ -0,0 +1,66 @@+# The first definition of an English word, from the Free Dictionary API.++name = "define"+version = 1+namespace = "pyrtan"+description = "What an English word means: its first definition, part of speech and pronunciation."+tags = ["reference", "language", "read-only"]+license = "MIT"+hosts = ["api.dictionaryapi.dev"]+start = "fetch"++[[triggers]]+pattern = "define {word:word}"+[[triggers]]+pattern = "what does {word:word} mean"+[[triggers]]+pattern = "[the] (meaning|definition) of {word:word}"++[inputs.word]+type = "str"+pattern = "^[A-Za-z][A-Za-z'\\-]*$"+description = "One English word"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://api.dictionaryapi.dev/api/v2/entries/en/{{ inputs.word | lower | url }}" }+into = "resp"+extract = [+ { into = "word", json = "json[0].word" },+ { into = "phonetic", json = "json[0].phonetic" },+ { into = "part", json = "json[0].meanings[0].partOfSpeech" },+ { into = "definition", json = "json[0].meanings[0].definitions[0].definition" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "has-definition" },+ { when = { eq = 404 }, then = "unknown-word" },+ { else = "unexpected" },+]++[nodes.has-definition]+kind = "decide"+on = "definition"+branches = [+ { when = { exists = true }, then = "reply" },+ { else = "unknown-word" },+]++[nodes.reply]+kind = "done"+say = "{{ state.word }} ({{ state.part }}): {{ state.definition }}"+output = { word = "{{ state.word }}", phonetic = "{{ state.phonetic }}", part_of_speech = "{{ state.part }}", definition = "{{ state.definition }}" }++[nodes.unknown-word]+kind = "fail"+reason = "unknown-word"+say = "The dictionary has no entry for {{ inputs.word }}."++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "The dictionary answered {{ state.resp.status }}."diff --git a/examples/earthquakes/earthquakes.toml b/examples/earthquakes/earthquakes.toml new file mode 100644 index 0000000..ce4a1ec --- /dev/null +++ b/examples/earthquakes/earthquakes.toml@@ -0,0 +1,63 @@+# Earthquakes of magnitude 4.5 and up in the past day, from the USGS feed:+# how many, and the most recent one. The feed lists the newest first.++name = "earthquakes"+version = 1+namespace = "pyrtan"+description = "Magnitude 4.5+ earthquakes worldwide in the past day, from the USGS: how many, and the latest."+tags = ["science", "read-only"]+license = "MIT"+hosts = ["earthquake.usgs.gov"]+start = "fetch"++[[triggers]]+pattern = "[any] [recent] earthquakes [today]"+[[triggers]]+pattern = "[was there an] earthquake today"+[[triggers]]+pattern = "latest earthquake"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://earthquake.usgs.gov/earthquakes/feed/v1.0/summary/4.5_day.geojson" }+into = "resp"+extract = [+ { into = "count", json = "json.metadata.count" },+ { into = "mag", json = "json.features[0].properties.mag" },+ { into = "place", json = "json.features[0].properties.place" },+ { into = "tsunami", json = "json.features[0].properties.tsunami" },+ { into = "url", json = "json.features[0].properties.url" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "any" },+ { else = "unexpected" },+]++[nodes.any]+kind = "decide"+on = "count"+branches = [+ { when = { eq = 0 }, then = "none" },+ { when = { gt = 0 }, then = "reply" },+ { else = "unexpected" },+]++[nodes.reply]+kind = "done"+say = "{{ state.count }} earthquake(s) of magnitude 4.5+ in the past day. The latest: M{{ state.mag }}, {{ state.place }}. {{ state.url }}"+output = { count = "{{ state.count }}", latest = { magnitude = "{{ state.mag }}", place = "{{ state.place }}", tsunami = "{{ state.tsunami }}", url = "{{ state.url }}" } }++[nodes.none]+kind = "done"+say = "No earthquakes of magnitude 4.5+ anywhere in the past day."+output = { count = 0 }++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "The USGS feed answered {{ state.resp.status }}."diff --git a/examples/exchange-rate/exchange-rate.toml b/examples/exchange-rate/exchange-rate.toml new file mode 100644 index 0000000..1055216 --- /dev/null +++ b/examples/exchange-rate/exchange-rate.toml@@ -0,0 +1,87 @@+# Converting an amount between currencies at the European Central Bank's+# reference rate, from Frankfurter. Asked for one currency, the API answers+# with a table of one, so the value is read from it by its shape rather+# than by the currency's name.++name = "exchange-rate"+version = 1+namespace = "pyrtan"+description = "An amount in another currency at today's European Central Bank reference rate, from Frankfurter."+tags = ["money", "read-only"]+license = "MIT"+hosts = ["api.frankfurter.dev"]+start = "fetch"++[[triggers]]+pattern = "convert {amount:number} {from:word} (to|into|in) {to:word}"+[[triggers]]+pattern = "how much is {amount:number} {from:word} in {to:word}"+[[triggers]]+pattern = "[the] exchange rate (from|for) {from:word} to {to:word}"+[[triggers]]+pattern = "{from:word} to {to:word} (rate|exchange rate)"++# Kept as the digits read from the sentence, so 100 reads as 100, not 100.0.+[inputs.amount]+type = "str"+default = "1"+pattern = "^[0-9]+(\\.[0-9]+)?$"+description = "How much of the first currency, as in: 100 or 2.5"++[inputs.from]+type = "str"+pattern = "^[A-Za-z]{3}$"+description = "A three-letter currency code, as in: USD"++[inputs.to]+type = "str"+pattern = "^[A-Za-z]{3}$"+description = "A three-letter currency code, as in: EUR"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://api.frankfurter.dev/v1/latest?base={{ inputs.from | upper }}&symbols={{ inputs.to | upper }}&amount={{ inputs.amount }}" }+into = "resp"+extract = [+ { into = "value", regex = "\"rates\":\\s*\\{\\s*\"[A-Z]{3}\":\\s*(-?[0-9.]+(?:[eE][-+]?[0-9]+)?)\\s*\\}" },+ { into = "day", json = "json.date" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "has-rate" },+ { when = { eq = 404 }, then = "unknown-currency" },+ { when = { eq = 422 }, then = "same-currency" },+ { else = "unexpected" },+]++[nodes.has-rate]+kind = "decide"+on = "value"+branches = [+ { when = { exists = true }, then = "reply" },+ { else = "unexpected" },+]++[nodes.reply]+kind = "done"+say = "{{ inputs.amount }} {{ inputs.from | upper }} is {{ state.value }} {{ inputs.to | upper }} at the ECB rate of {{ state.day }}."+output = { amount = "{{ inputs.amount | float }}", from = "{{ inputs.from | upper }}", to = "{{ inputs.to | upper }}", value = "{{ state.value | float }}", date = "{{ state.day }}" }++[nodes.unknown-currency]+kind = "fail"+reason = "unknown-currency"+say = "The ECB publishes no rate between {{ inputs.from | upper }} and {{ inputs.to | upper }}; it covers about thirty currencies, by their three-letter codes."++[nodes.same-currency]+kind = "fail"+reason = "same-currency"+say = "{{ inputs.from | upper }} to {{ inputs.to | upper }} is the same currency."++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "Frankfurter answered {{ state.resp.status }}."diff --git a/examples/hn-search/hn-search.toml b/examples/hn-search/hn-search.toml new file mode 100644 index 0000000..91b39e7 --- /dev/null +++ b/examples/hn-search/hn-search.toml@@ -0,0 +1,77 @@+# Hacker News stories matching some words, through Algolia's HN Search+# API: how many match, and the three best by its ranking.++name = "hn-search"+version = 1+namespace = "pyrtan"+description = "Hacker News stories about something, through HN Search: the top three matches."+tags = ["news", "tech", "read-only"]+license = "MIT"+hosts = ["hn.algolia.com"]+start = "search"++[[triggers]]+pattern = "search hacker news for {query}"+[[triggers]]+pattern = "hacker news stories about {query}"+[[triggers]]+pattern = "what does hacker news say about {query}"++[inputs.query]+type = "str"+description = "Words to search story titles and text for"++[nodes.search]+kind = "observe"+http = { method = "GET", url = "https://hn.algolia.com/api/v1/search?query={{ inputs.query | url }}&tags=story&hitsPerPage=3" }+into = "resp"+extract = [+ { into = "total", json = "json.nbHits" },+ { into = "first", json = "json.hits[0].title" },+ { into = "first_url", json = "json.hits[0].url" },+ { into = "hits", json = "json.hits" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "any" },+ { else = "unexpected" },+]++[nodes.any]+kind = "decide"+on = "hits"+branches = [+ { when = { empty = true }, then = "nothing" },+ { else = "each" },+]++[nodes.each]+kind = "each"+over = "hits"+limit = 3+start = "one"+into = "stories"+next = "reply"++[nodes.one]+kind = "done"+output = { title = "{{ item.title }}", url = "{{ item.url }}", points = "{{ item.points }}", by = "{{ item.author }}", discussion = "https://news.ycombinator.com/item?id={{ item.objectID }}" }++[nodes.reply]+kind = "done"+say = "{{ state.total }} Hacker News stories match {{ inputs.query }}. Top: {{ state.first }} {{ state.first_url }}"+output = { query = "{{ inputs.query }}", total = "{{ state.total }}", stories = "{{ state.stories }}" }++[nodes.nothing]+kind = "done"+say = "No Hacker News stories match {{ inputs.query }}."+output = { query = "{{ inputs.query }}", total = 0, stories = [] }++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "HN Search answered {{ state.resp.status }}."diff --git a/examples/holidays/holidays.toml b/examples/holidays/holidays.toml new file mode 100644 index 0000000..51614aa --- /dev/null +++ b/examples/holidays/holidays.toml@@ -0,0 +1,65 @@+# The next public holiday in a country, from Nager.Date. The country is its+# two-letter ISO code; the API answers an unknown one with an error status.++name = "holidays"+version = 1+namespace = "pyrtan"+description = "The next public holiday in a country (by its two-letter code), from Nager.Date."+tags = ["calendar", "read-only"]+license = "MIT"+hosts = ["date.nager.at"]+start = "fetch"++[[triggers]]+pattern = "[when is the] next [public|bank] holiday in {country:word}"+[[triggers]]+pattern = "[public|bank] holidays in {country:word}"++[inputs.country]+type = "str"+pattern = "^[A-Za-z]{2}$"+description = "A two-letter country code, as in: NO, US, DE"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://date.nager.at/api/v3/NextPublicHolidays/{{ inputs.country | upper }}" }+into = "resp"+extract = [+ { into = "date", json = "json[0].date" },+ { into = "name", json = "json[0].name" },+ { into = "local", json = "json[0].localName" },+ { into = "then", json = "json[1].name" },+ { into = "then_date", json = "json[1].date" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "any" },+ { else = "unknown-country" },+]++[nodes.any]+kind = "decide"+on = "date"+branches = [+ { when = { exists = true }, then = "reply" },+ { else = "none-listed" },+]++[nodes.reply]+kind = "done"+say = "The next public holiday in {{ inputs.country | upper }} is {{ state.name }} ({{ state.local }}) on {{ state.date }}."+output = { country = "{{ inputs.country | upper }}", date = "{{ state.date }}", name = "{{ state.name }}", local_name = "{{ state.local }}", after = { date = "{{ state.then_date }}", name = "{{ state.then }}" } }++[nodes.unknown-country]+kind = "fail"+reason = "unknown-country"+say = "Nager.Date knows no country {{ inputs.country | upper }} (it answered {{ state.resp.status }}); use a two-letter code such as NO, US or DE."++[nodes.none-listed]+kind = "fail"+reason = "none-listed"+say = "Nager.Date lists no coming public holidays for {{ inputs.country | upper }}."diff --git a/examples/iss/iss.toml b/examples/iss/iss.toml new file mode 100644 index 0000000..bafdf13 --- /dev/null +++ b/examples/iss/iss.toml@@ -0,0 +1,82 @@+# Where the International Space Station is now, from wheretheiss.at+# (25544 is its NORAD catalog number).++name = "iss"+version = 1+namespace = "pyrtan"+description = "Where the International Space Station is right now: latitude, longitude, altitude and speed."+tags = ["science", "space", "read-only"]+license = "MIT"+hosts = ["api.wheretheiss.at"]+start = "fetch"++[[triggers]]+pattern = "where is the (iss|space station) [now]"+[[triggers]]+pattern = "(iss|space station) (position|location)"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://api.wheretheiss.at/v1/satellites/25544" }+into = "resp"+# The API gives a dozen decimals; the words keep two (for a position, about a kilometre), the output all of them.+extract = [+ { into = "lat", json = "json.latitude" },+ { into = "lon", json = "json.longitude" },+ { into = "alt", json = "json.altitude" },+ { into = "speed", json = "json.velocity" },+ { into = "light", json = "json.visibility" },+ { into = "lat2", regex = "\"latitude\":\\s*(-?\\d+(?:\\.\\d{1,2})?)" },+ { into = "lon2", regex = "\"longitude\":\\s*(-?\\d+(?:\\.\\d{1,2})?)" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "have-position" },+ { when = { eq = 429 }, then = "rate-limited" },+ { else = "unexpected" },+]++[nodes.have-position]+kind = "decide"+on = "lat"+branches = [+ { when = { type = "number" }, then = "light" },+ { else = "unexpected" },+]++[nodes.light]+kind = "decide"+on = "light"+branches = [+ { when = { eq = "daylight" }, then = "in-sun" },+ { when = { eq = "eclipsed" }, then = "in-shadow" },+ { else = "in-sun" },+]++[nodes.in-sun]+kind = "set"+set = { where = "in sunlight" }+next = "reply"+[nodes.in-shadow]+kind = "set"+set = { where = "in the Earth's shadow" }+next = "reply"++[nodes.reply]+kind = "done"+say = "The ISS is over {{ state.lat2 }}, {{ state.lon2 }}, {{ state.alt | int }} km up, moving at {{ state.speed | int }} km/h, {{ state.where }}."+output = { latitude = "{{ state.lat }}", longitude = "{{ state.lon }}", altitude_km = "{{ state.alt }}", speed_kmh = "{{ state.speed }}", visibility = "{{ state.light }}", map = "https://www.openstreetmap.org/?mlat={{ state.lat }}&mlon={{ state.lon }}&zoom=3" }++[nodes.rate-limited]+kind = "fail"+reason = "rate-limited"+say = "wheretheiss.at is asking for fewer requests (429); try again in a moment."++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "wheretheiss.at answered {{ state.resp.status }}."diff --git a/examples/packages/npm-package.toml b/examples/packages/npm-package.toml new file mode 100644 index 0000000..f046d7c --- /dev/null +++ b/examples/packages/npm-package.toml@@ -0,0 +1,57 @@+# The latest published version of an npm package, from the registry.+# Scoped names (@scope/name) are one path segment, so the slash is encoded.++name = "npm-package"+version = 1+namespace = "pyrtan"+description = "The latest version of an npm package, and what it says it is."+tags = ["software", "javascript", "read-only"]+license = "MIT"+hosts = ["registry.npmjs.org"]+start = "fetch"++[[triggers]]+pattern = "[what's the] latest version of {package:word} on npm"+[[triggers]]+pattern = "latest {package:word} on npm"+[[triggers]]+pattern = "npm (package|version) {package:word}"++[inputs.package]+type = "str"+pattern = "^(@[a-z0-9][a-z0-9._~-]*/)?[a-z0-9][a-z0-9._~-]*$"+description = "A package name, as in: react or @types/node"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://registry.npmjs.org/{{ inputs.package | url }}/latest" }+into = "resp"+extract = [+ { into = "version", json = "json.version" },+ { into = "about", json = "json.description" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "reply" },+ { when = { eq = 404 }, then = "no-package" },+ { else = "unexpected" },+]++[nodes.reply]+kind = "done"+say = "{{ inputs.package }} {{ state.version }} on npm: {{ state.about }} https://www.npmjs.com/package/{{ inputs.package }}"+output = { package = "{{ inputs.package }}", version = "{{ state.version }}", description = "{{ state.about }}", url = "https://www.npmjs.com/package/{{ inputs.package }}" }++[nodes.no-package]+kind = "fail"+reason = "no-package"+say = "npm has no package called {{ inputs.package }}."++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "The npm registry answered {{ state.resp.status }}."diff --git a/examples/packages/pypi-package.toml b/examples/packages/pypi-package.toml new file mode 100644 index 0000000..9810c7d --- /dev/null +++ b/examples/packages/pypi-package.toml@@ -0,0 +1,58 @@+# The latest released version of a Python package, from PyPI's JSON API.++name = "pypi-package"+version = 1+namespace = "pyrtan"+description = "The latest version of a Python package on PyPI, and what it says it is."+tags = ["software", "python", "read-only"]+license = "MIT"+hosts = ["pypi.org"]+start = "fetch"++[[triggers]]+pattern = "[what's the] latest version of {package:word} on pypi"+[[triggers]]+pattern = "latest {package:word} on pypi"+[[triggers]]+pattern = "(pypi|pip) (package|version) {package:word}"++[inputs.package]+type = "str"+pattern = "^[A-Za-z0-9]([A-Za-z0-9._-]*[A-Za-z0-9])?$"+description = "A project name, as in: requests"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://pypi.org/pypi/{{ inputs.package | url }}/json" }+into = "resp"+extract = [+ { into = "name", json = "json.info.name" },+ { into = "version", json = "json.info.version" },+ { into = "about", json = "json.info.summary" },+ { into = "url", json = "json.info.package_url" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "reply" },+ { when = { eq = 404 }, then = "no-package" },+ { else = "unexpected" },+]++[nodes.reply]+kind = "done"+say = "{{ state.name }} {{ state.version }} on PyPI: {{ state.about }} {{ state.url }}"+output = { package = "{{ state.name }}", version = "{{ state.version }}", summary = "{{ state.about }}", url = "{{ state.url }}" }++[nodes.no-package]+kind = "fail"+reason = "no-package"+say = "PyPI has no project called {{ inputs.package }}."++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "PyPI answered {{ state.resp.status }}."diff --git a/examples/sun/sun.toml b/examples/sun/sun.toml new file mode 100644 index 0000000..64aabed --- /dev/null +++ b/examples/sun/sun.toml@@ -0,0 +1,80 @@+# Today's sunrise and sunset for a place, in the place's own time, from+# Open-Meteo: geocode the name, then ask for the day's sun times with the+# place's time zone.++name = "sun"+version = 1+namespace = "pyrtan"+description = "Today's sunrise and sunset for a place, in its local time, from Open-Meteo."+tags = ["weather", "read-only"]+license = "MIT"+hosts = ["geocoding-api.open-meteo.com", "api.open-meteo.com"]+start = "geocode"++[[triggers]]+pattern = "[when is] (sunrise|sunset) in {place}"+[[triggers]]+pattern = "when does the sun (rise|set) in {place}"+[[triggers]]+pattern = "(sunrise|sun) times (in|for) {place}"++[inputs.place]+type = "str"+description = "A town or city name"++[nodes.geocode]+kind = "observe"+http = { method = "GET", url = "https://geocoding-api.open-meteo.com/v1/search?name={{ inputs.place | url }}&count=1&language=en&format=json" }+into = "geo"+extract = [+ { into = "hit", json = "json.results[0]" },+ { into = "lat", json = "json.results[0].latitude" },+ { into = "lon", json = "json.results[0].longitude" },+ { into = "name", json = "json.results[0].name" },+ { into = "country", json = "json.results[0].country" },+]+next = "found"++[nodes.found]+kind = "decide"+on = "hit"+branches = [+ { when = { exists = true }, then = "times" },+ { else = "unknown-place" },+]++[nodes.times]+kind = "observe"+http = { method = "GET", url = "https://api.open-meteo.com/v1/forecast?latitude={{ state.lat }}&longitude={{ state.lon }}&daily=sunrise,sunset&timezone=auto&forecast_days=1" }+into = "fc"+extract = [+ { into = "day", json = "json.daily.time[0]" },+ { into = "rise", regex = "T(\\d\\d:\\d\\d)", from = "json.daily.sunrise[0]" },+ { into = "set", regex = "T(\\d\\d:\\d\\d)", from = "json.daily.sunset[0]" },+ { into = "zone", json = "json.timezone" },+]+next = "have-times"++# Above the polar circles the sun may not rise or set at all that day.+[nodes.have-times]+kind = "decide"+on = "rise"+branches = [+ { when = { exists = true }, then = "reply" },+ { else = "no-times" },+]++[nodes.reply]+kind = "done"+say = "{{ state.name }}, {{ state.country }}, {{ state.day }}: sunrise {{ state.rise }}, sunset {{ state.set }} ({{ state.zone }})."+output = { place = "{{ state.name }}", country = "{{ state.country }}", date = "{{ state.day }}", sunrise = "{{ state.rise }}", sunset = "{{ state.set }}", timezone = "{{ state.zone }}" }++[nodes.unknown-place]+kind = "fail"+reason = "unknown-place"+say = "I could not find a place called {{ inputs.place }}."++[nodes.no-times]+kind = "fail"+reason = "no-times"+say = "Open-Meteo gave no sunrise for {{ state.name }} today (status {{ state.fc.status }}); near the poles the sun may not rise or set at all."diff --git a/examples/synonyms/synonyms.toml b/examples/synonyms/synonyms.toml new file mode 100644 index 0000000..5bbde12 --- /dev/null +++ b/examples/synonyms/synonyms.toml@@ -0,0 +1,62 @@+# Words that mean the same, from Datamuse, best first.++name = "synonyms"+version = 1+namespace = "pyrtan"+description = "Synonyms for an English word, best first, from Datamuse."+tags = ["reference", "language", "read-only"]+license = "MIT"+hosts = ["api.datamuse.com"]+start = "fetch"++[[triggers]]+pattern = "synonyms (for|of) {word:word}"+[[triggers]]+pattern = "(another|other) word for {word:word}"+[[triggers]]+pattern = "what is another word for {word:word}"++[inputs.word]+type = "str"+pattern = "^[A-Za-z][A-Za-z'\\-]*$"+description = "One English word"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://api.datamuse.com/words?rel_syn={{ inputs.word | lower | url }}&max=8" }+into = "resp"+extract = [+ { into = "words", regex = "\"word\":\\s*\"([^\"]+)\"", all = true },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "any" },+ { else = "unexpected" },+]++[nodes.any]+kind = "decide"+on = "words"+branches = [+ { when = { empty = true }, then = "none" },+ { else = "reply" },+]++[nodes.reply]+kind = "done"+say = "Synonyms for {{ inputs.word }}: {{ state.words | join }}."+output = { word = "{{ inputs.word }}", synonyms = "{{ state.words }}" }++[nodes.none]+kind = "fail"+reason = "no-synonyms"+say = "Datamuse knows no synonyms for {{ inputs.word }}."++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "Datamuse answered {{ state.resp.status }}."diff --git a/examples/wikipedia/wikipedia.toml b/examples/wikipedia/wikipedia.toml new file mode 100644 index 0000000..27fdb53 --- /dev/null +++ b/examples/wikipedia/wikipedia.toml@@ -0,0 +1,79 @@+# The opening paragraph of a Wikipedia article, from its REST API. A title+# that leads to several articles is a disambiguation page: the tree says so+# rather than reading out a list of meanings.++name = "wikipedia"+version = 1+namespace = "pyrtan"+description = "The summary of a Wikipedia article."+tags = ["reference", "read-only"]+license = "MIT"+hosts = ["en.wikipedia.org"]+start = "fetch"++[[triggers]]+pattern = "tell me about {topic}"+[[triggers]]+pattern = "who (is|was) {topic}"+[[triggers]]+pattern = "(wikipedia|wiki) {topic}"++[inputs.topic]+type = "str"+description = "An article title, as in: Ursula K. Le Guin"++[nodes.fetch]+kind = "observe"+http = { method = "GET", url = "https://en.wikipedia.org/api/rest_v1/page/summary/{{ inputs.topic | url }}" }+into = "resp"+extract = [+ { into = "kind", json = "json.type" },+ { into = "title", json = "json.title" },+ { into = "summary", json = "json.extract" },+ { into = "url", json = "json.content_urls.desktop.page" },+]+next = "status"++[nodes.status]+kind = "decide"+on = "resp.status"+branches = [+ { when = { eq = 200 }, then = "kind" },+ { when = { eq = 404 }, then = "no-article" },+ { when = { eq = 429 }, then = "rate-limited" },+ { else = "unexpected" },+]++[nodes.kind]+kind = "decide"+on = "kind"+branches = [+ { when = { eq = "disambiguation" }, then = "ambiguous" },+ { when = { exists = true }, then = "reply" },+ { else = "unexpected" },+]++[nodes.reply]+kind = "done"+say = "{{ state.title }}: {{ state.summary }} {{ state.url }}"+output = { title = "{{ state.title }}", summary = "{{ state.summary }}", url = "{{ state.url }}" }++[nodes.ambiguous]+kind = "fail"+reason = "ambiguous"+say = "{{ state.title }} could mean several things on Wikipedia; say which: {{ state.url }}"++[nodes.no-article]+kind = "fail"+reason = "no-article"+say = "Wikipedia has no article called {{ inputs.topic }}."++[nodes.rate-limited]+kind = "fail"+reason = "rate-limited"+say = "Wikipedia is asking for fewer requests (429); try again in a minute."++[nodes.unexpected]+kind = "fail"+reason = "unexpected-status"+say = "Wikipedia answered {{ state.resp.status }}."diff --git a/pyproject.toml b/pyproject.toml index e17be35..f4a8abd 100644 --- a/pyproject.toml +++ b/pyproject.toml@@ -1,6 +1,6 @@[project] name = "pyrtan"-version = "0.15.0"+version = "0.18.0"description = "Open-source total automation: a core set of expandable services that act on all forms of I/O through inspectable decision trees." readme = "README.md" license = "AGPL-3.0-or-later" diff --git a/site/README.md b/site/README.md index 21fcf95..b803331 100644 --- a/site/README.md +++ b/site/README.md@@ -10,7 +10,7 @@ static directory and publishes it.| `/console/` | the console (installable: manifest, service worker and `pyrtan.js`, the tree runtime, beside it). Until it joins a hub the browser is one: it holds the catalog's trees (`/catalog/trees.json`) and runs them in the page. To join any hub started with `--cors https://pyrtan.com`, give its URL and token; its Member tab then joins that hub, and runs its trees when joined with the token or as a key the hub grants `run`; its Identity tab holds that key | `pyrtan console --out` | | `/catalog/` | the trees people can `pyrtan pull`, as `namespace/name.toml`, `index.json` the command reads, and `trees.json`, the same trees as JSON for the console | `pyrtan index examples catalog --copy`, through `site/gate.py` | | `/theme.js` | the wordmark's colour picker the landing page and the catalog load (the console carries its own copy inline) | `src/pyrtan/js/theme.js` |-| `/rendezvous/network.json` | the default network: its name, and a secret that is public on purpose | `site/rendezvous/network.json` |+| `/rendezvous/network.json` | the default network: its name, and a secret that is public on purpose; built into pyrtan, published here as a mirror (and for the snapshot) | `site/rendezvous/network.json` || `/rendezvous/hubs.json` | the hub list: hubs operators added by pull request, which the snapshot asks first and `pyrtan join` tries first | `site/rendezvous/hubs.json` | | `/rendezvous/directory.json` | a snapshot of the default network's directory, from every hub the build can reach, refreshed hourly | `site/snapshot.py` | | `/rendezvous/reports.json` | the gap reports the hubs carry for catalog trees, refreshed hourly | `site/snapshot.py --reports` |@@ -64,9 +64,11 @@ reports` reads when given no hub.`site/rendezvous/network.json` is the default network: the name `pyrtan` and a secret that is public on purpose (it names the network,-it guards nothing; see `_meta/federation.md` §7a). `pyrtan join` with+it guards nothing; see `_meta/federation.md` §7a). pyrtan has the same+two values built in, and a test holds them equal; this file is a mirror,+and what the snapshot keeps the network's part by. `pyrtan join` withno invite, `pyrtan hub --join default` and `$PYRTAN_JOIN=default` put a-hub in it.+hub in it without reading it.pyrtan.com runs no hub of its own; it is these static files, and the hub list is how it learns that a hub exists. `site/rendezvous/hubs.json` diff --git a/site/rendezvous/network.json b/site/rendezvous/network.json index ac2765c..2aaa4a0 100644 --- a/site/rendezvous/network.json +++ b/site/rendezvous/network.json@@ -1,6 +1,6 @@{ "pyrtan-network": 1,- "about": "The default network: `pyrtan join` with no invite (or `pyrtan hub --join default`) puts a hub in it and follows the first hub in hubs.json that answers, then the snapshot's hubs. Its secret is public on purpose: it names the network rather than guarding it, so anyone may join, and following stays one-directional trust. A network of your own is `pyrtan network new NAME`; its invite is yours to share.",+ "about": "The default network, as built into pyrtan; this file mirrors it (and the snapshot keeps the network's part by it). `pyrtan join` with no invite (or `pyrtan hub --join default`) puts a hub in it and follows the first hub that accepts: one it already knows, else one from hubs.json, else from the snapshot. Its secret is public on purpose: it names the network rather than guarding it, so anyone may join, and following stays one-directional trust. A network of your own is `pyrtan network new NAME`; its invite is yours to share.","name": "pyrtan", "secret": "public-default-network-of-pyrtan-com" } diff --git a/src/pyrtan/__init__.py b/src/pyrtan/__init__.py index 3461609..233bac2 100644 --- a/src/pyrtan/__init__.py +++ b/src/pyrtan/__init__.py@@ -33,4 +33,4 @@ __all__ = ["Response", ]-__version__ = "0.15.0"+__version__ = "0.18.0"diff --git a/src/pyrtan/chat.py b/src/pyrtan/chat.py index 3f7927c..855d69f 100644 --- a/src/pyrtan/chat.py +++ b/src/pyrtan/chat.py@@ -27,6 +27,22 @@ activity. A hub answers a ``Meet`` only from a hub that follows it, pairsit only with its own seats, and never passes one on: one hop, as everywhere in the federation.+**Chat links (v0.17).** Following is trust: it routes sentences to the+other hub's trees, so a hub follows few. That made the pool of strangers+small. A hub started ``--chat-reach N`` also keeps up to N *chat links*+with other hubs whose lobbies are open: hubs it heard of on the LAN or+in its directory (public networks only), asked with a signed+``ChatLink`` that carries a pair secret, as a ``Follow`` does. A link is+for chat and nothing else: it is not a follow, so nothing is routed over+it, nothing is announced or kept, and no directory travels on it. Both+ends may send ``Meet`` over it, so the two lobbies share their+strangers. A hub accepts a link only when its operator also gave+``--chat-reach`` and its lobby is open to strangers anyway (``--chat-+public``, or a hub with no token), takes at most twice its reach, and+refuses them all when its federation is closed (``--closed`` or a+federation secret). Links live in memory and are made again after a+restart; one whose hub stops answering is dropped.+**Typing.** While a person types, their client says so (``/chat/typing``) and the stranger sees "Stranger is typing…"; while a bot's trees run, its partner sees "Bot is thinking…". It is a state, never an event: it goes out@@ -76,10 +92,16 @@ from collections import dequefrom dataclasses import dataclass, field from typing import TYPE_CHECKING, Any, Union+from .service import http_json+if TYPE_CHECKING: from .federation import Remote from .hub import Hub+LINK_TRIES = 2 # links tried per round (a round is the hub's poll)+LINK_REFUSED = 86400.0 # a hub that refused a link is not asked again for a day+LINK_FAILS = 3 # Meets in a row a linked hub may fail before the link is dropped+WANTS = ("anyone", "people", "bot") SEAT_IDLE = 60.0 # a seat nobody polls for this long is let go BOT_AFTER = 8.0 # an "anyone" seat that meets no stranger in this long meets a bot@@ -207,8 +229,12 @@ class Seat:class Lobby:- def __init__(self, hub: "Hub", *, typing: bool = True) -> None:+ def __init__(self, hub: "Hub", *, typing: bool = True, reach: int = 0) -> None:self.hub = hub+ self.reach = max(0, int(reach)) # chat links to keep with other open lobbies (--chat-reach)+ self.links: dict[str, "Remote"] = {} # hub id -> the link: chat only, never a follow+ self._refused: dict[str, tuple[float, str]] = {} # hub id -> (when it may be asked again, why it said no)+ self._fails: dict[str, int] = {}self.typing = typing # the hub's switch: off, nobody's typing is shared self.seats: dict[str, Seat] = {} self.refs: dict[str, Seat] = {}@@ -231,7 +257,7 @@ class Lobby:"waiting": sum(1 for s in seats if s.state == "waiting"), "talking": sum(1 for s in seats if s.state == "talking"), "bots": 0 if guest else sum(1 for h in self.hub.holders() if h.link != "down" and h.trees),- "hubs": sum(1 for r in fed.following.values() if r.link != "down") if fed is not None else 0,+ "hubs": (sum(1 for r in fed.following.values() if r.link != "down") if fed is not None else 0) + len(self.links),"typing": self.typing, }@@ -352,7 +378,7 @@ class Lobby:self._pair(seat, other) return now = time.time()- far = fed is not None and bool(fed.following) and now - seat.asked >= self.meet_every+ far = fed is not None and bool(fed.following or self.links) and now - seat.asked >= self.meet_everyif far: seat.busy, seat.asked = True, now if far:@@ -370,11 +396,14 @@ class Lobby:def _meet_far(self, seat: Seat) -> bool: fed = self.hub.federation assert fed is not None- hubs = [r for r in fed.following.values() if r.kind == "hub" and r.state == "accepted" and r.link != "down"]+ followed = [r for r in fed.following.values() if r.kind == "hub" and r.state == "accepted" and r.link != "down"]+ hubs = followed + [r for url, r in list(self.links.items()) if url not in {f.url for f in followed}]random.shuffle(hubs) ref = seat.ref for remote in hubs[:MEET_HUBS]: ok, _, answer = fed.deliver(remote, fed._activity("Meet", {"ref": ref, "interests": seat.interests}))+ if remote.url in self.links:+ self._link_answered(remote.url, ok)if not (ok and isinstance(answer, dict) and answer.get("met") is True and isinstance(answer.get("ref"), str) and REF.match(answer["ref"])): continue with self.cond:@@ -483,7 +512,7 @@ class Lobby:nap = min(nap, seat.heard - now) # wake when the indicator goes out if seat.state == "waiting": # Wake to ask the hubs we follow again, or to meet a bot.- if self.hub.federation is not None and self.hub.federation.following:+ if self.hub.federation is not None and (self.hub.federation.following or self.links):nap = min(nap, seat.asked + self.meet_every - now) if seat.want == "anyone": nap = min(nap, seat.since + self.bot_after - now)@@ -633,6 +662,94 @@ class Lobby:extra = {k: answer[k] for k in ("status", "reason", "tree", "peer", "via", "output", "trace") if answer.get(k) is not None} self._add(seat, "bot", text, **extra)+ # ------------------------------------------------------------ chat links++ def open(self) -> bool:+ """Whether strangers may chat here anyway: a public lobby, or a hub with no token."""+ return bool(self.hub.chat_public or not self.hub.token)++ def link_for(self, hub: str) -> "Remote | None":+ return self.links.get(hub.rstrip("/"))++ def links_view(self) -> dict[str, Any]:+ now = time.time()+ return {"reach": self.reach, "open": self.open(), "links": [{"url": r.url, "name": r.name, "since": r.since} for r in list(self.links.values())],+ "refused": [{"url": u, "reason": why, "until": until} for u, (until, why) in list(self._refused.items()) if until > now]}++ def accept_link(self, remote: "Remote") -> tuple[int, dict[str, Any]]:+ """Another hub asks for a chat link (it signed with the pair secret it sent)."""+ if self.reach <= 0:+ return 403, {"status": "error", "reason": "not-linking", "say": "This hub keeps no chat links (its operator did not give --chat-reach)."}+ if not self.open():+ return 403, {"status": "error", "reason": "private-lobby", "say": "This hub's lobby is not open to strangers, so it links with nobody for chat."}+ with self.cond:+ if remote.url not in self.links and len(self.links) >= 2 * self.reach:+ return 503, {"status": "error", "reason": "links-full", "say": f"This hub keeps {len(self.links)} chat links already."}+ self.links[remote.url] = remote+ self._fails.pop(remote.url, None)+ return 202, {"status": "ok", "linked": True}++ def link(self, url: str) -> bool:+ """Ask the hub at ``url`` for a chat link: a ``ChatLink`` carrying a+ fresh pair secret, signed with it. True when it took it."""+ from .federation import Remote++ fed = self.hub.federation+ if fed is None:+ return False+ url = url.rstrip("/")+ try:+ status, actor = http_json("GET", f"{url}/actor", timeout=5.0)+ except Exception:+ status, actor = 0, None+ if status != 200 or not isinstance(actor, dict) or actor.get("type") != "Hub" or not isinstance(actor.get("inbox"), str) or str(actor.get("id", "")).rstrip("/") != url:+ self._refused[url] = (time.time() + LINK_REFUSED, "no-actor")+ return False+ remote = Remote(url, actor["inbox"], secrets.token_hex(32), str(actor.get("name") or ""), state="accepted", kind="hub")+ ok, _, answer = fed.deliver(remote, fed._activity("ChatLink", {"type": "Lobby", "id": fed.id}, secret=remote.secret, inbox=fed.inbox, name=self.hub.name))+ if not (ok and isinstance(answer, dict) and answer.get("linked") is True):+ self._refused[url] = (time.time() + LINK_REFUSED, str((answer.get("reason") if isinstance(answer, dict) else None) or "no-answer"))+ return False+ remote.last_ok = time.time()+ with self.cond:+ self.links[url] = remote+ self._fails.pop(url, None)+ return True++ def reach_out(self) -> int:+ """Keep up to ``reach`` links: try a hub or two it heard of on the+ LAN, then in its directory (public networks only), that it neither+ follows nor is followed by nor links with already. Called on the+ hub's poll; returns the links made."""+ fed = self.hub.federation+ if fed is None or self.reach <= 0 or not self.open() or len(self.links) >= self.reach:+ return 0+ now = time.time()+ public = fed._public_networks()+ with fed._lock:+ heard = sorted((e for e in fed.directory_entries.values() if e.get("network") in public and e.get("kind", "hub") == "hub"), key=lambda e: -e["seen"])+ taken = set(fed.following) | {u for u, r in fed.followers.items() if r.kind == "hub"}+ nearby = fed.lan.hubs() if fed.lan is not None else []+ made = tried = 0+ for url in dict.fromkeys(nearby + [e["id"].rstrip("/") for e in heard]):+ if tried >= LINK_TRIES or len(self.links) >= self.reach:+ break+ if url == fed.id or url in taken or url in self.links or self._refused.get(url, (0.0, ""))[0] > now:+ continue+ tried += 1+ made += 1 if self.link(url) else 0+ return made++ def _link_answered(self, url: str, ok: bool) -> None:+ with self.cond:+ if ok:+ self._fails.pop(url, None)+ return+ self._fails[url] = self._fails.get(url, 0) + 1+ if self._fails[url] >= LINK_FAILS:+ self.links.pop(url, None) # it stopped answering: make another in its place+ self._fails.pop(url, None)+# ------------------------------------------------------------ federation def on_activity(self, kind: str, remote: "Remote", obj: Any, *, follower: bool) -> tuple[int, dict[str, Any]]: diff --git a/src/pyrtan/cli.py b/src/pyrtan/cli.py index 5c297d0..0304109 100644 --- a/src/pyrtan/cli.py +++ b/src/pyrtan/cli.py@@ -12,6 +12,8 @@pyrtan hub --trees DIR and hold trees itself: a swarm in one process pyrtan hub --federate --advertise URL [--follow URL ...] [--listed] and between hubs pyrtan hub --chat and keep a lobby: strangers and bots, Omegle-style+ pyrtan hub --lan --join default and find the hubs on this LAN by multicast, with nobody to ask+ pyrtan hub --chat-public --chat-reach 8 and link its lobby with other open lobbies, for chat onlypyrtan follow [URL] --at HUB [--listed] follow another hub (and route to it); default: join the default network pyrtan unfollow URL --at HUB pyrtan peers --at HUB services, and hubs followed and following@@ -172,7 +174,7 @@ def main(argv: list[str] | None = None) -> int:p.add_argument("--federate", action="store_true", help="answer /actor and /inbox so other hubs can follow this one, and follow others") p.add_argument("--advertise", default=os.environ.get("PYRTAN_ADVERTISE"), help="the URL other hubs reach this hub at (implies --federate; default $PYRTAN_ADVERTISE, else this host and port)") p.add_argument("--follow", action="append", default=[h for h in os.environ.get("PYRTAN_FOLLOW", "").replace(",", " ").split() if h], metavar="URL", help="a hub to follow on start (repeatable; default $PYRTAN_FOLLOW; to join the default network, --join default)")- p.add_argument("--join", action="append", default=[h for h in os.environ.get("PYRTAN_JOIN", "").replace(",", " ").split() if h], metavar="INVITE", help="a network to be in on start: 'default' for the public one pyrtan.com publishes, or an invite NAME:SECRET[@HUB] (repeatable; default $PYRTAN_JOIN)")+ p.add_argument("--join", action="append", default=[h for h in os.environ.get("PYRTAN_JOIN", "").replace(",", " ").split() if h], metavar="INVITE", help="a network to be in on start: 'default' for the public one built into pyrtan, or an invite NAME:SECRET[@HUB] (repeatable; default $PYRTAN_JOIN)")p.add_argument("--federation-secret", default=os.environ.get("PYRTAN_FEDERATION_SECRET"), help="only hubs that sign their Follow with this secret may follow (default $PYRTAN_FEDERATION_SECRET; without it, follows are open)") p.add_argument("--closed", action="store_true", help="take no follows at all (this hub can still follow others)") p.add_argument("--state", metavar="FILE", default=os.environ.get("PYRTAN_STATE"), help="keep who this hub follows and is followed by, and the directory, in this file across restarts (default $PYRTAN_STATE)")@@ -182,6 +184,8 @@ def main(argv: list[str] | None = None) -> int:p.add_argument("--chat", action="store_true", default=os.environ.get("PYRTAN_CHAT", "").lower() in ("1", "true", "yes"), help="keep a lobby at /chat where people meet a random stranger here or at a hub this one follows, or a bot made of the trees it routes to (default $PYRTAN_CHAT)") p.add_argument("--allow", action="append", default=[a for a in os.environ.get("PYRTAN_ALLOW", "").split() if a], metavar="WHO=ABILITIES", help="grant an identity (pyr:… or name@domain, resolved and pinned now) abilities here: ask, chat, run, operate (repeatable; default $PYRTAN_ALLOW, space-separated)") p.add_argument("--grants", metavar="FILE", default=os.environ.get("PYRTAN_GRANTS"), help="keep the grants in this file (default $PYRTAN_GRANTS, else grants.json beside --state, else in memory)")+ p.add_argument("--lan", action="store_true", default=os.environ.get("PYRTAN_LAN", "").lower() in ("1", "true", "yes"), help="find hubs on this local network with no one to ask: announce this hub over UDP multicast (TTL 1) and try the hubs heard there when joining a network; needs --advertise with a LAN address (default $PYRTAN_LAN)")+ p.add_argument("--chat-reach", type=int, metavar="N", default=int(os.environ.get("PYRTAN_CHAT_REACH", "0") or 0), help="keep up to N chat links with other open lobbies this hub hears of (LAN, directory), so its strangers can meet theirs: for chat only, never a follow; needs an open lobby (--chat-public, or no token) (default $PYRTAN_CHAT_REACH or 0)")p.add_argument("--chat-public", action="store_true", default=os.environ.get("PYRTAN_CHAT_PUBLIC", "").lower() in ("1", "true", "yes"), help="open the lobby to callers without the token (implies --chat): they meet people only, never a bot, and everything else still needs the token (default $PYRTAN_CHAT_PUBLIC)") p.add_argument("--chat-typing", action=argparse.BooleanOptionalAction, default=os.environ.get("PYRTAN_CHAT_TYPING", "").lower() not in ("0", "false", "no", "off"), help="the lobby's typing indicator: \"Stranger is typing…\" for anyone who keeps theirs on (default on; $PYRTAN_CHAT_TYPING=0 turns it off)") p.add_argument("--report-gaps", action="store_true", default=os.environ.get("PYRTAN_REPORT_GAPS", "").lower() in ("1", "true", "yes"), help="put the gaps this swarm's catalog trees meet (tree id, node, value; never inputs) into its announces, so they reach the tree's author through the federation and pyrtan.com (default $PYRTAN_REPORT_GAPS)")@@ -1036,10 +1040,10 @@ def cmd_hub(args: argparse.Namespace) -> int:try: hub, server = serve_hub( args.host, args.port, name=args.name, peers=args.peer, poll_every=args.poll, quiet=args.quiet, token=args.token, cors=args.cors,- federate=args.federate or bool(args.advertise) or bool(follow) or bool(join) or args.listed, advertise=args.advertise, federation_secret=args.federation_secret,+ federate=args.federate or bool(args.advertise) or bool(follow) or bool(join) or args.listed or args.lan or args.chat_reach > 0, advertise=args.advertise, federation_secret=args.federation_secret,state=args.state, open_follows=not args.closed, follow=follow, join=join, listed=args.listed, trees=[Path(t) for t in args.trees], handout=[Path(t) for t in args.handout], env=_kv(args.env), gaps=args.gaps, egress=args.egress,- store=args.store, replicas=args.replicas, report_gaps=args.report_gaps, unanswered=args.unanswered, suggestions=args.suggestions, watch=args.watch, chat=args.chat or args.chat_public, chat_typing=args.chat_typing, chat_public=args.chat_public,+ store=args.store, replicas=args.replicas, report_gaps=args.report_gaps, unanswered=args.unanswered, suggestions=args.suggestions, watch=args.watch, chat=args.chat or args.chat_public, chat_typing=args.chat_typing, chat_public=args.chat_public, lan=args.lan, chat_reach=args.chat_reach,allow=args.allow, grants=args.grants, ) except IdentityError as e:@@ -1053,6 +1057,11 @@ def cmd_hub(args: argparse.Namespace) -> int:local += f", handing out {len(hub.handout_registry.trees())} tree(s) to trusted members" if hub.lobby is not None: local += ", chatting at /chat" + (" (open to callers without the token, people only)" if hub.chat_public and args.token else "")+ if hub.lobby.reach:+ local += f", linking with up to {hub.lobby.reach} other open lobbies" if hub.lobby.open() else ", but --chat-reach links only an open lobby (--chat-public, or no token)"+ if hub.lan is not None:+ lan = hub.lan.view()+ local += f", on the LAN ({lan['group']}): " + (f"cannot join it: {hub.lan.error}" if hub.lan.error else f"heard {len(lan['hubs'])} hub(s)" + ("" if lan["announcing"] else "; listening only, since this hub's id is not a LAN address (give --advertise http://<its LAN address>:PORT)"))if len(hub.grants): local += f", granting {len(hub.grants)} identit{'y' if len(hub.grants) == 1 else 'ies'}" print(f"pyrtan hub at {server.url}: {len(hub.peers)} peer(s){local}{fed}" + _access_note(args), flush=True) diff --git a/src/pyrtan/federation.py b/src/pyrtan/federation.py index 44554c1..4a419b2 100644 --- a/src/pyrtan/federation.py +++ b/src/pyrtan/federation.py@@ -60,12 +60,25 @@ it is in that network. The directory heard over a link is kept with thelink's network and passed on only over links in the same one, so a private network's hubs never reach the public directory, and ``GET /directory`` shows them only to the operator. The default network-(``default_network``, published at pyrtan.com) has a public secret: it-names the network, it does not guard it, and nothing here needs it to.+(``default_network``) has a public secret: it names the network, it does+not guard it, and nothing here needs it to. Its name and secret are+built in, so joining it asks nobody for them; pyrtan.com publishes the+same two values as a mirror, not as the authority.++**Joining** the default network tries what the hub already knows before+anything else: the hubs it follows in it, then the hubs its directory+has heard in it, most recent first. Only when none of those accepts does+it read the hub list and then the snapshot, and only as far as it needs.+So a hub that has been in the network once never needs pyrtan.com again,+and any hub's public ``/directory`` or any mirror of the list will do as+a starting point (``$PYRTAN_HUB_LIST``). A hub started ``--lan`` also+hears the hubs on its own network say they are there (:mod:`pyrtan.lan`)+and tries those right after the ones it follows: no internet needed.A follow outside any network (and every member's) works as before.-**Chat.** A hub with a lobby (``--chat``) asks the hubs it follows for a-stranger with ``Meet``, and the two seats talk in ``Say`` and end with+**Chat.** A hub with a lobby (``--chat``) asks the hubs it follows, and+the ones it keeps a chat link with (``ChatLink``, ``--chat-reach``: for+chat only, never a follow), for a stranger with ``Meet``, and the two seats talk in ``Say`` and end with``Part``, with ``Typing`` while someone types, both ways between any two hubs that share a secret; a ``Say`` is logged without its text and a ``Typing`` not at all. See :mod:`pyrtan.chat`.@@ -90,7 +103,7 @@ import uuidfrom collections import deque from dataclasses import dataclass, field from pathlib import Path-from typing import Any, Callable+from typing import Any, Callable, Iterablefrom . import __version__ from .service import http_json@@ -98,7 +111,10 @@ from .store import BlobStore, StoreError, check_key, digest as blob_digestDEFAULT_HUB_LIST = "https://pyrtan.com/rendezvous/hubs.json" DEFAULT_SNAPSHOT = "https://pyrtan.com/rendezvous/directory.json"-DEFAULT_NETWORK = "https://pyrtan.com/rendezvous/network.json"+# The default network, built in: public on purpose, so it names the network+# and guards nothing. site/rendezvous/network.json publishes the same values.+DEFAULT_NETWORK_NAME = "pyrtan"+DEFAULT_NETWORK_SECRET = "public-default-network-of-pyrtan-com"NETWORK_NAME = re.compile(r"^[a-z0-9][a-z0-9-]{0,39}$") INVITE = re.compile(r"^([a-z0-9][a-z0-9-]{0,39}):([A-Za-z0-9_-]{16,128})(?:@(https?://\S+))?$") SIGNATURE_WINDOW = 300.0 # seconds a signed message stays valid@@ -110,7 +126,7 @@ DIRECTORY_TTL = 7 * 86400 # an entry unheard of for this long is droppedDIRECTORY_MAX = 500 HEARD_UP = 600.0 # a directory entry seen this recently reads "up" HEARD_STALE = 86400.0-ACTIVITY_TYPES = ("Follow", "Accept", "Reject", "Undo", "Announce", "Keep", "Kept", "Fetch", "Ask", "Answer", "Meet", "Say", "Typing", "Part")+ACTIVITY_TYPES = ("Follow", "Accept", "Reject", "Undo", "Announce", "Keep", "Kept", "Fetch", "Ask", "Answer", "Meet", "Say", "Typing", "Part", "ChatLink")CHAT_TYPES = ("Meet", "Say", "Typing", "Part") # a lobby's: two strangers on two hubs UNLOGGED = ("Typing",) # said every couple of seconds while someone types: not worth a log line ASK_TIMEOUT = 30.0 # seconds a hub waits for a member to answer an Ask@@ -128,47 +144,51 @@ SIGNED_PATHS = ("/inbox", "/run", "/intent", "/resume", "/capabilities")# ---------------------------------------------------------------- the hub list-def rendezvous_hubs() -> list[str]:- """Hubs to try first, from ``$PYRTAN_RENDEZVOUS`` (one hub),- ``$PYRTAN_HUB_LIST`` (a URL or file holding ``{"hubs": [...]}``), or- the hub list at pyrtan.com. Never raises: no list is an empty list."""- single = os.environ.get("PYRTAN_RENDEZVOUS")- if single:- return [single.rstrip("/")]- source = os.environ.get("PYRTAN_HUB_LIST") or DEFAULT_HUB_LIST+def _read_json(source: str) -> Any:+ """A URL or a file, as JSON; None when it cannot be had."""try: if source.startswith(("http://", "https://")): status, body = http_json("GET", source, timeout=10.0)- if status != 200:- return []- else:- body = json.loads(Path(source).read_text(encoding="utf-8"))+ return body if status == 200 else None+ return json.loads(Path(source).read_text(encoding="utf-8"))except Exception:- return []+ return None+++def hub_urls(body: Any) -> list[str]:+ """The hubs a list names: ``{"hubs": ["https://…", …]}`` (a hub list)+ or ``{"hubs": [{"id": "https://…", "seen": …}, …]}`` (a hub's+ ``/directory``, or a snapshot), the latter most recently heard first."""if not isinstance(body, dict) or not isinstance(body.get("hubs"), list): return []- return [h.rstrip("/") for h in body["hubs"] if isinstance(h, str) and h.startswith(("http://", "https://"))]+ rows = body["hubs"]+ if all(isinstance(h, dict) for h in rows):+ rows = sorted(rows, key=lambda h: -float(h.get("seen", 0) or 0))+ urls = [h if isinstance(h, str) else h.get("id") if isinstance(h, dict) else None for h in rows]+ return list(dict.fromkeys(u.rstrip("/") for u in urls if isinstance(u, str) and u.startswith(("http://", "https://"))))+++def rendezvous_hubs() -> list[str]:+ """Hubs to try first, from ``$PYRTAN_RENDEZVOUS`` (one hub),+ ``$PYRTAN_HUB_LIST`` (any number of URLs or files, separated by spaces+ or commas: hub lists, a hub's ``/directory``, a snapshot or a mirror of+ one, in order), or the hub list at pyrtan.com. Never raises: nothing+ readable is an empty list."""+ single = os.environ.get("PYRTAN_RENDEZVOUS")+ if single:+ return [single.rstrip("/")]+ sources = os.environ.get("PYRTAN_HUB_LIST", "").replace(",", " ").split() or [DEFAULT_HUB_LIST]+ out: list[str] = []+ for source in sources:+ out.extend(u for u in hub_urls(_read_json(source)) if u not in out)+ return outdef snapshot_hubs() -> list[str]: """Hubs from the directory snapshot pyrtan.com bakes hourly (``$PYRTAN_SNAPSHOT`` overrides the URL or names a file), most recently heard first. What a new hub can try when no listed hub answers."""- source = os.environ.get("PYRTAN_SNAPSHOT") or DEFAULT_SNAPSHOT- try:- if source.startswith(("http://", "https://")):- status, body = http_json("GET", source, timeout=10.0)- if status != 200:- return []- else:- body = json.loads(Path(source).read_text(encoding="utf-8"))- except Exception:- return []- if not isinstance(body, dict) or not isinstance(body.get("hubs"), list):- return []- hubs = [h for h in body["hubs"] if isinstance(h, dict) and isinstance(h.get("id"), str) and h["id"].startswith(("http://", "https://"))]- hubs.sort(key=lambda h: -float(h.get("seen", 0) or 0))- return [h["id"].rstrip("/") for h in hubs]+ return hub_urls(_read_json(os.environ.get("PYRTAN_SNAPSHOT") or DEFAULT_SNAPSHOT))# ---------------------------------------------------------------- networks@@ -221,20 +241,14 @@ def parse_invite(text: str) -> tuple[Network, str | None]:def default_network() -> Network | None:- """The public network pyrtan.com publishes (``$PYRTAN_NETWORK``- overrides the URL or names a file). Its secret is public on purpose:- it names the network rather than guarding it. None when unreadable."""- source = os.environ.get("PYRTAN_NETWORK") or DEFAULT_NETWORK- try:- if source.startswith(("http://", "https://")):- status, body = http_json("GET", source, timeout=10.0)- if status != 200:- return None- else:- body = json.loads(Path(source).read_text(encoding="utf-8"))- except Exception:- return None- net = Network.from_state(body)+ """The default network: built in, so finding it asks nobody. Its secret+ is public on purpose: it names the network rather than guarding it.+ ``$PYRTAN_NETWORK`` (a URL or a file shaped like pyrtan.com's+ ``network.json``) names another; None when that cannot be read."""+ source = os.environ.get("PYRTAN_NETWORK")+ if not source:+ return Network(DEFAULT_NETWORK_NAME, DEFAULT_NETWORK_SECRET, public=True)+ net = Network.from_state(_read_json(source))if net is not None: net.public = True return net@@ -371,10 +385,13 @@ def _shape_report(e: dict[str, Any], reporter: str, now: float) -> dict[str, Anydef _logged(activity: dict[str, Any]) -> dict[str, Any]:- """An activity as the outbox and inbox logs keep it: a chat line without its text."""+ """An activity as the outbox and inbox logs keep it: a chat line without+ its text, a chat link without its secret."""obj = activity.get("object") if activity.get("type") == "Say" and isinstance(obj, dict) and "text" in obj: return {**activity, "object": {**obj, "text": f"<{len(str(obj['text']))} characters>"}}+ if activity.get("type") == "ChatLink" and "secret" in activity:+ return {**activity, "secret": "<secret>"}return activity@@ -406,6 +423,7 @@ class Federation:self._handed: dict[str, dict[str, dict[str, Any]]] = {} # member -> key -> {sha256, at, confirmed} self._asks: dict[str, dict[str, Any]] = {} # run id -> {member, event, answer} self.chat: Any = None # the hub's lobby, when it keeps one (pyrtan.chat.Lobby)+ self.lan: Any = None # what the hub hears on the LAN, when started --lan (pyrtan.lan.Lan)self.ask_timeout = ASK_TIMEOUT self.following: dict[str, Remote] = {} self.followers: dict[str, Remote] = {}@@ -514,9 +532,22 @@ class Federation:raise ValueError(f"this hub is already in a network named {name!r}") return self.add_network(Network(name, _secrets.token_urlsafe(24), public))- def join(self, net: Network, hubs: list[str]) -> Remote | None:+ def known_hubs(self, network: str) -> list[str]:+ """The hubs this one already knows in a network (by id), to try+ before asking anyone: the ones it follows in it, then the ones it+ heard on the LAN say they are in it (``--lan``), then the ones its+ directory heard in it, most recently heard first."""+ now = time.time()+ with self._lock:+ followed = [r.url for r in self.following.values() if r.kind == "hub" and r.network == network and r.state == "accepted"]+ heard = sorted((e for e in self.directory_entries.values() if e.get("network") == network and e.get("kind", "hub") == "hub" and now - e["seen"] < DIRECTORY_TTL), key=lambda e: -e["seen"])+ nearby = self.lan.hubs(network) if self.lan is not None else []+ return [u for u in dict.fromkeys(followed + nearby + [e["id"].rstrip("/") for e in heard]) if u != self.id]++ def join(self, net: Network, hubs: Iterable[str]) -> Remote | None:"""Be in ``net`` and follow the first of ``hubs`` that accepts a- follow in it. None when none does: the hub is then a network of+ follow in it (an iterable, so a caller can fetch more only when the+ first ones fail). None when none does: the hub is then a network ofone, which others can join by following it.""" net = self.add_network(net) for url in hubs:@@ -1151,6 +1182,8 @@ class Federation:self.inbox_log.append({**_logged(activity), "received": time.time()}) if kind == "Follow": return self._on_follow(actor, activity, h, trusted, trusted_by=trusted_by, signer=signer)+ if kind == "ChatLink":+ return self._on_chat_link(actor, activity, h)followed = self.following.get(actor) follower = self.followers.get(actor) if kind in ("Announce", "Keep", "Kept", "Fetch", "Answer", *CHAT_TYPES):@@ -1160,6 +1193,11 @@ class Federation:remote = followed or follower else: remote = followed if kind in ("Accept", "Reject") else follower+ linked = False+ if remote is None and kind in CHAT_TYPES and self.chat is not None:+ # A chat link: chat may travel on it, and nothing else does.+ remote = self.chat.link_for(actor)+ linked = remote is not Noneif remote is None: return 403, {"status": "error", "reason": "unknown-actor", "say": f"{actor} is not a hub this one follows or is followed by."} problem = verify(remote.secret, "POST", "/inbox", activity, h)@@ -1167,6 +1205,11 @@ class Federation:# Mutual follow: the announce may be signed with the other pair. remote = follower if remote is followed else followed problem = verify(remote.secret, "POST", "/inbox", activity, h)+ if problem and kind in CHAT_TYPES and self.chat is not None and not linked:+ # A follow made after a chat link: the line may be signed with the link's secret.+ link = self.chat.link_for(actor)+ if link is not None and verify(link.secret, "POST", "/inbox", activity, h) is None:+ remote, linked, problem = link, True, Noneif problem: return 401, {"status": "error", "reason": "unsigned", "say": problem} if kind in CHAT_TYPES:@@ -1175,7 +1218,7 @@ class Federation:if remote.kind != "hub": return 403, {"status": "error", "reason": "not-a-hub", "say": "Strangers meet hub to hub; a member chats through its hub's lobby."} remote.last_ok = time.time()- return self.chat.on_activity(kind, remote, activity.get("object"), follower=remote is follower)+ return self.chat.on_activity(kind, remote, activity.get("object"), follower=remote is follower or linked)if kind == "Accept": with self._lock: remote.state = "accepted"@@ -1270,6 +1313,28 @@ class Federation:self._save() return gone+ def _on_chat_link(self, actor: str, activity: dict[str, Any], h: dict[str, str]) -> tuple[int, dict[str, Any]]:+ """A hub asks for a chat link: checked as an open ``Follow`` is (signed+ with the pair secret it carries; its inbox is the one its ``/actor``+ names), then the lobby decides. A closed federation links with nobody."""+ pair, inbox = activity.get("secret"), activity.get("inbox")+ if not isinstance(pair, str) or len(pair) < 16 or not isinstance(inbox, str) or not actor.startswith(("http://", "https://")):+ return 400, {"status": "error", "reason": "bad-activity", "say": "A ChatLink comes from a hub and carries a pair secret and its inbox."}+ if self.chat is None:+ return 404, {"status": "error", "reason": "no-chat", "say": f"{self.id} keeps no lobby."}+ if self.secret or not self.open_follows:+ return 403, {"status": "error", "reason": "closed", "say": "This hub links only with hubs its operator chose."}+ problem = verify(pair, "POST", "/inbox", activity, h)+ if problem:+ return 401, {"status": "error", "reason": "unsigned", "say": problem}+ try:+ status, doc = http_json("GET", f"{actor}/actor", timeout=10.0)+ except Exception as e:+ return 400, {"status": "error", "reason": "no-actor", "say": f"{actor}/actor did not answer: {e}"}+ if status != 200 or not isinstance(doc, dict) or doc.get("id", "").rstrip("/") != actor or doc.get("inbox") != inbox:+ return 400, {"status": "error", "reason": "no-actor", "say": f"{actor}/actor does not describe this hub."}+ return self.chat.accept_link(Remote(actor, inbox, pair, str(activity.get("name") or doc.get("name") or "")[:80], state="accepted", last_ok=time.time(), kind="hub"))+def _on_follow(self, actor: str, activity: dict[str, Any], h: dict[str, str], trusted: bool = False, *, trusted_by: str | None = None, signer: Any = None) -> tuple[int, dict[str, Any]]: pair = activity.get("secret") inbox = activity.get("inbox") diff --git a/src/pyrtan/hub.py b/src/pyrtan/hub.py index 615851b..005a230 100644 --- a/src/pyrtan/hub.py +++ b/src/pyrtan/hub.py@@ -60,11 +60,12 @@ import threadingfrom pathlib import Path import time from dataclasses import dataclass, field-from typing import Any+from typing import Any, Iterable, Iteratorfrom . import __version__ from .chat import Lobby, is_chat_path-from .federation import MAX_HOPS, Federation, Remote, bootstrap_candidates, default_network, first_live_hub, parse_invite, sign+from .lan import Lan+from .federation import MAX_HOPS, Federation, Remote, default_network, first_live_hub, parse_invite, rendezvous_hubs, sign, snapshot_hubsfrom . import identity from .grants import Grants, ability_for, parse_may from .identity import IdentityError@@ -145,13 +146,14 @@ class _HubResolver:class Hub:- def __init__(self, *, name: str = "hub", poll_every: float = 10.0, token: str | None = None, local: Service | None = None, handout: Registry | None = None, unanswered: str | Path | None = None, chat: bool = False, chat_typing: bool = True, chat_public: bool = False, grants: Grants | None = None, audience: str | None = None, suggestions: str | Path | None = None) -> None:+ def __init__(self, *, name: str = "hub", poll_every: float = 10.0, token: str | None = None, local: Service | None = None, handout: Registry | None = None, unanswered: str | Path | None = None, chat: bool = False, chat_typing: bool = True, chat_public: bool = False, grants: Grants | None = None, audience: str | None = None, chat_reach: int = 0, suggestions: str | Path | None = None) -> None:self.name = name self.grants = grants if grants is not None else Grants() self.grants.on_change = self._grant_changed self._audience = audience- self.lobby = Lobby(self, typing=chat_typing) if chat or chat_public else None+ self.lobby = Lobby(self, typing=chat_typing, reach=chat_reach) if chat or chat_public else Noneself.chat_public = chat_public # callers without the token may chat, with people only+ self.lan: Lan | None = None # hubs heard on the local network (--lan)self.handout_registry = handout # trees for trusted members only: this hub hands them out and does not run them self.poll_every = poll_every self.token = token or None@@ -303,6 +305,8 @@ class Hub:if self.federation is not None: self.federation.refresh_all() self.federation.announce() # only when what we hold changed+ if self.lobby is not None:+ self.lobby.reach_out() # chat links, for chat only (--chat-reach)def start_polling(self) -> None: def loop() -> None:@@ -312,8 +316,18 @@ class Hub:self._thread = threading.Thread(target=loop, daemon=True) self._thread.start()+ def on_lan(self) -> Lan:+ """Announce this hub on the LAN and listen for others (``--lan``)."""+ fed = self.federation+ assert fed is not None, "a hub is federated before it goes on the LAN"+ self.lan = Lan(lambda: {"id": fed.id, "name": self.name, "networks": sorted(n.id for n in fed.networks.values() if n.public)}).start()+ fed.lan = self.lan+ return self.lan+def stop(self) -> None: self._stop.set()+ if self.lan is not None:+ self.lan.stop()if self.local is not None: self.local.service.stop()@@ -576,10 +590,14 @@ class Hub:out: dict[str, Any] = {"peers": ([self.local.to_dict()] if self.local else []) + [p.to_dict() for p in self.peers.values()]} if fed is not None: out.update(fed.to_dict())+ if self.lobby is not None and self.lobby.reach:+ out["chat_links"] = self.lobby.links_view()return 200, out if fed is not None: if method == "GET" and path == "/actor": return 200, fed.actor()+ if method == "GET" and path == "/lan":+ return (200, self.lan.view()) if self.lan is not None else (404, {"status": "error", "reason": "no-lan", "say": "This hub is not on the LAN; start it with --lan."})if method == "GET" and path == "/directory": # A private network's part only to the operator (or to anyone, on a hub with no token). return 200, fed.directory(private=not self.token or operator)@@ -702,7 +720,7 @@ class Hub:if self.lobby is not None: endpoints += ["/chat", "/chat/start", "/chat/poll", "/chat/say", "/chat/typing", "/chat/settings", "/chat/next", "/chat/leave"] if fed is not None:- endpoints += ["/actor", "/inbox", "/mailbox", "/directory", "/reports", "/follow", "/unfollow", "/networks", "/join", "/leave", "/listed", "/following", "/followers", "/outbox", "/keep", "/fetch", "/kept/{key}"]+ endpoints += ["/actor", "/inbox", "/mailbox", "/directory", "/lan", "/reports", "/follow", "/unfollow", "/networks", "/join", "/leave", "/listed", "/following", "/followers", "/outbox", "/keep", "/fetch", "/kept/{key}"]return 200, {"pyrtan": "hub", "name": self.name, "version": __version__, "auth": "bearer" if self.token else "none", "federation": fed.id if fed else None, "chat": self.lobby is not None, "chat_public": self.lobby is not None and self.chat_public, "audience": self.audience, "grants": len(self.grants), "chat_bots": self.lobby is not None and self._trusted(headers, may), "endpoints": endpoints} if method == "POST" and path == "/register": url = body.get("url")@@ -727,13 +745,26 @@ def _rank(peer: Local | Peer | Remote) -> int:def join_network(fed: Federation, invite: Any) -> tuple[int, dict[str, Any]]: """Join a network from an invite (``name:secret[@hub]``), or the default- network pyrtan.com publishes when ``invite`` is empty or ``default``,- and follow a hub in it. When the default network cannot be read, the- first listed hub that answers is followed outside any network, as- before networks."""+ network (built in) when ``invite`` is empty or ``default``, and follow a+ hub in it: for the default network, the hubs this one already knows in+ it first, then the hub list, then the snapshot, each fetched only when+ the ones before it did not do. When ``$PYRTAN_NETWORK`` names a default+ network that cannot be read, the first listed hub that answers is+ followed outside any network, as before networks."""+ tried: list[str] = []if invite in (None, "", "default"): net = default_network()- hubs = bootstrap_candidates()+ if net is not None:+ known = net.id++ def candidates() -> Iterator[str]:+ for source in (lambda: fed.known_hubs(known), rendezvous_hubs, snapshot_hubs):+ for url in source():+ if url not in tried and url != fed.id:+ tried.append(url)+ yield url++ hubs: Iterable[str] = candidates()if net is None: url = first_live_hub() if url is None:@@ -749,12 +780,13 @@ def join_network(fed: Federation, invite: Any) -> tuple[int, dict[str, Any]]:except ValueError as e: return 400, {"status": "error", "reason": "bad-invite", "say": str(e)} hubs = [url] if url else []+ tried.extend(h for h in hubs if h.rstrip("/") != fed.id)try: remote = fed.join(net, hubs) except ValueError as e: return 400, {"status": "error", "reason": "bad-network", "say": str(e)} net = fed.networks[net.name]- hubs = [h for h in hubs if h.rstrip("/") != fed.id]+ hubs = triedif remote is not None: say = f"in {net.name}, following {remote.url}" elif hubs:@@ -792,6 +824,8 @@ def serve_hub(report_gaps: bool = False, unanswered: str | Path | None = None, watch: bool = False,+ lan: bool = False,+ chat_reach: int = 0,chat: bool = False, suggestions: str | Path | None = None, allow: list[str] | None = None,@@ -811,10 +845,10 @@ def serve_hub(for who, may in specs: granted.allow(who, may) local = Service(Registry.from_paths(trees), name=f"{name}", env=env, gaps=gaps, egress=egress, watch=watch) if trees else None- hub = Hub(name=name, poll_every=poll_every, token=token, local=local, handout=Registry.from_paths(handout) if handout else None, unanswered=unanswered, chat=chat, chat_typing=chat_typing, chat_public=chat_public, grants=granted, suggestions=suggestions)+ hub = Hub(name=name, poll_every=poll_every, token=token, local=local, handout=Registry.from_paths(handout) if handout else None, unanswered=unanswered, chat=chat, chat_typing=chat_typing, chat_public=chat_public, grants=granted, chat_reach=chat_reach, suggestions=suggestions)server = Server(hub.route, host, port, quiet, token=token, cors=cors, federated=federate or bool(advertise), public=is_chat_path if chat_public else None, keyed=True).start() hub._audience = (advertise or server.url).rstrip("/")- if federate or advertise or join:+ if federate or advertise or join or lan or chat_reach:hub.federate(advertise or server.url, secret=federation_secret, state=state, open_follows=open_follows, listed=listed, store=store or (str(Path(state).with_name("store")) if state else None), replicas=replicas, report_gaps=report_gaps) for url in peers or []: hub.add_peer(url)@@ -823,8 +857,14 @@ def serve_hub(hub.federation.follow(url) # type: ignore[union-attr] except ValueError as e: print(f"pyrtan hub: cannot follow {url}: {e}", flush=True)+ if lan:+ hub.on_lan()+ if join and hub.lan is not None and hub.lan.error is None:+ hub.lan.wait(1.5) # the hubs nearby answer the first question in that time: join one of themfor invite in join or []: _, answer = join_network(hub.federation, invite) # type: ignore[arg-type] print(f"pyrtan hub: join {answer.get('network') or invite.split(':', 1)[0]}: {answer.get('say')}", flush=True)+ if join and hub.lan is not None:+ hub.lan.send() # now in its networks: say so at once, not at the next beaconhub.start_polling() return hub, server diff --git a/src/pyrtan/js/pyrtan.js b/src/pyrtan/js/pyrtan.js index cce53b4..5c15758 100644 --- a/src/pyrtan/js/pyrtan.js +++ b/src/pyrtan/js/pyrtan.js@@ -124,6 +124,7 @@return MISSING; } const pyQuote = (s) => encodeURIComponent(String(s)).replace(/[!'()*]/g, (c) => "%" + c.charCodeAt(0).toString(16).toUpperCase());+ const asText = (v) => (Array.isArray(v) || isObj(v) ? pyJson(v) : pyStr(v)); // a value as it reads inside a sentenceconst pyStr = (v) => (typeof v === "string" ? v : isBool(v) ? (v ? "True" : "False") : v === null || v === undefined ? "None" : Array.isArray(v) || isObj(v) ? pyrepr(v) : String(v)); const FILTERS = { url: (v) => pyQuote(pyStr(v)),@@ -137,6 +138,7 @@first: (v) => ((Array.isArray(v) || typeof v === "string") && v.length ? v[0] : MISSING), last: (v) => ((Array.isArray(v) || typeof v === "string") && v.length ? v[v.length - 1] : MISSING), length: (v) => (Array.isArray(v) || typeof v === "string" ? v.length : isObj(v) ? Object.keys(v).length : MISSING),+ join: (v) => (Array.isArray(v) ? v.map(asText).join(", ") : asText(v)),title: (v) => pyStr(v).toLowerCase().replace(/(^|[^a-z0-9'])([a-z])/gi, (m, pre, c) => pre + c.toUpperCase()).replace(/'([a-z])/gi, (m, c) => "'" + c.toUpperCase()), }; function renderValue(expr, scope) {@@ -161,8 +163,7 @@return template.replace(PLACEHOLDER, (m, expr) => { const value = renderValue(expr, scope); if (isMissing(value)) { if (strict) throw new TemplateError(`${m} resolves to nothing`); return ""; }- if (Array.isArray(value) || isObj(value)) return pyJson(value);- return pyStr(value);+ return asText(value);}); } if (Array.isArray(template)) return template.map((t) => render(t, scope, strict));@@ -589,7 +590,8 @@} function trim(text) { text = text.replace(/^[^\p{L}\p{N}]+|[\s?!.,;:]+$/gu, "");- text = text.replace(/\s*[,;!?]+\s*/g, " ");+ // A comma between digits (1,000) is part of a number and stays.+ text = text.replace(/\s*[,;!?]+\s*/g, (m, at, s) => (m === "," && /[0-9]/.test(s[at - 1] || "") && /[0-9]/.test(s[at + 1] || "") ? m : " "));text = text.replace(/(?<=[A-Za-z])\.(?=\s|$)/g, ""); return text.replace(/\s+/gu, " ").trim(); } diff --git a/src/pyrtan/lan.py b/src/pyrtan/lan.py new file mode 100644 index 0000000..c1a4db7 --- /dev/null +++ b/src/pyrtan/lan.py@@ -0,0 +1,210 @@+"""The LAN: hubs on the same network find each other, with nobody to ask.++A hub started ``--lan`` says it is here, on a UDP multicast group that+never leaves the local network (TTL 1): its federation id (the URL other+hubs reach it at), its name, and the public networks it is in. It says+so when it starts, with a question that makes every other hub on the LAN+answer at once, and again every thirty seconds. It keeps what it hears+for three minutes. Joining a network then tries the hubs it heard on the+LAN in that network right after the ones it already follows, before its+directory, the hub list or the snapshot: two boxes in one house find+each other with the internet unplugged.++A beacon is a hint, exactly as a directory entry is: anyone on the LAN+can send one, and nothing is routed to a hub because it said it was+there. Following it is still the one act of trust, and the follow checks+the hub's ``/actor`` and is signed with the network's secret like any+other. Beacons carry no secret and no private network: only public+network ids, so a private network's hubs are not named to the LAN.++A hub announces itself only when its id is an address others can reach:+not ``127.0.0.1``, ``localhost`` or ``0.0.0.0``. Give it one with+``--advertise http://<its LAN address>:8700``. In Docker, multicast+needs ``network_mode: host``.++Standard library only. ``$PYRTAN_LAN_GROUP`` (``ADDRESS:PORT``) moves the+group, which is how the tests keep to themselves.+"""++from __future__ import annotations++import json+import os+import socket+import struct+import threading+import time+from typing import Any, Callable+from urllib.parse import urlsplit++GROUP = "239.255.70.84" # administratively scoped: stays on the site+PORT = 8770+EVERY = 30.0 # seconds between beacons+HEARD_FOR = 180.0 # a hub not heard for this long is forgotten+HEARD_MAX = 64 # hubs kept at most+ANSWER_EVERY = 2.0 # a question is answered at most this often+BEACON_MAX = 2048 # bytes+UNREACHABLE = {"127.0.0.1", "localhost", "0.0.0.0", "::1", "::"}+++def group() -> tuple[str, int]:+ """The multicast group, from ``$PYRTAN_LAN_GROUP`` or the default."""+ text = os.environ.get("PYRTAN_LAN_GROUP", "")+ host, _, port = text.rpartition(":")+ try:+ return (host, int(port)) if host and port else (GROUP, PORT)+ except ValueError:+ return GROUP, PORT+++def reachable(url: str) -> bool:+ """Whether another box could reach a hub at ``url``."""+ try:+ parts = urlsplit(url)+ except ValueError:+ return False+ return parts.scheme in ("http", "https") and bool(parts.hostname) and parts.hostname not in UNREACHABLE and not parts.hostname.startswith("127.")+++class Lan:+ """Announce this hub on the LAN and keep what other hubs announce.++ ``me`` returns what to say (``{"id", "name", "networks"}``), or None+ to stay quiet; it is asked each time, so a hub that joins a network+ says so in its next beacon."""++ def __init__(self, me: Callable[[], dict[str, Any] | None], *, every: float = EVERY, address: tuple[str, int] | None = None) -> None:+ self.me = me+ self.every = every+ self.address = address or group()+ self.heard: dict[str, dict[str, Any]] = {}+ self.error: str | None = None+ self._answered = 0.0+ self._lock = threading.Lock()+ self._stop = threading.Event()+ self._sock: socket.socket | None = None+ self._threads: list[threading.Thread] = []++ # ------------------------------------------------------------ what is said++ def beacon(self, *, query: bool = False) -> bytes | None:+ """What this hub says; None when it has nothing to say. A hub that+ cannot be reached still asks who is there: a question with no id."""+ me = self.me()+ if not me or not reachable(str(me.get("id", ""))):+ return json.dumps({"pyrtan-lan": 1, "query": True}).encode("utf-8") if query else None+ body = {"pyrtan-lan": 1, "id": str(me["id"]).rstrip("/"), "name": str(me.get("name", ""))[:80], "networks": [n for n in me.get("networks", []) if isinstance(n, str)][:16]}+ if query:+ body["query"] = True+ data = json.dumps(body, separators=(",", ":")).encode("utf-8")+ return data if len(data) <= BEACON_MAX else None++ def take(self, data: bytes, sender: str, now: float | None = None) -> bool | None:+ """A datagram heard from ``sender``: True when it named a hub (and+ False when it named this one); None when it was not a beacon. A+ question in it is answered, at most every couple of seconds."""+ now = time.time() if now is None else now+ try:+ body = json.loads(data[:BEACON_MAX].decode("utf-8"))+ except (UnicodeDecodeError, json.JSONDecodeError):+ return None+ if not isinstance(body, dict) or body.get("pyrtan-lan") != 1:+ return None+ me = self.me() or {}+ hub = body["id"].rstrip("/") if isinstance(body.get("id"), str) else None+ if hub == str(me.get("id", "")).rstrip("/"):+ return False+ if body.get("query") is True and now - self._answered >= ANSWER_EVERY:+ self._answered = now+ self.send()+ if hub is None or not reachable(hub) or len(hub) > 200:+ return None+ networks = [n for n in body.get("networks", []) if isinstance(n, str) and len(n) <= 64][:16] if isinstance(body.get("networks"), list) else []+ with self._lock:+ self.heard[hub] = {"id": hub, "name": str(body.get("name", ""))[:80], "networks": networks, "from": sender, "seen": now}+ if len(self.heard) > HEARD_MAX:+ for old in sorted(self.heard.values(), key=lambda e: e["seen"])[: len(self.heard) - HEARD_MAX]:+ self.heard.pop(old["id"], None)+ return True++ def hubs(self, network: str | None = None, now: float | None = None) -> list[str]:+ """The hubs heard lately, most recent first; ``network`` keeps the+ ones that said they are in it."""+ now = time.time() if now is None else now+ with self._lock:+ fresh = [e for e in self.heard.values() if now - e["seen"] < HEARD_FOR and (network is None or network in e["networks"])]+ return [e["id"] for e in sorted(fresh, key=lambda e: -e["seen"])]++ def view(self) -> dict[str, Any]:+ now = time.time()+ with self._lock:+ rows = [{**e, "age_s": int(now - e["seen"])} for e in self.heard.values() if now - e["seen"] < HEARD_FOR]+ me = self.me() or {}+ return {"group": f"{self.address[0]}:{self.address[1]}", "announcing": self.beacon() is not None, "id": me.get("id"), "error": self.error, "hubs": sorted(rows, key=lambda e: e["age_s"])}++ # ------------------------------------------------------------ the socket++ def send(self, *, query: bool = False) -> bool:+ """Say this hub is here (``query``: and ask who else is)."""+ data = self.beacon(query=query)+ if data is None or self._sock is None:+ return False+ try:+ self._sock.sendto(data, self.address)+ return True+ except OSError as e:+ self.error = str(e)+ return False++ def start(self) -> "Lan":+ """Join the group, ask who is there, and keep listening and+ announcing in the background. A LAN that will not have it (no+ multicast route, the port refused) leaves ``error`` set and the+ hub running without it."""+ host, port = self.address+ try:+ s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)+ s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)+ if hasattr(socket, "SO_REUSEPORT"):+ s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1) # several hubs on one box share the port+ s.bind(("", port))+ s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, struct.pack("4sl", socket.inet_aton(host), socket.INADDR_ANY))+ s.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 1) # never past the first router+ s.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_LOOP, 1) # hubs on this same box hear it too+ s.settimeout(1.0)+ except OSError as e:+ self.error = f"cannot join {host}:{port}: {e}"+ return self+ self._sock = s+ self._threads = [threading.Thread(target=self._listen, daemon=True), threading.Thread(target=self._announce, daemon=True)]+ for t in self._threads:+ t.start()+ return self++ def wait(self, seconds: float) -> None:+ """Give the answers to the first question a moment to arrive."""+ self._stop.wait(seconds)++ def _listen(self) -> None:+ while not self._stop.is_set() and self._sock is not None:+ try:+ data, (sender, _) = self._sock.recvfrom(BEACON_MAX + 1)+ except socket.timeout:+ continue+ except OSError:+ return+ self.take(data, sender)++ def _announce(self) -> None:+ self.send(query=True)+ while not self._stop.wait(self.every):+ self.send()++ def stop(self) -> None:+ self._stop.set()+ if self._sock is not None:+ try:+ self._sock.close()+ except OSError:+ pass+ self._sock = Nonediff --git a/src/pyrtan/nlp.py b/src/pyrtan/nlp.py index 34e33f6..a2120b1 100644 --- a/src/pyrtan/nlp.py +++ b/src/pyrtan/nlp.py@@ -92,6 +92,7 @@ DURATION_UNITS = {"w": 604800, "wk": 604800, "wks": 604800, "week": 604800, "weeks": 604800, }+DIGITS = "0123456789"WORD = re.compile(r"[A-Za-z][A-Za-z'’]*|\d+(?:[.,]\d+)*|\S") HOST_RE = re.compile(r"^(?:https?://)?((?:[a-z0-9](?:[a-z0-9-]{0,61}[a-z0-9])?\.)+[a-z]{2,63}|localhost|(?:\d{1,3}\.){3}\d{1,3})(?::\d{1,5})?(?:/\S*)?$", re.IGNORECASE) URL_RE = re.compile(r"^https?://[^\s<>\"']+$", re.IGNORECASE)@@ -113,13 +114,20 @@ def fold(text: str) -> str:def trim(text: str) -> str: """Drop punctuation that carries nothing: around the whole sentence, and- a comma or full stop between words."""+ a comma or full stop between words. A comma between digits (1,000) is+ part of a number and stays."""text = re.sub(r"^[\s\W_]+|[\s?!.,;:]+$", "", text)- text = re.sub(r"\s*[,;!?]+\s*", " ", text)+ text = re.sub(r"\s*[,;!?]+\s*", lambda m: m.group(0) if _in_number(m) else " ", text)text = re.sub(r"(?<=[A-Za-z])\.(?=\s|$)", "", text) return re.sub(r"\s+", " ", text).strip()+def _in_number(m: re.Match[str]) -> bool:+ s, i, j = m.string, m.start(), m.end()+ return m.group(0) == "," and 0 < i and j < len(s) and s[i - 1] in DIGITS and s[j] in DIGITS+++def strip_fillers(text: str, keep: frozenset[str] = frozenset()) -> tuple[str, list[str]]: """Greetings, politeness and request wrappers at either edge, repeatedly. A filler whose first word is in ``keep`` stays: a trigger somewhere diff --git a/src/pyrtan/page.py b/src/pyrtan/page.py index 1e9a979..58894d3 100644 --- a/src/pyrtan/page.py +++ b/src/pyrtan/page.py@@ -96,6 +96,15 @@ PAGE = r"""<!doctype html>.status { font-weight:600 } .done{color:var(--ok)} .failed,.error{color:var(--bad)} .asked{color:var(--ask)} .say { font-size:18px; margin:6px 0 } pre { margin:8px 0 0; padding:10px 12px; background:var(--bg); border:1px solid var(--line); border-radius:8px; overflow:auto; font:13px/1.45 ui-monospace,SFMono-Regular,Menlo,monospace; white-space:pre-wrap }+ .suggest { position:relative; flex:1 1 180px; min-width:0 }+ form.row .suggest input[type=text] { position:relative; width:100%; background:transparent }+ .suggest .ghost { position:absolute; inset:0; padding:10px 12px; border:1px solid transparent; border-radius:8px; background:var(--card); white-space:pre; overflow:hidden; pointer-events:none }+ .suggest .ghost .typed { visibility:hidden } .suggest .ghost .rest { color:var(--mute) }+ .suggestions { position:absolute; z-index:10; left:0; right:0; top:calc(100% + 4px); list-style:none; margin:0; padding:4px 0; background:var(--card); border:1px solid var(--line); border-radius:8px; box-shadow:0 6px 18px rgba(0,0,0,.12); max-height:320px; overflow-y:auto }+ .suggestions li { padding:7px 12px; cursor:pointer; display:flex; gap:8px; align-items:baseline }+ .suggestions li[aria-selected=true], .suggestions li:hover { background:var(--soft) }+ .suggestions li b { font-weight:600 } .suggestions li .text { flex:1; min-width:0; overflow-wrap:anywhere }+ .suggestions li .from { color:var(--mute); font-size:13px }details summary { cursor:pointer; color:var(--mute) } ul.tries { list-style:none; padding:0; margin:0; display:flex; flex-wrap:wrap; gap:8px } ul.tries li a { display:inline-block; padding:6px 10px; border:1px solid var(--line); border-radius:999px; background:var(--card); color:var(--fg); text-decoration:none; font-size:14px }@@ -164,7 +173,11 @@ PAGE = r"""<!doctype html><section id="tab-ask"> <form class="row" id="ask">- <input type="text" id="text" placeholder="what's the weather in Oslo" autofocus autocomplete="off">+ <div class="suggest">+ <div class="ghost" id="ghost" aria-hidden="true"><span class="typed"></span><span class="rest"></span></div>+ <input type="text" id="text" placeholder="what's the weather in Oslo" autofocus autocomplete="off" spellcheck="false" role="combobox" aria-autocomplete="both" aria-expanded="false" aria-controls="suggestions">+ <ul class="suggestions" id="suggestions" role="listbox" aria-label="Suggestions" hidden></ul>+ </div><button type="submit" id="go">Ask</button> </form> <div class="card" style="margin-bottom:24px">@@ -380,6 +393,17 @@ const examples = {"site-status": "is example.com up", "github-release": "latest release of astral-sh/uv", "hacker-news-item": "hacker news item 1",+ "wikipedia": "tell me about Ada Lovelace",+ "define": "what does serendipity mean",+ "sun": "when is sunset in Oslo",+ "air-quality": "air quality in Delhi",+ "npm-package": "latest version of react on npm",+ "pypi-package": "latest requests on pypi",+ "hn-search": "search hacker news for sqlite",+ "books": "who wrote The Left Hand of Darkness",+ "holidays": "next holiday in US",+ "exchange-rate": "convert 100 USD to EUR",+ "synonyms": "synonyms for happy",}; // Served by a Pyrtan, only its token is kept, under its own origin; served elsewhere, the hub URL and token. const KEY = "pyrtan.console:" + location.origin;@@ -491,10 +515,18 @@ async function loadDirectory() {} window.addEventListener("hashchange", () => showTab((location.hash || "#ask").slice(1)));+// A trigger as a sentence: the first of each choice, an input's default or "…"+// for its blank, and the optional words kept or dropped.+function phrase(t, pat, optional) {+ return pat.replace(/\[([^\]|]*)[^\]]*\]/g, optional ? "$1" : "").replace(/\(([^|)]*)\|[^)]*\)/g, "$1").replace(/\{(\w+)(?::\w+)?\}/g, (m, n) => (t.inputs && t.inputs[n] && t.inputs[n].default != null) ? t.inputs[n].default : "…").replace(/\s+/g, " ").trim();+}function sample(t) { if (examples[t.name]) return examples[t.name];- const pat = (t.triggers || [])[0]; if (!pat) return null;- return pat.replace(/\[[^\]]*\]/g, "").replace(/\((\w+)\|[^)]*\)/g, "$1").replace(/\{(\w+)(?::\w+)?\}/g, (m, n) => (t.inputs && t.inputs[n] && t.inputs[n].default != null) ? t.inputs[n].default : "…").replace(/\s+/g, " ").trim();+ const pat = (t.triggers || [])[0]; return pat ? phrase(t, pat, false) : null;+}+// Every sentence a tree answers to, for the ask box to suggest; one that starts with a blank suggests nothing.+function samples(t) {+ return [examples[t.name], ...(t.triggers || []).map(p => phrase(t, p, true))].filter(s => s && !s.startsWith("…"));} async function detect() {@@ -620,6 +652,7 @@ async function load() {function renderCaps(c) { const body = $("#caps tbody"); const tries = $("#tries"); body.innerHTML = ""; tries.innerHTML = "";+ suggest.from((c.trees || []).filter(t => !local || (runsHere(t) && !anySite(t))));for (const t of c.trees || []) { const tr = document.createElement("tr"); const here = !local ? "" : !runsHere(t) ? `<div class="gap">needs a hub: ${esc((t.sources||[]).filter(x => x !== "http").join(", "))}</div>` : anySite(t) ? '<div class="gap">any site; most will not let a page read them</div>' : "";@@ -627,6 +660,9 @@ function renderCaps(c) {body.appendChild(tr); const s = sample(t); if (s && !s.includes("…") && (!local || (runsHere(t) && !anySite(t)))) { const li = document.createElement("li"); li.innerHTML = `<a href="#" data-q="${esc(s)}">${esc(s)}</a>`; tries.appendChild(li); } }+ // The greyed suggestion in the empty box (Tab takes it) is one this swarm answers.+ const first = tries.querySelector("a[data-q]"), offered = [...tries.querySelectorAll("a[data-q]")].map(a => a.dataset.q);+ if (first && !offered.includes($("#text").placeholder)) $("#text").placeholder = first.dataset.q;} async function loadPeers() {@@ -703,6 +739,7 @@ function renderTrace(tr) {async function ask(snapshot, answer) { const text = $("#text").value.trim(); if (!text && !snapshot) return;+ if (!snapshot) { suggest.remember(text); suggest.close(); }$("#go").disabled = true; const t0 = performance.now(); let r;@@ -735,6 +772,105 @@ async function ask(snapshot, answer) {} $("#ask").addEventListener("submit", (e) => { e.preventDefault(); ask(); });++// ---------------------------------------------------------------- suggestions in the ask box+// The sentences this swarm's trees answer to, and the ones asked in this+// browser before. The best that starts with what is typed shows greyed after+// it (Tab or → takes it; Tab in the empty box takes the placeholder), and up to+// eight drop down under the box: ↑ ↓ pick, Enter asks, Esc closes. A sentence+// with a blank ("latest release of …") fills the box up to the blank instead of asking.+const ASKED_KEY = "pyrtan.console.asked";+const suggest = {+ pool: [], items: [], active: -1, typed: "", open: false,+ input: $("#text"), list: $("#suggestions"),+ recent() { try { const r = JSON.parse(localStorage.getItem(ASKED_KEY) || "[]"); return Array.isArray(r) ? r.filter(s => typeof s === "string") : []; } catch (e) { return []; } },+ remember(text) {+ const r = [text, ...this.recent().filter(s => s.toLowerCase() !== text.toLowerCase())].slice(0, 20);+ try { localStorage.setItem(ASKED_KEY, JSON.stringify(r)); } catch (e) {}+ },+ from(trees) {+ const seen = new Set(); this.pool = [];+ for (const t of trees) for (const text of samples(t)) { const k = text.toLowerCase(); if (!seen.has(k)) { seen.add(k); this.pool.push({text, tree: t.name}); } }+ },+ // Recent first, then the trees': those that start with what is typed, then those with every typed word starting one of theirs.+ match(q) {+ const low = q.toLowerCase(), words = low.split(/\s+/).filter(Boolean), seen = new Set(), out = [];+ const add = (e) => { const k = e.text.toLowerCase(); if (!seen.has(k) && out.length < 8) { seen.add(k); out.push(e); } };+ const all = [...this.recent().map(text => ({text, recent: true})), ...this.pool];+ for (const e of all) if (e.text.toLowerCase().startsWith(low)) add(e);+ for (const e of all) { const ws = e.text.toLowerCase().split(/\s+/); if (words.every(w => ws.some(x => x.startsWith(w)))) add(e); }+ return out;+ },+ fill(e) { const i = e.text.indexOf("…"); return i < 0 ? e.text : e.text.slice(0, i); },+ // The rest of the first suggestion that starts with what is typed, while the caret is at the end.+ completion() {+ const el = this.input, v = el.value, low = v.toLowerCase();+ if (!v || document.activeElement !== el || el.selectionStart !== v.length || el.selectionEnd !== v.length) return "";+ const e = this.items.find(e => { const f = this.fill(e); return f.length > v.length && f.toLowerCase().startsWith(low); });+ return e ? this.fill(e).slice(v.length) : "";+ },+ ghost() {+ const rest = this.input.scrollWidth <= this.input.clientWidth ? this.completion() : ""; // scrolled text would not line up+ $("#ghost .typed").textContent = rest ? this.input.value : "";+ $("#ghost .rest").textContent = rest;+ },+ render() {+ const v = this.typed, low = v.toLowerCase();+ this.list.innerHTML = "";+ this.items.forEach((e, i) => {+ const li = document.createElement("li");+ li.id = "suggestion-" + i; li.setAttribute("role", "option"); li.setAttribute("aria-selected", String(i === this.active));+ const head = v && e.text.toLowerCase().startsWith(low) ? v.length : v ? 0 : e.text.length; // what is typed plain, the rest bold+ li.innerHTML = `<span class="text">${esc(e.text.slice(0, head))}<b>${esc(e.text.slice(head))}</b></span><span class="from">${e.recent ? "recent" : esc(e.tree)}</span>`;+ li.addEventListener("click", () => this.choose(e));+ this.list.appendChild(li);+ });+ const shown = this.open && this.items.length > 0;+ this.list.hidden = !shown; this.input.setAttribute("aria-expanded", String(shown));+ if (shown && this.active >= 0) this.input.setAttribute("aria-activedescendant", "suggestion-" + this.active); else this.input.removeAttribute("aria-activedescendant");+ },+ refresh() {+ this.typed = this.input.value; this.active = -1;+ this.items = this.open ? this.match(this.typed) : [];+ this.render(); this.ghost();+ },+ close() { this.open = false; this.refresh(); },+ choose(e) {+ this.input.value = this.fill(e); this.input.focus();+ if (e.text.includes("…")) { this.open = true; this.refresh(); } else ask();+ },+ key(ev) {+ if (ev.isComposing || ev.ctrlKey || ev.altKey || ev.metaKey) return;+ const shown = !this.list.hidden;+ if (ev.key === "ArrowDown" || ev.key === "ArrowUp") {+ ev.preventDefault();+ if (!shown) { this.open = true; this.refresh(); return; }+ // -1 is what was typed: the list wraps through it, as a search box's does.+ const n = this.items.length + 1;+ this.active = ((this.active + 1 + (ev.key === "ArrowDown" ? 1 : -1)) % n + n) % n - 1;+ this.input.value = this.active < 0 ? this.typed : this.fill(this.items[this.active]);+ this.render(); this.ghost();+ } else if ((ev.key === "Tab" && !ev.shiftKey) || ev.key === "ArrowRight") {+ const v = this.input.value, rest = this.completion();+ const take = rest ? v + rest : ev.key === "Tab" && !v ? this.input.placeholder : "";+ if (!take) return; // nothing to complete: Tab moves on, → moves the caret+ ev.preventDefault(); this.input.value = take; this.open = true; this.refresh();+ } else if (ev.key === "Escape" && shown) {+ ev.preventDefault(); this.input.value = this.typed; this.close();+ } else if (ev.key === "Enter" && shown && this.active >= 0 && this.items[this.active].text.includes("…")) {+ ev.preventDefault(); this.choose(this.items[this.active]);+ }+ },+};+suggest.input.addEventListener("input", () => { suggest.open = true; suggest.refresh(); });+suggest.input.addEventListener("keydown", (e) => suggest.key(e));+suggest.input.addEventListener("keyup", () => suggest.ghost());+suggest.input.addEventListener("mousedown", () => { if (!suggest.open) { suggest.open = true; suggest.refresh(); } });+suggest.input.addEventListener("click", () => suggest.ghost());+suggest.input.addEventListener("focus", () => suggest.ghost());+suggest.input.addEventListener("blur", () => suggest.close());+suggest.list.addEventListener("mousedown", (e) => e.preventDefault()); // keep the box focused, so the click lands+window.addEventListener("resize", () => suggest.ghost());$("#tries").addEventListener("click", (e) => { const a = e.target.closest("a[data-q]"); if (!a) return; e.preventDefault(); $("#text").value = a.dataset.q; location.hash = "#ask"; ask(); }); $("#connect-form").addEventListener("submit", (e) => { e.preventDefault();@@ -1119,7 +1255,7 @@ const agentsUI = {} let kept = ""; try { const out = await this.agent.send([item]); kept = (out.refused || []).length ? `refused by the hub: ${out.refused[0].why}` : `kept at ${this.where()}'s /suggestions`; }- catch (e) { kept = `not kept: ${e.message}${/404/.test(e.message) ? " (a hub older than 0.15 has no /suggestions; download it instead)" : ""}`; }+ catch (e) { kept = `not kept: ${e.message}${/404/.test(e.message) ? " (a hub older than 0.18 has no /suggestions; download it instead)" : ""}`; }this.agent.decide(id, "accepted"); item.note = [applied, kept].filter(Boolean).join(" · "); if (applied) { try { renderCaps(await json("capabilities")); await this.agent.trees(true); } catch (e) {} } diff --git a/src/pyrtan/template.py b/src/pyrtan/template.py index 8288ac0..4b55d69 100644 --- a/src/pyrtan/template.py +++ b/src/pyrtan/template.py@@ -85,6 +85,13 @@ def _get_index(cur: Any, index: int) -> Any:return MISSING+def _text(value: Any) -> str:+ """A value as it reads inside a sentence: a list or table as JSON, the rest as itself."""+ if isinstance(value, (dict, list)):+ return json.dumps(value, ensure_ascii=False)+ return str(value)++FILTERS = { "url": lambda v: urllib.parse.quote(str(v), safe=""), "json": lambda v: json.dumps(v, ensure_ascii=False),@@ -98,6 +105,8 @@ FILTERS = {"last": lambda v: v[-1] if isinstance(v, (list, tuple, str)) and v else MISSING, "length": lambda v: len(v) if hasattr(v, "__len__") else MISSING, "title": lambda v: str(v).title(),+ # A list in a sentence: "a, b, c". Its items read as they would on their own.+ "join": lambda v: ", ".join(_text(x) for x in v) if isinstance(v, (list, tuple)) else _text(v),}@@ -143,9 +152,7 @@ def render(template: Any, scope: dict[str, Any], *, strict: bool = True) -> Any:if strict: raise TemplateError(f"{m.group(0)} resolves to nothing") return ""- if isinstance(value, (dict, list)):- return json.dumps(value, ensure_ascii=False)- return str(value)+ return _text(value)return PLACEHOLDER.sub(sub, template) if isinstance(template, list): diff --git a/tests/conformance/make_vectors.py b/tests/conformance/make_vectors.py index d8014cb..f1ee1f4 100644 --- a/tests/conformance/make_vectors.py +++ b/tests/conformance/make_vectors.py@@ -77,6 +77,36 @@ CASES: list[dict] = [{"intent": "how are you today?"}, {"intent": "are you a bot"}, {"intent": "goodbye"},+ # Trees that run in a browser too.+ {"intent": "tell me about Ursula K. Le Guin"},+ {"intent": "wiki Mercury"},+ {"intent": "who is Xyzzyplugh Notaperson"},+ {"intent": "who was Alan Turing"},+ {"intent": "what does serendipity mean"},+ {"intent": "define zzzqx"},+ {"intent": "when is sunset in Oslo"},+ {"intent": "sunrise in Longyearbyen"},+ {"intent": "air quality in Oslo"},+ {"intent": "how is the air in Delhi"},+ {"intent": "latest version of @types/node on npm"},+ {"intent": "npm package nope-not-a-pkg-zz"},+ {"intent": "latest requests on pypi"},+ {"intent": "pip version nope-not-a-pkg-zz"},+ {"intent": "any earthquakes today"},+ {"intent": "where is the iss"},+ {"intent": "search hacker news for sqlite"},+ {"intent": "search hacker news for qqqzzzxxyyy"},+ {"intent": "who wrote The Left Hand of Darkness"},+ {"intent": "find the book qqqzzzxxyyy"},+ {"intent": "next holiday in no"},+ {"intent": "next holiday in XX"},+ {"intent": "tell me a joke"},+ {"intent": "convert 100 usd to eur"},+ {"intent": "convert 1,000 EUR into JPY"},+ {"intent": "usd to xyz rate"},+ {"intent": "exchange rate from usd to usd"},+ {"intent": "synonyms for happy"},+ {"intent": "another word for qqqzzz"},]@@ -91,7 +121,7 @@ LANGUAGE_TREES: list[dict] = [_echo("remind", ["remind me {when:date} at {at:time}", "remind me in {wait:duration}"], {"when": {"type": "str", "default": ""}, "at": {"type": "str", "default": ""}, "wait": {"type": "int", "default": 0}}), _echo("count", ["count to {n:number}", "add {x:float} and {y:int}"], {"n": {"type": "float", "default": 0}, "x": {"type": "float", "default": 0.0}, "y": {"type": "int", "default": 0}}), _echo("ping", ["ping {host:host}", "fetch {url:url}", "mail {to:email}"], {"host": {"type": "str", "default": ""}, "url": {"type": "str", "default": ""}, "to": {"type": "str", "default": ""}}),- _echo("books", ["(find|search) books by {author}", "top story on book club"], {"author": {"type": "str", "default": ""}, "count": {"type": "int", "default": 3}},+ _echo("library", ["(find|search) books by {author}", "top story on book club"], {"author": {"type": "str", "default": ""}, "count": {"type": "int", "default": 3}},language={"synonyms": {"books": ["novels", "paperback books"], "Find": "look up"}, "typos": True}), _echo("tie-a", ["shout {x:word}"], {"x": {"type": "str", "default": ""}}), _echo("tie-b", ["shout {y:word}"], {"y": {"type": "str", "default": ""}}),@@ -124,6 +154,7 @@ NLP = [("clean", "Hey, can you please tell me what's up?"), ("clean", "“Curly” quotes — and dashes"), ("clean", "ＦＵＬＬ width"), ("clean", "I'm sure it's a dozen"), ("clean", "one hundred and twenty three things"), ("clean", "a hundred and one"), ("clean", "Thanks"), ("clean", "well, um, okay pyrtan: weather."), ("clean", "St. Ives. Then Mr. Smith"),+ ("clean", "convert 1,000.5 EUR, then 2, 3"), ("clean", "a,b and 1 ,2 and 3, 4"),("singular", "stories"), ("singular", "boxes"), ("singular", "news"), ("singular", "glass"), ("singular", "cats"), ("singular", "bus"), ("plural", "story"), ("plural", "box"), ("plural", "day"), ("plural", "status"), ("parse_number", "twenty-one"), ("parse_number", "1,234"), ("parse_number", "-3.25"), ("parse_number", "lots"), diff --git a/tests/conformance/vectors.json b/tests/conformance/vectors.json index f961a4d..5cd9fdc 100644 --- a/tests/conformance/vectors.json +++ b/tests/conformance/vectors.json@@ -158,36 +158,65 @@"name": "this browser", "trees": [ {- "description": "echo for books",- "hosts": [],+ "description": "Current air quality for a place: the European AQI, its band, and PM2.5 and PM10, from Open-Meteo.",+ "hosts": [+ "geocoding-api.open-meteo.com",+ "air-quality-api.open-meteo.com"+ ],"inputs": {- "author": {+ "place": {+ "description": "A town or city name",+ "required": true,"type": "str"- },- "count": {- "default": 3,- "type": "int"} },- "language": {- "synonyms": {- "books": [- "novels",- "paperback books"- ],- "find": [- "look up"- ]- },- "typos": true+ "language": {},+ "license": "MIT",+ "name": "air-quality",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "weather",+ "health",+ "read-only"+ ],+ "triggers": [+ "[what's the] air quality (in|for) {place}",+ "how is the air in {place}",+ "(aqi|pollution) in {place}"+ ],+ "version": 1+ },+ {+ "description": "A book from Open Library: the best match for a title or an author, who wrote it, and when it first came out.",+ "hosts": [+ "openlibrary.org"+ ],+ "inputs": {+ "title": {+ "description": "A title or an author, as in: The Left Hand of Darkness",+ "required": true,+ "type": "str"+ }},- "license": "",+ "language": {},+ "license": "MIT","name": "books",- "sources": [],- "tags": [],+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "reference",+ "books",+ "read-only"+ ],"triggers": [- "(find|search) books by {author}",- "top story on book club"+ "who wrote {title}",+ "find the book {title}",+ "books by {title}"], "version": 1 },@@ -216,6 +245,61 @@], "version": 1 },+ {+ "description": "A random dad joke, from icanhazdadjoke.com.",+ "hosts": [+ "icanhazdadjoke.com"+ ],+ "inputs": {},+ "language": {},+ "license": "MIT",+ "name": "dad-joke",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "fun",+ "read-only"+ ],+ "triggers": [+ "tell me a [dad] joke",+ "[a] dad joke",+ "make me laugh"+ ],+ "version": 1+ },+ {+ "description": "What an English word means: its first definition, part of speech and pronunciation.",+ "hosts": [+ "api.dictionaryapi.dev"+ ],+ "inputs": {+ "word": {+ "description": "One English word",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "define",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "reference",+ "language",+ "read-only"+ ],+ "triggers": [+ "define {word:word}",+ "what does {word:word} mean",+ "[the] (meaning|definition) of {word:word}"+ ],+ "version": 1+ },{ "commands": [ "df"@@ -248,6 +332,71 @@], "version": 2 },+ {+ "description": "Magnitude 4.5+ earthquakes worldwide in the past day, from the USGS: how many, and the latest.",+ "hosts": [+ "earthquake.usgs.gov"+ ],+ "inputs": {},+ "language": {},+ "license": "MIT",+ "name": "earthquakes",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "science",+ "read-only"+ ],+ "triggers": [+ "[any] [recent] earthquakes [today]",+ "[was there an] earthquake today",+ "latest earthquake"+ ],+ "version": 1+ },+ {+ "description": "An amount in another currency at today's European Central Bank reference rate, from Frankfurter.",+ "hosts": [+ "api.frankfurter.dev"+ ],+ "inputs": {+ "amount": {+ "default": "1",+ "description": "How much of the first currency, as in: 100 or 2.5",+ "type": "str"+ },+ "from": {+ "description": "A three-letter currency code, as in: USD",+ "required": true,+ "type": "str"+ },+ "to": {+ "description": "A three-letter currency code, as in: EUR",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "exchange-rate",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "money",+ "read-only"+ ],+ "triggers": [+ "convert {amount:number} {from:word} (to|into|in) {to:word}",+ "how much is {amount:number} {from:word} in {to:word}",+ "[the] exchange rate (from|for) {from:word} to {to:word}",+ "{from:word} to {to:word} (rate|exchange rate)"+ ],+ "version": 1+ },{ "description": "echo for fixed", "hosts": [],@@ -363,6 +512,124 @@], "version": 2 },+ {+ "description": "Hacker News stories about something, through HN Search: the top three matches.",+ "hosts": [+ "hn.algolia.com"+ ],+ "inputs": {+ "query": {+ "description": "Words to search story titles and text for",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "hn-search",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "news",+ "tech",+ "read-only"+ ],+ "triggers": [+ "search hacker news for {query}",+ "hacker news stories about {query}",+ "what does hacker news say about {query}"+ ],+ "version": 1+ },+ {+ "description": "The next public holiday in a country (by its two-letter code), from Nager.Date.",+ "hosts": [+ "date.nager.at"+ ],+ "inputs": {+ "country": {+ "description": "A two-letter country code, as in: NO, US, DE",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "holidays",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "calendar",+ "read-only"+ ],+ "triggers": [+ "[when is the] next [public|bank] holiday in {country:word}",+ "[public|bank] holidays in {country:word}"+ ],+ "version": 1+ },+ {+ "description": "Where the International Space Station is right now: latitude, longitude, altitude and speed.",+ "hosts": [+ "api.wheretheiss.at"+ ],+ "inputs": {},+ "language": {},+ "license": "MIT",+ "name": "iss",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "science",+ "space",+ "read-only"+ ],+ "triggers": [+ "where is the (iss|space station) [now]",+ "(iss|space station) (position|location)"+ ],+ "version": 1+ },+ {+ "description": "echo for library",+ "hosts": [],+ "inputs": {+ "author": {+ "type": "str"+ },+ "count": {+ "default": 3,+ "type": "int"+ }+ },+ "language": {+ "synonyms": {+ "books": [+ "novels",+ "paperback books"+ ],+ "find": [+ "look up"+ ]+ },+ "typos": true+ },+ "license": "",+ "name": "library",+ "sources": [],+ "tags": [],+ "triggers": [+ "(find|search) books by {author}",+ "top story on book club"+ ],+ "version": 1+ },{ "description": "Error lines in the tail of a log file on this box.", "hosts": [],@@ -428,6 +695,37 @@], "version": 2 },+ {+ "description": "The latest version of an npm package, and what it says it is.",+ "hosts": [+ "registry.npmjs.org"+ ],+ "inputs": {+ "package": {+ "description": "A package name, as in: react or @types/node",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "npm-package",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "software",+ "javascript",+ "read-only"+ ],+ "triggers": [+ "[what's the] latest version of {package:word} on npm",+ "latest {package:word} on npm",+ "npm (package|version) {package:word}"+ ],+ "version": 1+ },{ "description": "asks, then answers", "hosts": [],@@ -478,9 +776,40 @@"version": 1 }, {- "description": "echo for remind",- "hosts": [],- "inputs": {+ "description": "The latest version of a Python package on PyPI, and what it says it is.",+ "hosts": [+ "pypi.org"+ ],+ "inputs": {+ "package": {+ "description": "A project name, as in: requests",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "pypi-package",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "software",+ "python",+ "read-only"+ ],+ "triggers": [+ "[what's the] latest version of {package:word} on pypi",+ "latest {package:word} on pypi",+ "(pypi|pip) (package|version) {package:word}"+ ],+ "version": 1+ },+ {+ "description": "echo for remind",+ "hosts": [],+ "inputs": {"at": { "type": "str" },@@ -648,6 +977,68 @@], "version": 1 },+ {+ "description": "Today's sunrise and sunset for a place, in its local time, from Open-Meteo.",+ "hosts": [+ "geocoding-api.open-meteo.com",+ "api.open-meteo.com"+ ],+ "inputs": {+ "place": {+ "description": "A town or city name",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "sun",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "weather",+ "read-only"+ ],+ "triggers": [+ "[when is] (sunrise|sunset) in {place}",+ "when does the sun (rise|set) in {place}",+ "(sunrise|sun) times (in|for) {place}"+ ],+ "version": 1+ },+ {+ "description": "Synonyms for an English word, best first, from Datamuse.",+ "hosts": [+ "api.datamuse.com"+ ],+ "inputs": {+ "word": {+ "description": "One English word",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "synonyms",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "reference",+ "language",+ "read-only"+ ],+ "triggers": [+ "synonyms (for|of) {word:word}",+ "(another|other) word for {word:word}",+ "what is another word for {word:word}"+ ],+ "version": 1+ },{ "description": "echo for tie-a", "hosts": [],@@ -725,6 +1116,36 @@"how [hot|cold|warm] is it in {place}" ], "version": 2+ },+ {+ "description": "The summary of a Wikipedia article.",+ "hosts": [+ "en.wikipedia.org"+ ],+ "inputs": {+ "topic": {+ "description": "An article title, as in: Ursula K. Le Guin",+ "required": true,+ "type": "str"+ }+ },+ "language": {},+ "license": "MIT",+ "name": "wikipedia",+ "namespace": "pyrtan",+ "sources": [+ "http"+ ],+ "tags": [+ "reference",+ "read-only"+ ],+ "triggers": [+ "tell me about {topic}",+ "who (is|was) {topic}",+ "(wikipedia|wiki) {topic}"+ ],+ "version": 1} ] },@@ -2546,25 +2967,39 @@"status": "ok", "suggestions": 3, "trees": [+ "air-quality","books", "count",+ "dad-joke",+ "define","disk-space",+ "earthquakes",+ "exchange-rate","fixed", "github-release", "hacker-news", "hacker-news-item",+ "hn-search",+ "holidays",+ "iss",+ "library","log-errors", "modem-check",+ "npm-package","order", "ping",+ "pypi-package","remind", "rendered-headline", "room-temperature", "site-status", "small-talk",+ "sun",+ "synonyms","tie-a", "tie-b",- "weather"+ "weather",+ "wikipedia"], "unanswered": 1, "watches": 0@@ -2707,7 +3142,7 @@"version": 1 }, {- "description": "echo for books",+ "description": "echo for library","hosts": [], "inputs": { "author": {@@ -2729,7 +3164,7 @@}, "typos": true },- "name": "books",+ "name": "library","nodes": { "reply": { "kind": "done",@@ -2961,6 +3396,22 @@[] ] },+ {+ "fn": "clean",+ "text": "convert 1,000.5 EUR, then 2, 3",+ "value": [+ "convert 1,000.5 EUR then 2 3",+ []+ ]+ },+ {+ "fn": "clean",+ "text": "a,b and 1 ,2 and 3, 4",+ "value": [+ "a b and 1 2 and 3 4",+ []+ ]+ },{ "fn": "singular", "text": "stories",@@ -4912,18 +5363,51 @@}, { "expect": {- "cleaned": "count to 1 000.5",- "considered": [],- "matches": [],+ "cleaned": "count to 1,000.5",+ "considered": [+ {+ "inputs": {+ "n": 1000.5+ },+ "notes": [+ "n: '1,000.5' read as 1000.5"+ ],+ "rejected": false,+ "specificity": 7,+ "tree": "count",+ "trigger": "count to {n:number}"+ }+ ],+ "matches": [+ {+ "inputs": {+ "n": 1000.5+ },+ "notes": [+ "n: '1,000.5' read as 1000.5"+ ],+ "rejected": false,+ "specificity": 7,+ "tree": "count",+ "trigger": "count to {n:number}"+ }+ ],"now": "2026-10-25T12:00:00+00:00", "step": { "kind": "intent", "node": "intent", "observed": {- "cleaned": "count to 1 000.5",- "steps": []+ "cleaned": "count to 1,000.5",+ "inputs": {+ "n": 1000.5+ },+ "notes": [+ "n: '1,000.5' read as 1000.5"+ ],+ "steps": [],+ "trigger": "count to {n:number}"},- "summary": "'count to 1,000.5' -> 'count to 1 000.5'; no trigger matched"+ "summary": "'count to 1,000.5'; count via 'count to {n:number}' (7 literal chars); n: '1,000.5' read as 1000.5"}, "steps": [], "text": "count to 1,000.5"@@ -5351,7 +5835,7 @@"notes": [], "rejected": false, "specificity": 17,- "tree": "books",+ "tree": "library","trigger": "(find|search) books by {author}" } ],@@ -5363,7 +5847,7 @@"notes": [], "rejected": false, "specificity": 17,- "tree": "books",+ "tree": "library","trigger": "(find|search) books by {author}" } ],@@ -5379,7 +5863,7 @@"steps": [], "trigger": "(find|search) books by {author}" },- "summary": "'find books by Ursula Le Guin'; books via '(find|search) books by {author}' (17 literal chars)"+ "summary": "'find books by Ursula Le Guin'; library via '(find|search) books by {author}' (17 literal chars)"}, "steps": [], "text": "find books by Ursula Le Guin"@@ -5397,7 +5881,7 @@"notes": [], "rejected": false, "specificity": 17,- "tree": "books",+ "tree": "library","trigger": "(find|search) books by {author}" } ],@@ -5409,7 +5893,7 @@"notes": [], "rejected": false, "specificity": 17,- "tree": "books",+ "tree": "library","trigger": "(find|search) books by {author}" } ],@@ -5425,7 +5909,7 @@"steps": [], "trigger": "(find|search) books by {author}" },- "summary": "'search novels by Le Guin'; books via '(find|search) books by {author}' (17 literal chars)"+ "summary": "'search novels by Le Guin'; library via '(find|search) books by {author}' (17 literal chars)"}, "steps": [], "text": "search novels by Le Guin"@@ -5443,7 +5927,7 @@"notes": [], "rejected": false, "specificity": 17,- "tree": "books",+ "tree": "library","trigger": "(find|search) books by {author}" } ],@@ -5455,7 +5939,7 @@"notes": [], "rejected": false, "specificity": 17,- "tree": "books",+ "tree": "library","trigger": "(find|search) books by {author}" } ],@@ -5471,7 +5955,7 @@"steps": [], "trigger": "(find|search) books by {author}" },- "summary": "'look up paperback books by Le Guin'; books via '(find|search) books by {author}' (17 literal chars)"+ "summary": "'look up paperback books by Le Guin'; library via '(find|search) books by {author}' (17 literal chars)"}, "steps": [], "text": "look up paperback books by Le Guin"@@ -5511,7 +5995,7 @@], "rejected": false, "specificity": 17,- "tree": "books",+ "tree": "library","trigger": "(find|search) books by {author}" } ],@@ -5525,7 +6009,7 @@], "rejected": false, "specificity": 17,- "tree": "books",+ "tree": "library","trigger": "(find|search) books by {author}" } ],@@ -5544,7 +6028,7 @@"steps": [], "trigger": "(find|search) books by {author}" },- "summary": "'serch books by Le Guin'; books via '(find|search) books by {author}' (17 literal chars); typo: serch->search"+ "summary": "'serch books by Le Guin'; library via '(find|search) books by {author}' (17 literal chars); typo: serch->search"}, "steps": [], "text": "serch books by Le Guin"@@ -5560,7 +6044,7 @@"notes": [], "rejected": false, "specificity": 18,- "tree": "books",+ "tree": "library","trigger": "top story on book club" } ],@@ -5570,7 +6054,7 @@"notes": [], "rejected": false, "specificity": 18,- "tree": "books",+ "tree": "library","trigger": "top story on book club" } ],@@ -5584,7 +6068,7 @@"steps": [], "trigger": "top story on book club" },- "summary": "'top stories on book club'; books via 'top story on book club' (18 literal chars)"+ "summary": "'top stories on book club'; library via 'top story on book club' (18 literal chars)"}, "steps": [], "text": "top stories on book club"@@ -8665,233 +9149,3845 @@}, "intent": "goodbye", "tree": "small-talk"- }- ],- "trees": {- "disk-space": {- "commands": [- "df"- ],- "description": "How full a filesystem is, from df on the box this tree runs on.",- "inputs": {- "mount": {- "default": "/",- "description": "A mount point, or any path on the filesystem to measure",- "type": "str"- }- },- "license": "MIT",- "name": "disk-space",- "namespace": "pyrtan",- "nodes": {- "df": {- "extract": [+ },+ {+ "expect": {+ "inputs": {+ "topic": "Ursula K Le Guin"+ },+ "output": {+ "summary": "Ursula Kroeber Le Guin was an American author. She is best known for her works of speculative fiction.",+ "title": "Ursula K. Le Guin",+ "url": "https://en.wikipedia.org/wiki/Ursula_K._Le_Guin"+ },+ "say": "Ursula K. Le Guin: Ursula Kroeber Le Guin was an American author. She is best known for her works of speculative fiction. https://en.wikipedia.org/wiki/Ursula_K._Le_Guin",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [{- "group": 1,- "into": "percent",- "regex": "(\\d+)%"+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "tell me about Ursula K Le Guin",+ "inputs": {+ "topic": "Ursula K Le Guin"+ },+ "steps": [],+ "trigger": "tell me about {topic}"+ },+ "summary": "'tell me about Ursula K. Le Guin' -> 'tell me about Ursula K Le Guin'; wikipedia via 'tell me about {topic}' (11 literal chars)"}, {- "group": 1,- "into": "mounted",- "regex": "%\\s+(\\S+)\\s*$"- }- ],- "into": "df",- "kind": "observe",- "next": "ran",- "shell": {- "args": [- "-P",- "{{ inputs.mount }}"- ],- "command": "df"- }- },- "failed": {- "kind": "fail",- "reason": "df-failed",- "say": "df could not measure {{ inputs.mount }}: {{ state.df.stderr | strip }}"- },- "fine": {- "kind": "done",- "output": {- "mounted": "{{ state.mounted }}",- "percent": "{{ state.pct }}"- },- "say": "{{ state.mounted }} is {{ state.pct }}% full."- },- "full": {- "kind": "fail",- "reason": "disk-full",- "say": "{{ state.mounted }} is {{ state.pct }}% full."- },- "level": {- "branches": [+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "kind": "standard",+ "summary": "Ursula Kroeber Le Guin was an American author. She is best known for her works of speculative fiction.",+ "title": "Ursula K. Le Guin",+ "url": "https://en.wikipedia.org/wiki/Ursula_K._Le_Guin"+ },+ "summary": "GET https://en.wikipedia.org/api/rest_v1/page/summary/Ursula%20K%20Le%20Guin -> 200; kind=standard, title=Ursula K. Le Guin, summary=Ursula Kroeber Le Guin was an American author. She is best known for her works …, url=https://en.wikipedia.org/wiki/Ursula_K._Le_Guin"+ },{- "then": "fine",- "when": {- "lt": 80- }+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "kind",+ "node": "status",+ "observed": 200,+ "summary": "state.resp.status = 200"}, {- "then": "tight",- "when": {- "lt": 95- }+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "reply",+ "node": "kind",+ "observed": "standard",+ "summary": "state.kind = standard"}, {- "else": "full"+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Ursula K. Le Guin: Ursula Kroeber Le Guin was an American author. She is best known for her works of speculative fiction. https://en.wikipedia.org/wiki/Ursula_K._Le_Guin"} ],- "kind": "decide",- "on": "pct"+ "tree": "wikipedia"},- "number": {- "kind": "set",- "next": "level",+ "tree": "wikipedia"+ },+ "inputs": {+ "topic": "Ursula K Le Guin"+ },+ "intent": "tell me about Ursula K. Le Guin",+ "tree": "wikipedia"+ },+ {+ "expect": {+ "inputs": {+ "topic": "Mercury"+ },+ "reason": "ambiguous",+ "say": "Mercury could mean several things on Wikipedia; say which: https://en.wikipedia.org/wiki/Mercury",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "wiki Mercury",+ "inputs": {+ "topic": "Mercury"+ },+ "steps": [],+ "trigger": "(wikipedia|wiki) {topic}"+ },+ "summary": "'wiki Mercury'; wikipedia via '(wikipedia|wiki) {topic}' (13 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "kind": "disambiguation",+ "summary": "Mercury most commonly refers to:Mercury (planet), the closest planet to the Sun\n",+ "title": "Mercury",+ "url": "https://en.wikipedia.org/wiki/Mercury"+ },+ "summary": "GET https://en.wikipedia.org/api/rest_v1/page/summary/Mercury -> 200; kind=disambiguation, title=Mercury, summary=Mercury most commonly refers to:Mercury (planet), the closest planet to the Sun\n, url=https://en.wikipedia.org/wiki/Mercury"+ },+ {+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "kind",+ "node": "status",+ "observed": 200,+ "summary": "state.resp.status = 200"+ },+ {+ "chosen": "eq 'disambiguation'",+ "depth": 0,+ "kind": "decide",+ "next": "ambiguous",+ "node": "kind",+ "observed": "disambiguation",+ "summary": "state.kind = disambiguation"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "ambiguous",+ "summary": "ambiguous: Mercury could mean several things on Wikipedia; say which: https://en.wikipedia.org/wiki/Mercury"+ }+ ],+ "tree": "wikipedia"+ },+ "tree": "wikipedia"+ },+ "inputs": {+ "topic": "Mercury"+ },+ "intent": "wiki Mercury",+ "tree": "wikipedia"+ },+ {+ "expect": {+ "inputs": {+ "topic": "Xyzzyplugh Notaperson"+ },+ "reason": "no-article",+ "say": "Wikipedia has no article called Xyzzyplugh Notaperson.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "who is Xyzzyplugh Notaperson",+ "inputs": {+ "topic": "Xyzzyplugh Notaperson"+ },+ "steps": [],+ "trigger": "who (is|was) {topic}"+ },+ "summary": "'who is Xyzzyplugh Notaperson'; wikipedia via 'who (is|was) {topic}' (8 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "kind": "Internal error",+ "summary": null,+ "title": null,+ "url": null+ },+ "summary": "GET https://en.wikipedia.org/api/rest_v1/page/summary/Xyzzyplugh%20Notaperson -> 404; kind=Internal error, title=None, summary=None, url=None"+ },+ {+ "chosen": "eq 404",+ "depth": 0,+ "kind": "decide",+ "next": "no-article",+ "node": "status",+ "observed": 404,+ "summary": "state.resp.status = 404"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "no-article",+ "summary": "no-article: Wikipedia has no article called Xyzzyplugh Notaperson."+ }+ ],+ "tree": "wikipedia"+ },+ "tree": "wikipedia"+ },+ "inputs": {+ "topic": "Xyzzyplugh Notaperson"+ },+ "intent": "who is Xyzzyplugh Notaperson",+ "tree": "wikipedia"+ },+ {+ "expect": {+ "inputs": {+ "topic": "Alan Turing"+ },+ "reason": "rate-limited",+ "say": "Wikipedia is asking for fewer requests (429); try again in a minute.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "who was Alan Turing",+ "inputs": {+ "topic": "Alan Turing"+ },+ "steps": [],+ "trigger": "who (is|was) {topic}"+ },+ "summary": "'who was Alan Turing'; wikipedia via 'who (is|was) {topic}' (8 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "kind": null,+ "summary": null,+ "title": null,+ "url": null+ },+ "summary": "GET https://en.wikipedia.org/api/rest_v1/page/summary/Alan%20Turing -> 429; kind=None, title=None, summary=None, url=None"+ },+ {+ "chosen": "eq 429",+ "depth": 0,+ "kind": "decide",+ "next": "rate-limited",+ "node": "status",+ "observed": 429,+ "summary": "state.resp.status = 429"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "rate-limited",+ "summary": "rate-limited: Wikipedia is asking for fewer requests (429); try again in a minute."+ }+ ],+ "tree": "wikipedia"+ },+ "tree": "wikipedia"+ },+ "inputs": {+ "topic": "Alan Turing"+ },+ "intent": "who was Alan Turing",+ "tree": "wikipedia"+ },+ {+ "expect": {+ "inputs": {+ "word": "serendipity"+ },+ "output": {+ "definition": "A combination of events which have come together by chance to make a surprisingly good or wonderful outcome.",+ "part_of_speech": "noun",+ "phonetic": "/ˌsɛ.ɹən.ˈdɪ.pɪ.ti/",+ "word": "serendipity"+ },+ "say": "serendipity (noun): A combination of events which have come together by chance to make a surprisingly good or wonderful outcome.",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "what does serendipity mean",+ "inputs": {+ "word": "serendipity"+ },+ "steps": [],+ "trigger": "what does {word:word} mean"+ },+ "summary": "'what does serendipity mean'; define via 'what does {word:word} mean' (12 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "definition": "A combination of events which have come together by chance to make a surprisingly good or wonderful outcome.",+ "part": "noun",+ "phonetic": "/ˌsɛ.ɹən.ˈdɪ.pɪ.ti/",+ "word": "serendipity"+ },+ "summary": "GET https://api.dictionaryapi.dev/api/v2/entries/en/serendipity -> 200; word=serendipity, phonetic=/ˌsɛ.ɹən.ˈdɪ.pɪ.ti/, part=noun, definition=A combination of events which have come together by chance to make a surprising…"+ },+ {+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "has-definition",+ "node": "status",+ "observed": 200,+ "summary": "state.resp.status = 200"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "reply",+ "node": "has-definition",+ "observed": "A combination of events which have come together by chance to make a surprisingly good or wonderful outcome.",+ "summary": "state.definition = A combination of events which have come together by chance to make a surprising…"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "serendipity (noun): A combination of events which have come together by chance to make a surprisingly good or wonderful outcome."+ }+ ],+ "tree": "define"+ },+ "tree": "define"+ },+ "inputs": {+ "word": "serendipity"+ },+ "intent": "what does serendipity mean",+ "tree": "define"+ },+ {+ "expect": {+ "inputs": {+ "word": "zzzqx"+ },+ "reason": "unknown-word",+ "say": "The dictionary has no entry for zzzqx.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "define zzzqx",+ "inputs": {+ "word": "zzzqx"+ },+ "steps": [],+ "trigger": "define {word:word}"+ },+ "summary": "'define zzzqx'; define via 'define {word:word}' (6 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "definition": null,+ "part": null,+ "phonetic": null,+ "word": null+ },+ "summary": "GET https://api.dictionaryapi.dev/api/v2/entries/en/zzzqx -> 404; word=None, phonetic=None, part=None, definition=None"+ },+ {+ "chosen": "eq 404",+ "depth": 0,+ "kind": "decide",+ "next": "unknown-word",+ "node": "status",+ "observed": 404,+ "summary": "state.resp.status = 404"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "unknown-word",+ "summary": "unknown-word: The dictionary has no entry for zzzqx."+ }+ ],+ "tree": "define"+ },+ "tree": "define"+ },+ "inputs": {+ "word": "zzzqx"+ },+ "intent": "define zzzqx",+ "tree": "define"+ },+ {+ "expect": {+ "inputs": {+ "place": "Oslo"+ },+ "output": {+ "country": "Norway",+ "date": "2026-10-28",+ "place": "Oslo",+ "sunrise": "07:28",+ "sunset": "16:32",+ "timezone": "Europe/Oslo"+ },+ "say": "Oslo, Norway, 2026-10-28: sunrise 07:28, sunset 16:32 (Europe/Oslo).",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "geocode",+ "node": "intent",+ "observed": {+ "cleaned": "when is sunset in Oslo",+ "inputs": {+ "place": "Oslo"+ },+ "steps": [],+ "trigger": "[when is] (sunrise|sunset) in {place}"+ },+ "summary": "'when is sunset in Oslo'; sun via '[when is] (sunrise|sunset) in {place}' (21 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "found",+ "node": "geocode",+ "observed": {+ "country": "Norway",+ "hit": {+ "country": "Norway",+ "latitude": 59.91,+ "longitude": 10.75,+ "name": "Oslo"+ },+ "lat": 59.91,+ "lon": 10.75,+ "name": "Oslo"+ },+ "summary": "GET https://geocoding-api.open-meteo.com/v1/search?name=Oslo&count=1&language=en&format=json -> 200; hit={'name': 'Oslo', 'country': 'Norway', 'latitude': 59.91, 'longitude': 10.75}, lat=59.91, lon=10.75, name=Oslo, country=Norway"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "times",+ "node": "found",+ "observed": {+ "country": "Norway",+ "latitude": 59.91,+ "longitude": 10.75,+ "name": "Oslo"+ },+ "summary": "state.hit = {'name': 'Oslo', 'country': 'Norway', 'latitude': 59.91, 'longitude': 10.75}"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "have-times",+ "node": "times",+ "observed": {+ "day": "2026-10-28",+ "rise": "07:28",+ "set": "16:32",+ "zone": "Europe/Oslo"+ },+ "summary": "GET https://api.open-meteo.com/v1/forecast?latitude=59.91&longitude=10.75&daily=sunrise,sunset&timezone=auto&forecast_days=1 -> 200; day=2026-10-28, rise=07:28, set=16:32, zone=Europe/Oslo"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "reply",+ "node": "have-times",+ "observed": "07:28",+ "summary": "state.rise = 07:28"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Oslo, Norway, 2026-10-28: sunrise 07:28, sunset 16:32 (Europe/Oslo)."+ }+ ],+ "tree": "sun"+ },+ "tree": "sun"+ },+ "inputs": {+ "place": "Oslo"+ },+ "intent": "when is sunset in Oslo",+ "tree": "sun"+ },+ {+ "expect": {+ "inputs": {+ "place": "Longyearbyen"+ },+ "reason": "no-times",+ "say": "Open-Meteo gave no sunrise for Longyearbyen today (status 200); near the poles the sun may not rise or set at all.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "geocode",+ "node": "intent",+ "observed": {+ "cleaned": "sunrise in Longyearbyen",+ "inputs": {+ "place": "Longyearbyen"+ },+ "steps": [],+ "trigger": "[when is] (sunrise|sunset) in {place}"+ },+ "summary": "'sunrise in Longyearbyen'; sun via '[when is] (sunrise|sunset) in {place}' (21 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "found",+ "node": "geocode",+ "observed": {+ "country": "Svalbard and Jan Mayen",+ "hit": {+ "country": "Svalbard and Jan Mayen",+ "latitude": 78.22,+ "longitude": 15.65,+ "name": "Longyearbyen"+ },+ "lat": 78.22,+ "lon": 15.65,+ "name": "Longyearbyen"+ },+ "summary": "GET https://geocoding-api.open-meteo.com/v1/search?name=Longyearbyen&count=1&language=en&format=json -> 200; hit={'name': 'Longyearbyen', 'country': 'Svalbard and Jan Mayen', 'latitude': 78.22…, lat=78.22, lon=15.65, name=Longyearbyen, country=Svalbard and Jan Mayen"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "times",+ "node": "found",+ "observed": {+ "country": "Svalbard and Jan Mayen",+ "latitude": 78.22,+ "longitude": 15.65,+ "name": "Longyearbyen"+ },+ "summary": "state.hit = {'name': 'Longyearbyen', 'country': 'Svalbard and Jan Mayen', 'latitude': 78.22…"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "have-times",+ "node": "times",+ "observed": {+ "day": "2026-12-21",+ "rise": null,+ "set": null,+ "zone": "Arctic/Longyearbyen"+ },+ "summary": "GET https://api.open-meteo.com/v1/forecast?latitude=78.22&longitude=15.65&daily=sunrise,sunset&timezone=auto&forecast_days=1 -> 200; day=2026-12-21, rise=None, set=None, zone=Arctic/Longyearbyen"+ },+ {+ "chosen": "else",+ "depth": 0,+ "kind": "decide",+ "next": "no-times",+ "node": "have-times",+ "summary": "state.rise = <missing>"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "no-times",+ "summary": "no-times: Open-Meteo gave no sunrise for Longyearbyen today (status 200); near the poles the sun may not rise or set at all."+ }+ ],+ "tree": "sun"+ },+ "tree": "sun"+ },+ "inputs": {+ "place": "Longyearbyen"+ },+ "intent": "sunrise in Longyearbyen",+ "tree": "sun"+ },+ {+ "expect": {+ "inputs": {+ "place": "Oslo"+ },+ "output": {+ "aqi": 21,+ "band": "fair",+ "country": "Norway",+ "place": "Oslo",+ "pm10": 10.2,+ "pm2_5": 3.8+ },+ "say": "Oslo, Norway: air quality fair (European AQI 21), PM2.5 3.8 µg/m³, PM10 10.2 µg/m³.",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "geocode",+ "node": "intent",+ "observed": {+ "cleaned": "air quality in Oslo",+ "inputs": {+ "place": "Oslo"+ },+ "steps": [],+ "trigger": "[what's the] air quality (in|for) {place}"+ },+ "summary": "'air quality in Oslo'; air-quality via \"[what's the] air quality (in|for) {place}\" (24 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "found",+ "node": "geocode",+ "observed": {+ "country": "Norway",+ "hit": {+ "country": "Norway",+ "latitude": 59.91,+ "longitude": 10.75,+ "name": "Oslo"+ },+ "lat": 59.91,+ "lon": 10.75,+ "name": "Oslo"+ },+ "summary": "GET https://geocoding-api.open-meteo.com/v1/search?name=Oslo&count=1&language=en&format=json -> 200; hit={'name': 'Oslo', 'country': 'Norway', 'latitude': 59.91, 'longitude': 10.75}, lat=59.91, lon=10.75, name=Oslo, country=Norway"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "air",+ "node": "found",+ "observed": {+ "country": "Norway",+ "latitude": 59.91,+ "longitude": 10.75,+ "name": "Oslo"+ },+ "summary": "state.hit = {'name': 'Oslo', 'country': 'Norway', 'latitude': 59.91, 'longitude': 10.75}"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "band",+ "node": "air",+ "observed": {+ "aqi": 21,+ "pm10": 10.2,+ "pm25": 3.8+ },+ "summary": "GET https://air-quality-api.open-meteo.com/v1/air-quality?latitude=59.91&longitude=10.75¤t=european_aqi,pm2_5,pm10 -> 200; aqi=21, pm25=3.8, pm10=10.2"+ },+ {+ "chosen": "lte 40",+ "depth": 0,+ "kind": "decide",+ "next": "fair",+ "node": "band",+ "observed": 21,+ "summary": "state.aqi = 21"+ },+ {+ "depth": 0,+ "kind": "set",+ "next": "reply",+ "node": "fair",+ "observed": {+ "band": "fair"+ },+ "summary": "band=fair"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Oslo, Norway: air quality fair (European AQI 21), PM2.5 3.8 µg/m³, PM10 10.2 µg/m³."+ }+ ],+ "tree": "air-quality"+ },+ "tree": "air-quality"+ },+ "inputs": {+ "place": "Oslo"+ },+ "intent": "air quality in Oslo",+ "tree": "air-quality"+ },+ {+ "expect": {+ "inputs": {+ "place": "Delhi"+ },+ "output": {+ "aqi": 87,+ "band": "very poor",+ "country": "India",+ "place": "Delhi",+ "pm10": 118.3,+ "pm2_5": 61.4+ },+ "say": "Delhi, India: air quality very poor (European AQI 87), PM2.5 61.4 µg/m³, PM10 118.3 µg/m³.",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "geocode",+ "node": "intent",+ "observed": {+ "cleaned": "how is the air in Delhi",+ "inputs": {+ "place": "Delhi"+ },+ "steps": [],+ "trigger": "how is the air in {place}"+ },+ "summary": "'how is the air in Delhi'; air-quality via 'how is the air in {place}' (13 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "found",+ "node": "geocode",+ "observed": {+ "country": "India",+ "hit": {+ "country": "India",+ "latitude": 28.65,+ "longitude": 77.23,+ "name": "Delhi"+ },+ "lat": 28.65,+ "lon": 77.23,+ "name": "Delhi"+ },+ "summary": "GET https://geocoding-api.open-meteo.com/v1/search?name=Delhi&count=1&language=en&format=json -> 200; hit={'name': 'Delhi', 'country': 'India', 'latitude': 28.65, 'longitude': 77.23}, lat=28.65, lon=77.23, name=Delhi, country=India"+ },+ {+ "chosen": "exists True",+ "depth": 0,+ "kind": "decide",+ "next": "air",+ "node": "found",+ "observed": {+ "country": "India",+ "latitude": 28.65,+ "longitude": 77.23,+ "name": "Delhi"+ },+ "summary": "state.hit = {'name': 'Delhi', 'country': 'India', 'latitude': 28.65, 'longitude': 77.23}"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "band",+ "node": "air",+ "observed": {+ "aqi": 87,+ "pm10": 118.3,+ "pm25": 61.4+ },+ "summary": "GET https://air-quality-api.open-meteo.com/v1/air-quality?latitude=28.65&longitude=77.23¤t=european_aqi,pm2_5,pm10 -> 200; aqi=87, pm25=61.4, pm10=118.3"+ },+ {+ "chosen": "lte 100",+ "depth": 0,+ "kind": "decide",+ "next": "very-poor",+ "node": "band",+ "observed": 87,+ "summary": "state.aqi = 87"+ },+ {+ "depth": 0,+ "kind": "set",+ "next": "reply",+ "node": "very-poor",+ "observed": {+ "band": "very poor"+ },+ "summary": "band=very poor"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "Delhi, India: air quality very poor (European AQI 87), PM2.5 61.4 µg/m³, PM10 118.3 µg/m³."+ }+ ],+ "tree": "air-quality"+ },+ "tree": "air-quality"+ },+ "inputs": {+ "place": "Delhi"+ },+ "intent": "how is the air in Delhi",+ "tree": "air-quality"+ },+ {+ "expect": {+ "inputs": {+ "package": "@types/node"+ },+ "output": {+ "description": "TypeScript definitions for node",+ "package": "@types/node",+ "url": "https://www.npmjs.com/package/@types/node",+ "version": "26.6.5"+ },+ "say": "@types/node 26.6.5 on npm: TypeScript definitions for node https://www.npmjs.com/package/@types/node",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "latest version of @types/node on npm",+ "inputs": {+ "package": "@types/node"+ },+ "steps": [],+ "trigger": "[what's the] latest version of {package:word} on npm"+ },+ "summary": "'latest version of @types/node on npm'; npm-package via \"[what's the] latest version of {package:word} on npm\" (29 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "about": "TypeScript definitions for node",+ "version": "26.6.5"+ },+ "summary": "GET https://registry.npmjs.org/%40types%2Fnode/latest -> 200; version=26.6.5, about=TypeScript definitions for node"+ },+ {+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "reply",+ "node": "status",+ "observed": 200,+ "summary": "state.resp.status = 200"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "@types/node 26.6.5 on npm: TypeScript definitions for node https://www.npmjs.com/package/@types/node"+ }+ ],+ "tree": "npm-package"+ },+ "tree": "npm-package"+ },+ "inputs": {+ "package": "@types/node"+ },+ "intent": "latest version of @types/node on npm",+ "tree": "npm-package"+ },+ {+ "expect": {+ "inputs": {+ "package": "nope-not-a-pkg-zz"+ },+ "reason": "no-package",+ "say": "npm has no package called nope-not-a-pkg-zz.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "npm package nope-not-a-pkg-zz",+ "inputs": {+ "package": "nope-not-a-pkg-zz"+ },+ "steps": [],+ "trigger": "npm (package|version) {package:word}"+ },+ "summary": "'npm package nope-not-a-pkg-zz'; npm-package via 'npm (package|version) {package:word}' (17 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "about": null,+ "version": null+ },+ "summary": "GET https://registry.npmjs.org/nope-not-a-pkg-zz/latest -> 404; version=None, about=None"+ },+ {+ "chosen": "eq 404",+ "depth": 0,+ "kind": "decide",+ "next": "no-package",+ "node": "status",+ "observed": 404,+ "summary": "state.resp.status = 404"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "no-package",+ "summary": "no-package: npm has no package called nope-not-a-pkg-zz."+ }+ ],+ "tree": "npm-package"+ },+ "tree": "npm-package"+ },+ "inputs": {+ "package": "nope-not-a-pkg-zz"+ },+ "intent": "npm package nope-not-a-pkg-zz",+ "tree": "npm-package"+ },+ {+ "expect": {+ "inputs": {+ "package": "requests"+ },+ "output": {+ "package": "requests",+ "summary": "Python HTTP for Humans.",+ "url": "https://pypi.org/project/requests/",+ "version": "2.34.2"+ },+ "say": "requests 2.34.2 on PyPI: Python HTTP for Humans. https://pypi.org/project/requests/",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "latest requests on pypi",+ "inputs": {+ "package": "requests"+ },+ "steps": [],+ "trigger": "latest {package:word} on pypi"+ },+ "summary": "'latest requests on pypi'; pypi-package via 'latest {package:word} on pypi' (12 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "about": "Python HTTP for Humans.",+ "name": "requests",+ "url": "https://pypi.org/project/requests/",+ "version": "2.34.2"+ },+ "summary": "GET https://pypi.org/pypi/requests/json -> 200; name=requests, version=2.34.2, about=Python HTTP for Humans., url=https://pypi.org/project/requests/"+ },+ {+ "chosen": "eq 200",+ "depth": 0,+ "kind": "decide",+ "next": "reply",+ "node": "status",+ "observed": 200,+ "summary": "state.resp.status = 200"+ },+ {+ "depth": 0,+ "kind": "done",+ "next": null,+ "node": "reply",+ "summary": "requests 2.34.2 on PyPI: Python HTTP for Humans. https://pypi.org/project/requests/"+ }+ ],+ "tree": "pypi-package"+ },+ "tree": "pypi-package"+ },+ "inputs": {+ "package": "requests"+ },+ "intent": "latest requests on pypi",+ "tree": "pypi-package"+ },+ {+ "expect": {+ "inputs": {+ "package": "nope-not-a-pkg-zz"+ },+ "reason": "no-package",+ "say": "PyPI has no project called nope-not-a-pkg-zz.",+ "status": "failed",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "pip version nope-not-a-pkg-zz",+ "inputs": {+ "package": "nope-not-a-pkg-zz"+ },+ "steps": [],+ "trigger": "(pypi|pip) (package|version) {package:word}"+ },+ "summary": "'pip version nope-not-a-pkg-zz'; pypi-package via '(pypi|pip) (package|version) {package:word}' (21 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "about": null,+ "name": null,+ "url": null,+ "version": null+ },+ "summary": "GET https://pypi.org/pypi/nope-not-a-pkg-zz/json -> 404; name=None, version=None, about=None, url=None"+ },+ {+ "chosen": "eq 404",+ "depth": 0,+ "kind": "decide",+ "next": "no-package",+ "node": "status",+ "observed": 404,+ "summary": "state.resp.status = 404"+ },+ {+ "depth": 0,+ "kind": "fail",+ "next": null,+ "node": "no-package",+ "summary": "no-package: PyPI has no project called nope-not-a-pkg-zz."+ }+ ],+ "tree": "pypi-package"+ },+ "tree": "pypi-package"+ },+ "inputs": {+ "package": "nope-not-a-pkg-zz"+ },+ "intent": "pip version nope-not-a-pkg-zz",+ "tree": "pypi-package"+ },+ {+ "expect": {+ "inputs": {},+ "output": {+ "count": 19,+ "latest": {+ "magnitude": 5.1,+ "place": "7 km ENE of Pangyan, Philippines",+ "tsunami": 0,+ "url": "https://earthquake.usgs.gov/earthquakes/eventpage/us7000ttkk"+ }+ },+ "say": "19 earthquake(s) of magnitude 4.5+ in the past day. The latest: M5.1, 7 km ENE of Pangyan, Philippines. https://earthquake.usgs.gov/earthquakes/eventpage/us7000ttkk",+ "status": "done",+ "trace": {+ "gaps": [],+ "steps": [+ {+ "depth": 0,+ "kind": "intent",+ "next": "fetch",+ "node": "intent",+ "observed": {+ "cleaned": "any earthquakes today",+ "inputs": {},+ "steps": [],+ "trigger": "[any] [recent] earthquakes [today]"+ },+ "summary": "'any earthquakes today'; earthquakes via '[any] [recent] earthquakes [today]' (25 literal chars)"+ },+ {+ "depth": 0,+ "kind": "observe",+ "next": "status",+ "node": "fetch",+ "observed": {+ "count": 19,+ "mag": 5.1,+ "place": "7 km ENE of Pangyan, Philippines",+ "tsunami": 0,+ "url": "https://earthquake.usgs.gov/earthquakes/eventpage/us7000ttkk"+ },+ "summary": "GET https://earthq
The diff is cut here; the rest is in the clone.