Merge main (networks, 0.10.0) into chat; chat is now release 0.11.0
Claude · 2026-10-27 · 78500c19e1766d53a51213b6f139f60a05f96daa · parents bf4031b 78e426c
Main shipped its own 0.10.0 (networks), so the chat lobby is 0.11.0. Conflicts were additive: the hub imports both chat and the network helpers, /api lists both sets of endpoints and says whether it chats, the console keeps the new theme and the chat styles, and pyproject takes main's description.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rD7jd9JrpLd1HKE8P1h3G
Changes
.github/workflows/pages.yml | 5 +- .gitignore | 1 + Dockerfile | 2 +- README.md | 71 ++++- _meta/chat.md | 6 +- _meta/design.md | 2 +- _meta/federation.md | 68 +++- deploy/README.md | 15 +- deploy/openmediavault/docker-compose.yml | 10 +- pyproject.toml | 4 +- site/README.md | 45 +-- site/build.sh | 22 +- site/catalog/index.html | 14 +- site/index.html | 28 +- site/rendezvous/network.json | 6 + site/snapshot.py | 91 ++++-- site/source.py | 479 ++++++++++++++++++++++++++++ src/pyrtan/__init__.py | 6 +- src/pyrtan/cli.py | 105 ++++-- src/pyrtan/federation.py | 333 ++++++++++++++++--- src/pyrtan/hub.py | 80 ++++- src/pyrtan/js/pyrtan.js | 2 +- src/pyrtan/js/theme.js | 134 ++++++++ src/pyrtan/page.py | 26 +- tests/conftest.py | 7 + tests/test_federation.py | 179 ++++++++++- tests/test_service_hub.py | 3 + tests/test_site_source.py | 44 +++ .../scratchpad/out/catalog/github/github-release.toml | 63 ---- .../scratchpad/out/catalog/hacker-news/hacker-news-item.toml | 66 ---- .../scratchpad/out/catalog/hacker-news/hacker-news.toml | 79 ----- .../scratchpad/out/catalog/site-status/site-status.toml | 62 ---- .../scratchpad/out/catalog/weather/weather.toml | 133 -------- uv.lock | 2 +- 34 files changed, 1600 insertions(+), 593 deletions(-)
diff --git a/.github/workflows/pages.yml b/.github/workflows/pages.yml index 80ca37e..f8454fe 100644 --- a/.github/workflows/pages.yml +++ b/.github/workflows/pages.yml@@ -1,6 +1,7 @@# pyrtan.com: this repository's GitHub Pages site. On every push to main # and once an hour, build the landing page, the console (from this-# checkout's pyrtan), the catalog (from examples/ and catalog/, through the+# checkout's pyrtan), the source browser (files, releases, branches and+# history from git), the catalog (from examples/ and catalog/, through the# gate), the seed list and a snapshot of the directory and of the gap # reports as the seeds know them, and publish them. Set the # repository's Pages source to "GitHub Actions" and its custom domain to@@ -29,6 +30,8 @@ jobs:runs-on: ubuntu-latest steps: - uses: actions/checkout@v4+ with:+ fetch-depth: 0 # every branch and the whole history, for /source- uses: astral-sh/setup-uv@v5 - run: uv sync - run: uv run pytest -q diff --git a/.gitignore b/.gitignore index 2af4a18..87ac1ce 100644 --- a/.gitignore +++ b/.gitignore@@ -7,3 +7,4 @@ build/*.egg-info/ .env .env.*+tmp/diff --git a/Dockerfile b/Dockerfile index 0c6c422..66b0fda 100644 --- a/Dockerfile +++ b/Dockerfile@@ -12,7 +12,7 @@ RUN pip install --no-cache-dir --prefix=/install .FROM python:3.12-slim LABEL org.opencontainers.image.title="pyrtan" \- org.opencontainers.image.description="A core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees." \+ org.opencontainers.image.description="A core set of expandable services that act on all forms of I/O through inspectable decision trees." \org.opencontainers.image.source="https://github.com/b-i-t-8/pyrtan" \ org.opencontainers.image.licenses="AGPL-3.0-or-later" COPY --from=build /install /usr/local diff --git a/README.md b/README.md index e81e86a..00cd2dc 100644 --- a/README.md +++ b/README.md@@ -1,7 +1,7 @@# Pyrtan **Open-source total automation: a core set of expandable services that act on all forms of I/O-through hardcoded, inspectable decision trees.**+through inspectable decision trees.**Pyrtan looks like an assistant from the outside. You give it a sentence; it reads or changes something through whatever form of I/O it has an adapter@@ -162,7 +162,9 @@ hosted box is not lent out. The design is in [`_meta/io.md`](_meta/io.md).| `pyrtan serve DIR --port 8701` | Hosts the trees as a service: `/health`, `/capabilities`, `/intent`, `/run`, `/resume`, `/gaps`, `/unanswered`. Reloads trees when their files change. `--gaps FILE` keeps gaps across restarts; `--egress public` refuses private addresses. | | `pyrtan hub --port 8700` | Routes between services. Services register with `--hub URL`; peers carry a `link` of `up`, `stale` or `down`. `--trees DIR` holds trees in the hub itself; `--federate` lets other hubs follow it and it follow them; `--listed` puts it in the directory; `--chat` keeps a lobby where people meet strangers and bots. | | `pyrtan keep KEY [FILE] --at HUB` | Gives your hub a blob to keep; its keepers get copies. `kept` shows who holds what; `fetch KEY` gets a blob back from your hub or whoever holds it. |-| `pyrtan follow [URL] --at HUB` | Makes your hub follow another (default: the first live seed from pyrtan.com): it then routes sentences to that hub's trees too. `--listed` also puts your hub in the directory. `unfollow` undoes it; `peers` lists services, hubs followed and followers; `directory` lists every hub that asked to be found, as the swarm knows it. |+| `pyrtan follow [URL] --at HUB` | Makes your hub follow another: it then routes sentences to that hub's trees too. With no URL, `pyrtan join`. `--network NAME` follows within a network of yours; `--listed` also puts your hub in the directory. `unfollow` undoes it; `peers` lists services, hubs followed and followers; `directory` lists every hub that asked to be found, as the swarm knows it. |+| `pyrtan join [INVITE] --at HUB` | Puts your hub in a network and follows a hub in it: the default network pyrtan.com publishes, or the one an invite (`NAME:SECRET@HUB`) names. |+| `pyrtan network new NAME --at HUB` | Starts a network of your own, a network of one, and prints its invite. `network list` shows each network with its invite; `network leave NAME` cuts the links made in it. || `pyrtan ask "…" --at URL` | Answers a sentence through a running service or hub. | | `pyrtan chat --at HUB` | Talks to a random stranger or a bot through a hub started `--chat`. `--bot` for a bot, `--people` for people only, `--interests` to meet someone who shares one, `--trace` to see the path behind each bot line. | | `pyrtan gaps --at URL` | Every decision that met a value with no branch: the branches to write next. |@@ -191,7 +193,10 @@ Open <http://127.0.0.1:8700/> for the console: a sentence box, the answer,the output and the trace; everything the swarm can answer; each peer and its `link`; and the gaps. The same console lives at <https://pyrtan.com/console/> and can be pointed at any hub started with-`--cors https://pyrtan.com`. To run the swarm on a NAS or any Docker host,+`--cors https://pyrtan.com`. The wordmark is two colours, **Pyr** the+primary and **tan** the secondary: click either half, right-click it, or+press and hold it on a phone to pick that half's colour, kept in the+browser. To run the swarm on a NAS or any Docker host,see [`deploy/README.md`](deploy/README.md). ## A federation@@ -220,21 +225,57 @@ ask. **Federation is one hop:** a hub announces what it holds itself,never what it learned from others, and refuses to forward a request that already crossed a hub, so there is nothing to loop. **Every message is signed**, and a hub started with `--federation-secret` takes follows only-from hubs that know the secret; `--closed` takes none. `--state FILE`-keeps the relationships across restarts.+from hubs that know the secret; `--closed` takes none outside its+networks. `--state FILE` keeps the relationships and networks across+restarts.Hubs find each other without a server. A hub started `--listed` puts an entry for itself (name, URL, the trees it holds) in every announce it sends, every hub keeps the entries it hears and passes them on in both directions of every follow, and `pyrtan directory` (or the console, or-any hub's `/directory`) shows the result. pyrtan.com publishes only-`rendezvous/seeds.json`, a static list of hubs to follow first, and an-hourly snapshot of what they know; `pyrtan follow` with no URL takes-the first seed that answers, then the first hub from the snapshot. An entry is a hint,-not a route: to use a hub you found, follow it. A hub can also hold trees+any hub's `/directory`) shows the result. An entry is a hint, not a+route: to use a hub you found, follow it. A hub can also hold treesitself, `pyrtan hub --trees DIR`, so a whole federated swarm can be one process on one box.+### Networks++A **network** is a name and a secret its hubs sign their follows with. A+hub can be in any number of them, and any hub can start one:++```+pyrtan network new home --at https://alpha.example # a network of one; prints its invite+pyrtan join home:Qx…@https://alpha.example --at https://beta.example # beta is in home, following alpha+pyrtan join --at https://gamma.example # gamma is in the default network+```++A follow made in a network names it, by an id drawn from the name and+the secret, and is signed with the secret, so only a hub holding the+invite gets in, and two networks that happen to share a name never+meet. What a hub hears of the directory over a link stays in that+link's network: a private network's hubs are passed on only within it,+shown at `/directory` only to the operator, and never handed to a+keeper. Leaving a network (`pyrtan network leave`) cuts the links made+in it and forgets what was heard there. A follow made outside any+network, and a member's, works as it always has.++The **default network** is the one pyrtan.com publishes in+`rendezvous/network.json`: `pyrtan`, with a secret that is public on+purpose. It names the network rather than guarding it: anyone may join,+and what keeps a stranger's hub harmless is that following is still the+one act of trust. `pyrtan join` (or `pyrtan hub --join default`, or+`$PYRTAN_JOIN=default`) puts a hub in it and follows the first hub in+`rendezvous/seeds.json` that answers, then the first from the snapshot;+when none does, the hub is the network so far, and others join through+it. No hub has to be always up: every hour pyrtan.com's build asks+every hub it knows of for its directory (the seeds, then everyone the+last snapshot or a hub names), keeps the default network's part, and+publishes it as `rendezvous/directory.json`, so the directory stays+current while any listed hub is up, and pyrtan.com itself is static+files on GitHub Pages. To be a seed, a hub in the default network,+started `--listed` and `--cors https://pyrtan.com`, goes into+`seeds.json` by pull request.+## Members, and what they keep A browser or a phone cannot be reached, so it joins a hub the other way@@ -520,15 +561,17 @@ _meta/ the design record, the tree format reference, and thedeploy/ Dockerfile at the root; a compose stack for OpenMediaVault and a systemd unit site/ pyrtan.com: the landing page, the seed list, the catalog- viewer, and the build the pages workflow publishes+ viewer, the source browser (files, releases, branches and+ history), and the build the pages workflow publishes``` ## Status-0.10.0. The core, the service, the hub, the command, the console, the+0.11.0. The core, the service, the hub, the command, the console, thecatalog (namespaces, publishing by pull request through the gate, gap reports gossiped back to authors), the language rules, federation-between hubs, the gossiped directory, members (a browser or a phone as+between hubs, networks (any hub starts one; pyrtan.com publishes the+default), the gossiped directory, members (a browser or a phone aspart of the swarm), the blobs they keep and the trees they run (`pyrtan.js`, conformant with the Python runtime), six forms of I/O (HTTP, a browser, files, programs, serial lines, MQTT queues), each@@ -541,8 +584,6 @@ and its own box, and follow other swarms. Not yet built: hosting.See [`_meta/design.md`](_meta/design.md) for the runtime's design and [`_meta/platform.md`](_meta/platform.md) for pyrtan.com.-Pyrtan is a product of [Hillvon.io](https://hillvon.io).-## License Pyrtan is free software under the GNU Affero General Public License, diff --git a/_meta/chat.md b/_meta/chat.md index 5b58b0c..0ac637d 100644 --- a/_meta/chat.md +++ b/_meta/chat.md@@ -2,7 +2,7 @@| | | |---|---|-| **Status** | **Built, v0.10.** The lobby, bots made of trees, strangers across a follow, the console's Chat tab and `pyrtan chat`; tests with two hubs over real sockets and the console driven in a browser. |+| **Status** | **Built, v0.11.** The lobby, bots made of trees, strangers across a follow, the console's Chat tab and `pyrtan chat`; tests with two hubs over real sockets and the console driven in a browser. || **Date** | 2026-10-27 | | **Decision** | A hub started `--chat` keeps a lobby where anyone who can reach it is paired at random with another person, on this hub or on a hub it follows, or with a bot. A bot is a holder of trees, and what it says is what `/intent` would have said: the chat adds no model and no new kind of knowledge. Nothing is kept. | | **Where** | `src/pyrtan/chat.py` (the lobby), `src/pyrtan/hub.py` (`/chat`, `intent(only=…)`), `src/pyrtan/federation.py` (`Meet`, `Say`, `Part`), `src/pyrtan/page.py` (the Chat tab), `src/pyrtan/cli.py` (`pyrtan chat`, `hub --chat`), `examples/small-talk/`. |@@ -130,12 +130,12 @@ meanwhile; two seats that ask each other's hubs at once both getorder, one at a time, and a reply for a conversation that has moved on is dropped. - Two hubs on different versions: one without a lobby answers a `Meet`- `no-chat`, one from before v0.10 refuses the activity type; either is+ `no-chat`, one from before v0.11 refuses the activity type; either is`met: false` to the asker. ## 6. Sequencing-- **v0.10, built:** the lobby, bots, strangers across a follow, the Chat+- **v0.11, built:** the lobby, bots, strangers across a follow, the Chattab, `pyrtan chat`, `small-talk`, `trigger_inputs`. - **Later:** "Stranger is typing…" (a transient field on the poll, and a `Typing` activity across hubs); a member's lobby that works offline diff --git a/_meta/design.md b/_meta/design.md index 494c4bb..9792584 100644 --- a/_meta/design.md +++ b/_meta/design.md@@ -1,4 +1,4 @@-# Design — Pyrtan: a core set of expandable services that act on all forms of I/O through hardcoded decision trees+# Design — Pyrtan: a core set of expandable services that act on all forms of I/O through inspectable decision trees| | | |---|---| diff --git a/_meta/federation.md b/_meta/federation.md index c5be390..cc12cea 100644 --- a/_meta/federation.md +++ b/_meta/federation.md@@ -212,18 +212,24 @@ visitor's browser.**Seeds.** pyrtan.com publishes `rendezvous/seeds.json`, a static list of hub URLs maintained by pull request, the role of a torrent's bootstrap-nodes. `pyrtan follow` with no URL follows the first seed whose `/actor`-answers; `pyrtan hub --follow seed` does the same on start. A seed is an-ordinary federated hub whose operator keeps it up; the first one is the-author's NAS. `$PYRTAN_SEEDS` points at another list (a URL or a file),+nodes. `pyrtan join` (and `pyrtan follow` with no URL) puts a hub in the+default network (§7a) and follows the first seed whose `/actor` answers;+`pyrtan hub --join default` (or `--follow seed`) does the same on start.+A seed is an ordinary federated hub in the default network, up when its+operator's box is; nothing needs it to be up always (below). `$PYRTAN_SEEDS` points at another list (a URL or a file),`$PYRTAN_RENDEZVOUS` at one hub. **The snapshot.** Seeds are home boxes, and home boxes go off. So the site's build, which GitHub runs on every push and once an hour, asks each-seed's `/directory` and bakes the merged answer into-`rendezvous/directory.json` with the time, keeping the last snapshot's-entries for a seed that did not answer and dropping anything a week old-(`site/snapshot.py`, standard library, no pyrtan). The landing page+seed's `/directory`, then every other hub it knows of (from the last+snapshot, and from what the hubs it asked named), in parallel rounds up+to a bound, and bakes the merged answer into `rendezvous/directory.json`+with the time, keeping the default network's part of each directory (and+what was heard outside any network), keeping the last snapshot's entries+for hubs that did not answer and dropping anything a week old+(`site/snapshot.py`, standard library, no pyrtan). The build is the+seed's directory work, done by GitHub: the directory stays current while+any listed hub is up, whether or not a seed is. The landing pageshows it when no seed answers the visitor's browser; `pyrtan directory` falls back to it; `pyrtan follow` with no URL tries its hubs after the seeds (`$PYRTAN_SNAPSHOT` overrides). A new hub can therefore bootstrap@@ -245,6 +251,52 @@ because nobody can reach its inbox. To be routable it needs an address:a port forward behind TLS, or a tunnel (Cloudflare Tunnel, Tailscale Funnel) that costs nothing and opens no port. The deploy guide says how.+## 7a. Networks++v0.10 lets one hub be in several federations at once and lets any hub+start one. A **network** is a name and a secret. `pyrtan network new+home` makes one on a hub, a network of one, and prints its invite,+`home:<secret>@<the hub's URL>`; `pyrtan join <invite>` on another hub+puts it in the network and follows the hub the invite names in it. A+hub keeps its networks in its state file.++**Follows.** A Follow made in a network carries `network`, an id drawn+from the name and the secret (`network_id`, a hash: it gives neither+away, and two networks that share a name have different ids), and is+signed with the network's secret, not the pair secret. The followed hub+takes it only if it is in a network with that id and the signature holds+under its secret; otherwise `403 not-in-network` or `401 unsigned`. A+Follow without `network` is judged as before (§2): open, the+`--federation-secret`, or refused by `--closed`, which now means "no+follows outside my networks". Both sides remember the network on the+link (`Remote.network`). A pair of hubs has one link, so a follow in a+second network moves the link there.++**Directory.** Each entry remembers the network of the link it arrived+over, and gossip over a link carries only that network's entries (and+the hub's own, if listed: being listed is the hub's choice, everywhere).+So what a hub hears in a private network is passed on only within it.+`GET /directory` without the operator's token shows what was heard+outside any network or in a public one; with it, everything, each row+naming its networks. The `directory` blob a hub's keepers hold is the+public part only. `pyrtan network leave` unfollows the hubs followed in+the network, drops those that followed in it and forgets its entries.++**The default network.** pyrtan.com publishes `rendezvous/network.json`:+the name `pyrtan` and a secret that is public on purpose. A secret on a+public page guards nothing, and is not meant to: it names the network,+so the public directory is one network among others and a private one+never mixes into it. What makes a stranger's hub harmless is what always+did: following is the one act of trust (§1), being followed grants only+the right to ask, and the operator's token guards the rest. A hub whose+operator wants a guarded federation makes a network and shares its+invite privately.++**Not taken.** Keys per network (an Ed25519 key a network signs members+with) would let a network expel a member without changing its secret.+Today expelling means a new network and new invites. That is the price+of HMAC, and small for networks of friends.+## 8. Members A browser or a phone cannot be followed, because nobody can reach it. It diff --git a/deploy/README.md b/deploy/README.md index 90721da..0a0c7ca 100644 --- a/deploy/README.md +++ b/deploy/README.md@@ -66,11 +66,16 @@ To let this swarm answer for another (a friend's hub, or any hub in thedirectory), give the hub a public address, with a tunnel that opens no port (`cloudflared tunnel --url http://127.0.0.1:8700`, or `tailscale funnel 8700`) or a port forward behind TLS, then set in the compose-file's environment `PYRTAN_ADVERTISE` (that address), `PYRTAN_FOLLOW:-seed` and `PYRTAN_LISTED: "true"`, with `PYRTAN_TOKEN` set. The hub then-follows the first seed from pyrtan.com, routes to it, and is listed in-the directory every hub gossips; `pyrtan follow URL --at <your hub>---token …` follows any other. Following is one-directional trust: your+file's environment `PYRTAN_ADVERTISE` (that address), `PYRTAN_JOIN:+default` and `PYRTAN_LISTED: "true"`, with `PYRTAN_TOKEN` set and+`PYRTAN_EGRESS: public` (hubs that follow yours may ask it to run trees,+and `any` would let them point a tree at your LAN). The hub then joins+the default network pyrtan.com publishes, follows the first seed in it,+routes to it, and is listed in the directory every hub gossips; `pyrtan+follow URL --at <your hub> --token …` follows any other. For a network+of your own, `pyrtan network new home --at <your hub> --token …` prints+an invite; a friend's hub joins with `pyrtan join <invite>`, or by+adding it to its `PYRTAN_JOIN`. Following is one-directional trust: yourhub routes to the hubs you follow, and hubs that follow yours may ask it. See [`../_meta/federation.md`](../_meta/federation.md). diff --git a/deploy/openmediavault/docker-compose.yml b/deploy/openmediavault/docker-compose.yml index 41851df..9e106f0 100644 --- a/deploy/openmediavault/docker-compose.yml +++ b/deploy/openmediavault/docker-compose.yml@@ -34,10 +34,14 @@ x-pyrtan: &pyrtan# PYRTAN_TOKEN: "change-me-to-something-long" PYRTAN_EGRESS: any # Federation: once the hub is reachable from outside (a tunnel or a- # port-forward with TLS), give it its public URL and it follows the- # seeds at pyrtan.com and lists itself in the directory. Needs the token.+ # port-forward with TLS), give it its public URL and it joins the+ # default network pyrtan.com publishes, follows a seed in it, and lists+ # itself in the directory. Needs the token, and PYRTAN_EGRESS: public+ # above: hubs that follow this one may ask it to run trees.+ # PYRTAN_JOIN may also hold invites to networks of your own+ # (NAME:SECRET@HUB, from `pyrtan network new`), space-separated.# PYRTAN_ADVERTISE: "https://hub.example.net"- # PYRTAN_FOLLOW: "seed"+ # PYRTAN_JOIN: "default"# PYRTAN_LISTED: "true" # PYRTAN_REPORT_GAPS: "true" # opt in: the gaps your catalog trees meet (tree, node, value; never inputs) reach their authors via pyrtan.com # PYRTAN_WATCH: "true" # run the trees' [[watches]]: a clock, a file, a topic or a serial line starts a run diff --git a/pyproject.toml b/pyproject.toml index 5c829e9..20f6aa5 100644 --- a/pyproject.toml +++ b/pyproject.toml@@ -1,7 +1,7 @@[project] name = "pyrtan"-version = "0.10.0"-description = "Open-source total automation: a core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees."+version = "0.11.0"+description = "Open-source total automation: a core set of expandable services that act on all forms of I/O through inspectable decision trees."readme = "README.md" license = "AGPL-3.0-or-later" license-files = ["LICENSE"] diff --git a/site/README.md b/site/README.md index 74d4c8f..9e68a8e 100644 --- a/site/README.md +++ b/site/README.md@@ -9,8 +9,10 @@ static directory and publishes it.| `/` | the landing page, with the live directory read from the seeds in the visitor's browser | `site/index.html` | | `/console/` | the console (installable: manifest, service worker and `pyrtan.js`, the tree runtime, beside it); point it at any hub started with `--cors https://pyrtan.com`; its Member tab joins that hub, and runs its trees when joined with the token | `pyrtan console --out` | | `/catalog/` | the trees people can `pyrtan pull`, as `namespace/name.toml`, and `index.json` the command reads | `pyrtan index examples catalog --copy`, through `site/gate.py` |+| `/theme.js` | the wordmark's colour picker the landing page and the catalog load (the console carries its own copy inline) | `src/pyrtan/js/theme.js` |+| `/rendezvous/network.json` | the default network: its name, and a secret that is public on purpose | `site/rendezvous/network.json` || `/rendezvous/seeds.json` | hubs to follow first | `site/rendezvous/seeds.json` |-| `/rendezvous/directory.json` | a snapshot of the directory as the seeds know it, refreshed hourly | `site/snapshot.py` |+| `/rendezvous/directory.json` | a snapshot of the default network's directory, from every hub the build can reach, refreshed hourly | `site/snapshot.py` || `/rendezvous/reports.json` | the gap reports the seeds carry for catalog trees, refreshed hourly | `site/snapshot.py --reports` | `site/build.sh out` builds the same thing locally; open `out/index.html`.@@ -56,23 +58,30 @@ reports the seeds carry (from hubs that opted in with `--report-gaps`),merged hourly, which the catalog page shows beside each tree and `pyrtan reports` reads when given no hub.-## The seeds--`site/rendezvous/seeds.json` lists hubs to follow first. A seed is any-federated hub its operator keeps up, started with `--listed` and `--cors-https://pyrtan.com`, reachable from outside (a tunnel or a port forward-behind TLS; [`deploy/README.md`](../deploy/README.md) says how). Put-yours in by pull request. The landing page asks each seed's `/directory`-from the visitor's browser, so the directory on pyrtan.com is live and-needs no backend; the console link on each row opens-`/console/?hub=<url>`. When no seed answers (every seed is a home box,-and home boxes go off), the page shows the snapshot instead, marked with-its time: `site/snapshot.py` asks the seeds on every build, hourly on-the schedule, keeps the last published snapshot's entries for any seed-that is down, and drops entries unheard of for a week. `pyrtan follow`-with no URL tries the snapshot's hubs after the seeds, and `pyrtan-directory` falls back to it, so a new hub can bootstrap from any listed-hub that happens to be up, not only from a seed.+## The default network and the seeds++`site/rendezvous/network.json` is the default network: the name+`pyrtan` and a secret that is public on purpose (it names the network,+it guards nothing; see `_meta/federation.md` §7a). `pyrtan join` with+no invite, `pyrtan hub --join default` and `$PYRTAN_JOIN=default` put a+hub in it.++`site/rendezvous/seeds.json` lists hubs to follow first. A seed is a+federated hub in the default network (`--join default`), started with+`--listed` and `--cors https://pyrtan.com`, reachable from outside (a+tunnel or a port forward behind TLS; [`deploy/README.md`](../deploy/README.md)+says how). It need not be up always. Put yours in by pull request. The+landing page asks each seed's `/directory` from the visitor's browser,+so the directory on pyrtan.com is live and needs no backend; the+console link on each row opens `/console/?hub=<url>`. When no seed+answers, the page shows the snapshot instead, marked with its time:+`site/snapshot.py` asks the seeds on every build, hourly on the+schedule, then every other hub it knows of, keeps the default network's+part of what they say, keeps the last published snapshot's entries for+any hub that is down, and drops entries unheard of for a week. `pyrtan+join` tries the snapshot's hubs after the seeds, and `pyrtan directory`+falls back to it, so a new hub can join through any listed hub that+happens to be up, not only a seed.## Later: hosting diff --git a/site/build.sh b/site/build.sh index e2f584f..18f0bc3 100755 --- a/site/build.sh +++ b/site/build.sh@@ -1,18 +1,28 @@#!/bin/sh # Assemble pyrtan.com into one static directory: the landing page, the-# console, the catalog (built from examples/ and catalog/), the seed list. Run from the+# console, the catalog (built from examples/ and catalog/), the seed list, the source. Run from the# repository root with pyrtan installed (`uv run site/build.sh`); the Pages # workflow runs the same thing. set -eu out="${1:-out}" rm -rf "$out"-mkdir -p "$out/console" "$out/catalog" "$out/rendezvous"+mkdir -p "$out/console" "$out/catalog" "$out/rendezvous" "$out/source"out="$(cd "$out" && pwd)"-cp site/index.html site/CNAME "$out/"-cp site/rendezvous/seeds.json "$out/rendezvous/"-# The directory as the seeds know it now, keeping the last published-# snapshot's entries when a seed is down (SNAPSHOT_PREVIOUS may add more).+cp site/CNAME "$out/"+# The source: files, releases, branches and history, from this checkout's git.+python3 site/source.py "$out/source"+# The landing page carries the version of the pyrtan it was built with.+version="$(pyrtan --version | cut -d' ' -f2)"+sed "s/__VERSION__/$version/g" site/index.html > "$out/index.html"+# The wordmark's colour picker, shared by the landing page and the catalog (the console inlines it).+cp src/pyrtan/js/theme.js "$out/"+cp site/rendezvous/seeds.json site/rendezvous/network.json "$out/rendezvous/"+# The directory as the hubs know it now (the seeds, then every hub the+# last snapshot or a hub names), the default network's part only, keeping+# the last published snapshot's entries for hubs that are down+# (SNAPSHOT_PREVIOUS may add more).python3 site/snapshot.py site/rendezvous/seeds.json "$out/rendezvous/directory.json" \+ --network site/rendezvous/network.json \--previous https://pyrtan.com/rendezvous/directory.json \ --previous https://b-i-t-8.github.io/pyrtan/rendezvous/directory.json \ ${SNAPSHOT_PREVIOUS:+--previous "$SNAPSHOT_PREVIOUS"} \ diff --git a/site/catalog/index.html b/site/catalog/index.html index 892c031..db974fb 100644 --- a/site/catalog/index.html +++ b/site/catalog/index.html@@ -5,9 +5,12 @@<meta name="viewport" content="width=device-width, initial-scale=1"> <title>Pyrtan catalog</title> <meta name="description" content="Trees you can pull into your own Pyrtan swarm, checked and indexed from the repository.">+<meta name="theme-color" content="#c2410c">+<script src="../theme.js"></script><style>- :root { --bg:#fbfaf7; --fg:#1d1d1b; --mute:#6b6a64; --line:#e3e1da; --card:#ffffff; --accent:#2b5db8; --soft:#f1efe9 }- @media (prefers-color-scheme: dark) { :root { --bg:#141513; --fg:#ecebe6; --mute:#9d9c95; --line:#2b2c29; --card:#1c1d1b; --accent:#7fa6f0; --soft:#222420 } }+ :root { --bg:#fbfaf7; --fg:#1d1d1b; --mute:#6b6a64; --line:#e3e1da; --card:#ffffff; --pyr:#c2410c; --pyr-fg:#fff; --tan:#a8804f; --tan-fg:#fff; --soft:#f1efe9 }+ @media (prefers-color-scheme: dark) { :root { --bg:#141513; --fg:#ecebe6; --mute:#9d9c95; --line:#2b2c29; --card:#1c1d1b; --pyr:#fb8a4c; --pyr-fg:#101318; --tan:#d2b48c; --tan-fg:#101318; --soft:#222420 } }+ :root { --accent:var(--pyr); --accent-fg:var(--pyr-fg) }* { box-sizing:border-box } body { margin:0; background:var(--bg); color:var(--fg); font:16px/1.5 system-ui,-apple-system,"Segoe UI",sans-serif } main { max-width:920px; margin:0 auto; padding:32px 16px 64px }@@ -19,16 +22,19 @@code { font:13px ui-monospace,SFMono-Regular,Menlo,monospace; background:var(--soft); padding:1px 5px; border-radius:4px } pre { margin:8px 0 0; padding:8px 12px; background:var(--soft); border-radius:8px; font:13px/1.45 ui-monospace,SFMono-Regular,Menlo,monospace; overflow:auto } a { color:var(--accent) }+ .pyr { color:var(--pyr) } .tan { color:var(--tan) }+ :focus-visible { outline:2px solid var(--tan); outline-offset:2px }+ ::selection { background:var(--tan); color:var(--tan-fg) }footer { color:var(--mute); font-size:13px; margin-top:32px } </style> </head> <body> <main>- <h1>Pyrtan catalog</h1>+ <h1><span class="pyr" data-theme-part="pyr">Pyr</span><span class="tan" data-theme-part="tan">tan</span> catalog</h1><p class="lede">Trees you can pull into your own swarm. Each is one TOML file, checked by the repository's tests, with the licence its author chose, under the namespace of whoever published it. <code>pyrtan pull NAMESPACE/NAME</code> verifies the file against <a href="index.json">index.json</a> before writing it. Where hubs that opted in have met a value a tree had no branch for, the <a href="../rendezvous/reports.json">gap reports</a> say so beside it.</p> <input type="search" id="q" placeholder="filter: weather, github, read-only…" autocomplete="off"> <div id="list"><p class="mute">Loading index.json…</p></div>- <footer>To publish a tree: <code>pyrtan publish my-tree.toml --into <checkout> --owner <your GitHub login></code>, then open a pull request; the gate checks it, a maintainer merges, and it is here on the next build. <a href="../">pyrtan.com</a> · <a href="https://github.com/b-i-t-8/pyrtan/tree/main/catalog">how</a> · <a href="https://github.com/b-i-t-8/pyrtan">source</a></footer>+ <footer>To publish a tree: <code>pyrtan publish my-tree.toml --into <checkout> --owner <your GitHub login></code>, then open a pull request; the gate checks it, a maintainer merges, and it is here on the next build. <a href="../">pyrtan.com</a> · <a href="../source/blob/catalog/README.md.html">how</a> · <a href="../source/">source</a></footer></main> <script> const esc = (s) => String(s).replace(/[&<>"]/g, c => ({"&":"&","<":"<",">":">",'"':"""}[c])); diff --git a/site/index.html b/site/index.html index 37bc9ef..392657e 100644 --- a/site/index.html +++ b/site/index.html@@ -4,10 +4,13 @@<meta charset="utf-8"> <meta name="viewport" content="width=device-width, initial-scale=1"> <title>Pyrtan</title>-<meta name="description" content="Open-source total automation: a core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees. No model in the loop.">+<meta name="description" content="Open-source total automation: a core set of expandable services that act on all forms of I/O through inspectable decision trees. No model in the loop.">+<meta name="theme-color" content="#c2410c">+<script src="theme.js"></script><style>- :root { --bg:#fbfaf7; --fg:#1d1d1b; --mute:#6b6a64; --line:#e3e1da; --card:#ffffff; --accent:#2b5db8; --accent-fg:#fff; --soft:#f1efe9 }- @media (prefers-color-scheme: dark) { :root { --bg:#141513; --fg:#ecebe6; --mute:#9d9c95; --line:#2b2c29; --card:#1c1d1b; --accent:#7fa6f0; --accent-fg:#101318; --soft:#222420 } }+ :root { --bg:#fbfaf7; --fg:#1d1d1b; --mute:#6b6a64; --line:#e3e1da; --card:#ffffff; --pyr:#c2410c; --pyr-fg:#fff; --tan:#a8804f; --tan-fg:#fff; --soft:#f1efe9 }+ @media (prefers-color-scheme: dark) { :root { --bg:#141513; --fg:#ecebe6; --mute:#9d9c95; --line:#2b2c29; --card:#1c1d1b; --pyr:#fb8a4c; --pyr-fg:#101318; --tan:#d2b48c; --tan-fg:#101318; --soft:#222420 } }+ :root { --accent:var(--pyr); --accent-fg:var(--pyr-fg) }* { box-sizing:border-box } body { margin:0; background:var(--bg); color:var(--fg); font:17px/1.55 system-ui,-apple-system,"Segoe UI",sans-serif } main { max-width:820px; margin:0 auto; padding:48px 16px 64px }@@ -16,6 +19,7 @@h2 { font-size:19px; margin:36px 0 10px } .actions { display:flex; flex-wrap:wrap; gap:10px; margin:0 0 8px } .actions a { display:inline-block; padding:10px 16px; border-radius:8px; text-decoration:none; font-weight:600; border:1px solid var(--line); background:var(--card); color:var(--fg) }+ .actions a:hover { border-color:var(--tan) }.actions a.primary { background:var(--accent); color:var(--accent-fg); border-color:var(--accent) } pre { padding:12px 14px; background:var(--card); border:1px solid var(--line); border-radius:10px; overflow:auto; font:13.5px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace; white-space:pre-wrap } code { font:14px ui-monospace,SFMono-Regular,Menlo,monospace; background:var(--soft); padding:1px 5px; border-radius:4px }@@ -24,6 +28,9 @@.card { background:var(--card); border:1px solid var(--line); border-radius:10px; padding:14px 16px } .card h3 { margin:0 0 6px; font-size:16px } .card p { margin:0; color:var(--mute); font-size:15px } a { color:var(--accent) }+ .pyr { color:var(--pyr) } .tan { color:var(--tan) }+ :focus-visible { outline:2px solid var(--tan); outline-offset:2px }+ ::selection { background:var(--tan); color:var(--tan-fg) }footer { color:var(--mute); font-size:14px; margin-top:48px; border-top:1px solid var(--line); padding-top:16px } table { width:100%; border-collapse:collapse; font-size:15px } td,th { text-align:left; padding:6px 8px; border-top:1px solid var(--line); vertical-align:top } th { color:var(--mute); font-weight:600; border-top:0 } .link { display:inline-block; width:9px; height:9px; border-radius:50%; margin-right:6px; vertical-align:middle; background:var(--mute) }@@ -33,14 +40,14 @@</head> <body> <main>- <h1>Pyrtan</h1>- <p class="lede">Open-source total automation: a core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees. It answers like an assistant. Inside, there is no model.</p>+ <h1><span class="pyr" data-theme-part="pyr">Pyr</span><span class="tan" data-theme-part="tan">tan</span></h1>+ <p class="lede">Open-source total automation: a core set of expandable services that act on all forms of I/O through inspectable decision trees. It answers like an assistant. Inside, there is no model.</p><div class="actions"> <a class="primary" href="console/">Open the console</a> <a href="catalog/">Browse the catalog</a> <a href="#directory">The directory</a>- <a href="https://github.com/b-i-t-8/pyrtan">Source on GitHub</a>+ <a href="source/">Browse the source</a></div> <pre>$ pyrtan run "what's the weather in Oslo" --trees examples@@ -72,17 +79,18 @@ pyrtan serve trees --port 8701 # then open http://127<p>Every hub and service serves the console at <code>/</code>. The same console is at <a href="console/">pyrtan.com/console</a>: point it at your own hub, started with <code>--token</code> and <code>--cors https://pyrtan.com</code>, and ask it from anywhere. Nothing about your swarm passes through this site; the page talks to your hub directly.</p> <h2 id="directory">The directory</h2>- <p>Hubs find each other without a server. A hub started <code>--listed</code> announces itself to the hubs it follows and that follow it; every hub keeps what it hears and passes it on. This page asks the <a href="rendezvous/seeds.json">seed hubs</a> from your browser and shows what they know; when none answers, it shows the <a href="rendezvous/directory.json">snapshot</a> pyrtan.com takes from them every hour. An entry is a hint, not a route: to use a hub, follow it from your own.</p>+ <p>Hubs find each other without a server. A hub started <code>--listed</code> announces itself to the hubs it follows and that follow it; every hub keeps what it hears and passes it on. This is the <a href="rendezvous/network.json">default network</a>, the one <code>pyrtan join</code> puts a hub in; any hub can also start a network of its own and share its invite, and a private network's hubs never show here. This page asks the <a href="rendezvous/seeds.json">seed hubs</a> from your browser and shows what they know; when none answers, it shows the <a href="rendezvous/directory.json">snapshot</a> pyrtan.com takes every hour from every hub it can reach. An entry is a hint, not a route: to use a hub, follow it from your own.</p><div class="card"><table id="hubs"><thead><tr><th>Heard</th><th>Hub</th><th>Holds</th><th></th></tr></thead><tbody><tr><td colspan="4" class="mute" id="hubs-note">Asking the seeds…</td></tr></tbody></table></div> <pre>pyrtan hub --federate --advertise https://your-hub --listed --token … # your hub, behind a tunnel or TLS-pyrtan follow --at https://your-hub --token … # follow the first seed; route to it+pyrtan join --at https://your-hub --token … # be in the default network+pyrtan network new home --at https://your-hub --token … # or start your own: prints its invitepyrtan directory # who the swarm knows</pre>- <p>To become a seed, add your hub's URL to <code>rendezvous/seeds.json</code> by pull request.</p>+ <p>To become a seed, add your hub's URL to <code>rendezvous/seeds.json</code> by pull request. It need not be up all the time: the hourly snapshot keeps the directory from every listed hub that is.</p><h2>Hosting</h2> <p>Self-hosting is the product. A hosted option, running the same open-source hub per customer with secrets, retention, scheduled re-runs and alerts around it, is planned; it will speak the same API, so a swarm can move either way without changing a caller.</p>- <footer>Pyrtan is free software under the AGPL-3.0-or-later, a product of <a href="https://hillvon.io">Hillvon.io</a>. Trees you write are your own.</footer>+ <footer>Pyrtan v__VERSION__. Trees you write are your own.</footer></main> <script> const esc = (s) => String(s).replace(/[&<>"]/g, c => ({"&":"&","<":"<",">":">",'"':"""}[c])); diff --git a/site/rendezvous/network.json b/site/rendezvous/network.json new file mode 100644 index 0000000..84c8b5b --- /dev/null +++ b/site/rendezvous/network.json@@ -0,0 +1,6 @@+{+ "pyrtan-network": 1,+ "about": "The default network: `pyrtan join` with no invite (or `pyrtan hub --join default`) puts a hub in it and follows the first seed in seeds.json that answers, then the snapshot's hubs. Its secret is public on purpose: it names the network rather than guarding it, so anyone may join, and following stays one-directional trust. A network of your own is `pyrtan network new NAME`; its invite is yours to share.",+ "name": "pyrtan",+ "secret": "public-default-network-of-pyrtan-com"+}diff --git a/site/snapshot.py b/site/snapshot.py index fb19ebd..3ff3327 100644 --- a/site/snapshot.py +++ b/site/snapshot.py@@ -1,10 +1,16 @@"""Bake a snapshot of the directory into the site. GitHub Actions runs this on every build and on a schedule: it asks each-seed hub for its ``/directory``, merges what they say with the snapshot-already published (so a build when every seed is down keeps the last good-one rather than publishing an empty list), drops entries nobody has heard-of for a week, and writes ``rendezvous/directory.json``. The landing page+seed hub for its ``/directory``, then every other hub it knows of (from+the last snapshot and from what the hubs it asked said), so the directory+stays fresh while any listed hub is up, even with every seed down. It+merges what they say with the snapshot already published (so a build when+nobody answers keeps the last good one rather than publishing an empty+list), drops entries nobody has heard of for a week, and writes+``rendezvous/directory.json``. It keeps only the default network's part+of each directory (``--network``, the ``network.json`` the site+publishes) and what was heard outside any network; a private network's+hubs never reach the snapshot. The landing pageshows the live directory when a seed answers the visitor's browser and this snapshot, marked with its time, when none does; ``pyrtan follow`` with no URL tries the hubs in it after the seeds.@@ -13,9 +19,10 @@ Standard library only; no pyrtan import, so it runs before anything isinstalled. python3 site/snapshot.py SEEDS OUT [--previous URL_OR_FILE ...]+ [--network URL_OR_FILE] [--max N][--reports OUT --previous-reports URL_OR_FILE ...]-With ``--reports`` it also asks each seed for ``/reports``, the gap+With ``--reports`` it also asks each hub that answered for ``/reports``, the gapreports hubs that opted in (``--report-gaps``) gossip for catalog trees, merges them the same way, and writes ``rendezvous/reports.json``; the catalog page shows them beside each tree.@@ -25,9 +32,11 @@ from __future__ import annotationsimport argparse import datetime as _dt+import hashlibimport json import sys import time+from concurrent.futures import ThreadPoolExecutorimport urllib.request from pathlib import Path from typing import Any@@ -35,6 +44,8 @@ from typing import AnyTTL = 7 * 86400 REPORT_TTL = 30 * 86400 TIMEOUT = 8.0+MAX_ASK = 200 # hubs asked per run, seeds included+PARALLEL = 16def fetch(source: str) -> Any:@@ -45,11 +56,21 @@ def fetch(source: str) -> Any:return json.loads(Path(source).read_text(encoding="utf-8"))-def merge(into: dict[str, dict[str, Any]], entries: Any, via: str, now: float) -> int:+def network_id(name: str, secret: str) -> str:+ """As pyrtan.federation.network_id; repeated so this runs before pyrtan is installed."""+ return hashlib.sha256(f"pyrtan-network\n{name}\n{secret}".encode("utf-8")).hexdigest()[:16]+++def merge(into: dict[str, dict[str, Any]], entries: Any, via: str, now: float, network: str | None = None) -> int:+ """Take a directory's rows; with ``network`` (an id), only those heard+ in it or outside any network."""taken = 0 for e in entries if isinstance(entries, list) else []: if not isinstance(e, dict) or not isinstance(e.get("id"), str) or not e["id"].startswith(("http://", "https://")): continue+ nets = e.get("networks")+ if network is not None and isinstance(nets, list) and nets and network not in nets:+ continuetry: seen = min(float(e.get("seen", 0)), now) except (TypeError, ValueError):@@ -95,7 +116,9 @@ def main(argv: list[str] | None = None) -> int:ap.add_argument("seeds", help="seeds.json, a path or URL") ap.add_argument("out", help="where to write directory.json") ap.add_argument("--previous", action="append", default=[], help="an earlier directory.json to keep entries from (URL or path; repeatable, first that answers wins)")- ap.add_argument("--reports", default=None, help="also write the gap reports the seeds carry to this reports.json")+ ap.add_argument("--network", default=None, help="the default network's network.json (URL or path): keep only its part of each directory")+ ap.add_argument("--max", type=int, default=MAX_ASK, help=f"hubs to ask at most, seeds included (default {MAX_ASK})")+ ap.add_argument("--reports", default=None, help="also write the gap reports the hubs carry to this reports.json")ap.add_argument("--previous-reports", action="append", default=[], help="an earlier reports.json to keep rows from (URL or path; repeatable, first that answers wins)") args = ap.parse_args(argv) now = time.time()@@ -122,29 +145,55 @@ def main(argv: list[str] | None = None) -> int:report["previous"] = source break+ network: str | None = None+ if args.network:+ try:+ n = fetch(args.network)+ network = network_id(n["name"], n["secret"])+ except Exception as e:+ print(f"snapshot: cannot read the default network from {args.network}: {e}; keeping every network's part", file=sys.stderr)try: seeds = [h.rstrip("/") for h in fetch(args.seeds).get("hubs", []) if isinstance(h, str)] except Exception as e: print(f"snapshot: cannot read seeds from {args.seeds}: {e}", file=sys.stderr) seeds = []- for seed in seeds:+ report["crawled"] = {"answered": [], "silent": []}++ def ask(hub: str) -> tuple[str, Any, Any, str | None]:try:- d = fetch(f"{seed}/directory")+ d = fetch(f"{hub}/directory")except Exception as e:- report["silent"].append({"seed": seed, "error": str(e)[:120]})- continue- if isinstance(d, dict) and isinstance(d.get("hubs"), list):- merge(hubs, d["hubs"], seed, now)- report["answered"].append(seed)- else:- report["silent"].append({"seed": seed, "error": "not a directory"})- if args.reports:+ return hub, None, None, str(e)[:120]+ r = None+ if args.reports and isinstance(d, dict):try:- r = fetch(f"{seed}/reports")- if isinstance(r, dict):- merge_reports(reports, r.get("reports"), now)+ r = fetch(f"{hub}/reports")except Exception: pass+ return hub, d, r, None++ # The seeds first, then every hub the last snapshot heard of, most+ # recent first, then whoever the hubs asked name: rounds of parallel+ # asks until nobody new is named or --max is reached.+ asked: set[str] = set()+ pending = list(dict.fromkeys(seeds + [h["id"] for h in sorted(hubs.values(), key=lambda h: -h["seen"])]))+ while pending and len(asked) < args.max:+ batch = [h for h in pending if h not in asked][: args.max - len(asked)]+ pending = []+ asked.update(batch)+ with ThreadPoolExecutor(max_workers=PARALLEL) as pool:+ results = list(pool.map(ask, batch))+ for hub, d, r, error in results:+ book = report if hub in seeds else report["crawled"]+ if error is None and isinstance(d, dict) and isinstance(d.get("hubs"), list):+ merge(hubs, d["hubs"], hub, now, network)+ book["answered"].append(hub)+ pending.extend(e["id"].rstrip("/") for e in d["hubs"] if isinstance(e, dict) and isinstance(e.get("id"), str) and e["id"].startswith(("http://", "https://")) and e["id"].rstrip("/") not in asked)+ if isinstance(r, dict):+ merge_reports(reports, r.get("reports"), now)+ else:+ book["silent"].append({"seed" if hub in seeds else "hub": hub, "error": error or "not a directory"})+ pending = list(dict.fromkeys(pending))out = { "pyrtan-directory": 1,@@ -155,7 +204,7 @@ def main(argv: list[str] | None = None) -> int:path = Path(args.out) path.parent.mkdir(parents=True, exist_ok=True) path.write_text(json.dumps(out, indent=2) + "\n", encoding="utf-8")- print(f"snapshot: {len(out['hubs'])} hub(s); {len(report['answered'])} seed(s) answered, {len(report['silent'])} silent" + (f"; kept entries from {report['previous']}" if report["previous"] else ""))+ print(f"snapshot: {len(out['hubs'])} hub(s); {len(report['answered'])} seed(s) answered, {len(report['silent'])} silent; {len(report['crawled']['answered'])} other hub(s) answered, {len(report['crawled']['silent'])} silent" + (f"; kept entries from {report['previous']}" if report["previous"] else ""))if args.reports: rows = sorted(reports.values(), key=lambda r: (-r["count"], -r["last"], r["id"])) rpath = Path(args.reports) diff --git a/site/source.py b/site/source.py new file mode 100644 index 0000000..6262d8e --- /dev/null +++ b/site/source.py@@ -0,0 +1,479 @@+"""The source, as pyrtan.com shows it.++ python3 site/source.py OUT [--branch main] [--max-commits 200]++Writes a static browser of this checkout into OUT (the site's ``source/``):+the files at HEAD, a page per directory and per file, README and Markdown+rendered; the releases, one per commit that changed the version in+``pyproject.toml``, each with its commit message as its notes and a+tarball of the tree at that commit; the branches the checkout knows+(``refs/remotes/origin/*``), each with what it holds that HEAD does not;+and the history, a page per commit with its diff. Only git and the+standard library; the Pages workflow checks out with full history so the+branches and releases are all there. Commits show their author's name,+not the address.+"""++from __future__ import annotations++import argparse+import html+import posixpath+import re+import subprocess+import sys+import tomllib+from pathlib import Path+from urllib.parse import quote++ROOT = Path(__file__).resolve().parents[1]+CLONE = "https://github.com/b-i-t-8/pyrtan.git"+MAX_BLOB = 1 << 20 # a file larger than this is listed, not shown+MAX_PATCH = 200_000 # a commit's diff is cut here++esc = html.escape+++def git(*args: str, check: bool = True) -> str:+ return subprocess.run(["git", *args], cwd=ROOT, capture_output=True, text=True, errors="replace", check=check).stdout+++def git_bytes(*args: str) -> bytes:+ return subprocess.run(["git", *args], cwd=ROOT, capture_output=True, check=True).stdout+++# -- pages ---------------------------------------------------------------++CSS = """+ :root { --bg:#fbfaf7; --fg:#1d1d1b; --mute:#6b6a64; --line:#e3e1da; --card:#ffffff; --pyr:#c2410c; --tan:#a8804f; --tan-fg:#fff; --soft:#f1efe9; --add:#e6f4ea; --del:#fbe9e7 }+ @media (prefers-color-scheme: dark) { :root { --bg:#141513; --fg:#ecebe6; --mute:#9d9c95; --line:#2b2c29; --card:#1c1d1b; --pyr:#fb8a4c; --tan:#d2b48c; --tan-fg:#101318; --soft:#222420; --add:#16301f; --del:#3a1d19 } }+ :root { --accent:var(--pyr) }+ * { box-sizing:border-box }+ body { margin:0; background:var(--bg); color:var(--fg); font:16px/1.55 system-ui,-apple-system,"Segoe UI",sans-serif }+ main { max-width:980px; margin:0 auto; padding:32px 16px 64px }+ header { display:flex; flex-wrap:wrap; gap:6px 18px; align-items:baseline; margin:0 0 20px; padding-bottom:12px; border-bottom:1px solid var(--line) }+ header b { font-size:18px } header nav { display:flex; flex-wrap:wrap; gap:4px 14px }+ header nav a.on { color:var(--fg); font-weight:600; text-decoration:none }+ h1 { font-size:22px; margin:0 0 6px; word-break:break-word } h2 { font-size:18px; margin:28px 0 8px }+ a { color:var(--accent) } .mute { color:var(--mute) } .small { font-size:14px }+ .pyr { color:var(--pyr) } .tan { color:var(--tan) }+ :focus-visible { outline:2px solid var(--tan); outline-offset:2px }+ ::selection { background:var(--tan); color:var(--tan-fg) }+ code, pre { font:13px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace }+ code { background:var(--soft); padding:1px 5px; border-radius:4px }+ pre { padding:12px 14px; background:var(--card); border:1px solid var(--line); border-radius:8px; overflow:auto }+ .box { background:var(--card); border:1px solid var(--line); border-radius:8px; overflow:auto }+ table { width:100%; border-collapse:collapse } td, th { text-align:left; padding:6px 10px; border-top:1px solid var(--line); vertical-align:top }+ .box tr:first-child td { border-top:0 } th { color:var(--mute); font-weight:600; font-size:14px }+ td.when { white-space:nowrap; color:var(--mute); font-size:14px; text-align:right }+ td.msg { color:var(--mute); font-size:14px; max-width:420px; overflow:hidden; text-overflow:ellipsis; white-space:nowrap }+ table.code td { border:0; padding:0 10px; white-space:pre; font:13px/1.5 ui-monospace,SFMono-Regular,Menlo,monospace }+ table.code td.n { text-align:right; user-select:none; width:1%; color:var(--mute) } table.code td.n a { color:inherit; text-decoration:none }+ table.code tr:target { background:var(--soft) }+ .add { background:var(--add) } .del { background:var(--del) } .hunk { color:var(--mute) }+ .notes { white-space:pre-wrap } .crumbs { font-size:15px; margin:0 0 12px; word-break:break-word }+ .md { background:var(--card); border:1px solid var(--line); border-radius:8px; padding:4px 22px 16px; margin-top:20px }+ .md table { margin:10px 0 } .md td, .md th { border:1px solid var(--line) } .md pre { background:var(--soft) }+ .release { background:var(--card); border:1px solid var(--line); border-radius:8px; padding:14px 18px; margin:0 0 14px }+ .release h2 { margin:0 0 4px } .tag { display:inline-block; font-size:12px; padding:1px 8px; border-radius:10px; background:var(--soft); color:var(--mute); margin-left:6px }+ details summary { cursor:pointer }+ @media (max-width:640px) { td.msg { display:none } }+"""+++class Site:+ def __init__(self, out: Path, branch: str, version: str):+ self.out, self.branch, self.version = out, branch, version++ def link(self, cur: str, target: str) -> str:+ """A relative href from page ``cur`` to page ``target`` (both under source/)."""+ rel = posixpath.relpath(target, posixpath.dirname(cur) or ".")+ return quote(rel, safe="/#.-_~")++ def write(self, page: str, title: str, body: str, on: str = "") -> None:+ nav = "".join(+ f'<a href="{self.link(page, href)}"{" class=on" if key == on else ""}>{label}</a>'+ for key, label, href in [("code", "Code", "index.html"), ("releases", "Releases", "releases.html"),+ ("branches", "Branches", "branches.html"), ("history", "History", "commits.html")])+ doc = f"""<!doctype html>+<html lang="en">+<head>+<meta charset="utf-8">+<meta name="viewport" content="width=device-width, initial-scale=1">+<title>{esc(title)} · Pyrtan source</title>+<meta name="theme-color" content="#c2410c">+<script src="{self.link(page, '../theme.js')}"></script>+<style>{CSS}</style>+</head>+<body>+<main>+<header><b><a href="{self.link(page, '../index.html')}"><span class="pyr">Pyr</span><span class="tan">tan</span></a> / <a href="{self.link(page, 'index.html')}">source</a></b><nav>{nav}</nav><span class="mute small">v{esc(self.version)} · {esc(self.branch)}</span></header>+{body}+</main>+</body>+</html>+"""+ path = self.out / page+ path.parent.mkdir(parents=True, exist_ok=True)+ path.write_text(doc, encoding="utf-8")+++def blob_page(path: str) -> str:+ return f"blob/{path}.html"+++def tree_page(path: str) -> str:+ return f"tree/{path}/index.html" if path else "index.html"+++def commit_page(sha: str) -> str:+ return f"commit/{sha}.html"+++# -- Markdown, enough for this repository's documents ---------------------++def md_inline(text: str, resolve) -> str:+ out, pos = [], 0+ for m in re.finditer(r"`([^`]+)`|\[([^\]]+)\]\(([^)\s]+)\)|\*\*([^*]+)\*\*", text):+ out.append(esc(text[pos:m.start()]))+ if m.group(1) is not None:+ out.append(f"<code>{esc(m.group(1))}</code>")+ elif m.group(2) is not None:+ out.append(f'<a href="{esc(resolve(m.group(3)), quote=True)}">{md_inline(m.group(2), resolve)}</a>')+ else:+ out.append(f"<b>{md_inline(m.group(4), resolve)}</b>")+ pos = m.end()+ out.append(esc(text[pos:]))+ return "".join(out)+++def markdown(src: str, resolve) -> str:+ lines, out, i = src.splitlines(), [], 0+ para: list[str] = []++ def flush():+ if para:+ out.append(f"<p>{md_inline(' '.join(para), resolve)}</p>")+ para.clear()++ while i < len(lines):+ line = lines[i]+ if line.startswith("```"):+ flush()+ j = i + 1+ while j < len(lines) and not lines[j].startswith("```"):+ j += 1+ out.append(f"<pre>{esc(chr(10).join(lines[i + 1:j]))}</pre>")+ i = j + 1+ continue+ if m := re.match(r"(#{1,6})\s+(.*)", line):+ flush()+ n = len(m.group(1))+ out.append(f"<h{n}>{md_inline(m.group(2), resolve)}</h{n}>")+ elif line.startswith("|"):+ flush()+ rows = []+ while i < len(lines) and lines[i].startswith("|"):+ cells = [c.strip() for c in lines[i].strip().strip("|").split("|")]+ if not all(re.fullmatch(r":?-+:?", c) for c in cells if c):+ rows.append(cells)+ i += 1+ out.append("<table>" + "".join("<tr>" + "".join(f"<td>{md_inline(c, resolve)}</td>" for c in r) + "</tr>" for r in rows) + "</table>")+ continue+ elif m := re.match(r"\s*([-*]|\d+\.)\s+(.*)", line):+ flush()+ tag = "ol" if m.group(1)[0].isdigit() else "ul"+ items = []+ while i < len(lines) and (m := re.match(r"\s*([-*]|\d+\.)\s+(.*)", lines[i])):+ item = [m.group(2)]+ i += 1+ while i < len(lines) and lines[i].startswith(" ") and not re.match(r"\s*([-*]|\d+\.)\s+", lines[i]):+ item.append(lines[i].strip())+ i += 1+ items.append(f"<li>{md_inline(' '.join(item), resolve)}</li>")+ out.append(f"<{tag}>{''.join(items)}</{tag}>")+ continue+ elif not line.strip():+ flush()+ else:+ para.append(line.strip())+ i += 1+ flush()+ return "".join(out)+++# -- what git knows --------------------------------------------------------++def files_at(ref: str) -> list[tuple[str, str, int]]:+ """(path, blob sha, size) for every file at ``ref``."""+ out = []+ for rec in git("ls-tree", "-r", "-l", "-z", ref).split("\0"):+ if not rec:+ continue+ meta, path = rec.split("\t", 1)+ mode, kind, sha, size = meta.split()+ if kind == "blob":+ out.append((path, sha, int(size) if size.isdigit() else 0))+ return out+++def last_commit(ref: str, path: str) -> tuple[str, str, str]:+ """(sha, subject, date) of the last commit at ``ref`` that touched ``path``."""+ rec = git("log", "-1", "--format=%H%x00%s%x00%as", ref, "--", path or ".").strip()+ return tuple(rec.split("\0")) if rec else ("", "", "") # type: ignore[return-value]+++def commits(*revs: str, limit: int, paths: tuple[str, ...] = ()) -> list[dict]:+ raw = git("log", f"-n{limit}", "--format=%H%x00%P%x00%an%x00%as%x00%s%x00%b%x1e", *revs, "--", *paths, check=False)+ out = []+ for rec in raw.split("\x1e"):+ rec = rec.strip("\n")+ if not rec:+ continue+ sha, parents, author, date, subject, body = rec.split("\0", 5)+ out.append({"sha": sha, "parents": parents.split(), "author": author, "date": date, "subject": subject, "body": body.strip()})+ return out+++def version_at(sha: str) -> str | None:+ text = git("show", f"{sha}:pyproject.toml", check=False)+ try:+ return tomllib.loads(text)["project"]["version"]+ except (tomllib.TOMLDecodeError, KeyError):+ return None+++def releases(ref: str) -> list[dict]:+ """The commits at ``ref`` that changed the project's version, newest first."""+ shallow = set(_shallow())+ out = []+ for c in commits("--no-merges", ref, limit=1000, paths=("pyproject.toml",)):+ if c["sha"] in shallow:+ continue # its parent is not in this checkout; the change is unknowable+ v = version_at(c["sha"])+ before = version_at(c["parents"][0]) if c["parents"] else None+ if v and v != before:+ out.append({**c, "version": v})+ return out+++def _shallow() -> list[str]:+ path = ROOT / git("rev-parse", "--git-path", "shallow").strip()+ return path.read_text().split() if path.is_file() else []+++def branches(default: str) -> list[str]:+ """The remote branches, the default first."""+ default = f"origin/{default}"+ refs = git("for-each-ref", "--format=%(refname:short)", "refs/remotes/origin").split()+ return sorted((r for r in refs if r not in ("origin", "origin/HEAD")), key=lambda r: (r != default, r))+++# -- building --------------------------------------------------------------++def notes(body: str, trailers: bool = True) -> str:+ """A commit message's body as HTML: wrapped paragraphs joined, lists, indented lines and+ trailers (``Co-Authored-By: ...``) kept line by line; ``trailers=False`` leaves trailers out."""+ out = []+ for para in re.split(r"\n\s*\n", body.strip()):+ lines = para.splitlines()+ if lines and all(re.match(r"[A-Za-z][\w-]*: \S", line) for line in lines):+ if trailers:+ out.append(f'<p class="notes mute small">{esc(para)}</p>')+ elif any(re.match(r"\s|[-*] |\d+\. ", line) for line in lines):+ out.append(f'<p class="notes">{esc(para)}</p>')+ else:+ out.append(f"<p>{esc(' '.join(line.strip() for line in lines))}</p>")+ return "".join(out)+++def when(date: str) -> str:+ return f'<td class="when">{esc(date)}</td>'+++def build(out: Path, branch: str, max_commits: int) -> int:+ version = version_at("HEAD") or "?"+ site = Site(out, branch, version)+ files = files_at("HEAD")+ head = git("rev-parse", "HEAD").strip()+ shown: dict[str, dict] = {} # every commit that gets a page++ # history: HEAD, then what each branch holds that HEAD does not+ history = commits("HEAD", limit=max_commits)+ for c in history:+ shown[c["sha"]] = c+ branch_rows = []+ for ref in branches(branch):+ name = ref.removeprefix("origin/")+ behind, ahead = (int(n) for n in git("rev-list", "--left-right", "--count", f"HEAD...{ref}").split())+ tip = commits(ref, limit=1)[0]+ own = commits(f"HEAD..{ref}", limit=50)+ for c in [tip, *own]:+ shown.setdefault(c["sha"], c)+ branch_rows.append((name, tip, ahead, behind, own))+ rels = releases("HEAD")+ for c in rels:+ shown.setdefault(c["sha"], c)++ def commit_link(cur: str, c: dict, text: str | None = None) -> str:+ return f'<a href="{site.link(cur, commit_page(c["sha"]))}">{esc(text or c["subject"])}</a>'++ def resolver(cur: str, base_dir: str):+ known = {p for p, _, _ in files}+ dirs = {posixpath.dirname(p) for p in known}+ dirs |= {d for p in list(dirs) for d in _parents(p)}++ def resolve(url: str) -> str:+ if re.match(r"[a-z]+:|#", url):+ return url+ target, _, frag = url.partition("#")+ target = posixpath.normpath(posixpath.join(base_dir, target)) if target else ""+ target = "" if target == "." else target.rstrip("/")+ if target in known:+ return site.link(cur, blob_page(target)) + (f"#{frag}" if frag else "")+ if target in dirs or target == "":+ return site.link(cur, tree_page(target))+ return url+ return resolve++ # Code: a page per directory, a page per file+ tree: dict[str, list[tuple[str, bool]]] = {}+ for path, _, _ in files:+ parts = path.split("/")+ for k in range(len(parts)):+ parent = "/".join(parts[:k])+ entry = ("/".join(parts[:k + 1]), k < len(parts) - 1)+ tree.setdefault(parent, [])+ if entry not in tree[parent]:+ tree[parent].append(entry)+ sizes = {p: (sha, size) for p, sha, size in files}++ for d, entries in tree.items():+ page = tree_page(d)+ rows = []+ if d:+ up = posixpath.dirname(d)+ rows.append(f'<tr><td><a href="{site.link(page, tree_page(up))}">..</a></td><td></td><td></td></tr>')+ for path, is_dir in sorted(entries, key=lambda e: (not e[1], e[0].lower())):+ sha, subject, date = last_commit("HEAD", path)+ name = posixpath.basename(path) + ("/" if is_dir else "")+ href = site.link(page, tree_page(path) if is_dir else blob_page(path))+ msg = f'<a href="{site.link(page, commit_page(sha))}">{esc(subject)}</a>' if sha in shown else esc(subject)+ rows.append(f'<tr><td><a href="{href}">{esc(name)}</a></td><td class="msg">{msg}</td>{when(date)}</tr>')+ top = ""+ if not d:+ latest = rels[0] if rels else None+ top = (f'<p class="mute small">{len(files)} files · {len(history)} commits · {len(branch_rows)} branches'+ + (f' · latest release <a href="releases.html">v{esc(latest["version"])}</a>' if latest else "")+ + (f' · last change {commit_link(page, history[0], history[0]["date"])}' if history else "")+ + f'</p><pre>git clone {esc(CLONE)}</pre>')+ crumbs = _crumbs(site, page, d)+ readme = next((p for p, is_dir in entries if not is_dir and posixpath.basename(p).lower() == "readme.md"), None)+ body = f'{crumbs}{top}<div class="box"><table>{"".join(rows)}</table></div>'+ if readme:+ text = git_bytes("cat-file", "blob", sizes[readme][0]).decode("utf-8", "replace")+ body += f'<div class="md">{markdown(text, resolver(page, d))}</div>'+ site.write(page, d or "Code", body, on="code")++ for path, sha, size in files:+ page = blob_page(path)+ c_sha, subject, date = last_commit("HEAD", path)+ meta = f'<p class="mute small">{size:,} bytes · last changed {date} in ' + (+ f'<a href="{site.link(page, commit_page(c_sha))}">{esc(subject)}</a>' if c_sha in shown else esc(subject)) + "</p>"+ if size > MAX_BLOB:+ content = '<p class="mute">Too large to show here; it is in the clone.</p>'+ else:+ data = git_bytes("cat-file", "blob", sha)+ if b"\0" in data[:8000]:+ content = '<p class="mute">A binary file; it is in the clone.</p>'+ else:+ text = data.decode("utf-8", "replace")+ if path.lower().endswith(".md"):+ content = f'<div class="md">{markdown(text, resolver(page, posixpath.dirname(path)))}</div>'+ else:+ rows = "".join(f'<tr id="L{n}"><td class="n"><a href="#L{n}">{n}</a></td><td>{esc(line)}</td></tr>'+ for n, line in enumerate(text.splitlines(), 1))+ content = f'<div class="box"><table class="code">{rows}</table></div>'+ site.write(page, path, _crumbs(site, page, path) + meta + content, on="code")++ # Releases+ page = "releases.html"+ (out / "releases").mkdir(parents=True, exist_ok=True)+ blocks = []+ for k, r in enumerate(rels):+ name = f"pyrtan-{r['version']}"+ subprocess.run(["git", "archive", "--format=tar.gz", f"--prefix={name}/", "-o", str(out / "releases" / f"{name}.tar.gz"), r["sha"]],+ cwd=ROOT, check=True)+ blocks.append(f'<div class="release" id="v{esc(r["version"])}"><h2>v{esc(r["version"])}{"<span class=tag>latest</span>" if k == 0 else ""}</h2>'+ f'<p class="mute small">{esc(r["date"])} · {commit_link(page, r, r["sha"][:7])} · '+ f'<a href="releases/{quote(name)}.tar.gz">{esc(name)}.tar.gz</a></p>'+ f'<p><b>{esc(r["subject"])}</b></p>{notes(r["body"], trailers=False)}</div>')+ site.write(page, "Releases", "<h1>Releases</h1>" + ("".join(blocks) or '<p class="mute">None yet.</p>'), on="releases")++ # Branches+ page = "branches.html"+ rows = []+ for name, tip, ahead, behind, own in branch_rows:+ mark = '<span class="tag">default</span>' if name == branch else ""+ detail = ""+ if own:+ items = "".join(f'<tr><td>{commit_link(page, c)}</td><td class="mute small">{esc(c["author"])}</td>{when(c["date"])}</tr>' for c in own)+ detail = f'<details><summary class="small">{len(own)} commit{"s" if len(own) != 1 else ""} not on {esc(branch)}</summary><table>{items}</table></details>'+ rows.append(f'<tr><td><code>{esc(name)}</code>{mark}<div class="small">{commit_link(page, tip)}</div>{detail}</td>'+ f'<td class="mute small">{ahead} ahead · {behind} behind</td>{when(tip["date"])}</tr>')+ site.write(page, "Branches", f'<h1>Branches</h1><p class="mute small">Ahead and behind are counted against {esc(branch)} at {commit_link(page, history[0], head[:7]) if history else esc(head[:7])}.</p>'+ f'<div class="box"><table>{"".join(rows) or "<tr><td class=mute>None in this checkout.</td></tr>"}</table></div>', on="branches")++ # History, and a page per commit+ page = "commits.html"+ rows = "".join(f'<tr><td>{commit_link(page, c)}<div class="mute small">{esc(c["author"])} · <code>{c["sha"][:7]}</code></div></td>{when(c["date"])}</tr>' for c in history)+ site.write(page, "History", f'<h1>History of {esc(branch)}</h1><div class="box"><table>{rows}</table></div>', on="history")++ for sha, c in shown.items():+ page = commit_page(sha)+ parents = " ".join(f'<a href="{site.link(page, commit_page(p))}">{p[:7]}</a>' if p in shown else f"<code>{p[:7]}</code>" for p in c["parents"])+ first = c["parents"][0] if c["parents"] else None+ stat = git("diff", "--stat=100", first, sha, check=False) if first else git("show", "--stat=100", "--format=", sha)+ patch = git("diff", first, sha, check=False) if first else git("show", "--format=", sha)+ cut = len(patch) > MAX_PATCH+ lines = []+ for line in patch[:MAX_PATCH].splitlines():+ cls = "hunk" if line.startswith("@@") else "add" if line.startswith("+") and not line.startswith("+++") else "del" if line.startswith("-") and not line.startswith("---") else ""+ lines.append(f'<div class="{cls}">{esc(line) or " "}</div>' if cls else esc(line) + "\n")+ body = (f'<h1>{esc(c["subject"])}</h1><p class="mute small">{esc(c["author"])} · {esc(c["date"])} · <code>{sha}</code>'+ + (f" · parents {parents}" if parents else "") + "</p>"+ + notes(c["body"])+ + f"<h2>Changes</h2><pre>{esc(stat)}</pre><pre>{''.join(lines)}</pre>"+ + ('<p class="mute">The diff is cut here; the rest is in the clone.</p>' if cut else ""))+ site.write(page, c["subject"], body, on="history")++ return len(files) + len(tree) + len(shown) + 3+++def _parents(path: str) -> list[str]:+ parts = path.split("/")+ return ["/".join(parts[:k]) for k in range(1, len(parts))]+++def _crumbs(site: Site, page: str, path: str) -> str:+ parts = path.split("/") if path else []+ links = [f'<a href="{site.link(page, "index.html")}">pyrtan</a>']+ for k, part in enumerate(parts):+ sub = "/".join(parts[:k + 1])+ links.append(esc(part) if k == len(parts) - 1 else f'<a href="{site.link(page, tree_page(sub))}">{esc(part)}</a>')+ return f'<p class="crumbs">{" / ".join(links)}</p>'+++def main(argv: list[str] | None = None) -> int:+ ap = argparse.ArgumentParser(description=__doc__.split("\n\n")[0])+ ap.add_argument("out", type=Path, help="where to write the browser (the site's source/)")+ ap.add_argument("--branch", default="main", help="the name of the branch HEAD is (default main)")+ ap.add_argument("--max-commits", type=int, default=200, help="how much of HEAD's history to show")+ args = ap.parse_args(argv)+ n = build(args.out, args.branch, args.max_commits)+ print(f"wrote {args.out}: {n} pages")+ return 0+++if __name__ == "__main__":+ sys.exit(main())diff --git a/src/pyrtan/__init__.py b/src/pyrtan/__init__.py index 7c7ee6c..b393bf8 100644 --- a/src/pyrtan/__init__.py +++ b/src/pyrtan/__init__.py@@ -1,5 +1,5 @@-"""Pyrtan: a core set of expandable services that act on all forms of I/O through hardcoded,-inspectable decision trees.+"""Pyrtan: a core set of expandable services that act on all forms of I/O through inspectable+decision trees.The public surface is small on purpose:@@ -33,4 +33,4 @@ __all__ = ["Response", ]-__version__ = "0.10.0"+__version__ = "0.11.0"diff --git a/src/pyrtan/cli.py b/src/pyrtan/cli.py index 68e1d38..b0f4e68 100644 --- a/src/pyrtan/cli.py +++ b/src/pyrtan/cli.py@@ -46,7 +46,7 @@ from typing import Anyfrom . import __version__ from .catalog import Catalog, CatalogError, build_index, catalog_url-from .federation import bootstrap_candidates, first_live_seed+from .federation import bootstrap_candidatesfrom .grow import Proposal, propose from .page import MANIFEST, PAGE, PYRTAN_JS, SERVICE_WORKER from .registry import Registry, default_paths@@ -59,7 +59,7 @@ from .tree import TreeError, check, load_treedef main(argv: list[str] | None = None) -> int:- parser = argparse.ArgumentParser(prog="pyrtan", description="A core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees.")+ parser = argparse.ArgumentParser(prog="pyrtan", description="A core set of expandable services that act on all forms of I/O through inspectable decision trees.")parser.add_argument("--version", action="version", version=f"pyrtan {__version__}") sub = parser.add_subparsers(dest="command", required=True)@@ -162,7 +162,8 @@ def main(argv: list[str] | None = None) -> int:_access_args(p) p.add_argument("--federate", action="store_true", help="answer /actor and /inbox so other hubs can follow this one, and follow others") p.add_argument("--advertise", default=os.environ.get("PYRTAN_ADVERTISE"), help="the URL other hubs reach this hub at (implies --federate; default $PYRTAN_ADVERTISE, else this host and port)")- p.add_argument("--follow", action="append", default=[h for h in os.environ.get("PYRTAN_FOLLOW", "").replace(",", " ").split() if h], metavar="URL", help="a hub to follow on start (repeatable; default $PYRTAN_FOLLOW; 'seed' means the first live seed)")+ p.add_argument("--follow", action="append", default=[h for h in os.environ.get("PYRTAN_FOLLOW", "").replace(",", " ").split() if h], metavar="URL", help="a hub to follow on start (repeatable; default $PYRTAN_FOLLOW; 'seed' means join the default network, as --join default)")+ p.add_argument("--join", action="append", default=[h for h in os.environ.get("PYRTAN_JOIN", "").replace(",", " ").split() if h], metavar="INVITE", help="a network to be in on start: 'default' for the public one pyrtan.com publishes, or an invite NAME:SECRET[@HUB] (repeatable; default $PYRTAN_JOIN)")p.add_argument("--federation-secret", default=os.environ.get("PYRTAN_FEDERATION_SECRET"), help="only hubs that sign their Follow with this secret may follow (default $PYRTAN_FEDERATION_SECRET; without it, follows are open)") p.add_argument("--closed", action="store_true", help="take no follows at all (this hub can still follow others)") p.add_argument("--state", metavar="FILE", default=os.environ.get("PYRTAN_STATE"), help="keep who this hub follows and is followed by, and the directory, in this file across restarts (default $PYRTAN_STATE)")@@ -174,11 +175,26 @@ def main(argv: list[str] | None = None) -> int:p.add_argument("--quiet", action="store_true") p = sub.add_parser("follow", help="follow another hub, through a running federated hub of yours")- p.add_argument("url", nargs="?", default=None, help="the other hub's URL (default: the first live seed from $PYRTAN_RENDEZVOUS, $PYRTAN_SEEDS or pyrtan.com)")+ p.add_argument("url", nargs="?", default=None, help="the other hub's URL (default: join the default network, as `pyrtan join`)")+ p.add_argument("--network", help="follow it in this network of yours, so the directory heard over the link stays in it")p.add_argument("--at", default=os.environ.get("PYRTAN_AT", "http://127.0.0.1:8700"), help="your hub (default $PYRTAN_AT or the local hub)") p.add_argument("--listed", action="store_true", help="also put your hub in the directory it gossips, so others can find it") _token_arg(p, "your hub's token, if any")+ p = sub.add_parser("join", help="put your hub in a network: the default one from pyrtan.com, or one an invite names")+ p.add_argument("invite", nargs="?", default="default", help="NAME:SECRET[@HUB] from `pyrtan network new` on another hub, or 'default' (the default)")+ p.add_argument("--at", default=os.environ.get("PYRTAN_AT", "http://127.0.0.1:8700"), help="your hub (default $PYRTAN_AT or the local hub)")+ p.add_argument("--listed", action="store_true", help="also put your hub in the directory it gossips, so others can find it")+ _token_arg(p, "your hub's token, if any")++ p = sub.add_parser("network", help="the networks your hub is in: start one, list them with their invites, leave one")+ p.add_argument("action", choices=["new", "list", "leave"])+ p.add_argument("name", nargs="?", help="the network's name (new, leave)")+ p.add_argument("--public", action="store_true", help="new: show this network's part of the directory to anyone, not only to you")+ p.add_argument("--at", default=os.environ.get("PYRTAN_AT", "http://127.0.0.1:8700"), help="your hub (default $PYRTAN_AT or the local hub)")+ p.add_argument("--json", action="store_true")+ _token_arg(p, "your hub's token, if any")+p = sub.add_parser("unfollow", help="stop following a hub") p.add_argument("url", help="the other hub's URL") p.add_argument("--at", default=os.environ.get("PYRTAN_AT", "http://127.0.0.1:8700"), help="your hub (default $PYRTAN_AT or the local hub)")@@ -262,7 +278,7 @@ def main(argv: list[str] | None = None) -> int:"check": cmd_check, "list": cmd_list, "run": cmd_run, "match": cmd_match, "ask": cmd_ask, "chat": cmd_chat, "serve": cmd_serve, "hub": cmd_hub, "gaps": cmd_gaps, "unanswered": cmd_unanswered, "new": cmd_new, "watch": cmd_watch, "watches": cmd_watches, "grow": cmd_grow, "search": cmd_search, "pull": cmd_pull, "index": cmd_index, "publish": cmd_publish, "console": cmd_console,- "follow": cmd_follow, "unfollow": cmd_unfollow, "peers": cmd_peers, "directory": cmd_directory, "reports": cmd_reports,+ "follow": cmd_follow, "join": cmd_join, "network": cmd_network, "unfollow": cmd_unfollow, "peers": cmd_peers, "directory": cmd_directory, "reports": cmd_reports,"keep": cmd_keep, "kept": cmd_kept, "fetch": cmd_fetch, } return commands[args.command](args)@@ -759,19 +775,12 @@ def _from_report(r: dict[str, Any]) -> dict[str, Any]:def cmd_hub(args: argparse.Namespace) -> int: from .hub import serve_hub- follow = []- for url in args.follow:- if url == "seed":- seed = first_live_seed()- if seed is None:- print("pyrtan hub: no seed answers; nothing followed", file=sys.stderr)- continue- url = seed- follow.append(url)+ follow = [url for url in args.follow if url != "seed"]+ join = list(args.join) + (["default"] if "seed" in args.follow and "default" not in args.join else [])hub, server = serve_hub( args.host, args.port, name=args.name, peers=args.peer, poll_every=args.poll, quiet=args.quiet, token=args.token, cors=args.cors,- federate=args.federate or bool(args.advertise) or bool(follow) or args.listed, advertise=args.advertise, federation_secret=args.federation_secret,- state=args.state, open_follows=not args.closed, follow=follow, listed=args.listed,+ federate=args.federate or bool(args.advertise) or bool(follow) or bool(join) or args.listed, advertise=args.advertise, federation_secret=args.federation_secret,+ state=args.state, open_follows=not args.closed, follow=follow, join=join, listed=args.listed,trees=[Path(t) for t in args.trees], handout=[Path(t) for t in args.handout], env=_kv(args.env), gaps=args.gaps, egress=args.egress, store=args.store, replicas=args.replicas, report_gaps=args.report_gaps, unanswered=args.unanswered, watch=args.watch, chat=args.chat, )@@ -849,14 +858,62 @@ def cmd_reports(args: argparse.Namespace) -> int:def cmd_follow(args: argparse.Namespace) -> int:- url = args.url- if not url:- url = first_live_seed()- if url is None:- tried = bootstrap_candidates()- print("pyrtan follow: no seed or snapshot hub answers" + (f" (tried {', '.join(tried)})" if tried else " (none listed; set $PYRTAN_SEEDS or $PYRTAN_RENDEZVOUS)") + "; give a hub URL", file=sys.stderr)- return 3- return _hub_post(args, "/follow", {"url": url, "listed": bool(args.listed)}, lambda b: f"{args.at} now follows {b['following']['url']} ({b['following']['state']}); it routes there once accepted" + (" and is listed in the directory" if b.get("listed") else ""))+ if not args.url:+ if args.network:+ print("pyrtan follow: --network needs a hub URL to follow", file=sys.stderr)+ return 2+ args.invite = "default"+ return cmd_join(args)+ payload = {"url": args.url, "listed": bool(args.listed)}+ if args.network:+ payload["network"] = args.network+ return _hub_post(args, "/follow", payload, lambda b: f"{args.at} now follows {b['following']['url']} ({b['following']['state']})" + (f" in {args.network}" if args.network else "") + "; it routes there once accepted" + (" and is listed in the directory" if b.get("listed") else ""))+++def cmd_join(args: argparse.Namespace) -> int:+ def said(b: dict[str, Any]) -> str:+ lines = [f"{args.at}: {b.get('say')}" + (" and listed in the directory" if args.listed else "")]+ if b.get("invite") and not b.get("public"):+ lines.append(f"invite others: pyrtan join {b['invite']}")+ return "\n".join(lines)++ code = _hub_post(args, "/join", {"invite": args.invite, "listed": bool(args.listed)}, said)+ if code == 1 and args.invite == "default":+ tried = bootstrap_candidates()+ print("pyrtan join: no seed or snapshot hub answers" + (f" (tried {', '.join(tried)})" if tried else " (none listed; set $PYRTAN_SEEDS or $PYRTAN_RENDEZVOUS)") + "; give an invite or a hub URL", file=sys.stderr)+ return 3+ return code+++def cmd_network(args: argparse.Namespace) -> int:+ if args.action in ("new", "leave") and not args.name:+ print(f"pyrtan network {args.action}: give the network's name", file=sys.stderr)+ return 2+ if args.action == "new":+ return _hub_post(args, "/networks", {"name": args.name, "public": bool(args.public)}, lambda b: f"{args.at} is in {b['network']} ({b['id']}), a network of one.\ninvite others: pyrtan join {b['invite']}\nthe invite holds the network's secret: share it only with the hubs you want in it")+ if args.action == "leave":+ return _hub_post(args, "/leave", {"network": args.name}, lambda b: f"{args.at} left {args.name}" if b.get("left") else f"{args.at} was not in {args.name}")+ at = args.at.rstrip("/")+ try:+ status, body = http_json("GET", f"{at}/networks", token=args.token)+ except Exception as e:+ print(f"pyrtan: {at}: {e}", file=sys.stderr)+ return 3+ if status == 401:+ print(f"pyrtan: {at} needs a token: pass --token or set $PYRTAN_TOKEN", file=sys.stderr)+ return 3+ if status != 200 or not isinstance(body, dict):+ print(f"pyrtan: {at} is not a federated hub ({status})", file=sys.stderr)+ return 3+ if args.json:+ print(json.dumps(body, indent=2, ensure_ascii=False))+ return 0+ nets = body.get("networks", [])+ print(f"{at} is in {len(nets)} network(s)")+ for n in nets:+ print(f"{n['name']:16} {n['id']} {'public' if n['public'] else 'private'} following {len(n['following'])}, followed by {len(n['followers'])}")+ print(f" invite: {n['invite']}")+ return 0def cmd_keep(args: argparse.Namespace) -> int: diff --git a/src/pyrtan/federation.py b/src/pyrtan/federation.py index eaf1ce7..5d949f2 100644 --- a/src/pyrtan/federation.py +++ b/src/pyrtan/federation.py@@ -42,11 +42,25 @@ started ``--listed`` puts an entry for itself (id, name, the trees itholds, when) in every ``Announce`` it sends; every hub keeps the entries it hears and passes them on in its own announces, in both directions of every follow. Nothing central holds the directory: every federated hub-does, and pyrtan.com publishes only a static list of *seeds*, hubs to-follow first. An entry is a hint (a hub two follows away cannot verify-who said it), kept until its hub has not been heard of for a week, and-routing never uses it: finding a hub in the directory and following it is-still the one act of trust.+does, and pyrtan.com publishes only static files: the default network,+a list of *seeds* (hubs to follow first), and an hourly snapshot. An+entry is a hint (a hub two follows away cannot verify who said it), kept+until its hub has not been heard of for a week, and routing never uses+it: finding a hub in the directory and following it is still the one act+of trust.++**Networks** are names and secrets. A hub can be in any number and start+one (``create_network``, a network of one); another joins with its+invite, ``name:secret@hub`` (``join``). A Follow made in a network names+it by ``network_id`` (from the name and the secret, so it gives neither+away) and is signed with its secret; the followed hub takes it only if+it is in that network. The directory heard over a link is kept with the+link's network and passed on only over links in the same one, so a+private network's hubs never reach the public directory, and ``GET+/directory`` shows them only to the operator. The default network+(``default_network``, published at pyrtan.com) has a public secret: it+names the network, it does not guard it, and nothing here needs it to.+A follow outside any network (and every member's) works as before.**Chat.** A hub with a lobby (``--chat``) asks the hubs it follows for a stranger with ``Meet``, and the two seats talk in ``Say`` and end with@@ -81,6 +95,9 @@ from .store import BlobStore, StoreError, check_key, digest as blob_digestDEFAULT_SEEDS = "https://pyrtan.com/rendezvous/seeds.json" DEFAULT_SNAPSHOT = "https://pyrtan.com/rendezvous/directory.json"+DEFAULT_NETWORK = "https://pyrtan.com/rendezvous/network.json"+NETWORK_NAME = re.compile(r"^[a-z0-9][a-z0-9-]{0,39}$")+INVITE = re.compile(r"^([a-z0-9][a-z0-9-]{0,39}):([A-Za-z0-9_-]{16,128})(?:@(https?://\S+))?$")SIGNATURE_WINDOW = 300.0 # seconds a signed message stays valid MAX_HOPS = 1 # federation is one hop; raise with care STALE_AFTER = 30.0@@ -149,6 +166,75 @@ def snapshot_hubs() -> list[str]:return [h["id"].rstrip("/") for h in hubs]+# ---------------------------------------------------------------- networks+++def network_id(name: str, secret: str) -> str:+ """What a Follow names a network by: derived from its name and secret,+ so two networks that share a name never meet, and the id gives the+ secret away to nobody."""+ return hashlib.sha256(f"pyrtan-network\n{name}\n{secret}".encode("utf-8")).hexdigest()[:16]+++@dataclass+class Network:+ """A set of hubs that sign their follows with one secret. ``public``+ marks one whose secret is published (the default network at+ pyrtan.com): its directory is shown to anyone, a private one's only+ to the operator."""++ name: str+ secret: str+ public: bool = False++ @property+ def id(self) -> str:+ return network_id(self.name, self.secret)++ def invite(self, hub: str | None = None) -> str:+ return f"{self.name}:{self.secret}" + (f"@{hub.rstrip('/')}" if hub else "")++ def to_state(self) -> dict[str, Any]:+ return {"name": self.name, "secret": self.secret, "public": self.public}++ @classmethod+ def from_state(cls, d: Any) -> "Network | None":+ if not isinstance(d, dict) or not isinstance(d.get("name"), str) or not isinstance(d.get("secret"), str):+ return None+ if not NETWORK_NAME.match(d["name"]) or len(d["secret"]) < 16:+ return None+ return cls(d["name"], d["secret"], bool(d.get("public", False)))+++def parse_invite(text: str) -> tuple[Network, str | None]:+ """``name:secret`` or ``name:secret@https://hub``: a network, and the+ hub to follow into it. Raises ValueError on anything else."""+ m = INVITE.match(text.strip()) if isinstance(text, str) else None+ if not m:+ raise ValueError("an invite is NAME:SECRET or NAME:SECRET@https://hub (a name of lowercase letters, digits and dashes; a secret of at least 16 letters, digits, dashes or underscores)")+ return Network(m.group(1), m.group(2)), (m.group(3).rstrip("/") if m.group(3) else None)+++def default_network() -> Network | None:+ """The public network pyrtan.com publishes (``$PYRTAN_NETWORK``+ overrides the URL or names a file). Its secret is public on purpose:+ it names the network rather than guarding it. None when unreadable."""+ source = os.environ.get("PYRTAN_NETWORK") or DEFAULT_NETWORK+ try:+ if source.startswith(("http://", "https://")):+ status, body = http_json("GET", source, timeout=10.0)+ if status != 200:+ return None+ else:+ body = json.loads(Path(source).read_text(encoding="utf-8"))+ except Exception:+ return None+ net = Network.from_state(body)+ if net is not None:+ net.public = True+ return net++def bootstrap_candidates() -> list[str]: """Where to look for a first hub: the seeds, then the snapshot's hubs.""" seeds = seed_urls()@@ -224,6 +310,7 @@ class Remote:since: float = field(default_factory=time.time) kind: str = "hub" # hub | member trusted: bool = False # a member that joined with the operator's token: it may run the hub's trees+ network: str | None = None # the id of the network the follow was made in; None for a follow outside any@property def link(self) -> str:@@ -239,14 +326,14 @@ class Remote:return "down" def to_dict(self) -> dict[str, Any]:- return {"url": self.url, "name": self.name, "kind": self.kind, "state": self.state, "link": self.link, "trees": [t["name"] for t in self.trees], "last_ok": self.last_ok, "last_error": self.last_error, "trusted": self.trusted}+ return {"url": self.url, "name": self.name, "kind": self.kind, "state": self.state, "link": self.link, "trees": [t["name"] for t in self.trees], "last_ok": self.last_ok, "last_error": self.last_error, "trusted": self.trusted, "network": self.network}def to_state(self) -> dict[str, Any]:- return {"url": self.url, "inbox": self.inbox, "secret": self.secret, "name": self.name, "state": self.state, "since": self.since, "kind": self.kind, "trusted": self.trusted}+ return {"url": self.url, "inbox": self.inbox, "secret": self.secret, "name": self.name, "state": self.state, "since": self.since, "kind": self.kind, "trusted": self.trusted, "network": self.network}@classmethod def from_state(cls, d: dict[str, Any]) -> "Remote":- return cls(d["url"], d["inbox"], d["secret"], d.get("name", ""), d.get("state", "pending"), since=float(d.get("since", time.time())), kind=d.get("kind", "hub"), trusted=bool(d.get("trusted", False)))+ return cls(d["url"], d["inbox"], d["secret"], d.get("name", ""), d.get("state", "pending"), since=float(d.get("since", time.time())), kind=d.get("kind", "hub"), trusted=bool(d.get("trusted", False)), network=d.get("network") if isinstance(d.get("network"), str) else None)def _shape_report(e: dict[str, Any], reporter: str, now: float) -> dict[str, Any] | None:@@ -310,7 +397,8 @@ class Federation:self.ask_timeout = ASK_TIMEOUT self.following: dict[str, Remote] = {} self.followers: dict[str, Remote] = {}- self.directory_entries: dict[str, dict[str, Any]] = {}+ self.networks: dict[str, Network] = {} # name -> network this hub is in+ self.directory_entries: dict[str, dict[str, Any]] = {} # (network id, hub id) -> entry; see _dkeyself.outbox: deque[dict[str, Any]] = deque(maxlen=50) self.inbox_log: deque[dict[str, Any]] = deque(maxlen=50) self._announced: tuple[str, float] = ("", 0.0) # digest and time of the last downward announce@@ -338,6 +426,7 @@ class Federation:with self._lock: data = { "listed": self.listed,+ "networks": [n.to_state() for n in self.networks.values()],"following": [r.to_state() for r in self.following.values()], "followers": [r.to_state() for r in self.followers.values()], "directory": list(self.directory_entries.values()),@@ -354,11 +443,15 @@ class Federation:except (OSError, json.JSONDecodeError): return self.listed = bool(data.get("listed", False))+ for d in data.get("networks", []):+ net = Network.from_state(d)+ if net is not None:+ self.networks[net.name] = netfor d in data.get("following", []): self.following[d["url"]] = Remote.from_state(d) for d in data.get("followers", []): self.followers[d["url"]] = Remote.from_state(d)- self.merge_directory(data.get("directory", []), via="state")+ self.merge_directory(data.get("directory", []), via="state", keep_tags=True)self.merge_reports(data.get("reports", []), via="state") keeping = data.get("keeping") if isinstance(keeping, dict):@@ -371,72 +464,204 @@ class Federation:self.directory_entries.pop(self.id, None) self._save()+ # ------------------------------------------------------------ networks+ # A network is a name and a secret its hubs sign their follows with. A+ # hub may be in any number, starts one with create_network, and joins+ # one with an invite (join). Every follow made in a network says so,+ # and the directory heard over it is kept apart and passed on only+ # within it, so a private network's hubs never reach the public one.+ # A follow outside any network (follow(url), a member) works as before.++ def network(self, name: str) -> Network:+ net = self.networks.get(name)+ if net is None:+ raise ValueError(f"this hub is in no network named {name!r}")+ return net++ def network_by_id(self, nid: Any) -> Network | None:+ return next((n for n in self.networks.values() if n.id == nid), None) if isinstance(nid, str) else None++ def add_network(self, net: Network) -> Network:+ """Be in ``net``. The same network again is a no-op; another one+ under a name already taken is refused."""+ with self._lock:+ current = self.networks.get(net.name)+ if current is not None:+ if current.secret != net.secret:+ raise ValueError(f"this hub is already in a different network named {net.name!r}; leave it first")+ current.public = current.public or net.public+ return current+ self.networks[net.name] = net+ self._save()+ return net++ def create_network(self, name: str, public: bool = False) -> Network:+ if not isinstance(name, str) or not NETWORK_NAME.match(name):+ raise ValueError("a network's name is lowercase letters, digits and dashes, at most 40")+ if name in self.networks:+ raise ValueError(f"this hub is already in a network named {name!r}")+ return self.add_network(Network(name, _secrets.token_urlsafe(24), public))++ def join(self, net: Network, hubs: list[str]) -> Remote | None:+ """Be in ``net`` and follow the first of ``hubs`` that accepts a+ follow in it. None when none does: the hub is then a network of+ one, which others can join by following it."""+ net = self.add_network(net)+ for url in hubs:+ url = url.rstrip("/")+ if url == self.id:+ continue+ current = self.following.get(url)+ if current is not None and current.network == net.id and current.state == "accepted":+ return current+ before = (current.network, current.state) if current is not None else None+ try:+ remote = self.follow(url, network=net.name)+ except ValueError:+ continue+ if remote.state == "accepted":+ return remote+ with self._lock:+ # Not accepted in this network: leave things as they were.+ if before is None:+ self.following.pop(url, None)+ else:+ remote.network, remote.state = before+ self._save()+ return None++ def leave_network(self, name: str) -> bool:+ """Leave a network: unfollow the hubs followed in it, drop the hubs+ that followed us in it, and forget its directory."""+ net = self.networks.get(name)+ if net is None:+ return False+ for url in [u for u, r in self.following.items() if r.network == net.id]:+ self.unfollow(url)+ with self._lock:+ for url in [u for u, r in self.followers.items() if r.network == net.id]:+ del self.followers[url]+ for key in [k for k, e in self.directory_entries.items() if e.get("network") == net.id]:+ del self.directory_entries[key]+ del self.networks[name]+ self._save()+ return True++ def networks_view(self) -> list[dict[str, Any]]:+ """The operator's view: each network with its invite."""+ with self._lock:+ return [{"name": n.name, "id": n.id, "public": n.public, "invite": n.invite(self.id), "following": sorted(u for u, r in self.following.items() if r.network == n.id), "followers": sorted(u for u, r in self.followers.items() if r.network == n.id)} for n in sorted(self.networks.values(), key=lambda n: n.name)]+# ------------------------------------------------------------ directory+ @staticmethod+ def _dkey(network: str | None, hub_id: str) -> str:+ return hub_id if network is None else f"{network} {hub_id}"+def own_entry(self) -> dict[str, Any] | None: if not self.listed: return None return {"id": self.id, "name": self.name, "kind": "hub", "trees": sorted({t.get("name") for t in self.capabilities() if isinstance(t, dict) and t.get("name")}), "seen": time.time()}- def gossip(self) -> list[dict[str, Any]]:- """What this hub tells others about the directory: itself, if- listed, and every live entry it has heard."""+ def gossip(self, network: str | None = None) -> list[dict[str, Any]]:+ """What this hub tells a hub it is linked to in ``network`` (an id;+ None for a link outside any) about the directory: itself, if+ listed, and every live entry it heard in that same network."""+ now = time.time()+ with self._lock:+ entries = [e for e in self.directory_entries.values() if e["id"] != self.id and e.get("network") == network and now - e["seen"] < DIRECTORY_TTL]+ own = self.own_entry()+ return ([own] if own else []) + sorted(({k: v for k, v in e.items() if k != "network"} for e in entries), key=lambda e: -e["seen"])++ def _public_networks(self) -> set[str | None]:+ return {None} | {n.id for n in self.networks.values() if n.public}++ def directory_backup(self) -> list[dict[str, Any]]:+ """The directory as a blob for keepers: the public part, tagged,+ so a private network's hubs are never handed to a keeper."""now = time.time()+ public = self._public_networks()with self._lock:- entries = [e for e in self.directory_entries.values() if e["id"] != self.id and now - e["seen"] < DIRECTORY_TTL]+ entries = [dict(e) for e in self.directory_entries.values() if e.get("network") in public and now - e["seen"] < DIRECTORY_TTL]own = self.own_entry() return ([own] if own else []) + sorted(entries, key=lambda e: -e["seen"])- def merge_directory(self, entries: Any, via: str) -> int:- """Take entries heard from ``via``: a newer ``seen`` replaces an- older one, our own entry is never overwritten, nothing from the- future is believed, and the table stays bounded. Returns how many- entries changed."""+ def merge_directory(self, entries: Any, via: str, network: str | None = None, keep_tags: bool = False) -> int:+ """Take entries heard from ``via`` over a link in ``network``: a+ newer ``seen`` replaces an older one, our own entry is never+ overwritten, nothing from the future is believed, and the table+ stays bounded. ``keep_tags`` is for our own copies (the state file,+ a blob handed back): each entry keeps the network it carries, if we+ are still in it. Returns how many entries changed."""if not isinstance(entries, list): return 0 now = time.time() changed = 0 with self._lock:+ known = {n.id for n in self.networks.values()}for e in entries: if not isinstance(e, dict) or not isinstance(e.get("id"), str) or not e["id"].startswith(("http://", "https://", MEMBER_PREFIX)): continue hub_id = e["id"].rstrip("/") if hub_id == self.id: continue+ net = network+ if keep_tags:+ net = e.get("network") if isinstance(e.get("network"), str) else None+ if net is not None and net not in known:+ continuetry: seen = min(float(e.get("seen", 0)), now) except (TypeError, ValueError): continue if now - seen >= DIRECTORY_TTL: continue- current = self.directory_entries.get(hub_id)+ key = self._dkey(net, hub_id)+ current = self.directory_entries.get(key)if current is not None and current["seen"] >= seen: continue- self.directory_entries[hub_id] = {+ self.directory_entries[key] = {"id": hub_id, "name": str(e.get("name") or "")[:80], "kind": "member" if hub_id.startswith(MEMBER_PREFIX) else "hub", "trees": [str(t)[:80] for t in e.get("trees", []) if isinstance(t, (str, int))][:200], "seen": seen, "via": via,+ "network": net,} changed += 1 if len(self.directory_entries) > DIRECTORY_MAX:- for hub_id in sorted(self.directory_entries, key=lambda k: self.directory_entries[k]["seen"])[: len(self.directory_entries) - DIRECTORY_MAX]:- del self.directory_entries[hub_id]+ for key in sorted(self.directory_entries, key=lambda k: self.directory_entries[k]["seen"])[: len(self.directory_entries) - DIRECTORY_MAX]:+ del self.directory_entries[key]if changed: self._save() return changed- def directory(self) -> dict[str, Any]:+ def directory(self, private: bool = True) -> dict[str, Any]:"""Every hub this one has heard of that asked to be found, this one- included if listed. ``link`` is how recently each announced."""+ included if listed: one row per hub, naming the networks it was+ heard in. ``link`` is how recently each announced. Without+ ``private`` (a caller without the operator's token), only what was+ heard outside any network or in a public one."""now = time.time()- hubs = []- for e in self.gossip():- hubs.append({"id": e["id"], "name": e["name"], "kind": e.get("kind", "hub"), "trees": e["trees"], "seen": e["seen"], "link": heard(e["seen"], now), "via": e.get("via", "self" if e["id"] == self.id else "")})+ visible = None if private else self._public_networks()+ names = {n.id: n.name for n in self.networks.values()}+ rows: dict[str, dict[str, Any]] = {}+ own = self.own_entry()+ if own:+ rows[self.id] = {**own, "via": "self", "networks": sorted(n.id for n in self.networks.values() if private or n.public)}+ with self._lock:+ entries = [dict(e) for e in self.directory_entries.values() if now - e["seen"] < DIRECTORY_TTL and (visible is None or e.get("network") in visible)]+ for e in sorted(entries, key=lambda e: -e["seen"]):+ row = rows.get(e["id"])+ if row is None:+ rows[e["id"]] = row = {**{k: v for k, v in e.items() if k != "network"}, "networks": []}+ if e.get("network") and e["network"] not in row["networks"]:+ row["networks"].append(e["network"])+ hubs = [{"id": r["id"], "name": r["name"], "kind": r.get("kind", "hub"), "trees": r["trees"], "seen": r["seen"], "link": heard(r["seen"], now), "via": r.get("via", ""), "networks": sorted(r["networks"])} for r in rows.values()]hubs.sort(key=lambda h: (h["id"] != self.id, {"up": 0, "stale": 1, "down": 2}[h["link"]], h["name"], h["id"]))- return {"hub": self.id, "name": self.name, "listed": self.listed, "count": len(hubs), "hubs": hubs}+ shown = {nid for h in hubs for nid in h["networks"]}+ return {"hub": self.id, "name": self.name, "listed": self.listed, "count": len(hubs), "hubs": hubs, "networks": {nid: names[nid] for nid in sorted(shown) if nid in names}}# ------------------------------------------------------------ gap reports # A hub that opted in (--report-gaps) puts the gaps its catalog trees met@@ -508,7 +733,7 @@ class Federation:return {"type": type_, "id": f"{self.id}/activities/{uuid.uuid4()}", "actor": self.id, "published": time.strftime("%Y-%m-%dT%H:%M:%SZ", time.gmtime()), "object": object_, **extra} def _announcement(self, for_remote: Remote | None = None) -> dict[str, Any]:- out = {"type": "Capabilities", "name": self.name, "trees": self.capabilities(), "directory": self.gossip()}+ out = {"type": "Capabilities", "name": self.name, "trees": self.capabilities(), "directory": self.gossip(for_remote.network if for_remote is not None else None)}reports = self.report_gossip() if reports: out["reports"] = reports@@ -537,7 +762,7 @@ class Federation:if ok and isinstance(answer, dict): if activity.get("type") == "Announce": # An inbox answers an Announce with its own gossip: one round trip, both directions.- self.merge_directory(answer.get("directory"), via=remote.url)+ self.merge_directory(answer.get("directory"), via=remote.url, network=remote.network)self.merge_reports(answer.get("reports"), via=remote.url) if activity.get("type") == "Keep" and answer.get("kept") == activity["object"].get("key"): self.confirm(remote.url, activity["object"]["key"], activity["object"]["sha256"])@@ -556,12 +781,16 @@ class Federation:remote.last_error = None return out- def follow(self, url: str) -> Remote:+ def follow(self, url: str, network: str | None = None) -> Remote:"""Follow the hub at ``url``: fetch its actor, make a pair secret,- send the Follow. The remote is pending until it Accepts."""+ send the Follow. The remote is pending until it Accepts. In a+ ``network`` (a name this hub is in), the Follow names it and is+ signed with its secret, and what the two hear of the directory over+ this link stays in it."""url = url.rstrip("/") if url == self.id: raise ValueError("a hub cannot follow itself")+ net = self.network(network) if network is not None else Nonetry: status, actor = http_json("GET", f"{url}/actor", timeout=10.0) except Exception as e:@@ -577,12 +806,14 @@ class Federation:remote.inbox = actor["inbox"] remote.name = actor.get("name", remote.name) remote.state = "pending"+ remote.network = net.id if net else Noneself._save()- activity = self._activity("Follow", url, secret=remote.secret, inbox=self.inbox, name=self.name)- # Under a shared federation secret the Follow is signed with it; in- # the open case with the pair secret, which proves nothing yet but- # keeps every message signed.- ok, error, _ = self.deliver(remote, activity, self.secret or remote.secret)+ extra = {"network": net.id} if net else {}+ activity = self._activity("Follow", url, secret=remote.secret, inbox=self.inbox, name=self.name, **extra)+ # In a network the Follow is signed with its secret; under a shared+ # federation secret with that; in the open case with the pair+ # secret, which proves nothing yet but keeps every message signed.+ ok, error, _ = self.deliver(remote, activity, net.secret if net else (self.secret or remote.secret))if not ok: remote.last_error = error elif remote.state == "accepted":@@ -600,7 +831,10 @@ class Federation:return True def _due(self, last: tuple[str, float], announcement: dict[str, Any], force: bool) -> tuple[bool, str]:- digest = hashlib.sha256(canonical({"trees": [{k: t.get(k) for k in ("name", "version", "triggers", "inputs", "language", "hosts")} for t in announcement["trees"]], "directory": [(e["id"], e["trees"]) for e in announcement["directory"]]})).hexdigest()+ with self._lock:+ # Every network's directory: a change in any is news for the links in it.+ heard = sorted((e.get("network") or "", e["id"], e["trees"]) for e in self.directory_entries.values())+ digest = hashlib.sha256(canonical({"trees": [{k: t.get(k) for k in ("name", "version", "triggers", "inputs", "language", "hosts")} for t in announcement["trees"]], "directory": [(e["id"], e["trees"]) for e in announcement["directory"]], "heard": heard})).hexdigest()due = force or digest != last[0] or time.time() - last[1] >= ANNOUNCE_EVERY return due, digest@@ -854,7 +1088,7 @@ class Federation:entry["holders"][who] = {"at": time.time(), "sha256": meta.get("sha256")} self._save() if key == "directory":- self.merge_directory(blob.get("data"), via=who)+ self.merge_directory(blob.get("data"), via=who, keep_tags=True)return True def _take_capabilities(self, remote: Remote, caps: dict[str, Any]) -> None:@@ -955,13 +1189,13 @@ class Federation:handed = self.handout() names = {t.get("name") for t in obj.get("trees", []) if isinstance(t, dict)} remote.trees = [handed[n].capability() for n in sorted(names & set(handed))]- taken = self.merge_directory(obj.get("directory", []), via=actor)+ taken = self.merge_directory(obj.get("directory", []), via=actor, network=remote.network)self.merge_reports(obj.get("reports", []), via=actor) # What it keeps for us that we no longer have: ask for it back. wanted = [k for k in obj.get("kept", []) if isinstance(k, str) and not self.store.get_own(k)] for k in wanted[:20]: self.deliver(remote, self._activity("Fetch", {"key": k}))- return 200, {"status": "ok", "trees": len(remote.trees) if remote is followed else 0, "merged": taken, "directory": self.gossip(), "reports": self.report_gossip(), "kept": [b["key"] for b in self.store.list_kept(actor)]}+ return 200, {"status": "ok", "trees": len(remote.trees) if remote is followed else 0, "merged": taken, "directory": self.gossip(remote.network), "reports": self.report_gossip(), "kept": [b["key"] for b in self.store.list_kept(actor)]}if kind == "Keep": obj = activity.get("object") if not isinstance(obj, dict) or "data" not in obj:@@ -1019,7 +1253,16 @@ class Federation:inbox = activity.get("inbox") if not isinstance(pair, str) or len(pair) < 16 or not isinstance(inbox, str): return 400, {"status": "error", "reason": "bad-activity", "say": "A Follow carries a pair secret and the follower's inbox."}- if self.secret:+ nid = activity.get("network")+ net = self.network_by_id(nid)+ if nid is not None:+ # A follow in a network: signed with its secret, and only into a network this hub is in.+ if net is None:+ return 403, {"status": "error", "reason": "not-in-network", "say": "This hub is not in the network this follow names."}+ problem = verify(net.secret, "POST", "/inbox", activity, h)+ if problem:+ return 401, {"status": "error", "reason": "unsigned", "say": f"A follow in {net.name} is signed with its secret: {problem}."}+ elif self.secret:problem = verify(self.secret, "POST", "/inbox", activity, h) if problem: return 401, {"status": "error", "reason": "unsigned", "say": f"This hub takes follows signed with its federation secret: {problem}."}@@ -1047,7 +1290,7 @@ class Federation:name = str(activity.get("name") or doc.get("name") or "") kind = "hub" with self._lock:- remote = Remote(actor, inbox, pair, name, state="accepted", last_ok=time.time(), kind=kind, trusted=bool(trusted and kind == "member"))+ remote = Remote(actor, inbox, pair, name, state="accepted", last_ok=time.time(), kind=kind, trusted=bool(trusted and kind == "member"), network=net.id if net else None)self.followers[actor] = remote self._handed.pop(actor, None) self._save()@@ -1061,4 +1304,4 @@ class Federation:def to_dict(self) -> dict[str, Any]: with self._lock:- return {"id": self.id, "listed": self.listed, "following": [r.to_dict() for r in self.following.values()], "followers": [r.to_dict() for r in self.followers.values()], "directory": len(self.directory_entries) + (1 if self.listed else 0), "keeping": len(self.keeping), "store": self.store.usage(), "replicas": self.replicas}+ return {"id": self.id, "listed": self.listed, "following": [r.to_dict() for r in self.following.values()], "followers": [r.to_dict() for r in self.followers.values()], "directory": len({e["id"] for e in self.directory_entries.values()}) + (1 if self.listed else 0), "keeping": len(self.keeping), "store": self.store.usage(), "replicas": self.replicas, "networks": [{"name": n.name, "id": n.id, "public": n.public} for n in self.networks.values()]}diff --git a/src/pyrtan/hub.py b/src/pyrtan/hub.py index 513ce49..ad4c19d 100644 --- a/src/pyrtan/hub.py +++ b/src/pyrtan/hub.py@@ -25,8 +25,9 @@ to; see :mod:`pyrtan.chat`.Federated (``--federate``), a hub also answers ``GET /actor``, ``POST /inbox``, ``POST /follow`` and ``/unfollow`` (operator), ``GET /following``,-``/followers`` and ``/outbox``, and routes to the hubs it follows; see-:mod:`pyrtan.federation`.+``/followers`` and ``/outbox``, ``GET`` and ``POST /networks``, ``POST+/join`` and ``/leave`` (operator: the networks it is in), and routes to+the hubs it follows; see :mod:`pyrtan.federation`.A service that holds several trees and a hub that routes between services answer the same requests, so a client need not know which it is talking to,@@ -47,7 +48,7 @@ from typing import Anyfrom . import __version__ from .chat import Lobby-from .federation import MAX_HOPS, Federation, Remote, sign+from .federation import MAX_HOPS, Federation, Remote, bootstrap_candidates, default_network, first_live_seed, parse_invite, signfrom .grow import Unanswered, no_match_outcome from .intent import Trigger, read from .page import MANIFEST, PAGE, PYRTAN_JS, SERVICE_WORKER@@ -150,7 +151,7 @@ class Hub:the URL they reach it at. ``store`` is where it keeps blobs, its own and others'; ``replicas`` how many keepers each of its own should have.""" auto = {- "directory": lambda: self.federation.gossip() if self.federation else [],+ "directory": lambda: self.federation.directory_backup() if self.federation else [],"gaps": lambda: self.gaps()["gaps"], "unanswered": lambda: self.unanswered_all()["unanswered"], }@@ -506,7 +507,8 @@ class Hub:if method == "GET" and path == "/actor": return 200, fed.actor() if method == "GET" and path == "/directory":- return 200, fed.directory()+ # A private network's part only to the operator (or to anyone, on a hub with no token).+ return 200, fed.directory(private=not self.token or bearer_ok(headers.get("Authorization") or headers.get("authorization"), self.token))if method == "GET" and path == "/reports": return 200, fed.reports_view() if method == "POST" and path == "/inbox":@@ -519,13 +521,33 @@ class Hub:url = body.get("url") if not isinstance(url, str) or not url.startswith(("http://", "https://")): return 400, {"status": "error", "reason": "bad-request", "say": "Send {\"url\": \"https://hub.example\"}."}+ network = body.get("network")+ if network is not None and not isinstance(network, str):+ return 400, {"status": "error", "reason": "bad-request", "say": "A network is named by a string."}if body.get("listed") is True and not fed.listed: fed.set_listed(True) try:- remote = fed.follow(url)+ remote = fed.follow(url, network=network)except ValueError as e: return 400, {"status": "error", "reason": "cannot-follow", "say": str(e)} return 200, {"status": "ok", "following": remote.to_dict(), "listed": fed.listed}+ if method == "GET" and path == "/networks":+ return 200, {"networks": fed.networks_view()}+ if method == "POST" and path == "/networks":+ try:+ net = fed.create_network(body.get("name"), public=body.get("public") is True)+ except ValueError as e:+ return 400, {"status": "error", "reason": "bad-network", "say": str(e)}+ return 200, {"status": "ok", "network": net.name, "id": net.id, "invite": net.invite(fed.id)}+ if method == "POST" and path == "/join":+ if body.get("listed") is True and not fed.listed:+ fed.set_listed(True)+ return join_network(fed, body.get("invite"))+ if method == "POST" and path == "/leave":+ name = body.get("network")+ if not isinstance(name, str):+ return 400, {"status": "error", "reason": "bad-request", "say": "Send {\"network\": \"name\"}."}+ return 200, {"status": "ok", "left": fed.leave_network(name)}if method == "POST" and path == "/listed": fed.set_listed(body.get("listed") is not False) if fed.listed:@@ -577,7 +599,7 @@ class Hub:if self.lobby is not None: endpoints += ["/chat", "/chat/start", "/chat/poll", "/chat/say", "/chat/next", "/chat/leave"] if fed is not None:- endpoints += ["/actor", "/inbox", "/mailbox", "/directory", "/reports", "/follow", "/unfollow", "/listed", "/following", "/followers", "/outbox", "/keep", "/fetch", "/kept/{key}"]+ endpoints += ["/actor", "/inbox", "/mailbox", "/directory", "/reports", "/follow", "/unfollow", "/networks", "/join", "/leave", "/listed", "/following", "/followers", "/outbox", "/keep", "/fetch", "/kept/{key}"]return 200, {"pyrtan": "hub", "name": self.name, "version": __version__, "auth": "bearer" if self.token else "none", "federation": fed.id if fed else None, "chat": self.lobby is not None, "endpoints": endpoints} if method == "POST" and path == "/register": url = body.get("url")@@ -600,6 +622,44 @@ def _rank(peer: Local | Peer | Remote) -> int:return {"up": 0, "pending": 1, "stale": 2, "down": 3}[peer.link] + (5 if isinstance(peer, Remote) and peer.kind == "member" else 10 if isinstance(peer, Remote) else 0) - (1 if isinstance(peer, Local) else 0)+def join_network(fed: Federation, invite: Any) -> tuple[int, dict[str, Any]]:+ """Join a network from an invite (``name:secret[@hub]``), or the default+ network pyrtan.com publishes when ``invite`` is empty or ``default``,+ and follow a hub in it. When the default network cannot be read, the+ first live seed is followed outside any network, as before networks."""+ if invite in (None, "", "default"):+ net = default_network()+ hubs = bootstrap_candidates()+ if net is None:+ url = first_live_seed()+ if url is None:+ return 200, {"status": "error", "reason": "no-network", "say": "Cannot read the default network, and no seed or snapshot hub answers; give an invite or a hub URL."}+ try:+ remote = fed.follow(url)+ except ValueError as e:+ return 200, {"status": "error", "reason": "cannot-follow", "say": str(e)}+ return 200, {"status": "ok", "network": None, "following": remote.to_dict(), "say": f"now follows {url}, outside any network (the default network could not be read)"}+ else:+ try:+ net, url = parse_invite(str(invite))+ except ValueError as e:+ return 400, {"status": "error", "reason": "bad-invite", "say": str(e)}+ hubs = [url] if url else []+ try:+ remote = fed.join(net, hubs)+ except ValueError as e:+ return 400, {"status": "error", "reason": "bad-network", "say": str(e)}+ net = fed.networks[net.name]+ hubs = [h for h in hubs if h.rstrip("/") != fed.id]+ if remote is not None:+ say = f"in {net.name}, following {remote.url}"+ elif hubs:+ say = f"in {net.name}; no hub in it accepted a follow yet (tried {', '.join(hubs)}), so this hub is a network of one until one does or another hub joins through it"+ else:+ say = f"in {net.name}, a network of one until another hub joins through this one"+ return 200, {"status": "ok", "network": net.name, "id": net.id, "public": net.public, "following": remote.to_dict() if remote else None, "invite": net.invite(fed.id), "say": say}++def serve_hub( host: str = "127.0.0.1", port: int = 8700,@@ -616,6 +676,7 @@ def serve_hub(state: str | None = None, open_follows: bool = True, follow: list[str] | None = None,+ join: list[str] | None = None,listed: bool = False, trees: list[str | Path] | None = None, handout: list[str | Path] | None = None,@@ -632,7 +693,7 @@ def serve_hub(local = Service(Registry.from_paths(trees), name=f"{name}", env=env, gaps=gaps, egress=egress, watch=watch) if trees else None hub = Hub(name=name, poll_every=poll_every, token=token, local=local, handout=Registry.from_paths(handout) if handout else None, unanswered=unanswered, chat=chat) server = Server(hub.route, host, port, quiet, token=token, cors=cors, federated=federate or bool(advertise)).start()- if federate or advertise:+ if federate or advertise or join:hub.federate(advertise or server.url, secret=federation_secret, state=state, open_follows=open_follows, listed=listed, store=store or (str(Path(state).with_name("store")) if state else None), replicas=replicas, report_gaps=report_gaps) for url in peers or []: hub.add_peer(url)@@ -641,5 +702,8 @@ def serve_hub(hub.federation.follow(url) # type: ignore[union-attr] except ValueError as e: print(f"pyrtan hub: cannot follow {url}: {e}", flush=True)+ for invite in join or []:+ _, answer = join_network(hub.federation, invite) # type: ignore[arg-type]+ print(f"pyrtan hub: join {answer.get('network') or invite.split(':', 1)[0]}: {answer.get('say')}", flush=True)hub.start_polling() return hub, server diff --git a/src/pyrtan/js/pyrtan.js b/src/pyrtan/js/pyrtan.js index 074a626..13502dd 100644 --- a/src/pyrtan/js/pyrtan.js +++ b/src/pyrtan/js/pyrtan.js@@ -912,5 +912,5 @@return outcome(d.status || "error", d.tree || fallbackTree, { say: d.say === undefined ? null : d.say, output: d.output === undefined ? null : d.output, reason: d.reason === undefined ? null : d.reason, question: d.question === undefined ? null : d.question, snapshot: d.snapshot === undefined ? null : d.snapshot, trace, inputs: d.inputs || {} }); }- return { MISSING, Tree, Runner, Trace, FetchTransport, RecordedTransport, Response, render, resolve, holds, extract, outcomeDict, fromDict, pyrepr, pyJson, html, TemplateError, TreeError, TransportError, ConditionError, SOURCES, adapterKey, version: "0.9.0" };+ return { MISSING, Tree, Runner, Trace, FetchTransport, RecordedTransport, Response, render, resolve, holds, extract, outcomeDict, fromDict, pyrepr, pyJson, html, TemplateError, TreeError, TransportError, ConditionError, SOURCES, adapterKey, version: "0.10.0" };}); diff --git a/src/pyrtan/js/theme.js b/src/pyrtan/js/theme.js new file mode 100644 index 0000000..ec06a69 --- /dev/null +++ b/src/pyrtan/js/theme.js@@ -0,0 +1,134 @@+// The wordmark and its two colours. "Pyr" is the primary colour (buttons,+// links, the current tab); "tan" is the secondary (focus rings, hover edges,+// selected text). Click either half of the wordmark, right-click it, or press+// and hold it on a touch screen to pick that half's colour. The choice is kept+// in this browser only, under this origin, and applies to every Pyrtan page+// served from it.+//+// A page gives the defaults as --pyr, --pyr-fg, --tan and --tan-fg on :root+// (light and dark), marks the halves with data-theme-part="pyr" and "tan",+// and loads this script in <head> so a saved choice applies before first paint.+(() => {+ const KEY = "pyrtan.theme";+ const PARTS = {+ pyr: {label: "Pyr", role: "primary", presets: [["ember", "#c2410c"], ["crimson", "#c92a2a"], ["rose", "#c2255c"], ["violet", "#7048e8"], ["blue", "#2b5db8"], ["teal", "#0c8599"], ["green", "#2f9e44"], ["slate", "#495057"]]},+ tan: {label: "tan", role: "secondary", presets: [["tan", "#c19a6b"], ["sand", "#d2b48c"], ["copper", "#b87333"], ["clay", "#a0522d"], ["gold", "#d4a017"], ["khaki", "#b5a642"], ["sage", "#8a9a5b"], ["steel", "#7d8ea3"]]},+ };+ const root = document.documentElement;+ const load = () => { try { return JSON.parse(localStorage.getItem(KEY) || "{}") || {}; } catch (e) { return {}; } };+ const save = (t) => { try { if (Object.keys(t).length) localStorage.setItem(KEY, JSON.stringify(t)); else localStorage.removeItem(KEY); } catch (e) {} };+ const valid = (c) => typeof c === "string" && /^#[0-9a-f]{6}$/i.test(c);+ // Text on a filled background of colour c: near-black on light colours, white on dark ones.+ function ink(c) {+ const lin = (i) => { const v = parseInt(c.slice(i, i + 2), 16) / 255; return v <= 0.03928 ? v / 12.92 : ((v + 0.055) / 1.055) ** 2.4; };+ return 0.2126 * lin(1) + 0.7152 * lin(3) + 0.0722 * lin(5) > 0.22 ? "#101318" : "#ffffff";+ }+ function apply(t) {+ for (const part in PARTS) {+ if (valid(t[part])) { root.style.setProperty("--" + part, t[part]); root.style.setProperty("--" + part + "-fg", ink(t[part])); }+ else { root.style.removeProperty("--" + part); root.style.removeProperty("--" + part + "-fg"); }+ }+ const meta = document.querySelector('meta[name="theme-color"]');+ if (meta) { const c = getComputedStyle(root).getPropertyValue("--pyr").trim(); if (c) meta.content = c; }+ }+ let theme = load();+ apply(theme);++ const style = document.createElement("style");+ style.textContent = `+ [data-theme-part] { cursor:pointer; border-radius:4px; -webkit-user-select:none; user-select:none; -webkit-touch-callout:none; touch-action:manipulation }+ [data-theme-part]:hover { text-decoration:underline; text-decoration-thickness:2px; text-underline-offset:4px }+ [data-theme-part][aria-expanded=true] { text-decoration:underline; text-decoration-thickness:2px; text-underline-offset:4px }+ .theme-pick { position:absolute; z-index:1000; width:min(264px, calc(100vw - 32px)); padding:12px; background:var(--card, #fff); color:var(--fg, #1d1d1b); border:1px solid var(--line, #e3e1da); border-radius:10px; box-shadow:0 8px 28px rgba(0,0,0,.18); font:14px/1.4 system-ui,-apple-system,"Segoe UI",sans-serif; letter-spacing:normal }+ .theme-pick[hidden] { display:none }+ .theme-pick .theme-head { font-weight:600; margin:0 0 8px } .theme-pick .theme-head span { color:var(--mute, #6b6a64); font-weight:normal }+ .theme-pick .theme-swatches { display:grid; grid-template-columns:repeat(4, 1fr); gap:8px; margin:0 0 10px }+ .theme-pick .theme-swatches button { aspect-ratio:1; width:100%; min-height:36px; padding:0; border:2px solid var(--line, #e3e1da); border-radius:8px; cursor:pointer }+ .theme-pick .theme-swatches button[aria-pressed=true] { border-color:var(--fg, #1d1d1b); box-shadow:inset 0 0 0 2px var(--card, #fff) }+ .theme-pick .theme-row { display:flex; align-items:center; justify-content:space-between; gap:8px }+ .theme-pick label { display:flex; align-items:center; gap:8px; cursor:pointer }+ .theme-pick input[type=color] { width:36px; height:28px; padding:0; border:1px solid var(--line, #e3e1da); border-radius:6px; background:none; cursor:pointer }+ .theme-pick .theme-reset { font:inherit; padding:6px 10px; border:1px solid var(--line, #e3e1da); border-radius:6px; background:var(--soft, #f1efe9); color:var(--fg, #1d1d1b); cursor:pointer }+ .theme-pick .theme-reset:disabled { opacity:.5; cursor:default }+ .theme-pick .theme-hint { color:var(--mute, #6b6a64); font-size:12px; margin:8px 0 0 }`;+ document.head.appendChild(style);++ let pick = null, openPart = null, openEl = null;++ function current(part) { return valid(theme[part]) ? theme[part].toLowerCase() : null; }+ function choose(part, color) {+ if (color) theme[part] = color.toLowerCase(); else delete theme[part];+ save(theme); apply(theme); render();+ }+ function render() {+ if (!pick || !openPart) return;+ const p = PARTS[openPart], cur = current(openPart);+ pick.setAttribute("aria-label", `${p.label} colour`);+ pick.querySelector(".theme-head").innerHTML = `${p.label} colour <span>· ${p.role}</span>`;+ pick.querySelector(".theme-swatches").innerHTML = p.presets.map(([name, c]) => `<button type="button" data-color="${c}" style="background:${c}" title="${name}" aria-label="${name}" aria-pressed="${cur === c}"></button>`).join("");+ pick.querySelector("input[type=color]").value = cur || getComputedStyle(root).getPropertyValue("--" + openPart).trim() || "#000000";+ pick.querySelector(".theme-reset").disabled = !cur;+ }+ function build() {+ pick = document.createElement("div");+ pick.className = "theme-pick"; pick.hidden = true; pick.setAttribute("role", "dialog");+ pick.innerHTML = `<div class="theme-head"></div><div class="theme-swatches"></div><div class="theme-row"><label><input type="color"> Custom</label><button type="button" class="theme-reset">Default</button></div><div class="theme-hint">Kept in this browser only.</div>`;+ pick.addEventListener("click", (e) => {+ const b = e.target.closest("button[data-color]");+ if (b) choose(openPart, b.dataset.color);+ else if (e.target.closest(".theme-reset")) choose(openPart, null);+ });+ pick.querySelector("input[type=color]").addEventListener("input", (e) => choose(openPart, e.target.value));+ document.body.appendChild(pick);+ }+ function place() {+ const r = openEl.getBoundingClientRect(), w = pick.offsetWidth;+ const left = Math.max(16, Math.min(r.left, document.documentElement.clientWidth - w - 16));+ pick.style.left = (left + window.scrollX) + "px";+ pick.style.top = (r.bottom + window.scrollY + 8) + "px";+ }+ function open(part, el) {+ if (!pick) build();+ if (openEl && openEl !== el) openEl.setAttribute("aria-expanded", "false");+ openPart = part; openEl = el; el.setAttribute("aria-expanded", "true");+ render(); pick.hidden = false; place();+ }+ function close(refocus) {+ if (!pick || pick.hidden) return;+ pick.hidden = true;+ if (openEl) { openEl.setAttribute("aria-expanded", "false"); if (refocus) openEl.focus(); }+ openPart = null; openEl = null;+ }+ const toggle = (part, el) => (openEl === el && !pick.hidden) ? close(false) : open(part, el);++ function bind(el) {+ const part = el.dataset.themePart;+ if (!PARTS[part] || el.dataset.themeBound) return;+ el.dataset.themeBound = "1";+ el.setAttribute("role", "button"); el.tabIndex = 0;+ el.setAttribute("aria-haspopup", "dialog"); el.setAttribute("aria-expanded", "false");+ el.title = `${PARTS[part].label} colour (${PARTS[part].role}): click, right-click or press and hold`;+ // Press and hold on touch and pen; a mouse has click and right-click.+ let timer = null, held = false;+ const cancel = () => { clearTimeout(timer); timer = null; };+ el.addEventListener("pointerdown", (e) => {+ held = false;+ if (e.pointerType === "mouse") return;+ cancel(); timer = setTimeout(() => { held = true; open(part, el); }, 450);+ });+ for (const ev of ["pointerup", "pointercancel", "pointerleave"]) el.addEventListener(ev, cancel);+ el.addEventListener("click", (e) => { e.preventDefault(); if (held) { held = false; return; } toggle(part, el); });+ el.addEventListener("contextmenu", (e) => { e.preventDefault(); cancel(); held = e.pointerType !== "mouse"; open(part, el); });+ el.addEventListener("keydown", (e) => { if (e.key === "Enter" || e.key === " ") { e.preventDefault(); toggle(part, el); } });+ }++ function start() {+ document.querySelectorAll("[data-theme-part]").forEach(bind);+ document.addEventListener("pointerdown", (e) => { if (pick && !pick.hidden && !pick.contains(e.target) && !e.target.closest("[data-theme-part]")) close(false); });+ document.addEventListener("keydown", (e) => { if (e.key === "Escape") close(true); });+ window.addEventListener("resize", () => { if (pick && !pick.hidden) place(); });+ }+ if (document.readyState === "loading") document.addEventListener("DOMContentLoaded", start); else start();+ // Another tab on the same origin changed the colours.+ window.addEventListener("storage", (e) => { if (e.key === KEY) { theme = load(); apply(theme); render(); } });+})();diff --git a/src/pyrtan/page.py b/src/pyrtan/page.py index 69dddb1..7a7c899 100644 --- a/src/pyrtan/page.py +++ b/src/pyrtan/page.py@@ -27,6 +27,8 @@ from . import __version__# The tree runtime in JavaScript, served beside the console so a member can run trees. PYRTAN_JS = (Path(__file__).parent / "js" / "pyrtan.js").read_text(encoding="utf-8")+# The wordmark's colour picker, inlined into the console and copied beside pyrtan.com's pages.+THEME_JS = (Path(__file__).parent / "js" / "theme.js").read_text(encoding="utf-8")PAGE = r"""<!doctype html> <html lang="en">@@ -37,10 +39,12 @@ PAGE = r"""<!doctype html><meta name="description" content="Pyrtan console: ask the swarm, read the trace, see what it can answer, find the gaps, and join a hub as a member that keeps its blobs."> <link rel="manifest" href="manifest.webmanifest"> <script src="pyrtan.js"></script>-<meta name="theme-color" content="#2b5db8">+<meta name="theme-color" content="#c2410c">+<script>__THEME__</script><style>- :root { --bg:#fbfaf7; --fg:#1d1d1b; --mute:#6b6a64; --line:#e3e1da; --card:#ffffff; --ok:#1f7a4d; --bad:#a3382b; --ask:#8a6400; --accent:#2b5db8; --accent-fg:#fff; --soft:#f1efe9 }- @media (prefers-color-scheme: dark) { :root { --bg:#141513; --fg:#ecebe6; --mute:#9d9c95; --line:#2b2c29; --card:#1c1d1b; --ok:#5fcf93; --bad:#f08a7a; --ask:#e2b84a; --accent:#7fa6f0; --accent-fg:#101318; --soft:#222420 } }+ :root { --bg:#fbfaf7; --fg:#1d1d1b; --mute:#6b6a64; --line:#e3e1da; --card:#ffffff; --ok:#1f7a4d; --bad:#a3382b; --ask:#8a6400; --pyr:#c2410c; --pyr-fg:#fff; --tan:#a8804f; --tan-fg:#fff; --soft:#f1efe9 }+ @media (prefers-color-scheme: dark) { :root { --bg:#141513; --fg:#ecebe6; --mute:#9d9c95; --line:#2b2c29; --card:#1c1d1b; --ok:#5fcf93; --bad:#f08a7a; --ask:#e2b84a; --pyr:#fb8a4c; --pyr-fg:#101318; --tan:#d2b48c; --tan-fg:#101318; --soft:#222420 } }+ :root { --accent:var(--pyr); --accent-fg:var(--pyr-fg) }* { box-sizing:border-box } body { margin:0; background:var(--bg); color:var(--fg); font:16px/1.5 system-ui,-apple-system,"Segoe UI",sans-serif } main { max-width:920px; margin:0 auto; padding:24px 16px 64px }@@ -49,6 +53,7 @@ PAGE = r"""<!doctype html>h2 { font-size:17px; margin:24px 0 8px } nav { display:flex; gap:4px; flex-wrap:wrap; margin:12px 0 16px } nav a { padding:6px 12px; border-radius:999px; color:var(--fg); text-decoration:none; border:1px solid var(--line); background:var(--card); font-size:14px }+ nav a:hover { border-color:var(--tan) }nav a[aria-current=page] { background:var(--accent); color:var(--accent-fg); border-color:var(--accent) } section[hidden] { display:none } p.lede { color:var(--mute); margin:0 0 16px }@@ -73,6 +78,9 @@ PAGE = r"""<!doctype html>.notice.bad { border-color:var(--bad) } footer { color:var(--mute); font-size:13px; margin-top:32px } footer a, .card a { color:var(--accent) }+ .pyr { color:var(--pyr) } .tan { color:var(--tan) }+ :focus-visible { outline:2px solid var(--tan); outline-offset:2px }+ ::selection { background:var(--tan); color:var(--tan-fg) }.chat-log { height:min(60vh,480px); overflow-y:auto; padding:2px 2px 8px; overflow-wrap:anywhere } .chat-log .line { margin:6px 0 } .chat-log .sys { color:var(--mute); font-size:14px }@@ -87,10 +95,10 @@ PAGE = r"""<!doctype html><body> <main> <header>- <h1>Pyrtan <small id="who">…</small></h1>+ <h1><span class="pyr" data-theme-part="pyr">Pyr</span><span class="tan" data-theme-part="tan">tan</span> <small id="who">…</small></h1><span class="mute" id="where"></span> </header>- <p class="lede">A core set of expandable services that act on all forms of I/O through hardcoded, inspectable decision trees. Ask a sentence a tree answers to; the answer comes back with the exact path that produced it. If nothing answers, you get the tree that would. No model is involved.</p>+ <p class="lede">A core set of expandable services that act on all forms of I/O through inspectable decision trees. Ask a sentence a tree answers to; the answer comes back with the exact path that produced it. If nothing answers, you get the tree that would. No model is involved.</p><section id="connect" hidden> <div class="card">@@ -245,7 +253,7 @@ POST /chat/start {"want": "anyone"|"people"|"bot"} (hub with --chat)</pre></div> </section>- <footer><a href="https://pyrtan.com">pyrtan.com</a> · <a href="https://github.com/b-i-t-8/pyrtan">source</a> · AGPL-3.0-or-later · console v__VERSION__<span id="hub-version"></span></footer>+ <footer><a href="https://pyrtan.com">pyrtan.com</a> · <a href="https://pyrtan.com/source/">source</a> · AGPL-3.0-or-later · console v__VERSION__<span id="hub-version"></span></footer></main> <script> const $ = (s) => document.querySelector(s);@@ -760,7 +768,7 @@ load();</script> </body> </html>-""".replace("__VERSION__", __version__)+""".replace("__VERSION__", __version__).replace("__THEME__", THEME_JS)MANIFEST = json.dumps({ "name": "Pyrtan",@@ -770,8 +778,8 @@ MANIFEST = json.dumps({"scope": "./", "display": "standalone", "background_color": "#fbfaf7",- "theme_color": "#2b5db8",- "icons": [{"src": "data:image/svg+xml," + urllib.parse.quote('<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64"><rect width="64" height="64" rx="12" fill="#2b5db8"/><path d="M18 46V18h14a9 9 0 0 1 0 18h-8" fill="none" stroke="#fff" stroke-width="6" stroke-linecap="round" stroke-linejoin="round"/></svg>'), "sizes": "any", "type": "image/svg+xml", "purpose": "any"}],+ "theme_color": "#c2410c",+ "icons": [{"src": "data:image/svg+xml," + urllib.parse.quote('<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 64 64"><rect width="64" height="64" rx="12" fill="#c2410c"/><path d="M18 46V18h14a9 9 0 0 1 0 18h-8" fill="none" stroke="#fff" stroke-width="6" stroke-linecap="round" stroke-linejoin="round"/></svg>'), "sizes": "any", "type": "image/svg+xml", "purpose": "any"}],}, indent=2) # The service worker keeps the console shell so the page opens offline (a diff --git a/tests/conftest.py b/tests/conftest.py index 7008d03..dd5d79a 100644 --- a/tests/conftest.py +++ b/tests/conftest.py@@ -24,3 +24,10 @@ def recorded() -> RecordedTransport:@pytest.fixture def runner(examples: Registry, recorded: RecordedTransport) -> Runner: return Runner(examples, recorded)+++@pytest.fixture(autouse=True)+def _no_default_network(monkeypatch, tmp_path):+ """No test reads the default network from pyrtan.com; one that wants a+ default network points $PYRTAN_NETWORK at a file of its own."""+ monkeypatch.setenv("PYRTAN_NETWORK", str(tmp_path / "no-default-network.json"))diff --git a/tests/test_federation.py b/tests/test_federation.py index f80a9ba..2463349 100644 --- a/tests/test_federation.py +++ b/tests/test_federation.py@@ -424,7 +424,8 @@ def test_snapshot_bakes_and_bootstraps(two, tmp_path, monkeypatch, capsys):assert snap["seeds"]["previous"] == str(old) ids = [h["id"] for h in snap["hubs"]] assert b.url in ids and "http://gone.example" in ids and "http://ancient.example" not in ids # live from the seed, kept from the snapshot, expired- assert next(h for h in snap["hubs"] if h["id"] == b.url)["via"] == a.url+ # B was named by A, so it was asked too, and its own word on itself is the newest.+ assert snap["seeds"]["crawled"]["answered"] == [b.url] and next(h for h in snap["hubs"] if h["id"] == b.url)["via"] == b.urldef cli(*argv): code = main(list(argv))@@ -850,3 +851,179 @@ def test_reports_command_and_snapshot(two, tmp_path, capsys, monkeypatch):monkeypatch.setenv("PYRTAN_REPORTS", str(tmp_path / "reports.json")) monkeypatch.delenv("PYRTAN_AT", raising=False) assert main(["reports"]) == 0 and "pyrtan/weather@1 code" in capsys.readouterr().out+++# ---------------------------------------------------------------- networks+++def test_invites_and_network_ids():+ from pyrtan.federation import Network, network_id, parse_invite++ net, hub = parse_invite("home:abcdefghijklmnop1234@https://nas.example.ts.net/")+ assert (net.name, net.secret, hub) == ("home", "abcdefghijklmnop1234", "https://nas.example.ts.net")+ assert parse_invite("home:abcdefghijklmnop1234")[1] is None+ assert net.invite("https://x/") == "home:abcdefghijklmnop1234@https://x" and net.invite() == "home:abcdefghijklmnop1234"+ for bad in ("home", "Home:abcdefghijklmnop1234", "home:short", "home:abcdefghijklmnop1234@ftp://x", ""):+ with pytest.raises(ValueError):+ parse_invite(bad)+ # The id comes from name and secret: same name, other secret, another network; and it holds neither.+ assert network_id("home", "a" * 20) != network_id("home", "b" * 20) == Network("home", "b" * 20).id+ assert "a" * 20 not in network_id("home", "a" * 20)+++def test_a_hub_starts_a_network_and_others_join_by_invite(tmp_path):+ from pyrtan.federation import Network++ for n in "abc":+ (tmp_path / n).mkdir()+ (tmp_path / n / f"{n}x.toml").write_text(ECHO.format(name=f"{n}x", word=f"{n}x"))+ a = Swarm("a", tmp_path / "a", token="token-a", open_follows=False) # takes no follows outside its networks+ b = Swarm("b", tmp_path / "b", token="token-b")+ c = Swarm("c", tmp_path / "c")+ try:+ status, made = a.post("/networks", {"name": "home"})+ assert status == 200 and made["network"] == "home" and made["invite"].startswith("home:") and made["invite"].endswith(f"@{a.url}")+ assert a.post("/networks", {"name": "home"})[0] == 400 and a.post("/networks", {"name": "Not OK"})[0] == 400+ assert a.fed.following == {} and a.fed.followers == {} # a network of one+ # B joins by invite: it is in home, follows A in it, and routes to A's trees.+ status, joined = b.post("/join", {"invite": made["invite"]})+ assert status == 200 and joined["network"] == "home" and joined["following"]["url"] == a.url and joined["following"]["state"] == "accepted"+ assert b.fed.following[a.url].network == a.fed.networks["home"].id == a.fed.followers[b.url].network+ assert b.post("/intent", {"text": "ax hi"})[1]["say"] == "ax: hi"+ # Outside the network A takes nobody (it is --closed); in another network of the same name, nobody either.+ assert c.fed.follow(a.url).state == "pending" and "closed" in c.fed.following[a.url].last_error+ c.fed.add_network(Network("home", "another-secret-entirely"))+ assert c.fed.follow(a.url, network="home").state == "pending"+ assert "not-in-network" in c.fed.following[a.url].last_error+ # A Follow naming home's id but signed with another secret is refused as unsigned.+ forged = {"type": "Follow", "id": "x", "actor": c.url, "object": a.url, "secret": "p" * 32, "inbox": f"{c.url}/inbox", "network": a.fed.networks["home"].id}+ status, answer = http_json("POST", f"{a.url}/inbox", forged, headers=sign("another-secret-entirely", c.url, "POST", "/inbox", forged))+ assert status == 401 and answer["reason"] == "unsigned"+ # The invite lets C in, and joining again changes nothing; a name taken by another network is refused.+ assert c.post("/leave", {"network": "home"})[1]["left"] is True and c.url not in a.fed.followers+ status, joined = c.post("/join", {"invite": made["invite"]})+ assert status == 200 and joined["following"]["state"] == "accepted"+ assert c.post("/join", {"invite": made["invite"]})[1]["following"]["url"] == a.url+ assert c.post("/join", {"invite": "home:yet-another-secret-xyz"})[0] == 400+ assert c.post("/join", {"invite": "nonsense"})[1]["reason"] == "bad-invite"+ # An invite with no hub: in the network, following nobody.+ status, alone = b.post("/join", {"invite": "solo:abcdefghijklmnopqrstu"})+ assert status == 200 and alone["following"] is None and "network of one" in alone["say"]+ _, view = b.get("/networks")+ assert {n["name"]: n["following"] for n in view["networks"]} == {"home": [a.url], "solo": []}+ assert all(n["invite"].endswith(f"@{b.url}") for n in view["networks"])+ assert http_json("GET", f"{b.url}/networks")[0] == 401 # the invites are the operator's+ # Leaving cuts the links made in it, both ways, and forgets it.+ assert b.post("/leave", {"network": "home"})[1]["left"] is True+ assert a.url not in b.fed.following and b.url not in a.fed.followers and "home" not in b.fed.networks+ assert b.post("/leave", {"network": "home"})[1]["left"] is False+ finally:+ for s in (a, b, c):+ s.stop()+++def test_a_private_network_never_reaches_the_public_directory(tmp_path):+ from pyrtan.federation import Network++ for n in "abc":+ (tmp_path / n).mkdir()+ (tmp_path / n / f"{n}x.toml").write_text(ECHO.format(name=f"{n}x", word=f"{n}x"))+ public = Network("pyrtan", "public-default-network-of-pyrtan-com", public=True)+ a = Swarm("a", tmp_path / "a", listed=True)+ b = Swarm("b", tmp_path / "b", token="token-b")+ c = Swarm("c", tmp_path / "c", listed=True)+ try:+ # A is in the public network; B joins it through A. C starts "home", and B joins that too.+ a.fed.add_network(public)+ assert b.fed.join(Network(public.name, public.secret, public=True), [a.url]).state == "accepted"+ home = c.fed.create_network("home")+ assert b.fed.join(Network("home", home.secret), [c.url]).state == "accepted"+ b.hub.refresh_all()+ # B hears A in the public network and C in home, and passes each on only within its own network.+ heard = {e["id"]: e["network"] for e in b.fed.directory_entries.values()}+ assert heard == {a.url: public.id, c.url: home.id}+ assert c.url not in {h["id"] for h in a.fed.directory()["hubs"]}+ assert a.url not in {h["id"] for h in c.fed.directory()["hubs"]}+ assert [e["id"] for e in b.fed.gossip(public.id)] == [a.url] and [e["id"] for e in b.fed.gossip(home.id)] == [c.url]+ # B's public /directory shows the public network; the operator also sees home.+ _, shown = http_json("GET", f"{b.url}/directory")+ assert {h["id"] for h in shown["hubs"]} == {a.url} and shown["networks"] == {public.id: "pyrtan"}+ _, full = b.get("/directory")+ assert {h["id"]: h["networks"] for h in full["hubs"]} == {a.url: [public.id], c.url: [home.id]}+ assert full["networks"] == {public.id: "pyrtan", home.id: "home"}+ # The directory blob B's keepers hold is the public part only.+ assert {e["id"] for e in b.fed.directory_backup()} == {a.url}+ # State keeps networks and the tags; an entry of a network since left is not taken back.+ state = tmp_path / "b-state.json"+ b.fed.state_path = state+ b.fed._save()+ again = Federation(b.url, state=state)+ assert set(again.networks) == {"pyrtan", "home"} and again.networks["pyrtan"].public+ assert {e["id"]: e["network"] for e in again.directory_entries.values()} == {a.url: public.id, c.url: home.id}+ stale = json.loads(state.read_text())+ stale["networks"] = [n for n in stale["networks"] if n["name"] != "home"]+ state.write_text(json.dumps(stale))+ assert {e["id"] for e in Federation(b.url, state=state).directory_entries.values()} == {a.url}+ finally:+ for s in (a, b, c):+ s.stop()+++def test_the_default_network_from_pyrtan_com(two, tmp_path, monkeypatch, capsys):+ import subprocess+ import sys as _sys++ from pyrtan.federation import default_network+ from pyrtan.hub import serve_hub++ a, b = two++ def run(*argv):+ code = main(list(argv))+ out = capsys.readouterr()+ return code, out.out, out.err++ network = ROOT / "site" / "rendezvous" / "network.json"+ monkeypatch.setenv("PYRTAN_NETWORK", str(network))+ seeds = tmp_path / "seeds.json"+ seeds.write_text(json.dumps({"hubs": ["http://127.0.0.1:9", b.url]}))+ monkeypatch.setenv("PYRTAN_SEEDS", str(seeds))+ monkeypatch.setenv("PYRTAN_SNAPSHOT", str(tmp_path / "no-snapshot.json"))+ monkeypatch.delenv("PYRTAN_RENDEZVOUS", raising=False)+ default = default_network()+ assert default is not None and default.name == "pyrtan" and default.public+ # The first hub in the default network is a network of one: nobody to follow yet.+ code, out, _ = run("join", "--at", b.url, "--token", "token-b", "--listed")+ assert code == 0 and "in pyrtan; no hub in it accepted a follow yet" in out and b.fed.following == {}+ # `pyrtan follow` with no URL is `pyrtan join`: A is in the default network, following the seed in it.+ code, out, _ = run("follow", "--at", a.url, "--token", "token-a")+ assert code == 0 and f"in pyrtan, following {b.url}" in out and "invite others" not in out # a public network needs no invite+ assert a.fed.following[b.url].network == default.id+ code, out, _ = run("network", "list", "--at", a.url, "--token", "token-a")+ assert code == 0 and f"pyrtan {default.id} public following 1" in out+ # Its own network, by command: the invite, then another hub joins with it.+ code, out, _ = run("network", "new", "home", "--at", b.url, "--token", "token-b")+ invite = next(line.split("pyrtan join ", 1)[1] for line in out.splitlines() if "pyrtan join " in line)+ assert code == 0 and invite.startswith("home:") and invite.endswith(f"@{b.url}")+ code, out, _ = run("join", invite, "--at", a.url, "--token", "token-a")+ assert code == 0 and "in home" in out and a.fed.following[b.url].network == b.fed.networks["home"].id+ assert run("follow", "--at", a.url, "--token", "token-a", "--network", "home")[0] == 2 # a network needs a hub to follow in it+ code, out, _ = run("network", "leave", "home", "--at", a.url, "--token", "token-a")+ assert code == 0 and "left home" in out and "home" not in a.fed.networks+ assert run("network", "new", "--at", a.url, "--token", "token-a")[0] == 2+ # A hub started with --join default (or --follow seed) is in it from the start.+ hub, srv = serve_hub(port=0, quiet=True, name="d", join=["default"])+ try:+ assert hub.federation is not None and hub.federation.following[b.url].network == default.id+ finally:+ hub.stop()+ srv.stop()+ # The snapshot keeps the default network's part of each directory and nobody else's.+ b.fed.merge_directory([{"id": "http://private.example", "name": "p", "trees": [], "seen": time.time()}], via="x", network="0123456789abcdef")+ b.fed.merge_directory([{"id": "http://public.example", "name": "q", "trees": [], "seen": time.time()}], via="x", network=default.id)+ b.hub.token = None # as anyone sees it, the private part included: the snapshot must still drop it+ out = tmp_path / "directory.json"+ done = subprocess.run([_sys.executable, "site/snapshot.py", str(seeds), str(out), "--network", str(network)], capture_output=True, text=True, cwd=str(ROOT))+ assert done.returncode == 0, done.stderr+ ids = {h["id"] for h in json.loads(out.read_text())["hubs"]}+ assert "http://public.example" in ids and "http://private.example" not in ids and b.url in idsdiff --git a/tests/test_service_hub.py b/tests/test_service_hub.py index c8c5e49..0b064d8 100644 --- a/tests/test_service_hub.py +++ b/tests/test_service_hub.py@@ -220,6 +220,9 @@ def test_demo_page_is_served_by_service_and_hub(example_service):assert resp.headers.get_content_type() == "text/html" page = resp.read() assert b"<title>Pyrtan</title>" in page and f"console v{__version__}".encode() in page+ # The wordmark's two halves pick their own colours; the picker is inline, not another request.+ assert b'data-theme-part="pyr">Pyr<' in page and b'data-theme-part="tan">tan<' in page+ assert b"__THEME__" not in page and b'localStorage.getItem(KEY)' in pagestatus, api = http_json("GET", f"{server.url}/api") assert status == 200 and api["pyrtan"] == "service" hub = Hub() diff --git a/tests/test_site_source.py b/tests/test_site_source.py new file mode 100644 index 0000000..4c6c969 --- /dev/null +++ b/tests/test_site_source.py@@ -0,0 +1,44 @@+"""site/source.py: the source browser pyrtan.com publishes at /source/."""++import importlib.util+import subprocess+import sys++from conftest import ROOT++spec = importlib.util.spec_from_file_location("site_source", ROOT / "site" / "source.py")+source = importlib.util.module_from_spec(spec)+spec.loader.exec_module(source)+++def test_builds_from_this_checkout(tmp_path):+ """Works on whatever git this checkout has, a shallow clone included."""+ out = tmp_path / "source"+ run = subprocess.run([sys.executable, "site/source.py", str(out)], capture_output=True, text=True, cwd=str(ROOT))+ assert run.returncode == 0, run.stderr+ for page in ("index.html", "releases.html", "branches.html", "commits.html", "tree/src/pyrtan/index.html", "blob/pyproject.toml.html"):+ assert (out / page).is_file(), page+ blob = (out / "blob/pyproject.toml.html").read_text()+ assert 'id="L1"' in blob and "[project]" in blob+ head = subprocess.run(["git", "rev-parse", "HEAD"], capture_output=True, text=True, cwd=str(ROOT)).stdout.strip()+ assert (out / f"commit/{head}.html").is_file()+ assert 'class="md"' in (out / "index.html").read_text() # the README, rendered+++def test_markdown_escapes_and_links():+ resolve = lambda url: "R:" + url+ html = source.markdown("# T <b>\n\nSee [the `docs`](docs/x.md) and **<i>**.\n\n```\n<script>\n```\n\n- one\n- two\n\n| a | b |\n|---|---|\n| 1 | 2 |", resolve)+ assert "<h1>T <b></h1>" in html+ assert '<a href="R:docs/x.md">the <code>docs</code></a>' in html+ assert "<b><i></b>" in html+ assert "<pre><script></pre>" in html+ assert "<ul><li>one</li><li>two</li></ul>" in html+ assert "<tr><td>1</td><td>2</td></tr>" in html and "---" not in html+++def test_notes_reflow_and_trailers():+ body = "One line\nwrapped here.\n\n- a list\n- kept\n\nCo-Authored-By: X <x@example.com>"+ assert "<p>One line wrapped here.</p>" in source.notes(body)+ assert "- a list\n- kept" in source.notes(body)+ assert "Co-Authored-By" in source.notes(body)+ assert "Co-Authored-By" not in source.notes(body, trailers=False)diff --git a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/github/github-release.toml b/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/github/github-release.toml deleted file mode 100644 index d32f374..0000000 --- a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/github/github-release.toml +++ /dev/null@@ -1,63 +0,0 @@-# Latest release of a GitHub repository, from the public API. Shows an-# optional token from the service's env and a decision on the status code.--name = "github-release"-version = 1-description = "The latest release of a GitHub repository."-tags = ["github", "read-only"]-license = "MIT"-hosts = ["api.github.com"]-start = "fetch"--[[triggers]]-pattern = "[what's the] latest release of {repo:word}"-[[triggers]]-pattern = "latest {repo:word} release"--[inputs.repo]-type = "str"-pattern = "^[A-Za-z0-9_.\\-]+/[A-Za-z0-9_.\\-]+$"-description = "owner/name"--[nodes.fetch]-kind = "observe"-http = { method = "GET", url = "https://api.github.com/repos/{{ inputs.repo }}/releases/latest", headers = { Accept = "application/vnd.github+json" } }-into = "resp"-extract = [- { into = "tag", json = "json.tag_name" },- { into = "name", json = "json.name" },- { into = "published", json = "json.published_at" },- { into = "url", json = "json.html_url" },- { into = "prerelease", json = "json.prerelease" },-]-next = "status"--[nodes.status]-kind = "decide"-on = "resp.status"-branches = [- { when = { eq = 200 }, then = "reply" },- { when = { eq = 404 }, then = "none" },- { when = { in = [403, 429] }, then = "rate-limited" },- { else = "unexpected" },-]--[nodes.reply]-kind = "done"-say = "{{ inputs.repo }}: {{ state.tag }} ({{ state.published }}) {{ state.url }}"-output = { repo = "{{ inputs.repo }}", tag = "{{ state.tag }}", name = "{{ state.name }}", published = "{{ state.published }}", url = "{{ state.url }}", prerelease = "{{ state.prerelease }}" }--[nodes.none]-kind = "fail"-reason = "no-release"-say = "{{ inputs.repo }} has no releases, or does not exist."--[nodes.rate-limited]-kind = "fail"-reason = "rate-limited"-say = "GitHub is rate-limiting unauthenticated requests ({{ state.resp.status }})."--[nodes.unexpected]-kind = "fail"-reason = "unexpected-response"-say = "GitHub answered {{ state.resp.status }}."diff --git a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/hacker-news/hacker-news-item.toml b/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/hacker-news/hacker-news-item.toml deleted file mode 100644 index eadbb09..0000000 --- a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/hacker-news/hacker-news-item.toml +++ /dev/null@@ -1,66 +0,0 @@-name = "hacker-news-item"-version = 1-description = "One Hacker News item by id: title, url, score, author."-tags = ["news", "read-only"]-license = "MIT"-hosts = ["hacker-news.firebaseio.com"]-start = "fetch"--[[triggers]]-pattern = "hacker news item {id:int}"--[inputs.id]-type = "int"-min = 1-description = "The item id"--[nodes.fetch]-kind = "observe"-http = { method = "GET", url = "https://hacker-news.firebaseio.com/v0/item/{{ inputs.id }}.json" }-into = "resp"-extract = [- { into = "title", json = "json.title" },- { into = "url", json = "json.url" },- { into = "score", json = "json.score" },- { into = "by", json = "json.by" },- { into = "type", json = "json.type" },-]-next = "kind"--[nodes.kind]-kind = "decide"-on = "type"-branches = [- { when = { eq = "story" }, then = "has-url" },- { when = { eq = "job" }, then = "has-url" },- { when = { exists = false }, then = "missing" },- { else = "not-a-story" },-]--[nodes.has-url]-kind = "decide"-on = "url"-branches = [- { when = { exists = true }, then = "reply" },- { else = "self-post" },-]--[nodes.self-post]-kind = "set"-set = { url = "https://news.ycombinator.com/item?id={{ inputs.id }}" }-next = "reply"--[nodes.reply]-kind = "done"-say = "{{ state.title }} ({{ state.score }} points, by {{ state.by }}) {{ state.url }}"-output = { id = "{{ inputs.id }}", title = "{{ state.title }}", url = "{{ state.url }}", score = "{{ state.score }}", by = "{{ state.by }}" }--[nodes.missing]-kind = "fail"-reason = "no-such-item"-say = "There is no Hacker News item {{ inputs.id }}."--[nodes.not-a-story]-kind = "fail"-reason = "not-a-story"-say = "Item {{ inputs.id }} is a {{ state.type }}, not a story."diff --git a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/hacker-news/hacker-news.toml b/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/hacker-news/hacker-news.toml deleted file mode 100644 index 826c223..0000000 --- a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/hacker-news/hacker-news.toml +++ /dev/null@@ -1,79 +0,0 @@-# Hacker News, through its public Firebase API. Two trees: this one answers-# the question, and `hacker-news-item` fetches one story, so that `each`-# can reuse it.--name = "hacker-news"-version = 1-description = "Top stories on Hacker News, from its public API."-tags = ["news", "read-only"]-license = "MIT"-hosts = ["hacker-news.firebaseio.com"]-start = "fetch-ids"--[[triggers]]-pattern = "[what's the] top story on hacker news"-inputs = { count = 1 }--[[triggers]]-pattern = "top {count:int} stories on hacker news"--[inputs.count]-type = "int"-default = 1-min = 1-max = 30-description = "How many stories to return"--[nodes.fetch-ids]-kind = "observe"-http = { method = "GET", url = "https://hacker-news.firebaseio.com/v0/topstories.json" }-into = "ids_resp"-extract = [{ into = "ids", json = "json" }]-next = "check"--[nodes.check]-kind = "decide"-on = "ids_resp.status"-branches = [- { when = { eq = 200 }, then = "any" },- { when = { gte = 500 }, then = "hn-down" },- { else = "unexpected" },-]--[nodes.any]-kind = "decide"-on = "ids"-branches = [- { when = { empty = true }, then = "nothing" },- { when = { type = "list" }, then = "stories" },- { else = "unexpected" },-]--[nodes.stories]-kind = "each"-over = "ids"-limit = "{{ inputs.count }}"-tree = "hacker-news-item"-with = { id = "{{ item }}" }-into = "stories"-next = "reply"--[nodes.reply]-kind = "done"-say = "Top of Hacker News: {{ state.stories[0].title }} — {{ state.stories[0].url }}"-output = { stories = "{{ state.stories }}" }--[nodes.nothing]-kind = "fail"-reason = "no-stories"-say = "Hacker News returned an empty front page."--[nodes.hn-down]-kind = "fail"-reason = "upstream-down"-say = "Hacker News answered {{ state.ids_resp.status }}; try again later."--[nodes.unexpected]-kind = "fail"-reason = "unexpected-response"-say = "Hacker News answered {{ state.ids_resp.status }} with something this tree does not recognise."diff --git a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/site-status/site-status.toml b/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/site-status/site-status.toml deleted file mode 100644 index 3d016af..0000000 --- a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/site-status/site-status.toml +++ /dev/null@@ -1,62 +0,0 @@-# Is a site up? The host comes from the sentence, so it cannot be checked-# at load time; the runtime refuses anything outside `hosts`, and this tree-# allows any host on purpose, with the wildcard spelled out.--name = "site-status"-version = 1-description = "Whether a web site answers, and how quickly."-tags = ["ops", "read-only"]-license = "MIT"-hosts = ["*"]-start = "probe"--[[triggers]]-pattern = "is {host:host} up"-[[triggers]]-pattern = "is {host:host} down"-[[triggers]]-pattern = "[check] status of {host:host}"--[inputs.host]-type = "str"-pattern = "(?i)^[a-z0-9.\\-]+(:[0-9]+)?$"-description = "A host name, with no scheme"--[nodes.probe]-kind = "observe"-http = { method = "GET", url = "https://{{ inputs.host | lower }}/" }-into = "resp"-timeout = 8-retries = 1-next = "status"--[nodes.status]-kind = "decide"-on = "resp.status"-branches = [- { when = { between = [200, 399] }, then = "up" },- { when = { between = [400, 499] }, then = "refusing" },- { when = { gte = 500 }, then = "erroring" },- { else = "odd" },-]--[nodes.up]-kind = "done"-say = "{{ inputs.host }} is up: {{ state.resp.status }} in {{ state.resp.elapsed_ms }} ms."-output = { host = "{{ inputs.host }}", up = true, status = "{{ state.resp.status }}", elapsed_ms = "{{ state.resp.elapsed_ms }}" }--[nodes.refusing]-kind = "done"-say = "{{ inputs.host }} answers but refuses the front page ({{ state.resp.status }}); the server is up."-output = { host = "{{ inputs.host }}", up = true, status = "{{ state.resp.status }}", note = "client error on /" }--[nodes.erroring]-kind = "fail"-reason = "server-error"-say = "{{ inputs.host }} is answering {{ state.resp.status }}: up, but broken."-output = { host = "{{ inputs.host }}", up = false, status = "{{ state.resp.status }}" }--[nodes.odd]-kind = "fail"-reason = "unexpected-status"-say = "{{ inputs.host }} answered {{ state.resp.status }}, which this tree has no branch for."diff --git a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/weather/weather.toml b/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/weather/weather.toml deleted file mode 100644 index 3e65419..0000000 --- a/tmp/claude-0/-home-user-pyrtan/c01cbcf1-7852-56a6-833f-ba19e8dea139/scratchpad/out/catalog/weather/weather.toml +++ /dev/null@@ -1,133 +0,0 @@-# Weather for a place, from Open-Meteo (no key needed). Two hops: geocode-# the name, then fetch the forecast for the coordinates. Each hop has its-# own decision so a failure says which hop failed.--name = "weather"-version = 1-description = "Current weather for a place, from Open-Meteo."-tags = ["weather", "read-only"]-license = "MIT"-hosts = ["geocoding-api.open-meteo.com", "api.open-meteo.com"]-start = "geocode"--# "forecast in Oslo" and "temperature in Oslo" read as "weather in Oslo".-[language]-synonyms = { weather = ["forecast", "temperature"] }--[[triggers]]-pattern = "[what's the] weather in {place}"-[[triggers]]-pattern = "[what's the] weather for {place}"-[[triggers]]-pattern = "how [hot|cold|warm] is it in {place}"--[inputs.place]-type = "str"-description = "A town or city name"--[inputs.units]-type = "str"-default = "celsius"-one_of = ["celsius", "fahrenheit"]--[nodes.geocode]-kind = "observe"-http = { method = "GET", url = "https://geocoding-api.open-meteo.com/v1/search?name={{ inputs.place | url }}&count=1&language=en&format=json" }-into = "geo"-extract = [- { into = "hit", json = "json.results[0]" },- { into = "lat", json = "json.results[0].latitude" },- { into = "lon", json = "json.results[0].longitude" },- { into = "name", json = "json.results[0].name" },- { into = "country", json = "json.results[0].country" },-]-next = "found"--[nodes.found]-kind = "decide"-on = "hit"-branches = [- { when = { exists = true }, then = "forecast" },- { else = "unknown-place" },-]--[nodes.forecast]-kind = "observe"-http = { method = "GET", url = "https://api.open-meteo.com/v1/forecast?latitude={{ state.lat }}&longitude={{ state.lon }}¤t=temperature_2m,relative_humidity_2m,wind_speed_10m,weather_code&temperature_unit={{ inputs.units }}" }-into = "fc"-extract = [- { into = "temp", json = "json.current.temperature_2m" },- { into = "humidity", json = "json.current.relative_humidity_2m" },- { into = "wind", json = "json.current.wind_speed_10m" },- { into = "code", json = "json.current.weather_code" },- { into = "unit", json = "json.current_units.temperature_2m" },-]-next = "have-temp"--[nodes.have-temp]-kind = "decide"-on = "temp"-branches = [- { when = { type = "number" }, then = "sky" },- { else = "no-forecast" },-]--# WMO weather codes, folded into words a person would use.-[nodes.sky]-kind = "decide"-on = "code"-branches = [- { when = { eq = 0 }, then = "clear" },- { when = { between = [1, 3] }, then = "cloudy" },- { when = { between = [45, 48] }, then = "fog" },- { when = { between = [51, 67] }, then = "rain" },- { when = { between = [71, 77] }, then = "snow" },- { when = { between = [80, 82] }, then = "rain" },- { when = { between = [85, 86] }, then = "snow" },- { when = { between = [95, 99] }, then = "storm" },- { else = "unknown-sky" },-]--[nodes.clear]-kind = "set"-set = { sky = "clear" }-next = "reply"-[nodes.cloudy]-kind = "set"-set = { sky = "cloudy" }-next = "reply"-[nodes.fog]-kind = "set"-set = { sky = "foggy" }-next = "reply"-[nodes.rain]-kind = "set"-set = { sky = "raining" }-next = "reply"-[nodes.snow]-kind = "set"-set = { sky = "snowing" }-next = "reply"-[nodes.storm]-kind = "set"-set = { sky = "stormy" }-next = "reply"-[nodes.unknown-sky]-kind = "set"-set = { sky = "weather code {{ state.code }}" }-next = "reply"--[nodes.reply]-kind = "done"-say = "{{ state.name }}, {{ state.country }}: {{ state.temp }}{{ state.unit }}, {{ state.sky }}, humidity {{ state.humidity }}%, wind {{ state.wind }} km/h."-output = { place = "{{ state.name }}", country = "{{ state.country }}", temperature = "{{ state.temp }}", unit = "{{ state.unit }}", sky = "{{ state.sky }}", humidity = "{{ state.humidity }}", wind_kmh = "{{ state.wind }}" }--[nodes.unknown-place]-kind = "fail"-reason = "unknown-place"-say = "I could not find a place called {{ inputs.place }}."--[nodes.no-forecast]-kind = "fail"-reason = "no-forecast"-say = "Open-Meteo found {{ state.name }} but returned no current temperature (status {{ state.fc.status }})."diff --git a/uv.lock b/uv.lock index 992ab3c..6bd7aff 100644 --- a/uv.lock +++ b/uv.lock@@ -147,7 +147,7 @@ wheels = [[[package]] name = "pyrtan"-version = "0.10.0"+version = "0.11.0"source = { editable = "." } [package.optional-dependencies]