Join without pyrtan.com: the default network is built in, and a hub tries the hubs it knows first; release 0.14.0
Claude · 2026-10-28 · 52873c330443221223d5c1ba2c1efa8daacbfb22 · parents 0a4767f
Joining the default network read its name and secret from pyrtan.com, and found a first hub only through pyrtan.com's hub list and snapshot, so a new hub, and every restart with --join default, depended on the site being up.
- The default network's name and secret are constants in pyrtan.federation (they are public on purpose). default_network() reads nothing unless $PYRTAN_NETWORK names another network. site/rendezvous/network.json stays as a mirror, and a test holds it equal to the built-in pair so the two can never split the network. - join_network tries, in order and lazily: the hubs it already follows in the network, the hubs its own directory heard in it (most recent first, Federation.known_hubs), the hub list, the snapshot. Each is fetched only if the ones before it did not do, so a restart, or a hub that has heard of anyone, asks pyrtan.com nothing. - $PYRTAN_HUB_LIST takes any number of sources, separated by spaces or commas, and each may be a hub list, any hub's /directory, a snapshot, or a mirror of one: pyrtan.com is one starting point among any. - Docs: README, site/README, the federation note, deploy docs and the join help say the network is built in and pyrtan.com mirrors it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rD7jd9JrpLd1HKE8P1h3G
Changes
README.md | 32 ++++++++------ _meta/federation.md | 23 +++++++--- deploy/README.md | 5 ++- deploy/openmediavault/docker-compose.yml | 2 +- pyproject.toml | 2 +- site/README.md | 8 ++-- site/rendezvous/network.json | 2 +- src/pyrtan/__init__.py | 2 +- src/pyrtan/cli.py | 2 +- src/pyrtan/federation.py | 120 ++++++++++++++++++++++++++++++--------------------- src/pyrtan/hub.py | 30 +++++++++---- tests/conftest.py | 5 ++- tests/test_federation.py | 50 +++++++++++++++++++++ uv.lock | 2 +- 14 files changed, 195 insertions(+), 90 deletions(-)
diff --git a/README.md b/README.md index 746f95a..ed1133c 100644 --- a/README.md +++ b/README.md@@ -163,7 +163,7 @@ hosted box is not lent out. The design is in [`_meta/io.md`](_meta/io.md).| `pyrtan hub --port 8700` | Routes between services. Services register with `--hub URL`; peers carry a `link` of `up`, `stale` or `down`. `--trees DIR` holds trees in the hub itself; `--federate` lets other hubs follow it and it follow them; `--listed` puts it in the directory; `--chat` keeps a lobby where people meet strangers and bots (`--no-chat-typing` turns its typing indicator off, `--chat-public` lets callers without the token in, to meet people only). | | `pyrtan keep KEY [FILE] --at HUB` | Gives your hub a blob to keep; its keepers get copies. `kept` shows who holds what; `fetch KEY` gets a blob back from your hub or whoever holds it. | | `pyrtan follow [URL] --at HUB` | Makes your hub follow another: it then routes sentences to that hub's trees too. With no URL, `pyrtan join`. `--network NAME` follows within a network of yours; `--listed` also puts your hub in the directory. `unfollow` undoes it; `peers` lists services, hubs followed and followers; `directory` lists every hub that asked to be found, as the swarm knows it. |-| `pyrtan join [INVITE] --at HUB` | Puts your hub in a network and follows a hub in it: the default network pyrtan.com publishes, or the one an invite (`NAME:SECRET@HUB`) names. |+| `pyrtan join [INVITE] --at HUB` | Puts your hub in a network and follows a hub in it: the default network (built in), or the one an invite (`NAME:SECRET@HUB`) names. || `pyrtan network new NAME --at HUB` | Starts a network of your own, a network of one, and prints its invite. `network list` shows each network with its invite; `network leave NAME` cuts the links made in it. | | `pyrtan ask "…" --at URL` | Answers a sentence through a running service or hub. | | `pyrtan chat --at HUB` | Talks to a random stranger or a bot through a hub started `--chat`. `--bot` for a bot, `--people` for people only, `--interests` to meet someone who shares one, `--trace` to see the path behind each bot line, `--no-typing` to hide "Stranger is typing…". |@@ -266,15 +266,21 @@ keeper. Leaving a network (`pyrtan network leave`) cuts the links madein it and forgets what was heard there. A follow made outside any network, and a member's, works as it always has.-The **default network** is the one pyrtan.com publishes in-`rendezvous/network.json`: `pyrtan`, with a secret that is public on-purpose. It names the network rather than guarding it: anyone may join,-and what keeps a stranger's hub harmless is that following is still the-one act of trust. `pyrtan join` (or `pyrtan hub --join default`, or-`$PYRTAN_JOIN=default`) puts a hub in it and follows the first hub in-`rendezvous/hubs.json` that answers, then the first from the snapshot;-when none does, the hub is the network so far, and others join through-it. pyrtan.com runs no hub: it is static files on GitHub Pages. Every+The **default network** is built into pyrtan: `pyrtan`, with a secret+that is public on purpose. It names the network rather than guarding+it: anyone may join, and what keeps a stranger's hub harmless is that+following is still the one act of trust. `pyrtan join` (or `pyrtan hub+--join default`, or `$PYRTAN_JOIN=default`) puts a hub in it and follows+the first hub that accepts, trying what the hub already knows before+asking anyone: the hubs it follows in the network, then the ones its+directory has heard in it, then the hub list, then the snapshot, each+fetched only if the ones before it did not do. So a hub that has been in+the network once never needs pyrtan.com again; when nobody accepts, the+hub is the network so far, and others join through it. `$PYRTAN_HUB_LIST`+takes any number of sources (a hub list, any hub's `/directory`, a+snapshot or a mirror of one), so pyrtan.com is one starting point among+any others. pyrtan.com runs no hub: it is static files on GitHub Pages,+and publishes the default network's name and secret only as a mirror. Everyhour its build asks every hub it knows of for its directory (the hub list, then everyone the last snapshot or a hub names), keeps the default network's part, and publishes it as `rendezvous/directory.json`, so the@@ -588,11 +594,11 @@ site/ pyrtan.com: the landing page, the hub list, the catalog## Status-0.13.0. The core, the service, the hub, the command, the console, the+0.14.0. The core, the service, the hub, the command, the console, thecatalog (namespaces, publishing by pull request through the gate, gap reports gossiped back to authors), the language rules, federation-between hubs, networks (any hub starts one; pyrtan.com publishes the-default), the gossiped directory, members (a browser or a phone as+between hubs, networks (any hub starts one; the default one is built+in), the gossiped directory, members (a browser or a phone aspart of the swarm), the blobs they keep and the trees they run (`pyrtan.js`, conformant with the Python runtime), six forms of I/O (HTTP, a browser, files, programs, serial lines, MQTT queues), each diff --git a/_meta/federation.md b/_meta/federation.md index 8f5aef4..ca794ad 100644 --- a/_meta/federation.md +++ b/_meta/federation.md@@ -215,11 +215,16 @@ the visitor's browser.request, the role of a torrent's bootstrap nodes, and it is the only way pyrtan.com learns that a hub exists. `pyrtan join` (and `pyrtan follow` with no URL) puts a hub in the default network (§7a) and follows the-first listed hub whose `/actor` answers; `pyrtan hub --join default`-does the same on start. A listed hub is an ordinary federated hub in-the default network, up when its operator's box is; nothing needs it to-be up always (below). `$PYRTAN_HUB_LIST` points at another list (a URL-or a file), `$PYRTAN_RENDEZVOUS` at one hub. (Until v0.13 this was+first hub that accepts, in this order, each fetched only if the ones+before it did not do: the hubs it already follows in the network, the+hubs its own directory heard in it (most recent first), the hub list,+the snapshot. `pyrtan hub --join default` does the same on start, so a+restart asks nobody. A listed hub is an ordinary federated hub in the+default network, up when its operator's box is; nothing needs it to be+up always (below). `$PYRTAN_HUB_LIST` names any number of sources,+separated by spaces or commas, each a URL or a file: a hub list, any+hub's `/directory`, a snapshot, or a mirror of one; `$PYRTAN_RENDEZVOUS`+names one hub. pyrtan.com is one starting point, not the only one. (Until v0.13 this was`seeds.json`, and pyrtan.com briefly ran a hub of its own on Fly.io; both are gone.)@@ -286,8 +291,12 @@ naming its networks. The `directory` blob a hub's keepers hold is thepublic part only. `pyrtan network leave` unfollows the hubs followed in the network, drops those that followed in it and forgets its entries.-**The default network.** pyrtan.com publishes `rendezvous/network.json`:-the name `pyrtan` and a secret that is public on purpose. A secret on a+**The default network** is built in (v0.14): the name `pyrtan` and a+secret that is public on purpose, so joining it reads nothing from+anyone. pyrtan.com publishes the same two in `rendezvous/network.json`+as a mirror, which a test holds equal to the built-in pair;+`$PYRTAN_NETWORK` names another (a URL or a file of that shape), which+is how the tests keep away from it. A secret on apublic page guards nothing, and is not meant to: it names the network, so the public directory is one network among others and a private one never mixes into it. What makes a stranger's hub harmless is what always diff --git a/deploy/README.md b/deploy/README.md index 73b5956..812a8cf 100644 --- a/deploy/README.md +++ b/deploy/README.md@@ -70,8 +70,9 @@ file's environment `PYRTAN_ADVERTISE` (that address), `PYRTAN_JOIN:default` and `PYRTAN_LISTED: "true"`, with `PYRTAN_TOKEN` set and `PYRTAN_EGRESS: public` (hubs that follow yours may ask it to run trees, and `any` would let them point a tree at your LAN). The hub then joins-the default network pyrtan.com publishes, follows the first hub in it-that answers (from pyrtan.com's hub list, then its snapshot),+the default network (built in), follows the first hub in it that+accepts (one it already knows, else from pyrtan.com's hub list, then+its snapshot),routes to it, and is listed in the directory every hub gossips; `pyrtan follow URL --at <your hub> --token …` follows any other. For a network of your own, `pyrtan network new home --at <your hub> --token …` prints diff --git a/deploy/openmediavault/docker-compose.yml b/deploy/openmediavault/docker-compose.yml index 6e502cc..148b87b 100644 --- a/deploy/openmediavault/docker-compose.yml +++ b/deploy/openmediavault/docker-compose.yml@@ -35,7 +35,7 @@ x-pyrtan: &pyrtanPYRTAN_EGRESS: any # Federation: once the hub is reachable from outside (a tunnel or a # port-forward with TLS), give it its public URL and it joins the- # default network pyrtan.com publishes, follows a hub in it, and lists+ # default network (built into pyrtan), follows a hub in it, and lists# itself in the directory. Needs the token, and PYRTAN_EGRESS: public # above: hubs that follow this one may ask it to run trees. # PYRTAN_JOIN may also hold invites to networks of your own diff --git a/pyproject.toml b/pyproject.toml index 7b1a867..36a29c3 100644 --- a/pyproject.toml +++ b/pyproject.toml@@ -1,6 +1,6 @@[project] name = "pyrtan"-version = "0.13.0"+version = "0.14.0"description = "Open-source total automation: a core set of expandable services that act on all forms of I/O through inspectable decision trees." readme = "README.md" license = "AGPL-3.0-or-later" diff --git a/site/README.md b/site/README.md index 76a30de..314d4d5 100644 --- a/site/README.md +++ b/site/README.md@@ -10,7 +10,7 @@ static directory and publishes it.| `/console/` | the console (installable: manifest, service worker and `pyrtan.js`, the tree runtime, beside it). Until it joins a hub the browser is one: it holds the catalog's trees (`/catalog/trees.json`) and runs them in the page. To join any hub started with `--cors https://pyrtan.com`, give its URL and token; its Member tab then joins that hub, and runs its trees when joined with the token | `pyrtan console --out` | | `/catalog/` | the trees people can `pyrtan pull`, as `namespace/name.toml`, `index.json` the command reads, and `trees.json`, the same trees as JSON for the console | `pyrtan index examples catalog --copy`, through `site/gate.py` | | `/theme.js` | the wordmark's colour picker the landing page and the catalog load (the console carries its own copy inline) | `src/pyrtan/js/theme.js` |-| `/rendezvous/network.json` | the default network: its name, and a secret that is public on purpose | `site/rendezvous/network.json` |+| `/rendezvous/network.json` | the default network: its name, and a secret that is public on purpose; built into pyrtan, published here as a mirror (and for the snapshot) | `site/rendezvous/network.json` || `/rendezvous/hubs.json` | the hub list: hubs operators added by pull request, which the snapshot asks first and `pyrtan join` tries first | `site/rendezvous/hubs.json` | | `/rendezvous/directory.json` | a snapshot of the default network's directory, from every hub the build can reach, refreshed hourly | `site/snapshot.py` | | `/rendezvous/reports.json` | the gap reports the hubs carry for catalog trees, refreshed hourly | `site/snapshot.py --reports` |@@ -62,9 +62,11 @@ reports` reads when given no hub.`site/rendezvous/network.json` is the default network: the name `pyrtan` and a secret that is public on purpose (it names the network,-it guards nothing; see `_meta/federation.md` §7a). `pyrtan join` with+it guards nothing; see `_meta/federation.md` §7a). pyrtan has the same+two values built in, and a test holds them equal; this file is a mirror,+and what the snapshot keeps the network's part by. `pyrtan join` withno invite, `pyrtan hub --join default` and `$PYRTAN_JOIN=default` put a-hub in it.+hub in it without reading it.pyrtan.com runs no hub of its own; it is these static files, and the hub list is how it learns that a hub exists. `site/rendezvous/hubs.json` diff --git a/site/rendezvous/network.json b/site/rendezvous/network.json index ac2765c..2aaa4a0 100644 --- a/site/rendezvous/network.json +++ b/site/rendezvous/network.json@@ -1,6 +1,6 @@{ "pyrtan-network": 1,- "about": "The default network: `pyrtan join` with no invite (or `pyrtan hub --join default`) puts a hub in it and follows the first hub in hubs.json that answers, then the snapshot's hubs. Its secret is public on purpose: it names the network rather than guarding it, so anyone may join, and following stays one-directional trust. A network of your own is `pyrtan network new NAME`; its invite is yours to share.",+ "about": "The default network, as built into pyrtan; this file mirrors it (and the snapshot keeps the network's part by it). `pyrtan join` with no invite (or `pyrtan hub --join default`) puts a hub in it and follows the first hub that accepts: one it already knows, else one from hubs.json, else from the snapshot. Its secret is public on purpose: it names the network rather than guarding it, so anyone may join, and following stays one-directional trust. A network of your own is `pyrtan network new NAME`; its invite is yours to share.","name": "pyrtan", "secret": "public-default-network-of-pyrtan-com" } diff --git a/src/pyrtan/__init__.py b/src/pyrtan/__init__.py index ec3238a..b69de05 100644 --- a/src/pyrtan/__init__.py +++ b/src/pyrtan/__init__.py@@ -33,4 +33,4 @@ __all__ = ["Response", ]-__version__ = "0.13.0"+__version__ = "0.14.0"diff --git a/src/pyrtan/cli.py b/src/pyrtan/cli.py index 8a79f27..cab716c 100644 --- a/src/pyrtan/cli.py +++ b/src/pyrtan/cli.py@@ -164,7 +164,7 @@ def main(argv: list[str] | None = None) -> int:p.add_argument("--federate", action="store_true", help="answer /actor and /inbox so other hubs can follow this one, and follow others") p.add_argument("--advertise", default=os.environ.get("PYRTAN_ADVERTISE"), help="the URL other hubs reach this hub at (implies --federate; default $PYRTAN_ADVERTISE, else this host and port)") p.add_argument("--follow", action="append", default=[h for h in os.environ.get("PYRTAN_FOLLOW", "").replace(",", " ").split() if h], metavar="URL", help="a hub to follow on start (repeatable; default $PYRTAN_FOLLOW; to join the default network, --join default)")- p.add_argument("--join", action="append", default=[h for h in os.environ.get("PYRTAN_JOIN", "").replace(",", " ").split() if h], metavar="INVITE", help="a network to be in on start: 'default' for the public one pyrtan.com publishes, or an invite NAME:SECRET[@HUB] (repeatable; default $PYRTAN_JOIN)")+ p.add_argument("--join", action="append", default=[h for h in os.environ.get("PYRTAN_JOIN", "").replace(",", " ").split() if h], metavar="INVITE", help="a network to be in on start: 'default' for the public one built into pyrtan, or an invite NAME:SECRET[@HUB] (repeatable; default $PYRTAN_JOIN)")p.add_argument("--federation-secret", default=os.environ.get("PYRTAN_FEDERATION_SECRET"), help="only hubs that sign their Follow with this secret may follow (default $PYRTAN_FEDERATION_SECRET; without it, follows are open)") p.add_argument("--closed", action="store_true", help="take no follows at all (this hub can still follow others)") p.add_argument("--state", metavar="FILE", default=os.environ.get("PYRTAN_STATE"), help="keep who this hub follows and is followed by, and the directory, in this file across restarts (default $PYRTAN_STATE)") diff --git a/src/pyrtan/federation.py b/src/pyrtan/federation.py index 90a1b10..775fed3 100644 --- a/src/pyrtan/federation.py +++ b/src/pyrtan/federation.py@@ -60,8 +60,18 @@ it is in that network. The directory heard over a link is kept with thelink's network and passed on only over links in the same one, so a private network's hubs never reach the public directory, and ``GET /directory`` shows them only to the operator. The default network-(``default_network``, published at pyrtan.com) has a public secret: it-names the network, it does not guard it, and nothing here needs it to.+(``default_network``) has a public secret: it names the network, it does+not guard it, and nothing here needs it to. Its name and secret are+built in, so joining it asks nobody for them; pyrtan.com publishes the+same two values as a mirror, not as the authority.++**Joining** the default network tries what the hub already knows before+anything else: the hubs it follows in it, then the hubs its directory+has heard in it, most recent first. Only when none of those accepts does+it read the hub list and then the snapshot, and only as far as it needs.+So a hub that has been in the network once never needs pyrtan.com again,+and any hub's public ``/directory`` or any mirror of the list will do as+a starting point (``$PYRTAN_HUB_LIST``).A follow outside any network (and every member's) works as before. **Chat.** A hub with a lobby (``--chat``) asks the hubs it follows for a@@ -90,7 +100,7 @@ import uuidfrom collections import deque from dataclasses import dataclass, field from pathlib import Path-from typing import Any, Callable+from typing import Any, Callable, Iterablefrom . import __version__ from .service import http_json@@ -98,7 +108,10 @@ from .store import BlobStore, StoreError, check_key, digest as blob_digestDEFAULT_HUB_LIST = "https://pyrtan.com/rendezvous/hubs.json" DEFAULT_SNAPSHOT = "https://pyrtan.com/rendezvous/directory.json"-DEFAULT_NETWORK = "https://pyrtan.com/rendezvous/network.json"+# The default network, built in: public on purpose, so it names the network+# and guards nothing. site/rendezvous/network.json publishes the same values.+DEFAULT_NETWORK_NAME = "pyrtan"+DEFAULT_NETWORK_SECRET = "public-default-network-of-pyrtan-com"NETWORK_NAME = re.compile(r"^[a-z0-9][a-z0-9-]{0,39}$") INVITE = re.compile(r"^([a-z0-9][a-z0-9-]{0,39}):([A-Za-z0-9_-]{16,128})(?:@(https?://\S+))?$") SIGNATURE_WINDOW = 300.0 # seconds a signed message stays valid@@ -127,47 +140,51 @@ SIGNED_PATHS = ("/inbox", "/run", "/intent", "/resume", "/capabilities")# ---------------------------------------------------------------- the hub list-def rendezvous_hubs() -> list[str]:- """Hubs to try first, from ``$PYRTAN_RENDEZVOUS`` (one hub),- ``$PYRTAN_HUB_LIST`` (a URL or file holding ``{"hubs": [...]}``), or- the hub list at pyrtan.com. Never raises: no list is an empty list."""- single = os.environ.get("PYRTAN_RENDEZVOUS")- if single:- return [single.rstrip("/")]- source = os.environ.get("PYRTAN_HUB_LIST") or DEFAULT_HUB_LIST+def _read_json(source: str) -> Any:+ """A URL or a file, as JSON; None when it cannot be had."""try: if source.startswith(("http://", "https://")): status, body = http_json("GET", source, timeout=10.0)- if status != 200:- return []- else:- body = json.loads(Path(source).read_text(encoding="utf-8"))+ return body if status == 200 else None+ return json.loads(Path(source).read_text(encoding="utf-8"))except Exception:- return []+ return None+++def hub_urls(body: Any) -> list[str]:+ """The hubs a list names: ``{"hubs": ["https://…", …]}`` (a hub list)+ or ``{"hubs": [{"id": "https://…", "seen": …}, …]}`` (a hub's+ ``/directory``, or a snapshot), the latter most recently heard first."""if not isinstance(body, dict) or not isinstance(body.get("hubs"), list): return []- return [h.rstrip("/") for h in body["hubs"] if isinstance(h, str) and h.startswith(("http://", "https://"))]+ rows = body["hubs"]+ if all(isinstance(h, dict) for h in rows):+ rows = sorted(rows, key=lambda h: -float(h.get("seen", 0) or 0))+ urls = [h if isinstance(h, str) else h.get("id") if isinstance(h, dict) else None for h in rows]+ return list(dict.fromkeys(u.rstrip("/") for u in urls if isinstance(u, str) and u.startswith(("http://", "https://"))))+++def rendezvous_hubs() -> list[str]:+ """Hubs to try first, from ``$PYRTAN_RENDEZVOUS`` (one hub),+ ``$PYRTAN_HUB_LIST`` (any number of URLs or files, separated by spaces+ or commas: hub lists, a hub's ``/directory``, a snapshot or a mirror of+ one, in order), or the hub list at pyrtan.com. Never raises: nothing+ readable is an empty list."""+ single = os.environ.get("PYRTAN_RENDEZVOUS")+ if single:+ return [single.rstrip("/")]+ sources = os.environ.get("PYRTAN_HUB_LIST", "").replace(",", " ").split() or [DEFAULT_HUB_LIST]+ out: list[str] = []+ for source in sources:+ out.extend(u for u in hub_urls(_read_json(source)) if u not in out)+ return outdef snapshot_hubs() -> list[str]: """Hubs from the directory snapshot pyrtan.com bakes hourly (``$PYRTAN_SNAPSHOT`` overrides the URL or names a file), most recently heard first. What a new hub can try when no listed hub answers."""- source = os.environ.get("PYRTAN_SNAPSHOT") or DEFAULT_SNAPSHOT- try:- if source.startswith(("http://", "https://")):- status, body = http_json("GET", source, timeout=10.0)- if status != 200:- return []- else:- body = json.loads(Path(source).read_text(encoding="utf-8"))- except Exception:- return []- if not isinstance(body, dict) or not isinstance(body.get("hubs"), list):- return []- hubs = [h for h in body["hubs"] if isinstance(h, dict) and isinstance(h.get("id"), str) and h["id"].startswith(("http://", "https://"))]- hubs.sort(key=lambda h: -float(h.get("seen", 0) or 0))- return [h["id"].rstrip("/") for h in hubs]+ return hub_urls(_read_json(os.environ.get("PYRTAN_SNAPSHOT") or DEFAULT_SNAPSHOT))# ---------------------------------------------------------------- networks@@ -220,20 +237,14 @@ def parse_invite(text: str) -> tuple[Network, str | None]:def default_network() -> Network | None:- """The public network pyrtan.com publishes (``$PYRTAN_NETWORK``- overrides the URL or names a file). Its secret is public on purpose:- it names the network rather than guarding it. None when unreadable."""- source = os.environ.get("PYRTAN_NETWORK") or DEFAULT_NETWORK- try:- if source.startswith(("http://", "https://")):- status, body = http_json("GET", source, timeout=10.0)- if status != 200:- return None- else:- body = json.loads(Path(source).read_text(encoding="utf-8"))- except Exception:- return None- net = Network.from_state(body)+ """The default network: built in, so finding it asks nobody. Its secret+ is public on purpose: it names the network rather than guarding it.+ ``$PYRTAN_NETWORK`` (a URL or a file shaped like pyrtan.com's+ ``network.json``) names another; None when that cannot be read."""+ source = os.environ.get("PYRTAN_NETWORK")+ if not source:+ return Network(DEFAULT_NETWORK_NAME, DEFAULT_NETWORK_SECRET, public=True)+ net = Network.from_state(_read_json(source))if net is not None: net.public = True return net@@ -506,9 +517,20 @@ class Federation:raise ValueError(f"this hub is already in a network named {name!r}") return self.add_network(Network(name, _secrets.token_urlsafe(24), public))- def join(self, net: Network, hubs: list[str]) -> Remote | None:+ def known_hubs(self, network: str) -> list[str]:+ """The hubs this one already knows in a network (by id), to try+ before asking anyone: the ones it follows in it, then the ones its+ directory heard in it, most recently heard first."""+ now = time.time()+ with self._lock:+ followed = [r.url for r in self.following.values() if r.kind == "hub" and r.network == network and r.state == "accepted"]+ heard = sorted((e for e in self.directory_entries.values() if e.get("network") == network and e.get("kind", "hub") == "hub" and now - e["seen"] < DIRECTORY_TTL), key=lambda e: -e["seen"])+ return [u for u in dict.fromkeys(followed + [e["id"].rstrip("/") for e in heard]) if u != self.id]++ def join(self, net: Network, hubs: Iterable[str]) -> Remote | None:"""Be in ``net`` and follow the first of ``hubs`` that accepts a- follow in it. None when none does: the hub is then a network of+ follow in it (an iterable, so a caller can fetch more only when the+ first ones fail). None when none does: the hub is then a network ofone, which others can join by following it.""" net = self.add_network(net) for url in hubs: diff --git a/src/pyrtan/hub.py b/src/pyrtan/hub.py index 3cf3f75..08c41f2 100644 --- a/src/pyrtan/hub.py +++ b/src/pyrtan/hub.py@@ -46,11 +46,11 @@ import threadingfrom pathlib import Path import time from dataclasses import dataclass, field-from typing import Any+from typing import Any, Iterable, Iteratorfrom . import __version__ from .chat import Lobby, is_chat_path-from .federation import MAX_HOPS, Federation, Remote, bootstrap_candidates, default_network, first_live_hub, parse_invite, sign+from .federation import MAX_HOPS, Federation, Remote, default_network, first_live_hub, parse_invite, rendezvous_hubs, sign, snapshot_hubsfrom .grow import Unanswered, no_match_outcome from .intent import Trigger, read from .page import MANIFEST, PAGE, PYRTAN_JS, SERVICE_WORKER@@ -631,13 +631,26 @@ def _rank(peer: Local | Peer | Remote) -> int:def join_network(fed: Federation, invite: Any) -> tuple[int, dict[str, Any]]: """Join a network from an invite (``name:secret[@hub]``), or the default- network pyrtan.com publishes when ``invite`` is empty or ``default``,- and follow a hub in it. When the default network cannot be read, the- first listed hub that answers is followed outside any network, as- before networks."""+ network (built in) when ``invite`` is empty or ``default``, and follow a+ hub in it: for the default network, the hubs this one already knows in+ it first, then the hub list, then the snapshot, each fetched only when+ the ones before it did not do. When ``$PYRTAN_NETWORK`` names a default+ network that cannot be read, the first listed hub that answers is+ followed outside any network, as before networks."""+ tried: list[str] = []if invite in (None, "", "default"): net = default_network()- hubs = bootstrap_candidates()+ if net is not None:+ known = net.id++ def candidates() -> Iterator[str]:+ for source in (lambda: fed.known_hubs(known), rendezvous_hubs, snapshot_hubs):+ for url in source():+ if url not in tried and url != fed.id:+ tried.append(url)+ yield url++ hubs: Iterable[str] = candidates()if net is None: url = first_live_hub() if url is None:@@ -653,12 +666,13 @@ def join_network(fed: Federation, invite: Any) -> tuple[int, dict[str, Any]]:except ValueError as e: return 400, {"status": "error", "reason": "bad-invite", "say": str(e)} hubs = [url] if url else []+ tried.extend(h for h in hubs if h.rstrip("/") != fed.id)try: remote = fed.join(net, hubs) except ValueError as e: return 400, {"status": "error", "reason": "bad-network", "say": str(e)} net = fed.networks[net.name]- hubs = [h for h in hubs if h.rstrip("/") != fed.id]+ hubs = triedif remote is not None: say = f"in {net.name}, following {remote.url}" elif hubs: diff --git a/tests/conftest.py b/tests/conftest.py index dd5d79a..c88a12c 100644 --- a/tests/conftest.py +++ b/tests/conftest.py@@ -28,6 +28,7 @@ def runner(examples: Registry, recorded: RecordedTransport) -> Runner:@pytest.fixture(autouse=True) def _no_default_network(monkeypatch, tmp_path):- """No test reads the default network from pyrtan.com; one that wants a- default network points $PYRTAN_NETWORK at a file of its own."""+ """No test is in the built-in default network unless it says so: one+ that wants a default network points $PYRTAN_NETWORK at a file of its+ own, or unsets it for the built-in one."""monkeypatch.setenv("PYRTAN_NETWORK", str(tmp_path / "no-default-network.json")) diff --git a/tests/test_federation.py b/tests/test_federation.py index fb0211c..07b5e14 100644 --- a/tests/test_federation.py +++ b/tests/test_federation.py@@ -1027,3 +1027,53 @@ def test_the_default_network_from_pyrtan_com(two, tmp_path, monkeypatch, capsys)assert done.returncode == 0, done.stderr ids = {h["id"] for h in json.loads(out.read_text())["hubs"]} assert "http://public.example" in ids and "http://private.example" not in ids and b.url in ids+++def test_the_default_network_is_built_in_and_pyrtan_com_only_mirrors_it(monkeypatch):+ import pyrtan.federation as federation++ monkeypatch.delenv("PYRTAN_NETWORK", raising=False)+ monkeypatch.setattr(federation, "http_json", lambda *a, **k: (_ for _ in ()).throw(AssertionError("asked the network for the default network")))+ net = federation.default_network()+ assert net is not None and net.public and net.name == "pyrtan"+ mirror = json.loads((ROOT / "site" / "rendezvous" / "network.json").read_text())+ assert (net.name, net.secret) == (mirror["name"], mirror["secret"]) # the site publishes the same network, or joining would split it+++def test_the_hub_list_reads_many_sources_of_either_shape(tmp_path, monkeypatch):+ from pyrtan.federation import hub_urls, rendezvous_hubs++ listed = tmp_path / "hubs.json"+ listed.write_text(json.dumps({"hubs": ["https://a.example/", "https://b.example"]}))+ directory = tmp_path / "directory.json" # a hub's /directory, or a snapshot: newest first+ directory.write_text(json.dumps({"hubs": [{"id": "https://old.example", "seen": 1}, {"id": "https://b.example", "seen": 3}, {"id": "https://new.example", "seen": 5}]}))+ monkeypatch.delenv("PYRTAN_RENDEZVOUS", raising=False)+ monkeypatch.setenv("PYRTAN_HUB_LIST", f"{listed}, {tmp_path / 'missing.json'} {directory}")+ assert rendezvous_hubs() == ["https://a.example", "https://b.example", "https://new.example", "https://old.example"]+ assert hub_urls({"hubs": ["ftp://x", 3, None]}) == [] and hub_urls(None) == []+++def test_joining_tries_the_hubs_it_knows_before_asking_anyone(two, monkeypatch):+ import pyrtan.hub as hubmod+ from pyrtan.federation import default_network++ a, b = two+ monkeypatch.delenv("PYRTAN_NETWORK", raising=False) # the built-in default network+ default = default_network()+ assert hubmod.join_network(b.fed, "default")[1]["say"].startswith("in pyrtan") # b is in it, a network of one++ # a has heard of b in the default network, from gossip: that is enough.+ a.fed.add_network(default)+ a.fed.merge_directory([{"id": b.url, "name": "b", "trees": [], "seen": time.time()}], via="http://someone.example", network=default.id)+ assert a.fed.known_hubs(default.id) == [b.url]++ def nobody(*_):+ raise AssertionError("asked pyrtan.com although a known hub answered")++ monkeypatch.setattr(hubmod, "rendezvous_hubs", nobody)+ monkeypatch.setattr(hubmod, "snapshot_hubs", nobody)+ status, out = hubmod.join_network(a.fed, "default")+ assert status == 200 and out["say"] == f"in pyrtan, following {b.url}" and a.fed.following[b.url].network == default.id+ # Joining again (a restart with --join default) is the follow it has, and asks nobody either.+ assert hubmod.join_network(a.fed, "default")[1]["following"]["url"] == b.url+ assert a.fed.known_hubs(default.id)[0] == b.urldiff --git a/uv.lock b/uv.lock index f9dfa38..9e0a363 100644 --- a/uv.lock +++ b/uv.lock@@ -147,7 +147,7 @@ wheels = [[[package]] name = "pyrtan"-version = "0.13.0"+version = "0.14.0"source = { editable = "." } [package.optional-dependencies]