On the LAN: hubs find each other by multicast, with nobody to ask; release 0.16.0
Claude · 2026-10-28 · 97bee79bc8464a4f33212ccd6ea54cbf787c719e · parents f346bb4
Joining the default network still needed a first contact: a hub the hub already knew, or one from pyrtan.com's list or snapshot. Two boxes in one house should find each other with the internet unplugged.
A hub started --lan (pyrtan.lan) announces itself on a UDP multicast group, 239.255.70.84:8770, administratively scoped with TTL 1, so it never passes a router: its id, its name and the public networks it is in. It asks who is there on start (every hub nearby answers at once, rate-limited), announces every thirty seconds, keeps what it hears for three minutes, and shows it at GET /lan (operator). Federation.known_hubs puts hubs heard on the LAN in a network right after the ones it follows and before the directory, so `pyrtan hub --lan --join default` joins through a hub in the same house before asking pyrtan.com anything; with --join it gives the first answers a moment, and announces again once joined.
A beacon is a hint like a directory entry: anyone on the LAN can send one, nothing is routed because of it, and the follow it leads to checks /actor and is signed with the network's secret as always. It carries no secret and names only public networks. A hub announces only an id others can reach (--advertise a LAN address; never 127.0.0.1 or 0.0.0.0); without one it still asks and listens. $PYRTAN_LAN_GROUP moves the group; the tests use their own.
Docs: the federation note, README, deploy docs and compose file (Docker needs network_mode: host for multicast).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_013rD7jd9JrpLd1HKE8P1h3G
Changes
README.md | 12 ++- _meta/federation.md | 20 +++++ deploy/README.md | 7 +- deploy/openmediavault/docker-compose.yml | 3 + pyproject.toml | 2 +- src/pyrtan/__init__.py | 2 +- src/pyrtan/cli.py | 9 ++- src/pyrtan/federation.py | 11 ++- src/pyrtan/hub.py | 25 +++++- src/pyrtan/lan.py | 210 +++++++++++++++++++++++++++++++++++++++++++++++++++ tests/test_lan.py | 143 +++++++++++++++++++++++++++++++++++ uv.lock | 2 +- 12 files changed, 431 insertions(+), 15 deletions(-)
diff --git a/README.md b/README.md index 6fcbf5e..8554c10 100644 --- a/README.md +++ b/README.md@@ -162,7 +162,7 @@ hosted box is not lent out. The design is in [`_meta/io.md`](_meta/io.md).| `pyrtan run --tree NAME --input k=v` | Runs a named tree. | | `pyrtan match "…" --trees DIR` | How a sentence reads, without running anything: the rewrites, every trigger's verdict, the winner. The tool for writing triggers. | | `pyrtan serve DIR --port 8701` | Hosts the trees as a service: `/health`, `/capabilities`, `/intent`, `/run`, `/resume`, `/gaps`, `/unanswered`. Reloads trees when their files change. `--gaps FILE` keeps gaps across restarts; `--egress public` refuses private addresses. |-| `pyrtan hub --port 8700` | Routes between services. Services register with `--hub URL`; peers carry a `link` of `up`, `stale` or `down`. `--trees DIR` holds trees in the hub itself; `--federate` lets other hubs follow it and it follow them; `--listed` puts it in the directory; `--chat` keeps a lobby where people meet strangers and bots (`--no-chat-typing` turns its typing indicator off, `--chat-public` lets callers without the token in, to meet people only). |+| `pyrtan hub --port 8700` | Routes between services. Services register with `--hub URL`; peers carry a `link` of `up`, `stale` or `down`. `--trees DIR` holds trees in the hub itself; `--federate` lets other hubs follow it and it follow them; `--listed` puts it in the directory; `--lan` finds hubs on the local network by multicast, with nobody to ask; `--chat` keeps a lobby where people meet strangers and bots (`--no-chat-typing` turns its typing indicator off, `--chat-public` lets callers without the token in, to meet people only). || `pyrtan keep KEY [FILE] --at HUB` | Gives your hub a blob to keep; its keepers get copies. `kept` shows who holds what; `fetch KEY` gets a blob back from your hub or whoever holds it. | | `pyrtan follow [URL] --at HUB` | Makes your hub follow another: it then routes sentences to that hub's trees too. With no URL, `pyrtan join`. `--network NAME` follows within a network of yours; `--listed` also puts your hub in the directory. `unfollow` undoes it; `peers` lists services, hubs followed and followers; `directory` lists every hub that asked to be found, as the swarm knows it. | | `pyrtan join [INVITE] --at HUB` | Puts your hub in a network and follows a hub in it: the default network (built in), or the one an invite (`NAME:SECRET@HUB`) names. |@@ -284,7 +284,11 @@ the network once never needs pyrtan.com again; when nobody accepts, thehub is the network so far, and others join through it. `$PYRTAN_HUB_LIST` takes any number of sources (a hub list, any hub's `/directory`, a snapshot or a mirror of one), so pyrtan.com is one starting point among-any others. pyrtan.com runs no hub: it is static files on GitHub Pages,+any others. On one LAN not even that is needed: hubs started `--lan`+(with `--advertise http://<LAN address>:PORT`) announce themselves over+UDP multicast that never passes a router, and `pyrtan join` tries the+hubs heard there right after the ones it follows, so two boxes in one+house find each other with the internet unplugged. pyrtan.com runs no hub: it is static files on GitHub Pages,and publishes the default network's name and secret only as a mirror. Every hour its build asks every hub it knows of for its directory (the hub list, then everyone the last snapshot or a hub names), keeps the default@@ -651,11 +655,11 @@ site/ pyrtan.com: the landing page, the hub list, the catalog## Status-0.15.0. The core, the service, the hub, the command, the console, the+0.16.0. The core, the service, the hub, the command, the console, thecatalog (namespaces, publishing by pull request through the gate, gap reports gossiped back to authors), the language rules, federation between hubs, networks (any hub starts one; the default one is built-in), the gossiped directory, members (a browser or a phone as+in; hubs on one LAN find each other by multicast), the gossiped directory, members (a browser or a phone aspart of the swarm), the blobs they keep and the trees they run (`pyrtan.js`, conformant with the Python runtime), six forms of I/O (HTTP, a browser, files, programs, serial lines, MQTT queues), each diff --git a/_meta/federation.md b/_meta/federation.md index 7a733d4..4cad130 100644 --- a/_meta/federation.md +++ b/_meta/federation.md@@ -245,6 +245,26 @@ after the listed ones (`$PYRTAN_SNAPSHOT` overrides). A new hub cantherefore bootstrap from any hub the snapshot knows of that is up, and the only way the directory is empty is if nobody has been up for a week. Nothing of ours polls: GitHub does.+**On the LAN (v0.16).** Two boxes in one house should not need the+internet to find each other. A hub started `--lan` says it is here on a+UDP multicast group (`239.255.70.84:8770`, administratively scoped, TTL+1, so it never passes a router): its id, its name and the public+networks it is in. It says so on start, with a question every other hub+on the LAN answers at once, and every thirty seconds after; what it+hears it keeps for three minutes (`GET /lan`, operator). `pyrtan join`+then tries the hubs heard on the LAN in that network right after the+ones it already follows, before the directory, the hub list and the+snapshot (`known_hubs`), so a hub started `--lan --join default` on a+LAN with another such hub is in the network with it, with nothing on+the internet asked. A beacon is a hint like a directory entry: anyone on+the LAN can send one, nothing is routed because of it, and the follow it+leads to is checked and signed like any other. It carries no secret and+names only public networks. A hub announces only an id others can reach+(`--advertise http://<LAN address>:PORT`; never `127.0.0.1` or+`0.0.0.0`); without one it still asks and listens, and can join a hub+it hears. In Docker, multicast needs `network_mode: host`. Standard+library only: `src/pyrtan/lan.py`.+**What an entry is not.** A hub two follows away cannot verify who made an entry: there are no public keys in the standard library, and HMAC proves things only between two hubs that share a secret. So an entry is diff --git a/deploy/README.md b/deploy/README.md index 812a8cf..ce17c71 100644 --- a/deploy/README.md +++ b/deploy/README.md@@ -74,7 +74,12 @@ the default network (built in), follows the first hub in it thataccepts (one it already knows, else from pyrtan.com's hub list, then its snapshot), routes to it, and is listed in the directory every hub gossips; `pyrtan-follow URL --at <your hub> --token …` follows any other. For a network+follow URL --at <your hub> --token …` follows any other. With other hubs+in the same house (a second NAS, a laptop running `pyrtan hub`), set+`PYRTAN_LAN: "true"` too, with `network_mode: host` on the hub service+and `PYRTAN_ADVERTISE` at the box's LAN address: the hubs find each+other by multicast, and join through each other before asking+pyrtan.com anything. For a networkof your own, `pyrtan network new home --at <your hub> --token …` prints an invite; a friend's hub joins with `pyrtan join <invite>`, or by adding it to its `PYRTAN_JOIN`. Following is one-directional trust: your diff --git a/deploy/openmediavault/docker-compose.yml b/deploy/openmediavault/docker-compose.yml index 148b87b..007a065 100644 --- a/deploy/openmediavault/docker-compose.yml +++ b/deploy/openmediavault/docker-compose.yml@@ -38,6 +38,9 @@ x-pyrtan: &pyrtan# default network (built into pyrtan), follows a hub in it, and lists # itself in the directory. Needs the token, and PYRTAN_EGRESS: public # above: hubs that follow this one may ask it to run trees.+ # PYRTAN_LAN: "true" finds the other hubs in the house by multicast,+ # with nobody to ask; it needs `network_mode: host` on the hub and+ # PYRTAN_ADVERTISE set to the box's LAN address (http://<nas-ip>:8700).# PYRTAN_JOIN may also hold invites to networks of your own # (NAME:SECRET@HUB, from `pyrtan network new`), space-separated. # PYRTAN_ADVERTISE: "https://hub.example.net" diff --git a/pyproject.toml b/pyproject.toml index e17be35..2851c24 100644 --- a/pyproject.toml +++ b/pyproject.toml@@ -1,6 +1,6 @@[project] name = "pyrtan"-version = "0.15.0"+version = "0.16.0"description = "Open-source total automation: a core set of expandable services that act on all forms of I/O through inspectable decision trees." readme = "README.md" license = "AGPL-3.0-or-later" diff --git a/src/pyrtan/__init__.py b/src/pyrtan/__init__.py index 3461609..a73334b 100644 --- a/src/pyrtan/__init__.py +++ b/src/pyrtan/__init__.py@@ -33,4 +33,4 @@ __all__ = ["Response", ]-__version__ = "0.15.0"+__version__ = "0.16.0"diff --git a/src/pyrtan/cli.py b/src/pyrtan/cli.py index 5eeeb77..41adb11 100644 --- a/src/pyrtan/cli.py +++ b/src/pyrtan/cli.py@@ -12,6 +12,7 @@pyrtan hub --trees DIR and hold trees itself: a swarm in one process pyrtan hub --federate --advertise URL [--follow URL ...] [--listed] and between hubs pyrtan hub --chat and keep a lobby: strangers and bots, Omegle-style+ pyrtan hub --lan --join default and find the hubs on this LAN by multicast, with nobody to askpyrtan follow [URL] --at HUB [--listed] follow another hub (and route to it); default: join the default network pyrtan unfollow URL --at HUB pyrtan peers --at HUB services, and hubs followed and following@@ -178,6 +179,7 @@ def main(argv: list[str] | None = None) -> int:p.add_argument("--chat", action="store_true", default=os.environ.get("PYRTAN_CHAT", "").lower() in ("1", "true", "yes"), help="keep a lobby at /chat where people meet a random stranger here or at a hub this one follows, or a bot made of the trees it routes to (default $PYRTAN_CHAT)") p.add_argument("--allow", action="append", default=[a for a in os.environ.get("PYRTAN_ALLOW", "").split() if a], metavar="WHO=ABILITIES", help="grant an identity (pyr:… or name@domain, resolved and pinned now) abilities here: ask, chat, run, operate (repeatable; default $PYRTAN_ALLOW, space-separated)") p.add_argument("--grants", metavar="FILE", default=os.environ.get("PYRTAN_GRANTS"), help="keep the grants in this file (default $PYRTAN_GRANTS, else grants.json beside --state, else in memory)")+ p.add_argument("--lan", action="store_true", default=os.environ.get("PYRTAN_LAN", "").lower() in ("1", "true", "yes"), help="find hubs on this local network with no one to ask: announce this hub over UDP multicast (TTL 1) and try the hubs heard there when joining a network; needs --advertise with a LAN address (default $PYRTAN_LAN)")p.add_argument("--chat-public", action="store_true", default=os.environ.get("PYRTAN_CHAT_PUBLIC", "").lower() in ("1", "true", "yes"), help="open the lobby to callers without the token (implies --chat): they meet people only, never a bot, and everything else still needs the token (default $PYRTAN_CHAT_PUBLIC)") p.add_argument("--chat-typing", action=argparse.BooleanOptionalAction, default=os.environ.get("PYRTAN_CHAT_TYPING", "").lower() not in ("0", "false", "no", "off"), help="the lobby's typing indicator: \"Stranger is typing…\" for anyone who keeps theirs on (default on; $PYRTAN_CHAT_TYPING=0 turns it off)") p.add_argument("--report-gaps", action="store_true", default=os.environ.get("PYRTAN_REPORT_GAPS", "").lower() in ("1", "true", "yes"), help="put the gaps this swarm's catalog trees meet (tree id, node, value; never inputs) into its announces, so they reach the tree's author through the federation and pyrtan.com (default $PYRTAN_REPORT_GAPS)")@@ -888,10 +890,10 @@ def cmd_hub(args: argparse.Namespace) -> int:try: hub, server = serve_hub( args.host, args.port, name=args.name, peers=args.peer, poll_every=args.poll, quiet=args.quiet, token=args.token, cors=args.cors,- federate=args.federate or bool(args.advertise) or bool(follow) or bool(join) or args.listed, advertise=args.advertise, federation_secret=args.federation_secret,+ federate=args.federate or bool(args.advertise) or bool(follow) or bool(join) or args.listed or args.lan, advertise=args.advertise, federation_secret=args.federation_secret,state=args.state, open_follows=not args.closed, follow=follow, join=join, listed=args.listed, trees=[Path(t) for t in args.trees], handout=[Path(t) for t in args.handout], env=_kv(args.env), gaps=args.gaps, egress=args.egress,- store=args.store, replicas=args.replicas, report_gaps=args.report_gaps, unanswered=args.unanswered, watch=args.watch, chat=args.chat or args.chat_public, chat_typing=args.chat_typing, chat_public=args.chat_public,+ store=args.store, replicas=args.replicas, report_gaps=args.report_gaps, unanswered=args.unanswered, watch=args.watch, chat=args.chat or args.chat_public, chat_typing=args.chat_typing, chat_public=args.chat_public, lan=args.lan,allow=args.allow, grants=args.grants, ) except IdentityError as e:@@ -905,6 +907,9 @@ def cmd_hub(args: argparse.Namespace) -> int:local += f", handing out {len(hub.handout_registry.trees())} tree(s) to trusted members" if hub.lobby is not None: local += ", chatting at /chat" + (" (open to callers without the token, people only)" if hub.chat_public and args.token else "")+ if hub.lan is not None:+ lan = hub.lan.view()+ local += f", on the LAN ({lan['group']}): " + (f"cannot join it: {hub.lan.error}" if hub.lan.error else f"heard {len(lan['hubs'])} hub(s)" + ("" if lan["announcing"] else "; listening only, since this hub's id is not a LAN address (give --advertise http://<its LAN address>:PORT)"))if len(hub.grants): local += f", granting {len(hub.grants)} identit{'y' if len(hub.grants) == 1 else 'ies'}" print(f"pyrtan hub at {server.url}: {len(hub.peers)} peer(s){local}{fed}" + _access_note(args), flush=True) diff --git a/src/pyrtan/federation.py b/src/pyrtan/federation.py index 3ace9f0..1f1bf74 100644 --- a/src/pyrtan/federation.py +++ b/src/pyrtan/federation.py@@ -71,7 +71,9 @@ has heard in it, most recent first. Only when none of those accepts doesit read the hub list and then the snapshot, and only as far as it needs. So a hub that has been in the network once never needs pyrtan.com again, and any hub's public ``/directory`` or any mirror of the list will do as-a starting point (``$PYRTAN_HUB_LIST``).+a starting point (``$PYRTAN_HUB_LIST``). A hub started ``--lan`` also+hears the hubs on its own network say they are there (:mod:`pyrtan.lan`)+and tries those right after the ones it follows: no internet needed.A follow outside any network (and every member's) works as before. **Chat.** A hub with a lobby (``--chat``) asks the hubs it follows for a@@ -417,6 +419,7 @@ class Federation:self._handed: dict[str, dict[str, dict[str, Any]]] = {} # member -> key -> {sha256, at, confirmed} self._asks: dict[str, dict[str, Any]] = {} # run id -> {member, event, answer} self.chat: Any = None # the hub's lobby, when it keeps one (pyrtan.chat.Lobby)+ self.lan: Any = None # what the hub hears on the LAN, when started --lan (pyrtan.lan.Lan)self.ask_timeout = ASK_TIMEOUT self.following: dict[str, Remote] = {} self.followers: dict[str, Remote] = {}@@ -527,13 +530,15 @@ class Federation:def known_hubs(self, network: str) -> list[str]: """The hubs this one already knows in a network (by id), to try- before asking anyone: the ones it follows in it, then the ones its+ before asking anyone: the ones it follows in it, then the ones it+ heard on the LAN say they are in it (``--lan``), then the ones itsdirectory heard in it, most recently heard first.""" now = time.time() with self._lock: followed = [r.url for r in self.following.values() if r.kind == "hub" and r.network == network and r.state == "accepted"] heard = sorted((e for e in self.directory_entries.values() if e.get("network") == network and e.get("kind", "hub") == "hub" and now - e["seen"] < DIRECTORY_TTL), key=lambda e: -e["seen"])- return [u for u in dict.fromkeys(followed + [e["id"].rstrip("/") for e in heard]) if u != self.id]+ nearby = self.lan.hubs(network) if self.lan is not None else []+ return [u for u in dict.fromkeys(followed + nearby + [e["id"].rstrip("/") for e in heard]) if u != self.id]def join(self, net: Network, hubs: Iterable[str]) -> Remote | None: """Be in ``net`` and follow the first of ``hubs`` that accepts a diff --git a/src/pyrtan/hub.py b/src/pyrtan/hub.py index f20dc71..dc38247 100644 --- a/src/pyrtan/hub.py +++ b/src/pyrtan/hub.py@@ -63,6 +63,7 @@ from typing import Any, Iterable, Iteratorfrom . import __version__ from .chat import Lobby, is_chat_path+from .lan import Lanfrom .federation import MAX_HOPS, Federation, Remote, default_network, first_live_hub, parse_invite, rendezvous_hubs, sign, snapshot_hubs from . import identity from .grants import Grants, ability_for, parse_may@@ -150,6 +151,7 @@ class Hub:self._audience = audience self.lobby = Lobby(self, typing=chat_typing) if chat or chat_public else None self.chat_public = chat_public # callers without the token may chat, with people only+ self.lan: Lan | None = None # hubs heard on the local network (--lan)self.handout_registry = handout # trees for trusted members only: this hub hands them out and does not run them self.poll_every = poll_every self.token = token or None@@ -309,8 +311,18 @@ class Hub:self._thread = threading.Thread(target=loop, daemon=True) self._thread.start()+ def on_lan(self) -> Lan:+ """Announce this hub on the LAN and listen for others (``--lan``)."""+ fed = self.federation+ assert fed is not None, "a hub is federated before it goes on the LAN"+ self.lan = Lan(lambda: {"id": fed.id, "name": self.name, "networks": sorted(n.id for n in fed.networks.values() if n.public)}).start()+ fed.lan = self.lan+ return self.lan+def stop(self) -> None: self._stop.set()+ if self.lan is not None:+ self.lan.stop()if self.local is not None: self.local.service.stop()@@ -576,6 +588,8 @@ class Hub:if fed is not None: if method == "GET" and path == "/actor": return 200, fed.actor()+ if method == "GET" and path == "/lan":+ return (200, self.lan.view()) if self.lan is not None else (404, {"status": "error", "reason": "no-lan", "say": "This hub is not on the LAN; start it with --lan."})if method == "GET" and path == "/directory": # A private network's part only to the operator (or to anyone, on a hub with no token). return 200, fed.directory(private=not self.token or operator)@@ -696,7 +710,7 @@ class Hub:if self.lobby is not None: endpoints += ["/chat", "/chat/start", "/chat/poll", "/chat/say", "/chat/typing", "/chat/settings", "/chat/next", "/chat/leave"] if fed is not None:- endpoints += ["/actor", "/inbox", "/mailbox", "/directory", "/reports", "/follow", "/unfollow", "/networks", "/join", "/leave", "/listed", "/following", "/followers", "/outbox", "/keep", "/fetch", "/kept/{key}"]+ endpoints += ["/actor", "/inbox", "/mailbox", "/directory", "/lan", "/reports", "/follow", "/unfollow", "/networks", "/join", "/leave", "/listed", "/following", "/followers", "/outbox", "/keep", "/fetch", "/kept/{key}"]return 200, {"pyrtan": "hub", "name": self.name, "version": __version__, "auth": "bearer" if self.token else "none", "federation": fed.id if fed else None, "chat": self.lobby is not None, "chat_public": self.lobby is not None and self.chat_public, "audience": self.audience, "grants": len(self.grants), "chat_bots": self.lobby is not None and self._trusted(headers, may), "endpoints": endpoints} if method == "POST" and path == "/register": url = body.get("url")@@ -800,6 +814,7 @@ def serve_hub(report_gaps: bool = False, unanswered: str | Path | None = None, watch: bool = False,+ lan: bool = False,chat: bool = False, allow: list[str] | None = None, grants: str | None = None,@@ -821,7 +836,7 @@ def serve_hub(hub = Hub(name=name, poll_every=poll_every, token=token, local=local, handout=Registry.from_paths(handout) if handout else None, unanswered=unanswered, chat=chat, chat_typing=chat_typing, chat_public=chat_public, grants=granted) server = Server(hub.route, host, port, quiet, token=token, cors=cors, federated=federate or bool(advertise), public=is_chat_path if chat_public else None, keyed=True).start() hub._audience = (advertise or server.url).rstrip("/")- if federate or advertise or join:+ if federate or advertise or join or lan:hub.federate(advertise or server.url, secret=federation_secret, state=state, open_follows=open_follows, listed=listed, store=store or (str(Path(state).with_name("store")) if state else None), replicas=replicas, report_gaps=report_gaps) for url in peers or []: hub.add_peer(url)@@ -830,8 +845,14 @@ def serve_hub(hub.federation.follow(url) # type: ignore[union-attr] except ValueError as e: print(f"pyrtan hub: cannot follow {url}: {e}", flush=True)+ if lan:+ hub.on_lan()+ if join and hub.lan is not None and hub.lan.error is None:+ hub.lan.wait(1.5) # the hubs nearby answer the first question in that time: join one of themfor invite in join or []: _, answer = join_network(hub.federation, invite) # type: ignore[arg-type] print(f"pyrtan hub: join {answer.get('network') or invite.split(':', 1)[0]}: {answer.get('say')}", flush=True)+ if join and hub.lan is not None:+ hub.lan.send() # now in its networks: say so at once, not at the next beaconhub.start_polling() return hub, server diff --git a/src/pyrtan/lan.py b/src/pyrtan/lan.py new file mode 100644 index 0000000..c1a4db7 --- /dev/null +++ b/src/pyrtan/lan.py@@ -0,0 +1,210 @@+"""The LAN: hubs on the same network find each other, with nobody to ask.++A hub started ``--lan`` says it is here, on a UDP multicast group that+never leaves the local network (TTL 1): its federation id (the URL other+hubs reach it at), its name, and the public networks it is in. It says+so when it starts, with a question that makes every other hub on the LAN+answer at once, and again every thirty seconds. It keeps what it hears+for three minutes. Joining a network then tries the hubs it heard on the+LAN in that network right after the ones it already follows, before its+directory, the hub list or the snapshot: two boxes in one house find+each other with the internet unplugged.++A beacon is a hint, exactly as a directory entry is: anyone on the LAN+can send one, and nothing is routed to a hub because it said it was+there. Following it is still the one act of trust, and the follow checks+the hub's ``/actor`` and is signed with the network's secret like any+other. Beacons carry no secret and no private network: only public+network ids, so a private network's hubs are not named to the LAN.++A hub announces itself only when its id is an address others can reach:+not ``127.0.0.1``, ``localhost`` or ``0.0.0.0``. Give it one with+``--advertise http://<its LAN address>:8700``. In Docker, multicast+needs ``network_mode: host``.++Standard library only. ``$PYRTAN_LAN_GROUP`` (``ADDRESS:PORT``) moves the+group, which is how the tests keep to themselves.+"""++from __future__ import annotations++import json+import os+import socket+import struct+import threading+import time+from typing import Any, Callable+from urllib.parse import urlsplit++GROUP = "239.255.70.84" # administratively scoped: stays on the site+PORT = 8770+EVERY = 30.0 # seconds between beacons+HEARD_FOR = 180.0 # a hub not heard for this long is forgotten+HEARD_MAX = 64 # hubs kept at most+ANSWER_EVERY = 2.0 # a question is answered at most this often+BEACON_MAX = 2048 # bytes+UNREACHABLE = {"127.0.0.1", "localhost", "0.0.0.0", "::1", "::"}+++def group() -> tuple[str, int]:+ """The multicast group, from ``$PYRTAN_LAN_GROUP`` or the default."""+ text = os.environ.get("PYRTAN_LAN_GROUP", "")+ host, _, port = text.rpartition(":")+ try:+ return (host, int(port)) if host and port else (GROUP, PORT)+ except ValueError:+ return GROUP, PORT+++def reachable(url: str) -> bool:+ """Whether another box could reach a hub at ``url``."""+ try:+ parts = urlsplit(url)+ except ValueError:+ return False+ return parts.scheme in ("http", "https") and bool(parts.hostname) and parts.hostname not in UNREACHABLE and not parts.hostname.startswith("127.")+++class Lan:+ """Announce this hub on the LAN and keep what other hubs announce.++ ``me`` returns what to say (``{"id", "name", "networks"}``), or None+ to stay quiet; it is asked each time, so a hub that joins a network+ says so in its next beacon."""++ def __init__(self, me: Callable[[], dict[str, Any] | None], *, every: float = EVERY, address: tuple[str, int] | None = None) -> None:+ self.me = me+ self.every = every+ self.address = address or group()+ self.heard: dict[str, dict[str, Any]] = {}+ self.error: str | None = None+ self._answered = 0.0+ self._lock = threading.Lock()+ self._stop = threading.Event()+ self._sock: socket.socket | None = None+ self._threads: list[threading.Thread] = []++ # ------------------------------------------------------------ what is said++ def beacon(self, *, query: bool = False) -> bytes | None:+ """What this hub says; None when it has nothing to say. A hub that+ cannot be reached still asks who is there: a question with no id."""+ me = self.me()+ if not me or not reachable(str(me.get("id", ""))):+ return json.dumps({"pyrtan-lan": 1, "query": True}).encode("utf-8") if query else None+ body = {"pyrtan-lan": 1, "id": str(me["id"]).rstrip("/"), "name": str(me.get("name", ""))[:80], "networks": [n for n in me.get("networks", []) if isinstance(n, str)][:16]}+ if query:+ body["query"] = True+ data = json.dumps(body, separators=(",", ":")).encode("utf-8")+ return data if len(data) <= BEACON_MAX else None++ def take(self, data: bytes, sender: str, now: float | None = None) -> bool | None:+ """A datagram heard from ``sender``: True when it named a hub (and+ False when it named this one); None when it was not a beacon. A+ question in it is answered, at most every couple of seconds."""+ now = time.time() if now is None else now+ try:+ body = json.loads(data[:BEACON_MAX].decode("utf-8"))+ except (UnicodeDecodeError, json.JSONDecodeError):+ return None+ if not isinstance(body, dict) or body.get("pyrtan-lan") != 1:+ return None+ me = self.me() or {}+ hub = body["id"].rstrip("/") if isinstance(body.get("id"), str) else None+ if hub == str(me.get("id", "")).rstrip("/"):+ return False+ if body.get("query") is True and now - self._answered >= ANSWER_EVERY:+ self._answered = now+ self.send()+ if hub is None or not reachable(hub) or len(hub) > 200:+ return None+ networks = [n for n in body.get("networks", []) if isinstance(n, str) and len(n) <= 64][:16] if isinstance(body.get("networks"), list) else []+ with self._lock:+ self.heard[hub] = {"id": hub, "name": str(body.get("name", ""))[:80], "networks": networks, "from": sender, "seen": now}+ if len(self.heard) > HEARD_MAX:+ for old in sorted(self.heard.values(), key=lambda e: e["seen"])[: len(self.heard) - HEARD_MAX]:+ self.heard.pop(old["id"], None)+ return True++ def hubs(self, network: str | None = None, now: float | None = None) -> list[str]:+ """The hubs heard lately, most recent first; ``network`` keeps the+ ones that said they are in it."""+ now = time.time() if now is None else now+ with self._lock:+ fresh = [e for e in self.heard.values() if now - e["seen"] < HEARD_FOR and (network is None or network in e["networks"])]+ return [e["id"] for e in sorted(fresh, key=lambda e: -e["seen"])]++ def view(self) -> dict[str, Any]:+ now = time.time()+ with self._lock:+ rows = [{**e, "age_s": int(now - e["seen"])} for e in self.heard.values() if now - e["seen"] < HEARD_FOR]+ me = self.me() or {}+ return {"group": f"{self.address[0]}:{self.address[1]}", "announcing": self.beacon() is not None, "id": me.get("id"), "error": self.error, "hubs": sorted(rows, key=lambda e: e["age_s"])}++ # ------------------------------------------------------------ the socket++ def send(self, *, query: bool = False) -> bool:+ """Say this hub is here (``query``: and ask who else is)."""+ data = self.beacon(query=query)+ if data is None or self._sock is None:+ return False+ try:+ self._sock.sendto(data, self.address)+ return True+ except OSError as e:+ self.error = str(e)+ return False++ def start(self) -> "Lan":+ """Join the group, ask who is there, and keep listening and+ announcing in the background. A LAN that will not have it (no+ multicast route, the port refused) leaves ``error`` set and the+ hub running without it."""+ host, port = self.address+ try:+ s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, socket.IPPROTO_UDP)+ s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)+ if hasattr(socket, "SO_REUSEPORT"):+ s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEPORT, 1) # several hubs on one box share the port+ s.bind(("", port))+ s.setsockopt(socket.IPPROTO_IP, socket.IP_ADD_MEMBERSHIP, struct.pack("4sl", socket.inet_aton(host), socket.INADDR_ANY))+ s.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_TTL, 1) # never past the first router+ s.setsockopt(socket.IPPROTO_IP, socket.IP_MULTICAST_LOOP, 1) # hubs on this same box hear it too+ s.settimeout(1.0)+ except OSError as e:+ self.error = f"cannot join {host}:{port}: {e}"+ return self+ self._sock = s+ self._threads = [threading.Thread(target=self._listen, daemon=True), threading.Thread(target=self._announce, daemon=True)]+ for t in self._threads:+ t.start()+ return self++ def wait(self, seconds: float) -> None:+ """Give the answers to the first question a moment to arrive."""+ self._stop.wait(seconds)++ def _listen(self) -> None:+ while not self._stop.is_set() and self._sock is not None:+ try:+ data, (sender, _) = self._sock.recvfrom(BEACON_MAX + 1)+ except socket.timeout:+ continue+ except OSError:+ return+ self.take(data, sender)++ def _announce(self) -> None:+ self.send(query=True)+ while not self._stop.wait(self.every):+ self.send()++ def stop(self) -> None:+ self._stop.set()+ if self._sock is not None:+ try:+ self._sock.close()+ except OSError:+ pass+ self._sock = Nonediff --git a/tests/test_lan.py b/tests/test_lan.py new file mode 100644 index 0000000..4edd8b0 --- /dev/null +++ b/tests/test_lan.py@@ -0,0 +1,143 @@+"""Hubs on one LAN find each other: beacons, the multicast socket, and+joining a network through a hub heard nearby."""++import json+import random+import time++import pytest++from pyrtan import lan as lanmod+from pyrtan.lan import HEARD_FOR, HEARD_MAX, Lan, reachable+from pyrtan.service import http_json+++def beacon(hub, networks=(), query=False, name="x"):+ body = {"pyrtan-lan": 1, "id": hub, "name": name, "networks": list(networks)}+ if query:+ body["query"] = True+ return json.dumps(body).encode()+++@pytest.fixture+def group(monkeypatch):+ """A group of the tests' own, so they never hear a real hub, nor each other."""+ port = random.randint(20000, 60000)+ monkeypatch.setenv("PYRTAN_LAN_GROUP", f"239.255.70.84:{port}")+ return ("239.255.70.84", port)+++def test_who_counts_as_reachable():+ assert reachable("http://192.168.1.20:8700") and reachable("https://nas.local")+ for url in ("http://127.0.0.1:8700", "http://localhost:8700", "http://0.0.0.0:8700", "http://127.0.1.1", "ftp://nas.local", "nas.local", ""):+ assert not reachable(url), url+++def test_beacons_are_heard_kept_and_forgotten():+ me = {"id": "http://192.168.1.10:8700", "name": "home", "networks": ["n1"]}+ lan = Lan(lambda: me)+ sent = []+ lan.send = lambda query=False: sent.append(query) or True+ now = 1000.0+ assert lan.take(beacon("http://192.168.1.20:8700/", ["n1"], name="nas"), "192.168.1.20", now) is True+ assert lan.take(beacon("http://192.168.1.30:8700", ["n2"]), "192.168.1.30", now + 1) is True+ assert lan.take(beacon(me["id"], ["n1"]), "192.168.1.10", now) is False # its own voice+ for junk in (b"not json", b"[]", json.dumps({"pyrtan-lan": 2, "id": "http://192.168.1.40"}).encode(), beacon("http://127.0.0.1:8700"), beacon("x" * 300)):+ assert lan.take(junk, "192.168.1.40", now) is None+ assert lan.hubs(now=now + 2) == ["http://192.168.1.30:8700", "http://192.168.1.20:8700"] # most recent first+ assert lan.hubs("n1", now=now + 2) == ["http://192.168.1.20:8700"]+ assert lan.heard["http://192.168.1.20:8700"]["name"] == "nas" and lan.heard["http://192.168.1.20:8700"]["from"] == "192.168.1.20"+ assert lan.hubs(now=now + HEARD_FOR + 5) == [] # not heard for three minutes: gone+ # A question is answered, but not more than every couple of seconds.+ assert sent == []+ lan.take(beacon("http://192.168.1.50:8700", query=True), "192.168.1.50", now + 10)+ lan.take(beacon("http://192.168.1.51:8700", query=True), "192.168.1.51", now + 10.5)+ lan.take(json.dumps({"pyrtan-lan": 1, "query": True}).encode(), "192.168.1.52", now + 13) # a hub that cannot be reached asks too+ assert sent == [False, False]+ # Kept to a bound.+ for i in range(HEARD_MAX + 10):+ lan.take(beacon(f"http://10.0.0.{i + 1}:8700"), f"10.0.0.{i + 1}", now + 20 + i)+ assert len(lan.heard) == HEARD_MAX and "http://10.0.0.1:8700" not in lan.heard+++def test_what_a_hub_says_and_when_it_stays_quiet():+ said = {"id": "http://192.168.1.10:8700/", "name": "home", "networks": ["n1", 7]}+ lan = Lan(lambda: said)+ body = json.loads(lan.beacon())+ assert body == {"pyrtan-lan": 1, "id": "http://192.168.1.10:8700", "name": "home", "networks": ["n1"]}+ assert json.loads(lan.beacon(query=True))["query"] is True+ said["id"] = "http://127.0.0.1:8700" # nobody else could reach it: it only asks+ assert lan.beacon() is None and json.loads(lan.beacon(query=True)) == {"pyrtan-lan": 1, "query": True}+ assert Lan(lambda: None).beacon() is None+++def test_two_hubs_hear_each_other_over_multicast(group):+ a = Lan(lambda: {"id": "http://192.0.2.101:8700", "name": "a", "networks": ["n1"]}, address=group)+ b = Lan(lambda: {"id": "http://192.0.2.102:8700", "name": "b", "networks": []}, address=group)+ try:+ a.start()+ if a.error:+ pytest.skip(f"no multicast here: {a.error}")+ b.start() # b asks who is there; a answers at once, and hears b's question too+ deadline = time.time() + 5+ while time.time() < deadline and not (a.hubs() and b.hubs()):+ time.sleep(0.05)+ assert b.hubs("n1") == ["http://192.0.2.101:8700"] and a.hubs() == ["http://192.0.2.102:8700"]+ assert a.view()["announcing"] is True and a.view()["hubs"][0]["name"] == "b"+ finally:+ a.stop()+ b.stop()+++def test_joining_takes_a_hub_heard_on_the_lan_before_anyone_else(monkeypatch, tmp_path):+ import pyrtan.hub as hubmod+ from pyrtan.federation import default_network+ from pyrtan.hub import Hub+ from pyrtan.service import Server++ monkeypatch.delenv("PYRTAN_NETWORK", raising=False) # the built-in default network+ hubs = []+ for name in ("a", "b"):+ hub = Hub(name=name, poll_every=60)+ srv = Server(hub.route, quiet=True, federated=True).start()+ hub.federate(srv.url)+ hubs.append((hub, srv))+ (a, a_srv), (b, b_srv) = hubs+ try:+ default = default_network()+ hubmod.join_network(b.federation, "default") # b is the network so far+ # a heard b on the LAN say it is in the default network (as a socket would hand it over).+ a.lan = Lan(lambda: {"id": a.federation.id, "name": "a", "networks": []})+ a.federation.lan = a.lan+ a.lan.heard[b.federation.id] = {"id": b.federation.id, "name": "b", "networks": [default.id], "from": "192.0.2.2", "seen": time.time()}+ a.federation.directory_entries["http://gossiped.example"] = {"id": "http://gossiped.example", "name": "g", "trees": [], "seen": time.time(), "network": default.id}+ assert a.federation.known_hubs(default.id) == [b.federation.id, "http://gossiped.example"] # the LAN before the directory++ def nobody(*_):+ raise AssertionError("asked beyond the LAN although a hub there answered")++ monkeypatch.setattr(hubmod, "rendezvous_hubs", nobody)+ monkeypatch.setattr(hubmod, "snapshot_hubs", nobody)+ status, out = hubmod.join_network(a.federation, "default")+ assert status == 200 and out["say"] == f"in pyrtan, following {b.federation.id}"+ finally:+ for hub, srv in hubs:+ hub.stop()+ srv.stop()+++def test_a_hub_on_the_lan_without_a_lan_address_listens_only(group, monkeypatch, tmp_path):+ from pyrtan.hub import serve_hub++ monkeypatch.setenv("PYRTAN_HUB_LIST", str(tmp_path / "none.json"))+ monkeypatch.setenv("PYRTAN_SNAPSHOT", str(tmp_path / "none.json"))+ hub, srv = serve_hub(port=0, quiet=True, name="solo", lan=True, token="t")+ try:+ assert hub.lan is not None and hub.federation is not None and hub.federation.lan is hub.lan+ status, view = http_json("GET", f"{srv.url}/lan", token="t")+ assert status == 200 and view["announcing"] is False and view["group"] == f"{group[0]}:{group[1]}" # 127.0.0.1 is nobody else's address+ assert http_json("GET", f"{srv.url}/lan")[0] == 401 # what the LAN said is the operator's+ finally:+ hub.stop()+ srv.stop()+ assert lanmod.group() == groupdiff --git a/uv.lock b/uv.lock index 97d82d2..9bf2632 100644 --- a/uv.lock +++ b/uv.lock@@ -304,7 +304,7 @@ wheels = [[[package]] name = "pyrtan"-version = "0.15.0"+version = "0.16.0"source = { editable = "." } [package.optional-dependencies]