Sovereign by Construction

Abstract

A voice assistant is, physically, a set of microphones and cameras with a network connection. In the dominant commercial architecture, the intelligence that interprets those signals runs in a datacenter, so the signals must leave the house. Vendors govern what happens next through policy: retention rules, training opt-outs, deletion requests. Policy can change. Architecture cannot change retroactively. Our position is that privacy should be a property of the architecture, not a clause in a contract.

This paper describes the architecture Elysium uses to make that position concrete. The system exposes three deployment placements (Cloud, Hybrid, Sovereign) as presets over a single per-stage placement policy. Two invariants hold in every placement: device control is executed locally, and raw video does not leave the home. In the Sovereign placement the complete product, including language-model inference, identity, and memory, runs on one machine on the home network, and the system fails closed: an error path may degrade capability, but it may not silently reroute data to a cloud service. Because claims of this kind are easy to make and hard to evaluate, the design treats verifiability as a requirement. The owner can run an egress test with the network cable physically disconnected, and the update mechanism applies cryptographically signed releases without ever initiating contact with the vendor. We describe each mechanism, report what a capable local configuration costs in hardware terms, and close with an explicit account of what is operational today and what is still in development.

1. Two ways to promise privacy

Privacy assurances in consumer smart-home systems come in two forms. The first is contractual: the vendor states what it collects, how long it retains it, and who may access it. The assurance is real but conditional. It depends on the vendor's current incentives, its security posture, the jurisdictions it operates in, and every future revision of its terms. Once data has crossed the home boundary, the resident's protection is whatever the policy says this quarter.

The second form is architectural: the data never crosses the boundary at all. Nothing needs to be retained correctly, deleted correctly, or subpoenaed narrowly, because it does not exist outside the house. The two forms differ in kind, not in degree. A policy is a promise about behavior; an architecture is a constraint on what behavior is possible.

The architectural form has historically carried a cost. Local processing meant weak processing: pattern-matched commands, no genuine language understanding, no reasoning. The recent availability of open-weight language models in the 10 to 30 billion parameter range, together with quantization methods that fit them into consumer GPU memory, changes that calculus. Section 5 quantifies it.

Anyone can claim "local." A product people trust lets them verify it.

2. One pipeline, three placements

Elysium runs the same product in three placements. They are not separate builds. They are presets over one pipeline in which each processing stage (speech recognition, language-model reasoning, vision, memory, identity) carries its own placement decision.

Three considerations determine where a stage runs, and keeping them separate is what keeps the behavior honest. The first is availability: what the installed hardware can actually serve. A model is offered only when its serving backend is reachable, so the system can never be configured into a placement that cannot exist. The second is the owner's policy: the chosen preset, plus any per-stage overrides. The third is a small set of invariants that sit above both and are not owner-configurable. Section 3 states them.

Switching placement is a runtime operation, performed from the dashboard rather than by reinstallation. The transition from Sovereign to Hybrid is deliberately the heaviest one. It is the only transition that opens a path to the outside, so it requires the local password to be re-entered and records the owner's consent to the specific categories of data that will begin to cross. On appliance units the toggle ships locked; a household opts into cloud reasoning, never out of it.

3. Invariants at the boundary

Two rules hold in every placement, including the thinnest cloud configuration.

1. Device control is local. Commands to lights, locks, and thermostats execute over local radio and LAN protocols. An internet outage may remove a reasoning capability; it does not remove control of the house.
2. Raw video does not leave the home. Camera frames are processed, if at all, on the hub. What may cross the boundary in the Hybrid placement is derived data: text and structured context, never the frame itself.

A third rule is scoped by placement. In Hybrid and Sovereign, microphone audio is reduced to text on the hub, so no audio recording crosses the boundary in either of the placements built around local perception. The thin Cloud preset is the exception: it may route speech to a remote recognizer, and it is documented as doing so.

Figure 1 summarizes what actually crosses in each architecture, compared with the conventional design.

4. Hybrid, stated precisely

Most households will run Hybrid, because it combines local perception with the strongest available reasoning. It is also the placement that most needs precise language, since it crosses the boundary by design.

In a Hybrid turn, the following derived data may leave the house: the message text or speech transcript, retrieved memory relevant to the request, summaries that carry context across conversations, the current state of devices, the intermediate context of tool calls, and the household's stated preferences. The following do not leave: camera frames, audio recordings, and the credential material that controls the home. The system maintains this list as a disclosure the owner can read on the hub itself, category by category, rather than as a blanket statement in a document filed elsewhere.

Two properties keep Hybrid from decaying into an ordinary cloud product. Perception stays physically local, so the raw-signal invariants of Section 3 are unaffected by where reasoning runs. And the crossing is inspectable: the disclosure list is part of the product surface.

5. Running the whole product at home

The Sovereign placement is only interesting if a genuinely capable system fits on hardware a household can own. It now does, and the margin is worth reporting.

The reference hub is a single workstation-class machine with one consumer GPU carrying 32 GB of memory. The full stack is sized to run concurrently on that budget: speech recognition (a large multilingual recognizer, about 3.5 GB), a 14-billion-parameter language model quantized to 4-bit weights (about 8 GB of weights, with a pre-allocated key-value cache sized for a 16k-token context that brings its footprint to roughly 17 GB), a vision model for camera perception (about 1 GB), neural speech synthesis (about 1.2 GB), and a sentence-embedding model for memory retrieval (about 0.3 GB). The allocation totals roughly 23 GB and leaves about 9 GB of headroom for concurrency and longer contexts.

Configurations are not hand-tuned per machine. The hub inventories its own hardware and selects the largest model configuration that fits within the required margins, from small models suited to compact units up to 30-billion-parameter mixture models on larger GPUs. The result is graded: more local hardware buys more local capability, and the placement semantics stay identical across the whole range.

A local 14-billion-parameter model does not reason at the level of a frontier datacenter model, and it is worth saying so plainly. The household trades peak reasoning quality for the property that no request ever leaves the house. Sovereign exists for the households that consider that trade obviously correct, and the per-stage design exists so that nobody else has to make it all or nothing.

6. Failing closed, and proving it

Failure behavior is where privacy architectures are usually falsified in practice. The common pattern is a silent fallback: the local path errors, the software helpfully retries against a cloud endpoint, and the promise dissolves precisely when nobody is watching. Elysium's placement layer rejects that pattern. In the Sovereign placement, an error is surfaced as an error. Reduced capability is an acceptable outcome; a silent reroute is not. During development this rule was violated once, by a fallback deep in a serving path, and the fix was treated as release-blocking. That is the severity the rule is meant to carry.

Enforcement is necessary but not sufficient, because a resident cannot read enforcement. The design therefore includes two verification mechanisms aimed at the owner rather than at the developer.

The egress test. The hub ships a self-test that inspects every running service and asserts that none holds an established connection to a public network address. The intended protocol is deliberately physical: disconnect the uplink, use the house for a while (the assistant keeps answering, sign-in keeps working, devices keep responding), then run the test and read the result. A sovereignty claim that survives a pulled cable is a different kind of claim than one that lives in a settings page.

Updates that never phone home. Software updates are a common covert channel; even nominally local products routinely initiate outbound connections to check for releases. The Sovereign update path inverts the direction. Releases are cryptographically signed by the vendor and verified on the hub against keys installed at manufacture. The hub never initiates contact, and a household can apply updates from removable media with no uplink at all. This mechanism has been exercised end to end on a development unit, including rejection and rollback when verification fails.

Alongside both mechanisms, the hub presents the per-category disclosure described in Section 4, so that what would cross in a more permissive placement is stated in the same place the verification lives.

7. From self-hosted system to appliance

Architectural privacy is only meaningful to the general public if it does not require a systems administrator. The deployment path is therefore staged. The current form is a hardened self-hosted installation on the owner's hardware; it is the configuration the system is developed and dogfooded on. The next form packages the same system as a one-click installation for popular home-server platforms. The final form is the appliance: a hub that arrives with the system already imaged and uses the signed offline update path from Section 6 for its entire service life. The appliance image and its update mechanism exist and have been proven on development hardware; what remains between here and a purchasable unit is manufacturing and support work rather than research.

The staging preserves one property throughout: every tier runs the same product with the same placement semantics. The appliance is not a diminished edition, and the self-hosted installation is not an unsupported one.

8. Status and limitations

The state of the system today, in one place.

Operational: the placement layer and its three presets, including runtime switching with re-authentication and recorded consent; local identity with offline sign-in; local persistence of memory, conversations, and configuration; local language-model inference with hardware-fitted model selection; fail-closed behavior in the Sovereign placement; the egress self-test; and the signed offline update path, exercised on a development unit.

In active integration: the on-hub perception stack (speech recognition and vision) and the room-level voice satellite that performs wake-word detection and local speech recognition are the phase currently being brought up on hardware, with the appliance tier following once that work completes. Because the invariants of Section 3 are enforced in the placement policy already, perception lands inside the boundary rather than being retrofitted to it.

Two limitations are intrinsic rather than temporary. A local model will trail the strongest datacenter models in reasoning quality at any given moment. And verifiability as offered here covers network egress and update provenance, not side channels. The trust anchor for updates remains the vendor's signing key, which is a far smaller surface to trust than a vendor's entire data-handling operation, though it is still trust the owner extends rather than verifies.

References

1. Connectivity Standards Alliance. Matter specification. The local-control protocol used for lights, locks, and sensors.
2. Thread Group. Thread specification. Low-power mesh networking for local device transport.
3. Radford, A., Kim, J. W., Xu, T., Brockman, G., McLeavey, C., and Sutskever, I. (2023). Robust Speech Recognition via Large-Scale Weak Supervision. The Whisper family of speech-recognition models.
4. Kwon, W., Li, Z., Zhuang, S., et al. (2023). Efficient Memory Management for Large Language Model Serving with PagedAttention. The vLLM serving system used for on-hub inference.
5. Internet Engineering Task Force. RFC 5545: iCalendar. The subscribed-calendar format read by the assistant.