Abstract
A home that serves the people in it has to know them: who is acting, whose morning routine this is, which room belongs to the twelve-year-old, and which requests are standing preferences and which are one-time commands. In the dominant commercial pattern, knowing the resident means assembling a profile of them in a datacenter, where it can be retained, correlated across services, and monetized. Personalization and surveillance become the same act, and the resident is asked to accept the second to get the first. This paper describes an architecture in which recognition, and the memory it produces, are a property of the home rather than a service rendered from outside it. The household model, the standing preferences (one active setting per room, with authorship recorded), and the memory of what the home has learned live on the sovereign machine inside the house and do not leave it. This is the companion property to the sovereign architecture described in our earlier paper: sovereignty keeps the reasoning at home, and this keeps the knowing at home. On that foundation the design makes three commitments. Enrollment is a mutual, revocable act: a person is added to the household deliberately and can be removed at will. The system optimizes for discretion: each person gets the run of the house that fits them, and a guest gets the lights and nothing of the private world. And what the home knows about each person is legible to the household and erasable by it. We describe the access model the product enforces today, state precisely which recognition signals are operational and which are a scheduled hardware phase, and close with the limits we consider intrinsic.
1. The personalization dilemma
A useful home assistant is personal by necessity. A shared, anonymous device can turn on a light, but it cannot know that "my usual" means the bedroom at 21 degrees, that the child's tablet should go quiet at nine, or which requests come from a resident with the run of the house and which from a guest who should get the lights and nothing more. Every capability that feels like care depends on the home knowing who it is serving.
The industry has met that requirement in one way: build a model of the resident on the vendor's servers. Voices become voiceprints held in an account. Faces become embeddings in a recognition service. Routines, locations, and preferences become a profile that is valuable precisely because it is detailed and durable. The resident receives real personalization, and in exchange a copy of their domestic life is retained somewhere they cannot see, governed by terms they did not write.
The unease people report about a home that "knows too much" is usually misdiagnosed as unease about recognition itself. It is better described by the theory of contextual integrity: information flows carry norms tied to the context they occur in, and a violation is not the collection of information but its flow to a party or purpose the context did not sanction. A butler who knows your schedule is not a privacy problem. A butler who sells it is. The uncanny line is crossed not when the home knows you, but when someone outside the home does. The remainder of this paper takes that distinction as a design specification rather than a slogan.
A home can know your family, or a company can. The architecture decides which.
2. Recognition as a local property
The sovereign placement described in our companion paper keeps inference, identity, and memory on a single machine on the home network, and holds a hard rule that raw audio and video do not leave the house. Personalization is built entirely inside that boundary. The people in the household, their names and relationships, their standing preferences, and the assistant's memory of what it has learned are stored on the hub and are read only by the software running there. There is no personalization service to call, because the thing that would be called lives in the same house as the person it describes.
This inverts the usual data flow. In the cloud pattern, the signal (a voice, a face, a routine) travels to the model, and a profile accumulates far from the person. Here the model is already where the person is, so the signal never travels and no external profile exists to accumulate. Figure 1 contrasts the two.
The practical consequence is that the privacy of personalization is settled by where the data is. A profile that is never assembled off-site cannot be breached off-site, correlated with a shopping history, or handed to a third party, because it does not exist off-site. The resident does not have to believe a retention policy. They have to believe an address.
3. Enrollment is mutual and revocable
Covert recognition is the default that makes people uneasy, and with reason: a camera that silently learns the faces of everyone who passes is building a record no one consented to. Elysium treats enrollment as an explicit, mutual act. A person is added to the household deliberately, the household can see who is enrolled, and any enrollment can be undone, after which the home stops treating that person as known.
Two properties follow from making enrollment first-class. Consent is legible: being recognized by the home is something a resident opted into and can point to, not something that happened to them. And recognition is reversible: removing a person removes the basis on which the home personalizes for them. A guest who was given temporary access for a weekend is not quietly remembered the following month. The design goal is that no one is ever recognized by the home without having chosen to be, and that the choice can be withdrawn as easily as it was made.
4. Discretion over identification
The obvious objective for a recognition system is maximal identification: knowing exactly who everyone is. It is the wrong objective for a home. What a household actually wants is discretion, the appropriate matching of access to person, and discretion needs far less certainty than identification does. The organizing metaphor is a good butler, who gives the family the run of the house, gives the children what is theirs, and gives a visitor the sitting room and the lights without ever showing them the study.
The product enforces this as tiered access rather than a single trusted-or-untrusted switch. A resident has broad reach across the home. A child has a limited slice of their own. A guest is granted a deliberately narrow set of capabilities, and anything outside it is declined in the open rather than silently ignored: when a guest asks for something they were not given, the assistant says so plainly and does not perform it, which keeps the boundary observable to everyone in the room. Table 1 shows the shape of the ladder.
| Access tier | Reach | Private memory | Example |
|---|---|---|---|
| Resident | The run of the house, across rooms and capabilities | Their own, and the shared household memory | "Set the bedroom to 21 and remember it." |
| Child | A limited slice: their own rooms and devices | Their own only | "Turn my lights down." |
| Guest | A narrow, granted set of capabilities: the lights in shared rooms | None | "Turn on the living-room lights." → done; "Remember that I like it warm." → declined, in the open |
Three choices make the ladder safe. A grant is specific, a named set of capabilities rather than a broad role. A declined request is a stated outcome and not a silent one, so a person always knows what they can and cannot do, and never has to guess whether the home ignored them or refused them. And a guest is not even joined to the home's live stream of device state, so the ambient activity of the household never reaches someone who was handed the lights. Discretion, built this way, is something the whole household can see working. The access model defines and enforces these tiers in code; under today's cloud sign-in the separation exercised in practice is guest scoping, with per-resident roles arriving alongside the sovereign, per-person sign-in.
5. What is recognized, and what it is for
Recognition in the home serves two ends: routing a request to the right person's access and preferences, and keeping continuity across days so a resident is not made to repeat themselves. The clearest form of continuity is the standing preference. A resident states a preference once, "keep the bedroom at 21 degrees," and the home holds it and re-applies it when the room is next occupied, without being asked again. This is the difference between a device that obeys and a home that remembers. A standing preference is a household rule: the home keeps one active setting per room and records who set it, and when a new preference from one resident displaces another's, it says so in the open rather than overwriting it silently. Reconciling genuinely competing preferences between residents is a limit we return to in Section 7.
The way the home tells people apart is chosen to be the least invasive that will do the job, and today that is a credential rather than a body. A resident is known by their sign-in, a guest by a short code that expires, and a person at a shared wall panel by tapping their name and entering a personal PIN. No biometric is read. The designed next step is voice: the wall hub is intended to perform wake-word detection and speech recognition on the device itself, so that what a person says becomes text on the hub and the audio never leaves it. Recognizing a specific person by their voice is a later phase still, and it is treated deliberately as a low-assurance signal for discretion rather than a proof of identity, a boundary Section 7 makes explicit.
Today the household model, the tiered access with guest scoping, and household standing preferences are exercised through the assistant's text interface, with identity established by credential. On-device speech recognition and the wall hub that performs it are in active hardware integration rather than shipped. The architecture is arranged so that voice, when it lands, feeds the access and preference machinery that already exists rather than requiring it to be rebuilt.
6. Legible and erasable
A home that knows its family should be able to say what it knows and to forget on request. Both follow from keeping the knowledge local and structured. The household can see the people the home has enrolled, the standing preferences it holds, and the memory it has formed, and can remove any of them. That memory is itself scoped: what a resident tells the home in confidence is theirs alone and is not shown to the rest of the household, while what belongs to the home as a whole is shared. Discretion holds among residents, not only against the outside world. Removal is real because there is one place the data lives, so erasing it on the hub erases it, with no second copy in a datacenter to reconcile against.
This is the same commitment that governs the assistant's reasoning, described in our earlier work: a system that acts on your behalf should be one you can inspect and correct. A home that cannot explain what it knows about you, or cannot forget it when asked, is a surveillance device with better manners. Legibility and erasure are what keep recognition on the right side of that line.
7. Status and limitations
Operational today: the household model of distinct people and relationships; household standing preferences (one active setting per room, recording who set it), remembered and enforced across days; tiered access with room-and-capability guest scoping and a declined-in-the-open contract; and local storage of memory and preferences in the sovereign placement. These are exercised through the assistant's text interface and are the machinery the rest of the paper describes.
In active integration: on-device wake-word detection and speech recognition, and the wall hub that performs them; recognizing a specific speaker is a later phase. Until voice lands, the home tells people apart by the account they are acting through.
When two residents hold genuinely competing standing preferences for the same room, the home today applies the most recent instruction and names the change, rather than arbitrating between them; principled per-resident arbitration is planned, not shipped.
Some limits are intrinsic rather than temporary. Recognition of this kind is probabilistic and low-assurance by design: a shared or imitated voice is not proof of identity, so recognition is used to grant comfort and appropriate access, never to stand in for a security credential where one is required. Locality protects the household against outside profiling; it does not resolve trust inside the household, because the access model deliberately gives residents real reach, and a home is only as private among its own members as those members choose to make it. And personalization improves with the memory it keeps, which is in tension with keeping little; the design's answer is to keep that memory legible and erasable rather than to pretend it is not kept.
References
- Nissenbaum, H. (2004). Privacy as Contextual Integrity. Washington Law Review, 79(1). The framework for treating a privacy violation as an inappropriate information flow rather than as collection per se.
- Solove, D. J. (2006). A Taxonomy of Privacy. University of Pennsylvania Law Review, 154(3). A structured account of the distinct harms grouped under "privacy," including aggregation and secondary use.
- Bai, Z., and Zhang, X.-L. (2021). Speaker Recognition Based on Deep Learning: An Overview. Neural Networks, 140. Survey of modern speaker-recognition methods and their accuracy-versus-assurance trade-offs.
- Elysium Labs. Sovereign by Construction. The companion paper describing the placement architecture, boundary invariants, and the cable-pulled egress test that this work builds on (Elysium research, 2026; /research/sovereign-by-construction).
- Connectivity Standards Alliance. Matter specification. The local-control protocol the design uses to act on the access decisions described here; actuation is validated against a Matter bridge to date.