app development context
This commit is contained in:
Илья Глазунов
@@ -0,0 +1,217 @@
|
||||
# Native Mastermind Architecture
|
||||
|
||||
## Scope
|
||||
|
||||
This document defines the production architecture for Mastermind. Domain language comes from [`../../CONTEXT.md`](../../CONTEXT.md), product scope from [`../product/mastermind-product-brief.md`](../product/mastermind-product-brief.md), and privacy invariants from [`../privacy/local-first-data-contract.md`](../privacy/local-first-data-contract.md).
|
||||
|
||||
The production application lives under `native/Mastermind`. `native/MastermindPOC` is a capability reference, not the production architecture. The Electron application is legacy source material and is not part of the production runtime.
|
||||
|
||||
## Runtime boundary
|
||||
|
||||
The application host is Swift-native:
|
||||
|
||||
- AppKit owns lifecycle, windows, menu bar behavior, permissions, and capture.
|
||||
- SwiftUI may render Companion Island content inside AppKit-controlled windows.
|
||||
- Swift concurrency isolates Collectors and processing from the main actor.
|
||||
- The host owns the encrypted Context Graph and all policy decisions.
|
||||
- Local model processes remain outside the host behind narrow protocols.
|
||||
- No Node or Electron process is required.
|
||||
|
||||
Permitted external local processes:
|
||||
|
||||
- an OpenAI-compatible Local Provider on loopback or a Unix socket;
|
||||
- a local ASR sidecar using the existing WebSocket protocol.
|
||||
|
||||
Mastermind owns a small multilingual embedding component because chat endpoints do not reliably expose embeddings.
|
||||
|
||||
## System shape
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
subgraph host [Swift native host]
|
||||
AppCoordinator --> MenuBar
|
||||
AppCoordinator --> CompanionIsland
|
||||
AppCoordinator --> CollectorSupervisor
|
||||
CollectorSupervisor --> ScreenCollector
|
||||
CollectorSupervisor --> AudioCollectors
|
||||
CollectorSupervisor --> WorkspaceCollector
|
||||
CollectorSupervisor --> CalendarCollector
|
||||
CollectorSupervisor --> TerminalCollector
|
||||
ScreenCollector --> LocalReduction
|
||||
AudioCollectors --> LocalReduction
|
||||
WorkspaceCollector --> LocalReduction
|
||||
CalendarCollector --> LocalReduction
|
||||
TerminalCollector --> LocalReduction
|
||||
LocalReduction --> ObservationLedger
|
||||
ObservationLedger --> KnowledgeProjector
|
||||
KnowledgeProjector --> ContextGraph
|
||||
ContextGraph --> Retriever
|
||||
Retriever --> AssistantSession
|
||||
AssistantSession --> ContextReceipt
|
||||
CollectorSupervisor --> ActivityLog
|
||||
AssistantSession --> ActivityLog
|
||||
end
|
||||
|
||||
AudioCollectors --> AsrSidecar
|
||||
AssistantSession --> LocalProvider
|
||||
EmbeddingModel --> KnowledgeProjector
|
||||
EmbeddingModel --> Retriever
|
||||
```
|
||||
|
||||
Every arrow carrying Source content stays on the Mac. Activity Log entries contain metadata, not Source content.
|
||||
|
||||
## Application components
|
||||
|
||||
### AppCoordinator
|
||||
|
||||
Owns lifecycle and composes the application. It restores the persisted paused state before starting Collectors, coordinates sleep/wake and Launch at Login, and never places heavy processing on the main actor.
|
||||
|
||||
### CompanionIslandController
|
||||
|
||||
Owns the hidden, revealed, and expanded window states on the primary display. The controller separates non-activating hover behavior from the key window used for text entry and management.
|
||||
|
||||
### MenuBarController
|
||||
|
||||
Owns the stable status glyph and detailed menu. Collector states are represented in menu content rather than by changing the glyph.
|
||||
|
||||
### CollectorSupervisor
|
||||
|
||||
Owns independent Collector lifecycles. It starts enabled Collectors, pauses them atomically, persists Pause All, applies power and thermal policy, and records Gaps. One Collector failure must not collapse the rest of the pipeline.
|
||||
|
||||
### ScreenCollector
|
||||
|
||||
Uses ScreenCaptureKit for pixels and Accessibility for application/window structure. The active display is the display containing the key window, falling back to pointer location and then the primary display. Changed-region detection, OCR, and deduplication reduce pixels locally before immediate disposal.
|
||||
|
||||
Mastermind windows are excluded from its own stream. Third-party exclusion is a best-effort window policy, not a security guarantee.
|
||||
|
||||
### AudioCollectors
|
||||
|
||||
Microphone and system audio are separate Collectors and remain separate. Both emit 16 kHz mono signed 16-bit PCM frames to VAD and local ASR. The pipeline labels channel role as User Speech or System Speech but does not infer a Person from voice.
|
||||
|
||||
### WorkspaceCollector
|
||||
|
||||
Observes explicitly connected roots. It respects `.gitignore`, configured excludes, binary detection, and size limits, and emits changes to Artifacts and repository metadata.
|
||||
|
||||
### CalendarCollector
|
||||
|
||||
Uses EventKit read-only for selected calendars and Reminders lists. External applications remain systems of record for Tasks and Events.
|
||||
|
||||
### TerminalCollector
|
||||
|
||||
Receives shell integration events for working directory, command, exit status, duration, and Git metadata. It does not persist stdout or stderr by default.
|
||||
|
||||
## Knowledge pipeline
|
||||
|
||||
### Observation ledger
|
||||
|
||||
Collectors append immutable Observations. Each record includes:
|
||||
|
||||
- stable identifier;
|
||||
- Source identifier and Collector kind;
|
||||
- observed time and source event time when available;
|
||||
- structured payload;
|
||||
- sensitivity metadata;
|
||||
- derivation version.
|
||||
|
||||
Observations are retained until age or size eviction. Corrections never mutate them.
|
||||
|
||||
### Knowledge projector
|
||||
|
||||
The projector derives versioned Assertions, entity relationships, short summaries, and Provenance. Multiple contradictory Assertions may coexist. A current view selects relevant Assertions without erasing history.
|
||||
|
||||
Only explicit user confirmation or correction promotes an Assertion to a Fact. A correction creates a new version and supersedes prior current views; it does not rewrite supporting Observations.
|
||||
|
||||
Entity resolution automatically links only high-confidence matches. Ambiguous entities remain separate and can be merged or split from a Context Receipt.
|
||||
|
||||
### Context store
|
||||
|
||||
The source of truth is encrypted SQLite/SQLCipher with explicit schema migrations. It stores:
|
||||
|
||||
- Sources and Collector configuration;
|
||||
- immutable Observations;
|
||||
- entities and relations;
|
||||
- versioned Assertions and Facts;
|
||||
- Provenance;
|
||||
- Assistant History;
|
||||
- Context Receipts;
|
||||
- Activity Log metadata;
|
||||
- retention and Provider policies.
|
||||
|
||||
Full-text and vector indexes are derived and rebuildable. They must not become unencrypted alternate stores.
|
||||
|
||||
The database key is generated locally and protected by Keychain. Provider secrets use separate Keychain entries. An encrypted, versioned archive supports manual export and import without Provider credentials.
|
||||
|
||||
### Retention
|
||||
|
||||
The default inferred-context budget is 90 days and 2 GB. Eviction considers `lastObservedAt` and utility. User-confirmed Facts are pinned. Assistant History has no automatic TTL and supports per-session and complete deletion.
|
||||
|
||||
Raw frames, audio, OCR, and full transcripts never enter the Context store. If local reduction cannot complete, the system records a Gap.
|
||||
|
||||
## Retrieval and assistance
|
||||
|
||||
The Retriever combines:
|
||||
|
||||
- structured entity and relationship queries;
|
||||
- temporal filtering;
|
||||
- full-text search;
|
||||
- multilingual vector similarity;
|
||||
- recency and confidence;
|
||||
- Provenance completeness.
|
||||
|
||||
Retrieved Source content is always untrusted data. It cannot alter system instructions or grant capabilities.
|
||||
|
||||
An Assistant Session binds to one Provider profile. The MVP has one active OpenAI-compatible Local Provider profile at a time and streams text responses. Failure leaves the query draft intact and preserves access to graph search, history, settings, and diagnostics.
|
||||
|
||||
Every answer produces a Context Receipt containing:
|
||||
|
||||
- Provider profile and locality;
|
||||
- supporting Sources, Observations, Assertions, and Facts;
|
||||
- time range and stale/conflict indicators;
|
||||
- any external-transfer record.
|
||||
|
||||
## State model
|
||||
|
||||
Capture and assistant state are orthogonal. Do not reproduce the POC's single `TrustStatus` enum, which loses information when screen, microphone, and system audio run simultaneously.
|
||||
|
||||
Model at least:
|
||||
|
||||
- one `CollectorState` per Collector: disabled, starting, running, paused, permission-needed, degraded, or failed;
|
||||
- an application pause state persisted across launches;
|
||||
- Companion Island presentation state;
|
||||
- Assistant Session state;
|
||||
- Local Provider and sidecar health;
|
||||
- Context Graph migration and maintenance state.
|
||||
|
||||
The menu and Activity Log project these states for the user.
|
||||
|
||||
## Failure rules
|
||||
|
||||
- Permission loss changes only the affected Collector to permission-needed.
|
||||
- Processor or sidecar failure creates Gaps; it never creates a raw-content backlog.
|
||||
- Database or encryption failure prevents new context writes and surfaces a blocking error.
|
||||
- Thermal and low-power pressure reduce sampling and local processing.
|
||||
- Quit stops all Collectors and flushes metadata.
|
||||
- Relaunch while paused remains paused.
|
||||
|
||||
## Future boundaries
|
||||
|
||||
### Cloud Provider
|
||||
|
||||
A future Cloud Provider must implement the Provider boundary and Provider Context Permission. It may receive only minimized, locally filtered context and must generate a Context Receipt for every request.
|
||||
|
||||
### Codex
|
||||
|
||||
Codex remains documentation-only until it exposes a supported no-tools integration. The MVP has no Codex code or UI.
|
||||
|
||||
### Agent actions
|
||||
|
||||
The MVP may emit an Action Proposal as text. Computer Control, Tool execution, approvals, and action auditing form a future bounded context and are not represented by executable stubs in the MVP.
|
||||
|
||||
## Migration direction
|
||||
|
||||
1. Treat `native/MastermindPOC` as evidence that the required macOS capture and window primitives are viable.
|
||||
2. Create the production application under `native/Mastermind`.
|
||||
3. Port validated capability code behind production state and protocol boundaries.
|
||||
4. Build the Context Graph vertical slice and primary acceptance scenario.
|
||||
5. Retire the POC after production parity checks pass.
|
||||
6. Keep Electron code as legacy reference only; do not migrate its data or runtime.
|
||||
Reference in New Issue
Block a user