# macOS Native AI Companion Reconstruction Plan

> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.

**Goal:** Reconstruct Mastermind from an Electron interview-helper-shaped app into a native macOS AI companion with reliable overlay, screen context, system audio, voice interaction, and explicit trust indicators.

**Architecture:** Build a Swift/AppKit-first macOS host that owns windows, capture, audio, permissions, menu bar status, and privacy boundaries. Keep AI providers and local sidecars behind protocol boundaries so the rewrite can progress incrementally instead of becoming a risky full replacement.

**Tech Stack:** Swift, AppKit, SwiftUI where appropriate, ScreenCaptureKit, AVAudioEngine, CoreAudio, Keychain, Application Support storage, URLSession streaming, WebSocket, local ASR sidecar protocol, OpenAI-compatible HTTP/SSE, optional Electron bridge during migration.

---

## Product Thesis

Mastermind should become an **AI companion for your Mac**, not a hidden interview helper.

The assistant is a personal, visible-to-the-user HUD and agent system. It can help during calls, presentations, coding, writing, research, and everyday computer use. It may stay out of the content the user is presenting or sharing, just like presenter notes, a timer, captions, or a local command palette.

The product should not implement stealth, anti-detection, process hiding, monitoring bypass, proctoring bypass, or policy evasion. If a workplace, school, exam, or managed system prohibits AI assistants, Mastermind should not help users hide that it is running.

## Core Boundary

**Allowed:** The assistant window does not appear in the user's own screen context, screenshots, screen-share content, or presentation output when technically possible.

**Not allowed:** Hiding the app process, bundle identifier, permissions, network use, Accessibility use, Screen Recording use, microphone use, or audio capture use from operating-system tools or managed environments.

Practical phrasing:

> Mastermind does not hide from the user or the system. It only avoids contaminating the content the user intentionally shares or asks the assistant to analyze.

## Target User Experience

### Normal Companion Mode

The assistant is available through voice, keyboard, and a small native overlay. It can answer questions, remember context, summarize active work, and trigger actions.

### Meeting Mode

The assistant can listen to microphone and system audio with clear status indicators. It can summarize discussion, draft follow-ups, keep agenda state, and help the user stay oriented.

### Presentation Mode

The user can share slides, a browser, or an app window while seeing local prompts, timing, plan notes, likely objections, and speaker guidance in Mastermind's HUD. The shared audience does not need to see the HUD.

### Screen Context Mode

Screen context is opt-in. The user explicitly asks the assistant to look at the screen, capture a snapshot, or follow a bounded live stream. Mastermind excludes its own UI from that context.

### Idle Mode

When not actively listening, viewing, or processing, Mastermind stays quiet and visibly idle in the menu bar.

## Trust Model

Trust is a first-class feature, not a settings afterthought.

Required visible signals:

- Menu bar icon is always present while the app runs.
- Status reflects actual state: idle, listening, reading screen, processing, paused, permission problem.
- Mic capture, system audio capture, and screen context have separate indicators.
- The user can pause capture immediately.
- The app exposes a local activity log showing recent context use: microphone, system audio, screen snapshot, screen stream, provider request.
- The app does not provide stealth labels, anti-detection switches, or hidden-running modes.

Recommended status vocabulary:

- `Idle`
- `Listening`
- `System Audio`
- `Screen Snapshot`
- `Screen Stream`
- `Agent Working`
- `Paused`
- `Permission Needed`
- `Error`

## Current Project Context

Current repository shape:

- Electron Forge app with JavaScript entry point at `src/index.js`.
- Window management and global shortcuts in `src/utils/window.js`.
- AI provider/session logic in `src/utils/gemini.js`, `src/utils/localai.js`, and `src/utils/localProviders.js`.
- Local ASR sidecar protocol already documented in `docs/local-sidecar-protocol.md`.
- JSON storage in `src/storage.js`.
- Lit-based UI under `src/components`.

Important existing behavior to preserve:

- Always-on-top assistant window.
- Global keyboard shortcuts.
- Click-through toggle.
- Hide/show assistant.
- Session history.
- Local mode with ASR sidecar.
- BYOK provider support.
- OpenAI-compatible provider support.
- Groq/Gemini provider support.
- Configurable prompt/profile/language.

Important behavior to replace:

- Electron-owned transparent window quirks.
- Chromium `getDisplayMedia` screen capture dependency.
- Fragile loopback/system-audio capture.
- Renderer-local storage access from main process.
- `nodeIntegration: true` and `contextIsolation: false`.
- Hidden coupling between capture, transcription, provider routing, and renderer events.

## Reconstruction Strategy

Recommended path: **Swift-native host first, full app migration second**.

Do not start with a full SwiftUI rewrite of every screen. The riskiest parts are windowing, screen capture, audio capture, permissions, and trust indicators. Prove those first in a native macOS host, then migrate provider and UI surfaces incrementally.

Migration shape:

```text
Phase 1: Native capability proof
  Swift/AppKit overlay
  menu bar status item
  ScreenCaptureKit screen context
  ScreenCaptureKit system audio
  AVAudioEngine microphone audio

Phase 2: Native companion shell
  state machine
  permissions
  trust indicators
  local sidecar bridge
  provider boundary

Phase 3: Agentic assistant
  modes
  memory
  tools
  activity log
  meeting and presentation flows

Phase 4: Retire Electron
  migrate settings/history
  package/notarize Swift app
  deprecate Electron runtime
```

## Proposed Native Architecture

```text
Mastermind.app
  AppCoordinator
    owns lifecycle, mode, permissions, menu bar status

  OverlayWindowController
    owns transparent HUD, click-through, Spaces behavior, focus behavior

  CaptureCoordinator
    owns screen context and system audio through ScreenCaptureKit

  MicrophoneCaptureEngine
    owns microphone stream through AVAudioEngine

  AudioPipeline
    owns channel separation, resampling, VAD, PCM frame output

  AssistantSession
    owns active conversation, transcript, memory references, provider routing

  ProviderClients
    Gemini, Groq, OpenAI-compatible, local LLM, local ASR sidecar

  TrustStatusStore
    owns visible state, activity log, pause/resume, capture indicators

  Storage
    Application Support for non-secret app data
    Keychain for secrets
```

## Agentic System Direction

The future assistant should be built around modes and tools, not a single chat box.

Core modes:

- General Companion
- Meeting Assistant
- Presentation Coach
- Coding Assistant
- Research Assistant
- Focus Assistant

Core agent abilities:

- Listen and summarize.
- Answer conversationally.
- See screen only when requested.
- Track agenda or presentation plan.
- Draft follow-up notes.
- Remember user preferences.
- Use local tools after explicit permission.
- Explain what context it used.

Agent boundaries:

- No autonomous destructive actions.
- No hidden capture.
- No silent screen streaming.
- No policy bypass tooling.
- No stealth process behavior.

## File Structure Target

Future native app structure:

```text
native/Mastermind/
  Mastermind.xcodeproj
  Mastermind/
    App/
      MastermindApp.swift
      AppCoordinator.swift
      AppMode.swift
      PermissionState.swift
    Status/
      MenuBarController.swift
      TrustStatus.swift
      ActivityLog.swift
    Overlay/
      OverlayWindowController.swift
      OverlayRootView.swift
      OverlayViewModel.swift
    Capture/
      CaptureCoordinator.swift
      ScreenContextCapture.swift
      SystemAudioCapture.swift
      CaptureExclusionPolicy.swift
    Audio/
      MicrophoneCaptureEngine.swift
      AudioPipeline.swift
      PCMFrame.swift
      VoiceActivityDetector.swift
    Assistant/
      AssistantSession.swift
      AssistantMode.swift
      AssistantEvent.swift
      AssistantMemory.swift
    Providers/
      ProviderClient.swift
      LocalAsrSidecarClient.swift
      OpenAICompatibleClient.swift
      GeminiClient.swift
      GroqClient.swift
    Storage/
      AppStorageStore.swift
      KeychainStore.swift
      MigrationStore.swift
    Settings/
      SettingsWindowController.swift
      SettingsRootView.swift
    Presentation/
      PresentationPlan.swift
      PresentationCoach.swift
```

Existing Electron files remain read-only during the proof phase except for bridge points explicitly required by a migration task.

## Task 1: Write Product Reconstruction Charter

**Files:**

- Create: `docs/product/macos-native-ai-companion-charter.md`

- [ ] **Step 1: Create the product charter**

Write a concise charter with these sections:

```markdown
# macOS Native AI Companion Charter

## Positioning

Mastermind is a native macOS AI companion for everyday computer work, meetings, presentations, research, and focused execution.

## What It Is

- A visible personal assistant for the user.
- A voice-first and context-aware companion.
- A local HUD for notes, guidance, and agent status.
- A privacy-conscious screen and audio context tool.

## What It Is Not

- A hidden interview helper.
- A proctoring bypass tool.
- An anti-detection tool.
- A process-hiding tool.
- A tool for hiding AI use from managed systems that prohibit it.

## Core Promise

Mastermind does not hide from the user or the system. It only avoids contaminating the content the user intentionally shares or asks the assistant to analyze.

## Trust Requirements

- Show a menu bar status item whenever running.
- Show when microphone, system audio, or screen context is active.
- Allow immediate pause.
- Keep a local activity log of context use.
- Store secrets in Keychain.
```

- [ ] **Step 2: Review the charter language**

Confirm the document includes the phrases `AI companion for your Mac`, `does not hide from the user or the system`, and `visible personal assistant`.

- [ ] **Step 3: Commit**

```bash
git add docs/product/macos-native-ai-companion-charter.md
git commit -m "docs: define macos ai companion direction"
```

## Task 2: Build Native Capability Proof Project

**Files:**

- Create: `native/Mastermind/Mastermind.xcodeproj`
- Create: `native/Mastermind/Mastermind/App/MastermindApp.swift`
- Create: `native/Mastermind/Mastermind/App/AppCoordinator.swift`
- Create: `native/Mastermind/Mastermind/Status/MenuBarController.swift`
- Create: `native/Mastermind/Mastermind/Overlay/OverlayWindowController.swift`
- Create: `native/Mastermind/Mastermind/Capture/CaptureCoordinator.swift`
- Create: `native/Mastermind/Mastermind/Audio/MicrophoneCaptureEngine.swift`
- Create: `native/Mastermind/Mastermind/Audio/AudioPipeline.swift`

- [ ] **Step 1: Create a minimal native macOS app**

Create a macOS app target named `Mastermind`. Use Swift, AppKit lifecycle, and SwiftUI only for simple views.

- [ ] **Step 2: Add a menu bar status item**

Implement a menu bar item that always appears while the app runs. The menu must include:

```text
Mastermind: Idle
Pause All Capture
Show Assistant
Hide Assistant
Settings
Quit Mastermind
```

- [ ] **Step 3: Add a transparent overlay window**

Create an AppKit-controlled floating overlay window with these properties:

```text
borderless
transparent background
always on top
visible across Spaces
does not steal focus when shown
can become click-through
can be hidden and restored
```

- [ ] **Step 4: Add screen context proof**

Use ScreenCaptureKit to capture the main display. The proof succeeds only when captured frames exclude the assistant overlay.

- [ ] **Step 5: Add system audio proof**

Use ScreenCaptureKit audio output to receive system audio buffers.

- [ ] **Step 6: Add microphone proof**

Use AVAudioEngine to receive microphone PCM buffers.

- [ ] **Step 7: Add 16 kHz PCM output proof**

Convert microphone and system audio streams into 16 kHz signed 16-bit PCM frames.

- [ ] **Step 8: Manual verification**

Run the native app and verify:

```text
menu bar item is visible
overlay is visible locally
overlay can become click-through
overlay does not appear in app-owned captured frames
system audio buffers arrive when another app plays audio
microphone buffers arrive when speaking
capture can be paused immediately
```

- [ ] **Step 9: Commit**

```bash
git add native/Mastermind
git commit -m "feat: add native macos capture proof"
```

## Task 3: Define Trust State Machine

**Files:**

- Create: `native/Mastermind/Mastermind/Status/TrustStatus.swift`
- Create: `native/Mastermind/Mastermind/Status/ActivityLog.swift`
- Modify: `native/Mastermind/Mastermind/Status/MenuBarController.swift`
- Modify: `native/Mastermind/Mastermind/App/AppCoordinator.swift`

- [ ] **Step 1: Define trust states**

Use these states exactly:

```swift
enum TrustStatus: Equatable {
    case idle
    case listening
    case systemAudio
    case screenSnapshot
    case screenStream
    case agentWorking
    case paused
    case permissionNeeded(String)
    case error(String)
}
```

- [ ] **Step 2: Define context activity events**

Use these events exactly:

```swift
enum ActivityEventKind: String, Codable {
    case microphoneStarted
    case microphoneStopped
    case systemAudioStarted
    case systemAudioStopped
    case screenSnapshotCaptured
    case screenStreamStarted
    case screenStreamStopped
    case providerRequestStarted
    case providerRequestFinished
    case capturePaused
}
```

- [ ] **Step 3: Wire status to menu bar copy**

Map trust states to visible labels:

```text
idle -> Mastermind: Idle
listening -> Mastermind: Listening
systemAudio -> Mastermind: System Audio
screenSnapshot -> Mastermind: Screen Snapshot
screenStream -> Mastermind: Screen Stream
agentWorking -> Mastermind: Agent Working
paused -> Mastermind: Paused
permissionNeeded -> Mastermind: Permission Needed
error -> Mastermind: Error
```

- [ ] **Step 4: Verify state changes manually**

Trigger each capture path and confirm the menu bar label changes before any capture data leaves the machine.

- [ ] **Step 5: Commit**

```bash
git add native/Mastermind/Mastermind/Status native/Mastermind/Mastermind/App
git commit -m "feat: add native trust status model"
```

## Task 4: Preserve Local ASR Sidecar Boundary

**Files:**

- Create: `native/Mastermind/Mastermind/Providers/ProviderClient.swift`
- Create: `native/Mastermind/Mastermind/Providers/LocalAsrSidecarClient.swift`
- Modify: `docs/local-sidecar-protocol.md`

- [ ] **Step 1: Define provider boundary**

Use this protocol as the native client boundary:

```swift
protocol ProviderClient {
    associatedtype Event

    func start() async throws
    func stop() async
    var events: AsyncStream<Event> { get }
}
```

- [ ] **Step 2: Mirror the current ASR sidecar protocol**

Implement the native client against the existing WebSocket flow:

```json
{
  "type": "start",
  "sampleRate": 16000,
  "channels": 1,
  "encoding": "pcm_s16le",
  "language": "en-US"
}
```

- [ ] **Step 3: Preserve event semantics**

Support these sidecar events:

```text
ready
partial
final
error
```

Only `final` transcript events should enter the assistant response pipeline by default.

- [ ] **Step 4: Document native compatibility**

Add a section to `docs/local-sidecar-protocol.md`:

```markdown
## Native macOS Client Compatibility

The Swift-native app uses the same WebSocket protocol as the Electron app. Audio frames are sent as raw 16 kHz mono signed 16-bit little-endian PCM. The sidecar does not need to know whether the client is Electron or Swift.
```

- [ ] **Step 5: Commit**

```bash
git add native/Mastermind/Mastermind/Providers docs/local-sidecar-protocol.md
git commit -m "feat: add native local asr sidecar client"
```

## Task 5: Design Native Audio Pipeline

**Files:**

- Create: `native/Mastermind/Mastermind/Audio/PCMFrame.swift`
- Create: `native/Mastermind/Mastermind/Audio/AudioPipeline.swift`
- Create: `native/Mastermind/Mastermind/Audio/VoiceActivityDetector.swift`
- Modify: `native/Mastermind/Mastermind/Audio/MicrophoneCaptureEngine.swift`
- Modify: `native/Mastermind/Mastermind/Capture/CaptureCoordinator.swift`

- [ ] **Step 1: Define PCM frame format**

Use this data model:

```swift
struct PCMFrame: Equatable {
    let source: AudioSource
    let sampleRate: Int
    let channels: Int
    let pcmS16LE: Data
    let timestamp: Date
}

enum AudioSource: String {
    case microphone
    case system
}
```

- [ ] **Step 2: Keep microphone and system audio separate**

The pipeline should not mix microphone and system audio before transcription. Separate streams preserve future diarization and meeting-context quality.

- [ ] **Step 3: Resample to 16 kHz**

Every frame sent to local ASR must be:

```text
sample rate: 16000
channels: 1
encoding: signed 16-bit little-endian PCM
```

- [ ] **Step 4: Add VAD boundary**

Voice activity detection should emit speech segments instead of forcing every buffer into transcription.

- [ ] **Step 5: Manual verification**

Verify:

```text
mic frames continue when system audio is silent
system frames continue when mic is silent
both streams can be paused together
either stream can be disabled independently
sidecar receives valid 16 kHz PCM frames
```

- [ ] **Step 6: Commit**

```bash
git add native/Mastermind/Mastermind/Audio native/Mastermind/Mastermind/Capture
git commit -m "feat: add native audio pipeline model"
```

## Task 6: Migrate Secrets and Storage Boundaries

**Files:**

- Create: `native/Mastermind/Mastermind/Storage/KeychainStore.swift`
- Create: `native/Mastermind/Mastermind/Storage/AppStorageStore.swift`
- Create: `native/Mastermind/Mastermind/Storage/MigrationStore.swift`
- Reference: `src/storage.js`

- [ ] **Step 1: Map current storage**

Preserve these current logical groups:

```text
config
credentials
preferences
keybinds
limits
history
```

- [ ] **Step 2: Move secrets to Keychain**

Store these values in Keychain:

```text
Gemini API key
Groq API key
OpenAI-compatible API key
local LLM API key
```

- [ ] **Step 3: Store non-secret data in Application Support**

Store these values in Application Support:

```text
preferences
profiles
keybinds
history
limits
activity log
presentation plans
assistant memory references
```

- [ ] **Step 4: Add one-way migration**

Read existing Electron JSON files from:

```text
~/Library/Application Support/cheating-daddy-config
```

Import values into native stores. Leave the old files untouched.

- [ ] **Step 5: Commit**

```bash
git add native/Mastermind/Mastermind/Storage
git commit -m "feat: add native storage migration boundary"
```

## Task 7: Build Presentation Coach Mode

**Files:**

- Create: `native/Mastermind/Mastermind/Presentation/PresentationPlan.swift`
- Create: `native/Mastermind/Mastermind/Presentation/PresentationCoach.swift`
- Modify: `native/Mastermind/Mastermind/Assistant/AssistantMode.swift`
- Modify: `native/Mastermind/Mastermind/Overlay/OverlayRootView.swift`

- [ ] **Step 1: Define presentation plan model**

Use this model:

```swift
struct PresentationPlan: Codable, Equatable {
    var title: String
    var sections: [PresentationSection]
}

struct PresentationSection: Codable, Equatable {
    var title: String
    var talkingPoints: [String]
    var expectedDurationSeconds: Int
}
```

- [ ] **Step 2: Add presentation mode**

Add a mode named `presentationCoach`.

- [ ] **Step 3: Show local guidance in HUD**

HUD should show:

```text
current section
next talking point
elapsed time
suggested transition
likely audience question
```

- [ ] **Step 4: Keep guidance out of shared content**

Use the same overlay exclusion policy as Screen Context Mode. The local HUD should be visible to the user and absent from app-owned captures.

- [ ] **Step 5: Commit**

```bash
git add native/Mastermind/Mastermind/Presentation native/Mastermind/Mastermind/Assistant native/Mastermind/Mastermind/Overlay
git commit -m "feat: add presentation coach mode"
```

## Task 8: Define Electron Retirement Gates

**Files:**

- Create: `docs/migration/electron-retirement-gates.md`

- [ ] **Step 1: Create retirement gate checklist**

Write this checklist:

```markdown
# Electron Retirement Gates

- [ ] Native overlay matches or exceeds Electron window behavior.
- [ ] Native menu bar status item is always visible while running.
- [ ] Native screen context excludes Mastermind UI.
- [ ] Native system audio capture works without third-party loopback drivers.
- [ ] Native microphone capture works independently from system audio.
- [ ] Native local ASR sidecar client can transcribe 16 kHz PCM.
- [ ] Native provider client can stream OpenAI-compatible responses.
- [ ] Native settings can import current Electron preferences.
- [ ] Native Keychain storage replaces JSON credential storage.
- [ ] Native package can be signed and notarized.
- [ ] Electron app remains available as fallback until native app covers core workflows.
```

- [ ] **Step 2: Commit**

```bash
git add docs/migration/electron-retirement-gates.md
git commit -m "docs: add electron retirement gates"
```

## Task 9: Package and Distribution Direction

**Files:**

- Create: `docs/migration/native-distribution.md`

- [ ] **Step 1: Document distribution requirements**

Include:

```text
Developer ID signing
notarization
Screen Recording permission messaging
Microphone permission messaging
optional Accessibility permission messaging
Sparkle or equivalent update path
DMG distribution
crash reporting decision
local log export
```

- [ ] **Step 2: Document permission copy**

Use transparent user-facing copy:

```text
Mastermind needs Screen Recording permission only when you ask it to use screen context.
Mastermind needs Microphone permission only when voice input or meeting listening is enabled.
Mastermind shows a menu bar status item whenever it is running.
```

- [ ] **Step 3: Commit**

```bash
git add docs/migration/native-distribution.md
git commit -m "docs: define native distribution requirements"
```

## Verification Checklist

Before claiming the reconstruction direction is ready for implementation:

- [ ] The charter clearly says this is an AI companion, not a hidden helper.
- [ ] The plan preserves local ASR sidecar compatibility.
- [ ] The plan does not require a full rewrite before validating native capture.
- [ ] The plan includes a visible menu bar status item.
- [ ] The plan separates user-visible HUD behavior from stealth behavior.
- [ ] The plan includes microphone, system audio, and screen context as separate states.
- [ ] The plan includes Keychain for secrets.
- [ ] The plan includes Electron retirement gates.
- [ ] The plan keeps current Electron app functional during migration.

## Recommended First Milestone

The first milestone should be **Native Capability Proof**, not full product migration.

Success definition:

```text
A Swift/AppKit app shows a local transparent overlay, displays an always-visible menu bar status item, captures screen frames without its own overlay, receives system audio, receives microphone audio, and emits 16 kHz PCM frames compatible with the existing local ASR sidecar protocol.
```

If this milestone fails, keep improving the Electron app while reassessing native capture options. If it succeeds, move provider clients and agent modes into the native shell incrementally.

