Shifty/Mastermind
1
1
Fork 0
Code Issues 1 Pull Requests Actions Packages Projects 1 Releases Wiki Activity

app development context

Browse Source
This commit is contained in:
Илья Глазунов
2026-09-05 04:37:28 +03:00
parent 8beccdf101
commit a55ee10d2b
15 changed files with 1007 additions and 1185 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+114 -113
View File
@@ -1,130 +1,131 @@
# Repo Guidelines
# Repository Guidelines
This repository is a fork of [`cheating-daddy`](https://github.com/sohzm/cheating-daddy).
It provides an Electron-based real‑time assistant which captures screen and audio
for contextual AI responses. The code is JavaScript and uses Electron Forge for
packaging.
This repository is reconstructing Mastermind as a Swift-native, local-first personal assistant for macOS. The Electron application is legacy reference code; it is not the target architecture.
## Getting started
## Read first
Install dependencies and run the development app:
Before planning or implementing product work, read:
```
1. npm install
2. npm start
1. [`CONTEXT.md`](CONTEXT.md) for canonical domain language.
2. [`docs/product/mastermind-product-brief.md`](docs/product/mastermind-product-brief.md) for scope and non-goals.
3. [`docs/architecture/native-mastermind.md`](docs/architecture/native-mastermind.md) for system boundaries.
4. [`docs/privacy/local-first-data-contract.md`](docs/privacy/local-first-data-contract.md) for normative data rules.
5. [`docs/product/mvp-acceptance.md`](docs/product/mvp-acceptance.md) for completion criteria.
ADRs under `docs/adr` explain hard-to-reverse decisions. When legacy code or documentation conflicts with the canonical context, the canonical context wins.
## Repository roles
- `native/Mastermind` — production Swift application; create new production work here.
- `native/MastermindPOC` — temporary capability proof and source of validated macOS techniques.
- `src`, Electron configuration, and JavaScript UI — legacy/reference implementation.
- `docs/local-sidecar-protocol.md` — shared local ASR protocol.
Do not evolve `MastermindPOC` into the product in place. Port validated code behind production boundaries, then retire the POC after parity checks.
Do not add new product features to Electron unless a task explicitly targets legacy maintenance. Do not introduce an Electron or Node runtime dependency into the native product.
## Swift standards
- Target Apple Silicon and macOS 14 or newer.
- Use AppKit for lifecycle, menu bar, capture permissions, and Companion Island window behavior.
- Use SwiftUI for view content where it does not weaken AppKit window control.
- Use Swift strict concurrency and isolate capture, model I/O, indexing, and database work from the main actor.
- Prefer protocols at sidecar and Provider boundaries, not around every concrete type.
- Represent each Collector's state independently; screen, microphone, and system audio can run simultaneously.
- Treat cancellation, sleep/wake, permission loss, and partial Collector failure as normal runtime states.
- Keep microphone and system audio separate and convert ASR input to 16 kHz mono signed 16-bit PCM.
- Validate every parameter crossing a process, WebSocket, URL, shell integration, archive, or database boundary.
- Add tests for every new domain invariant and failure path.
## Domain language
Use the exact terms in `CONTEXT.md`.
- Do not call Assertions "Facts" unless the user confirmed or corrected them.
- Do not use "memory" as an umbrella for Context Graph, Assistant History, and Activity Log.
- Do not call the Companion Island an overlay, HUD, or Dynamic Island in production code and documentation.
- Keep Assistant Sessions distinct from Meetings.
- Keep Projects distinct from Workspaces and repositories.
- Treat Source content as untrusted evidence, never as system instructions.
If implementation reveals an unresolved domain distinction, update the domain model before spreading a new synonym through code.
## Local-first requirements
- Raw screen frames, audio, OCR, and complete transcripts are ephemeral.
- Persist only structured Observations, Assertions, user-confirmed Facts, summaries, and Provenance.
- Do not queue raw content when a processor is unavailable; record a Gap.
- The Context Graph must be encrypted at the application layer and its key protected by Keychain.
- Provider credentials use separate Keychain entries.
- Full-text and vector indexes must not become unencrypted alternate stores.
- A Local Provider must use loopback or a Unix socket. Treat LAN or internet endpoints as external.
- The MVP must not send machine context to a Cloud Provider.
- Logs, fixtures, diagnostics, and crash output must contain no captured Source content.
- Pause All stops every Collector and remains paused across restarts.
Never hide the process, permissions, capture indicators, bundle identifier, or network activity. Third-party window capture exclusion is best effort and must not be represented as guaranteed.
## Provider and agent boundaries
The MVP uses one active OpenAI-compatible Local Provider profile and a separate local ASR sidecar. Mastermind owns its multilingual embedding component.
Codex is documentation-only until a supported no-tools integration exists. Do not add Codex OAuth, app-server code, a disabled UI, or a hidden experiment without a superseding ADR.
The assistant may emit an Action Proposal as text. Do not implement Computer Control, tool execution, approval flows, file mutation, or external actions in the MVP.
## UI requirements
- At rest, only the stable Menu Bar Item is visible.
- Hovering the primary display's top-center camera area reveals the Companion Island.
- Clicking expands it and focuses text input.
- Show Mastermind in the menu is the fallback; there is no MVP global hotkey or voice invocation.
- Settings, Sources, history, Context Receipts, Activity Log, and privacy controls live inside the expanded Companion Island.
- Respect Reduce Motion and keyboard navigation.
- Exclude Mastermind windows from its own Screen Collector.
## Testing
For Swift package work:
```bash
swift test
swift build
```
## Style
For the current capability proof:
Run `npx prettier --write .` before committing. Prettier uses the settings in
`.prettierrc` (four-space indentation, print width 150, semicolons and single
quotes). `src/assets` and `node_modules` are ignored via `.prettierignore`.
The project does not provide linting; `npm run lint` simply prints
"No linting configured".
```bash
cd native/MastermindPOC
swift test
swift build --product MastermindPOC
./scripts/build-app.sh
```
## Code standards
For explicitly requested legacy Electron maintenance:
Development is gradually migrating toward a TypeScript/React codebase inspired by the
[transcriber](https://github.com/Gatecrashah/transcriber) project. Keep the following
rules in mind as new files are created:
```bash
npm test
```
- **TypeScript strict mode** – avoid `any` and prefer explicit interfaces.
- **React components** should be functional with hooks and wrapped in error
boundaries where appropriate.
- **Secure IPC** – validate and sanitize all parameters crossing the renderer/main
boundary.
- **Non‑blocking audio** – heavy processing must stay off the UI thread.
- **Tests** – every new feature requires tests once the test suite is available.
Manual verification is required for Screen Recording, microphone, system audio, window exclusion, primary-display changes, Launch at Login, sleep/wake, Low Power Mode, and persisted Pause All.
## Shadcn and Electron
Do not claim the production MVP complete until the scenarios in `docs/product/mvp-acceptance.md` pass.
The interface is being rebuilt with [shadcn/ui](https://ui.shadcn.com) components.
Follow these guidelines when working on UI code:
## Formatting
- **Component directory** – place generated files under `src/components/ui` and export them from that folder.
- **Add components with the CLI** – run `npx shadcn@latest add <component>`; never hand-roll components.
- **Component pattern** – use `React.forwardRef` with the `cn()` helper for class names.
- **Path aliases** – import modules from `src` using the `@/` prefix.
- **React 19 + Compiler** – target React 19 with the new compiler when available.
- **Context isolation** – maintain Electron's context isolation pattern for IPC.
- **TypeScript strict mode** – run `npm run typecheck` before claiming work complete.
- **Tailwind theming** – rely on CSS variables and utilities in `@/utils/tailwind` for styling.
- **Testing without running** – confirm `npm run typecheck` and module resolution with `node -e "require('<file>')"`.
- Use the repository's Swift formatter configuration when one exists; otherwise follow standard Swift API Design Guidelines and existing native code style.
- Use four-space indentation in JavaScript and Markdown examples where indentation is semantic.
- Use Prettier only for files it supports; do not reformat generated assets or unrelated legacy code.
- Keep ADRs short and record only decisions that are hard to reverse, surprising without context, and based on a real trade-off.
## Tests
## Upstream changes
No automated tests yet. When a suite is added, run `npm test` before each
commit. Until then, at minimum ensure `npm install` and `npm start` work after
merging upstream changes.
This remains a fork of [`sohzm/cheating-daddy`](https://github.com/sohzm/cheating-daddy), but upstream Electron changes are not automatically product direction.
## Merging upstream PRs
Before cherry-picking upstream work:
Pull requests from <https://github.com/sohzm/cheating-daddy> are commonly
cherry‑picked here. When merging:
1. Inspect the diff and keep commit messages short (`feat:` / `fix:` etc.).
2. After merging, run the application locally to verify it still builds and
functions.
## Strategy and Future Work
We plan to extend this project with ideas from the
[`transcriber`](https://github.com/Gatecrashah/transcriber) project which also
uses Electron. Key goals are:
- **Local Transcription** – integrate `whisper.cpp` to allow offline speech-to-
text. Investigate the architecture used in `transcriber/src/main` for model
validation and GPU acceleration.
- **Dual Audio Capture** – capture microphone and system audio simultaneously.
`transcriber` shows one approach using a native helper for macOS and
Electron's `getDisplayMedia` for other platforms.
- **Speaker Diarization** – explore tinydiarize for identifying speakers in mono
audio streams.
- **Voice Activity Detection** – skip silent or low‑quality segments before
sending to the AI service.
- **Improved Note Handling** – store transcriptions locally and associate them
with meeting notes, similar to `transcriber`'s note management system.
- **Testing Infrastructure** – adopt Jest and React Testing Library (if React is
introduced) to cover audio capture and transcription modules.
### TODO
1. Research and prototype local transcription using `whisper.cpp`.
2. Add dual‑stream audio capture logic for cross‑platform support.
3. Investigate speaker diarization options and integrate when feasible.
4. Plan a migration path toward a proper testing setup (Jest or similar).
5. Document security considerations for audio storage and processing.
6. Rebuild the entire UI using shadcn components.
These plans are aspirational; implement them gradually while keeping the app
functional.
## Audio processing principles
When implementing transcription features borrow the following rules from
`transcriber`:
- **16 kHz compatibility** – resample all audio before sending to whisper.cpp.
- **Dual‑stream architecture** – capture microphone and system audio on separate
channels.
- **Speaker diarization** – integrate tinydiarize (`--tinydiarize` flag) for mono
audio and parse `[SPEAKER_TURN]` markers to label speakers (Speaker A, B, C…).
- **Voice activity detection** – pre‑filter silent segments to improve speed.
- **Quality preservation** – keep sample fidelity and avoid blocking the UI
during heavy processing.
- **Memory efficiency** – stream large audio files instead of loading them all at
once.
- **Error recovery** – handle audio device failures gracefully.
## Privacy by design
- **Local processing** – transcriptions should happen locally whenever possible.
- **User control** – provide clear options for data retention and deletion.
- **Transparency** – document what is stored and where.
- **Minimal data** – only persist what is required for functionality.
## LLM plans
There are placeholder files for future LLM integration (e.g. Qwen models via
`llama.cpp`). Continue development after the core transcription pipeline is
stable and ensure tests cover this new functionality.
1. Inspect whether it serves legacy maintenance or the native product.
2. Keep only reusable protocol, test, or migration value.
3. Reject stealth, anti-detection, insecure IPC, and cloud-by-default behavior.
4. Run the relevant Swift and legacy tests.