Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: Session Sync & Management
Overview
The syncThe process of discovering new/changed session files and updating the database engine discovers sessionA single conversation or interaction with an AI coding agent, typically one run/launch files from all configured AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) directories on disk, parses them using agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific parsers, sanitizes the extracted data, and writes it into the SQLiteEmbedded relational database used as the primary session archive database. It provides incremental and full resync modes, a file watcher for live updates, and supports remote syncSyncing sessions from other machines via SSH or HTTP from other machines via SSHSecure Shell and S3Simple Storage Service (Amazon S3-compatible)-compatible storage.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Sessions are automatically discovered and indexed without manual intervention |
| Operator | Can trigger resyncs and monitor syncThe process of discovering new/changed session files and updating the database status |
| Developer | Extensible provider system for adding new agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) parsers |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall discover sessionA single conversation or interaction with an AI coding agent, typically one run/launch files from all configured agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) directories on startup | Must | The system shall discover sessionA single conversation or interaction with an AI coding agent, typically one run/launch files from all configured agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) directories on startup |
| FR-2MustThe system shall parse sessionA single conversation or interaction with an AI coding agent, typically one run/launch files using agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific parsers to extract messages, tool calls, and usage data | Must | The system shall parse sessionA single conversation or interaction with an AI coding agent, typically one run/launch files using agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific parsers to extract messages, tool calls, and usage data |
| FR-3MustThe system shall sanitize parsed data (clamp tokens, blank timestamps, coerce roles, strip control characters) | Must | The system shall sanitize parsed data (clamp tokens, blank timestamps, coerce roles, strip control characters) |
| FR-4MustThe system shall write parsed sessions to SQLiteEmbedded relational database used as the primary session archive in batches of 100 with 8 parallel workers | Must | The system shall write parsed sessions to SQLiteEmbedded relational database used as the primary session archive in batches of 100 with 8 parallel workers |
| FR-5MustThe system shall support incremental syncThe process of discovering new/changed session files and updating the database (only changed files) and full resync | Must | The system shall support incremental syncThe process of discovering new/changed session files and updating the database (only changed files) and full resync |
| FR-6MustThe system shall provide a file watcher using fsnotifyGo library for filesystem event notification with configurable debounce | Must | The system shall provide a file watcher using fsnotifyGo library for filesystem event notification with configurable debounce |
| FR-7MustThe system shall run a periodic full scan every 15 minutes | Must | The system shall run a periodic full scan every 15 minutes |
| FR-8MustThe system shall track failed/unparseable files by path and mtime via a skip cache | Must | The system shall track failed/unparseable files by path and mtime via a skip cache |
| FR-9ShouldThe system shall support syncing sessions from remote machines via SSHSecure Shell | Should | The system shall support syncing sessions from remote machines via SSHSecure Shell |
| FR-10ShouldThe system shall support S3Simple Storage Service (Amazon S3-compatible)-compatible object storage as a sessionA single conversation or interaction with an AI coding agent, typically one run/launch source | Should | The system shall support S3Simple Storage Service (Amazon S3-compatible)-compatible object storage as a sessionA single conversation or interaction with an AI coding agent, typically one run/launch source |
| FR-11ShouldThe system shall emit SSEServer-Sent Events notifications after each write pass for live UI updates | Should | The system shall emit SSEServer-Sent Events notifications after each write pass for live UI updates |
| FR-12ShouldThe system shall auto-start the daemon on demand for commands that need fresh data | Should | The system shall auto-start the daemon on demand for commands that need fresh data |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustSync should process at least 100 sessions per second | Must | Performance | Sync should process at least 100 sessions per second |
| NFR-2MustFailed parses must not crash the syncThe process of discovering new/changed session files and updating the database engine; errors are tracked and skipped | Must | Reliability | Failed parses must not crash the syncThe process of discovering new/changed session files and updating the database engine; errors are tracked and skipped |
Acceptance Criteria
- FR-1MustThe system shall discover sessionA single conversation or interaction with an AI coding agent, typically one run/launch files from all configured agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) directories on startup
- Given agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) sessionA single conversation or interaction with an AI coding agent, typically one run/launch files exist on disk
- When the server starts
- Then all sessionA single conversation or interaction with an AI coding agent, typically one run/launch files are discovered and queued for parsing
- FR-2MustThe system shall parse sessionA single conversation or interaction with an AI coding agent, typically one run/launch files using agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific parsers to extract messages, tool calls, and usage data
- Given a sessionA single conversation or interaction with an AI coding agent, typically one run/launch file for a supported agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)
- When the syncThe process of discovering new/changed session files and updating the database engine processes it
- Then messages, tool calls, and usage data are extracted
- FR-5MustThe system shall support incremental syncThe process of discovering new/changed session files and updating the database (only changed files) and full resync
- Given a previously synced sessionA single conversation or interaction with an AI coding agent, typically one run/launch file is modified
- When the file watcher detects the change
- Then only that sessionA single conversation or interaction with an AI coding agent, typically one run/launch is re-parsed and updated
- FR-6MustThe system shall provide a file watcher using fsnotifyGo library for filesystem event notification with configurable debounce
- Given the server is running
- When a new sessionA single conversation or interaction with an AI coding agent, typically one run/launch file is created
- Then it is detected and synced within the debounce window
Open Questions
- What are the performance characteristics of S3Simple Storage Service (Amazon S3-compatible)-based sessionA single conversation or interaction with an AI coding agent, typically one run/launch discovery at scale?
Specification: Session Sync & Management
Overview
The syncThe process of discovering new/changed session files and updating the database engine (internal/syncThe process of discovering new/changed session files and updating the database/Engine) orchestrates sessionA single conversation or interaction with an AI coding agent, typically one run/launch discovery, parsing, and database writing. Discovery uses provider factories or legacy path resolution. Parsing delegates to per-agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) parsers in internal/parserCode that reads an agent-specific session file format and extracts structured data/. Writing batches sessions and messages into SQLiteEmbedded relational database used as the primary session archive with incremental update support.
Architecture
Agent Files (disk) → Provider Factory / Legacy Resolver
↓
Agent-specific Parser
↓
Sanitizer
↓
Batch Writer → SQLiteEmbedded relational database used as the primary session archive DB
↓
SSEServer-Sent Events Emitter → UI Clients
Data Models
ParsedSession
| Field | Type | Description |
|---|---|---|
| ID | string | Unique sessionA single conversation or interaction with an AI coding agent, typically one run/launch identifier |
| AgentID | AgentType | Agent that created the sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| Project | string | Project name |
| StartTime | time.Time | Session start timestamp |
| Messages | []ParsedMessage | Messages in the sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| ToolCalls | []ParsedToolCall | Tool invocations |
| UsageEvents | []ParsedUsageEvent | Token usage data |
SkipCacheEntry
| Field | Type | Description |
|---|---|---|
| Path | string | File path |
| ModTime | time.Time | Last modification time |
| Error | string | Parse error description |
APIApplication Programming Interface Contracts
POST /api/v1/syncThe process of discovering new/changed session files and updating the database
Request: Empty Response: SSEServer-Sent Events stream of syncThe process of discovering new/changed session files and updating the database progress events
POST /api/v1/resync
Request: Empty Response: SSEServer-Sent Events stream of full resync progress events
GET /api/v1/syncThe process of discovering new/changed session files and updating the database/status
Response: JSON with syncThe process of discovering new/changed session files and updating the database state, last syncThe process of discovering new/changed session files and updating the database time, pending count
Sequences
File Change Detection
fsnotifyGo library for filesystem event notification → Debounce (500ms) → Parse Changed Files → Batch Write → SSEServer-Sent Events Notify
Periodic Scan
Timer (15min) → Discover All Paths → Diff Against Known → Parse New/Changed → Write
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Batch size | 100 sessions, 8 workers | Balances throughput and memory |
| Debounce | 500ms | Prevents thundering herd on bulk file changes |
| Skip cache | File path + mtime | Avoids re-parsing unchanged files |
| Provider factory | Per-agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) pluggable discovery | Supports diverse agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) directory structures |
Risks and Unknowns
- S3Simple Storage Service (Amazon S3-compatible)-backed discovery latency with many objects
- File watcher scalability on directories with thousands of files
Test Plan: Session Sync & Management
Scope
Tests cover sessionA single conversation or interaction with an AI coding agent, typically one run/launch file discovery, agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific parsing, sanitization, database writing, incremental and full resync, file watcher, periodic scanning, skip cache, remote syncSyncing sessions from other machines via SSH or HTTP via SSHSecure Shell, S3Simple Storage Service (Amazon S3-compatible)-based discovery, and SSEServer-Sent Events notification.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Engine discovers sessionA single conversation or interaction with an AI coding agent, typically one run/launch files from configured directories | Provider config with sessionA single conversation or interaction with an AI coding agent, typically one run/launch files | All files discovered and queued |
| TC-2 | Parser extracts messages, tool calls, usage from sessionA single conversation or interaction with an AI coding agent, typically one run/launch files | Agent-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file | Structured ParsedSession output |
| TC-3 | Sanitizer clamps tokens, blanks timestamps, coerces roles | ParsedSession with edge case values | Sanitized ParsedSession |
| TC-4 | Skip cache tracks failed files by path and mtime | Failed parse result | Entry added to skip cache |
| TC-5 | File watcher detects new/changed files via fsnotifyGo library for filesystem event notification | fsnotifyGo library for filesystem event notification event stream | Changed files dispatched to parserCode that reads an agent-specific session file format and extracts structured data |
| TC-6 | CWDCurrent Working Directory, used for sync path filtering filter correctly scopes syncThe process of discovering new/changed session files and updating the database to working directory | CWDCurrent Working Directory, used for sync path filtering config + sessionA single conversation or interaction with an AI coding agent, typically one run/launch paths | Only matching paths synced |
| TC-7 | Periodic scan discovers new files after startup | File added between scans | File discovered on next scan |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-8 | Full syncThe process of discovering new/changed session files and updating the database engine integration with real directory | Session files on disk | Sessions parsed and written to DB |
| TC-9 | Incremental resync updates changed sessions | Previously synced sessionA single conversation or interaction with an AI coding agent, typically one run/launch modified | Session re-parsed and updated |
| TC-10 | Remote syncThe process of discovering new/changed session files and updating the database via SSHSecure Shell | SSHSecure Shell host configured | Remote sessions synced locally |
| TC-11 | S3Simple Storage Service (Amazon S3-compatible)-compatible storage as sessionA single conversation or interaction with an AI coding agent, typically one run/launch source | S3Simple Storage Service (Amazon S3-compatible) bucket configured | Sessions discovered and synced from S3Simple Storage Service (Amazon S3-compatible) |
Test Files
internal/syncThe process of discovering new/changed session files and updating the database/engine_test.go- Engine discovery, syncThe process of discovering new/changed session files and updating the database, and lifecycle testsinternal/syncThe process of discovering new/changed session files and updating the database/engine_integration_test.go- Integration tests with real directoriesinternal/syncThe process of discovering new/changed session files and updating the database/watcher_test.go- File watcher behavior testsinternal/syncThe process of discovering new/changed session files and updating the database/hash_test.go- File hashing for change detectioninternal/syncThe process of discovering new/changed session files and updating the database/parsediff_*_test.go- Parse diff comparison testsinternal/syncThe process of discovering new/changed session files and updating the database/s3_test.go- S3Simple Storage Service (Amazon S3-compatible)-based discovery testsinternal/syncThe process of discovering new/changed session files and updating the database/secret_*_test.go- Secret scanning integration during syncThe process of discovering new/changed session files and updating the databaseinternal/parserCode that reads an agent-specific session file format and extracts structured data/*_test.go- Per-agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) parserCode that reads an agent-specific session file format and extracts structured data tests (50+ agents)internal/parserCode that reads an agent-specific session file format and extracts structured data/provider_test.go- Provider factory testsinternal/parserCode that reads an agent-specific session file format and extracts structured data/discovery_test.go- Discovery path resolution testsinternal/syncThe process of discovering new/changed session files and updating the database/provider_process_test.go- Provider processing lifecycleinternal/syncThe process of discovering new/changed session files and updating the database/cwd_filter_test.go- CWDCurrent Working Directory, used for sync path filtering filter tests
Edge Cases and Failure Scenarios
| ID | Scenario | Expected Behavior |
|---|---|---|
| TC-12 | Malformed sessionA single conversation or interaction with an AI coding agent, typically one run/launch file | Parsing error tracked, syncThe process of discovering new/changed session files and updating the database continues |
| TC-13 | File removed during syncThe process of discovering new/changed session files and updating the database | Graceful skip, no crash |
| TC-14 | Directory with thousands of files | File watcher handles volume |
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall discover sessionA single conversation or interaction with an AI coding agent, typically one run/launch files from all configured agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) directories on startup | TC-1, TC-8 |
| FR-2MustThe system shall parse sessionA single conversation or interaction with an AI coding agent, typically one run/launch files using agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific parsers to extract messages, tool calls, and usage data | TC-2 |
| FR-3MustThe system shall sanitize parsed data (clamp tokens, blank timestamps, coerce roles, strip control characters) | TC-3 |
| FR-4MustThe system shall write parsed sessions to SQLiteEmbedded relational database used as the primary session archive in batches of 100 with 8 parallel workers | TC-8 |
| FR-5MustThe system shall support incremental syncThe process of discovering new/changed session files and updating the database (only changed files) and full resync | TC-9 |
| FR-6MustThe system shall provide a file watcher using fsnotifyGo library for filesystem event notification with configurable debounce | TC-5 |
| FR-7MustThe system shall run a periodic full scan every 15 minutes | TC-7 |
| FR-8MustThe system shall track failed/unparseable files by path and mtime via a skip cache | TC-4 |
| FR-9ShouldThe system shall support syncing sessions from remote machines via SSHSecure Shell | TC-10 |
| FR-10ShouldThe system shall support S3Simple Storage Service (Amazon S3-compatible)-compatible object storage as a sessionA single conversation or interaction with an AI coding agent, typically one run/launch source | TC-11 |
requirements
- What are the performance characteristics of S3-based session discovery at scale?
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: Recall System
Overview
The recall system extracts reusable facts from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history, ranks them by relevance, and assembles context for AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) prompts. It distills key information such as technical decisions, projectA named group of sessions, typically corresponding to a code repository conventions, environment setup, and solution patterns from past sessions into structured recall entries that can be queried and surfaced.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Reuse past learnings and avoid repeating solved problems |
| AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) user | Provide context to AI coding assistants from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history |
| Power user | Manually curate and review recall entries |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall extract structured recall entries from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages | Must | The system shall extract structured recall entries from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages |
| FR-2MustThe system shall rank recall entries by relevance to a given query | Must | The system shall rank recall entries by relevance to a given query |
| FR-3MustThe system shall assemble recall entries into a formatted context block | Must | The system shall assemble recall entries into a formatted context block |
| FR-4MustThe system shall support querying recall entries via the APIApplication Programming Interface | Must | The system shall support querying recall entries via the APIApplication Programming Interface |
| FR-5ShouldThe system shall support evidence tracking linking recall entries to source messages | Should | The system shall support evidence tracking linking recall entries to source messages |
| FR-6ShouldThe system shall support distillation configurations for extraction | Should | The system shall support distillation configurations for extraction |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustRecall queries should complete in under 1 second | Must | Performance | Recall queries should complete in under 1 second |
Acceptance Criteria
- FR-1MustThe system shall extract structured recall entries from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages
- Given sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages exist
- When recall extraction is run
- Then structured recall entries are created with source evidence
- FR-2MustThe system shall rank recall entries by relevance to a given query
- Given a query string
- When recall ranking is performed
- Then results are ordered by relevance score
Open Questions
- What is the optimal number of recall entries to include in context assembly?
Specification: Recall System
Overview
The recall system is implemented across internal/recall/ and internal/db/recall*.go. It extracts facts from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, stores them in SQLiteEmbedded relational database used as the primary session archive with evidence links to source messages, ranks them by query relevance using embeddingA vector representation of text used for semantic search similarity and keyword matching, and assembles formatted context blocks.
Architecture
Session Messages
→ Extraction (distill facts into recall_entries)
→ Storage (SQLiteEmbedded relational database used as the primary session archive: recall_entries, recall_evidence)
→ Ranking (query → ranked results)
→ Context Assembly (formatted prompt block)
Data Models
recall_entries
| Field | Type | Description |
|---|---|---|
| id | TEXT | Entry identifier |
| session_id | TEXT | Source sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| content | TEXT | Distilled fact content |
| type | TEXT | Entry type (decision, convention, pattern, etc.) |
| created_at | TEXT | Creation timestamp |
recall_evidence
| Field | Type | Description |
|---|---|---|
| entry_id | TEXT | Recall entry reference |
| message_id | TEXT | Source messageAn individual turn in a session (user prompt, assistant response, tool result) reference |
| relevance | REAL | Evidence relevance score |
APIApplication Programming Interface Contracts
GET /api/v1/recall/search
Parameters:
- q - query string
- limit - max results
Response: Ranked list of recall entries with evidence
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Storage | SQLiteEmbedded relational database used as the primary session archive tables | Co-located with sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| Ranking | Embedding + keyword hybrid | Balances semantic and exact matching |
Test Plan: Recall System
Scope
Tests cover the ranking algorithm, type definitions, and extraction logic. Integration tests verify SQLiteEmbedded relational database used as the primary session archive persistence and query functionality.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Rank entries by query relevance | Recall entries + query | Ordered by score descending |
| TC-2 | Type definitions validate correctly | Valid/invalid entry types | Correct validation behavior |
Test Files
internal/recall/rank_test.go- Ranking algorithm testsinternal/recall/rank_internal_test.go- Internal ranking helper testsinternal/recall/types_test.go- Type definition tests
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-2MustThe system shall rank recall entries by relevance to a given query | TC-1 |
| FR-4MustThe system shall support querying recall entries via the APIApplication Programming Interface | TC-1 |
requirements
- What is the optimal number of recall entries to include in context assembly?
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: Desktop App
Overview
The desktop app wraps the agentsview web server in a native application using TauriDesktop application framework wrapping web UIs, providing a native title bar, system tray integration, auto-start, and auto-update. It targets macOS (DMGApple Disk Image) and Windows, with the Go binary bundled as a sidecar process.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Install and run agentsview as a native desktop application |
| macOS user | Prefer DMGApple Disk Image installation with automatic updates |
| Windows user | Native Windows experience |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall provide a TauriDesktop application framework wrapping web UIs-based desktop wrapper for agentsview | Must | The system shall provide a TauriDesktop application framework wrapping web UIs-based desktop wrapper for agentsview |
| FR-2MustThe system shall bundle the Go server binary as a TauriDesktop application framework wrapping web UIs sidecar | Must | The system shall bundle the Go server binary as a TauriDesktop application framework wrapping web UIs sidecar |
| FR-3MustThe system shall support auto-start on login | Must | The system shall support auto-start on login |
| FR-4MustThe system shall support system tray integration with status icon | Must | The system shall support system tray integration with status icon |
| FR-5MustThe system shall support auto-update checking and installation | Must | The system shall support auto-update checking and installation |
| FR-6ShouldThe system shall provide a macOS DMGApple Disk Image installer | Should | The system shall provide a macOS DMGApple Disk Image installer |
| FR-7ShouldThe system shall provide a Windows installer | Should | The system shall provide a Windows installer |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustDesktop app should use minimal system resources when idle | Must | Performance | Desktop app should use minimal system resources when idle |
Acceptance Criteria
- FR-1MustThe system shall provide a TauriDesktop application framework wrapping web UIs-based desktop wrapper for agentsview
- Given the desktop app is installed
- When the app is launched
- Then the agentsview web UI opens in the default browser
- FR-3MustThe system shall support auto-start on login
- Given the desktop app is installed
- When the user logs in
- Then agentsview starts automatically
Open Questions
- Should there be a Linux desktop build or is web-only sufficient?
Specification: Desktop App
Overview
The desktop app uses TauriDesktop application framework wrapping web UIs v2 to wrap the agentsview Go web server. The Go binary is built with the desktop build tag and embedded as a TauriDesktop application framework wrapping web UIs sidecar. The app provides a system tray icon, auto-start via platform-specific mechanisms, and auto-update via GitHub Releases.
Architecture
TauriDesktop application framework wrapping web UIs Shell (Rust)
→ Sidecar: agentsview server (Go)
→ System Tray Icon
→ Auto-start (LaunchAgent / Registry)
→ Auto-update (GitHub Releases APIApplication Programming Interface)
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Desktop framework | TauriDesktop application framework wrapping web UIs v2 | Lightweight, cross-platform, Rust-based |
| Server integration | Sidecar process | Keep Go server as single binary |
| Auto-update | GitHub Releases | Reuses existing release infrastructure |
| Build | CI workflows | Separate workflows for macOS and Windows |
Build Configuration
- CI workflows:
.github/workflows/desktop-*.yml - Desktop source:
desktop/ - Build artifacts: DMGApple Disk Image (macOS), NSIS installer (Windows)
Test Plan: Desktop App
Scope
Tests cover the desktop sidecar management and icon handling in the Go binary. Frontend TauriDesktop application framework wrapping web UIs UI tests are covered by E2E test suites.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Desktop icon configuration | Icon settings | Correct icon paths and formats |
| TC-2 | Desktop sidecar management | Sidecar process state | Proper start/stop lifecycle |
Test Files
desktop_icon_test.go- Desktop icon configuration testsdesktop_sidecar_test.go- Sidecar lifecycle tests
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall provide a TauriDesktop application framework wrapping web UIs-based desktop wrapper for agentsview | TC-2 |
| FR-2MustThe system shall bundle the Go server binary as a TauriDesktop application framework wrapping web UIs sidecar | TC-2 |
requirements
- Should there be a Linux desktop build or is web-only sufficient?
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: Full-Text Search
Overview
The full-text search feature indexes all sessionA single conversation or interaction with an AI coding agent, typically one run/launch messageAn individual turn in a session (user prompt, assistant response, tool result) content using SQLiteEmbedded relational database used as the primary session archive FTS5Full-Text Search version 5 and provides fast text-based search across all sessions. It supports multiple query modes (AND, exact phrase, prefix), filtering by agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), projectA named group of sessions, typically corresponding to a code repository, date, and source scope, and returns ranked results with snippet highlights.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Quickly find past sessions by searching for specific terms, code snippets, or error messages |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall index all messageAn individual turn in a session (user prompt, assistant response, tool result) content using SQLiteEmbedded relational database used as the primary session archive FTS5Full-Text Search version 5 | Must | The system shall index all messageAn individual turn in a session (user prompt, assistant response, tool result) content using SQLiteEmbedded relational database used as the primary session archive FTS5Full-Text Search version 5 |
| FR-2MustThe system shall support AND-mode search (all terms must match) | Must | The system shall support AND-mode search (all terms must match) |
| FR-3MustThe system shall support exact phrase matching with double quotes | Must | The system shall support exact phrase matching with double quotes |
| FR-4MustThe system shall support prefix matching (trailing wildcard) | Must | The system shall support prefix matching (trailing wildcard) |
| FR-5MustThe system shall return ranked results with snippet highlights | Must | The system shall return ranked results with snippet highlights |
| FR-6MustThe system shall support filtering by agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), projectA named group of sessions, typically corresponding to a code repository, date range, machine, and git branch | Must | The system shall support filtering by agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), projectA named group of sessions, typically corresponding to a code repository, date range, machine, and git branch |
| FR-7MustThe system shall support cursor-based pagination | Must | The system shall support cursor-based pagination |
| FR-8ShouldThe system shall support regex-based content search as an alternative mode | Should | The system shall support regex-based content search as an alternative mode |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustSearch results should return in under 500ms for typical queries | Must | Performance | Search results should return in under 500ms for typical queries |
| NFR-2ShouldIndex should handle millions of messages | Should | Scalability | Index should handle millions of messages |
Acceptance Criteria
- FR-1MustThe system shall index all messageAn individual turn in a session (user prompt, assistant response, tool result) content using SQLiteEmbedded relational database used as the primary session archive FTS5Full-Text Search version 5
- Given messages are synced
- When a search query is submitted
- Then results include matching messages with snippet highlights
Open Questions
None.
Specification: Full-Text Search
Overview
Search uses SQLiteEmbedded relational database used as the primary session archive FTS5Full-Text Search version 5 virtual tables created during DB migration. The indexed content covers messageAn individual turn in a session (user prompt, assistant response, tool result) body text with configurable tokenization. Queries support AND, exact phrase, and prefix modes. Results include per-hit snippets built from the FTS5Full-Text Search version 5 snippet() function.
Data Models
FTS5Full-Text Search version 5 Virtual Table
| Column | Type | Description |
|---|---|---|
| content | TEXT | Message body text |
| session_id | TEXT | Parent sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | TEXT | Agent type |
| projectA named group of sessions, typically corresponding to a code repository | TEXT | Project name |
APIApplication Programming Interface Contracts
GET /api/v1/search
Query Parameters
| Field | Type | Required | Description |
|---|---|---|---|
| q | string | Yes | Search query |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | string | No | Filter by agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) |
| projectA named group of sessions, typically corresponding to a code repository | string | No | Filter by projectA named group of sessions, typically corresponding to a code repository |
| since | string | No | Start date (RFC3339) |
| until | string | No | End date (RFC3339) |
| limit | int | No | Results per page |
| cursor | string | No | Pagination cursor |
Response
| Field | Type | Description |
|---|---|---|
| results | []SearchResult | Ranked hits with snippets |
| next_cursor | string | Cursor for next page |
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Tokenizer | unicode61 | Good Unicode support for multi-language content |
| Query mode | AND by default | Most intuitive for user search; exact phrase with quotes |
Risks and Unknowns
- FTS5Full-Text Search version 5 index rebuild on version migration
Test Plan: Full-Text Search
Scope
Tests cover FTS5Full-Text Search version 5 index creation, query modes (AND, exact phrase, prefix), snippet generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions), filtering by agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)/projectA named group of sessions, typically corresponding to a code repository/date/machine/branch, cursor-based pagination, and regex search.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | AND-mode search returns only messages matching all terms | Multi-term query | Filtered results |
| TC-2 | Exact phrase search matches quoted terms | Quoted phrase query | Results matching exact phrase |
| TC-3 | Prefix search matches trailing wildcard | Prefix query | Results with matching prefix |
| TC-4 | Snippet generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) includes hit context | Search query + messageAn individual turn in a session (user prompt, assistant response, tool result) body | Snippet with highlighted terms |
| TC-5 | Filter by agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) returns only that agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)'s sessions | Agent filter param | Filtered results |
| TC-6 | Filter by projectA named group of sessions, typically corresponding to a code repository returns only that projectA named group of sessions, typically corresponding to a code repository's sessions | Project filter param | Filtered results |
| TC-7 | Date range filter scopes results correctly | Since/until params | Results only in date range |
| TC-8 | Cursor pagination returns next page of results | Cursor from first page | Next page results |
| TC-9 | Hybrid search merges FTS5Full-Text Search version 5 and semantic results | RRFReciprocal Rank Fusion mode query | Merged ranked results |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-10 | Search across indexed messages | Messages synced to DB | Results returned with snippets |
Test Files
internal/db/search_test.go- Core search query testsinternal/db/search_content_test.go- Content search testsinternal/db/search_content_scope_test.go- Scope filtering testsinternal/db/search_content_chunk_test.go- Chunk-level search testsinternal/db/search_content_bench_test.go- Search performance benchmarksinternal/server/search_test.go- HTTP search handler tests
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall index all messageAn individual turn in a session (user prompt, assistant response, tool result) content using SQLiteEmbedded relational database used as the primary session archive FTS5Full-Text Search version 5 | TC-10 |
| FR-2MustThe system shall support AND-mode search (all terms must match) | TC-1 |
| FR-3MustThe system shall support exact phrase matching with double quotes | TC-2 |
| FR-4MustThe system shall support prefix matching (trailing wildcard) | TC-3 |
| FR-5MustThe system shall return ranked results with snippet highlights | TC-4 |
| FR-6MustThe system shall support filtering by agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), projectA named group of sessions, typically corresponding to a code repository, date range, machine, and git branch | TC-5, TC-6, TC-7 |
| FR-7MustThe system shall support cursor-based pagination | TC-8 |
requirements
None.
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: Semantic Search
Overview
The semantic search feature provides meaning-based search across sessionA single conversation or interaction with an AI coding agent, typically one run/launch content using vector embeddings. An opt-in feature, it indexes sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages via an OpenAI-compatible embeddings endpoint, stores vectors in a separate sqlite-vecSQLite extension for vector similarity search database, and supports pure semantic and hybrid (RRFReciprocal Rank Fusion-merged) search modes.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Find sessions by meaning rather than exact text match |
| Power user | Configure custom embeddingA vector representation of text used for semantic search endpoints (model, dimensions, provider) |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall support building an embeddingA vector representation of text used for semantic search index from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messageAn individual turn in a session (user prompt, assistant response, tool result) content | Must | The system shall support building an embeddingA vector representation of text used for semantic search index from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messageAn individual turn in a session (user prompt, assistant response, tool result) content |
| FR-2MustThe system shall support multiple embeddingA vector representation of text used for semantic search generations (different models/dimensions) | Must | The system shall support multiple embeddingA vector representation of text used for semantic search generations (different models/dimensions) |
| FR-3MustThe system shall support activating and retiring generations | Must | The system shall support activating and retiring generations |
| FR-4MustThe system shall support pure semantic search (query → vector → similarity) | Must | The system shall support pure semantic search (query → vector → similarity) |
| FR-5MustThe system shall support hybrid search (RRFReciprocal Rank Fusion merge of FTS5Full-Text Search version 5 + semantic results) | Must | The system shall support hybrid search (RRFReciprocal Rank Fusion merge of FTS5Full-Text Search version 5 + semantic results) |
| FR-6MustThe system shall support configurable embeddingA vector representation of text used for semantic search endpoints, APIApplication Programming Interface keys, and models | Must | The system shall support configurable embeddingA vector representation of text used for semantic search endpoints, APIApplication Programming Interface keys, and models |
| FR-7ShouldThe system shall support incremental index updates (only new/changed content) | Should | The system shall support incremental index updates (only new/changed content) |
| FR-8ShouldThe system shall provide a background scheduler for automatic index builds | Should | The system shall provide a background scheduler for automatic index builds |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustEmbedding build should batch requests with configurable concurrency | Must | Performance | Embedding build should batch requests with configurable concurrency |
| NFR-2ShouldSemantic search is opt-in and does not affect core syncThe process of discovering new/changed session files and updating the database or FTS5Full-Text Search version 5 search | Should | Isolation | Semantic search is opt-in and does not affect core syncThe process of discovering new/changed session files and updating the database or FTS5Full-Text Search version 5 search |
Acceptance Criteria
- FR-1MustThe system shall support building an embeddingA vector representation of text used for semantic search index from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messageAn individual turn in a session (user prompt, assistant response, tool result) content
- Given messages are synced
- When an embeddings build is triggered
- Then a vector index is created for all messages
- FR-4MustThe system shall support pure semantic search (query → vector → similarity)
- Given an active embeddingA vector representation of text used for semantic search generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions)
- When a semantic search query is submitted
- Then results ranked by cosine similarity are returned
Open Questions
- What are the recommended default embeddingA vector representation of text used for semantic search providers and models?
Specification: Semantic Search
Overview
Semantic search uses a separate SQLiteEmbedded relational database used as the primary session archive database (vectors.db) with the sqlite-vecSQLite extension for vector similarity search extension. A messageAn individual turn in a session (user prompt, assistant response, tool result) mirror table tracks content changes for incremental updates. The encoder client sends content to an OpenAI-compatible embeddings APIApplication Programming Interface with configurable endpoint, model, dimensions, batch size, and concurrency.
Architecture
Main SQLiteEmbedded relational database used as the primary session archive DB → Message Mirror (hash-based refresh)
↓
Embeddings Encoder (HTTP client)
↓
vectors.db (sqlite-vecSQLite extension for vector similarity search)
↓
Query → Encode → Similarity Search
↓
FTS5Full-Text Search version 5 + Semantic → RRFReciprocal Rank Fusion Merge (hybrid mode)
Data Models
vector_messages
| Field | Type | Description |
|---|---|---|
| id | INTEGER | Primary key |
| message_id | TEXT | Source messageAn individual turn in a session (user prompt, assistant response, tool result) ID |
| content_hash | TEXT | Hash of messageAn individual turn in a session (user prompt, assistant response, tool result) content |
| session_id | TEXT | Parent sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| ordinal | INTEGER | Message position in sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| refreshed_at | TEXT | Last refresh timestamp |
embedding_generations
| Field | Type | Description |
|---|---|---|
| id | INTEGER | Primary key |
| model | TEXT | Embedding model name |
| dimensions | INTEGER | Vector dimensions |
| status | TEXT | building, active, retired |
| started_at | TEXT | Build start time |
| completed_at | TEXT | Build completion time |
APIApplication Programming Interface Contracts
POST /api/v1/embeddings/build
Response: SSEServer-Sent Events stream of build progress
GET /api/v1/embeddings/generations
Response: List of generations with status
POST /api/v1/embeddings/generations/{id}/activate
Response: Generation status
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Vector DB | Separate SQLiteEmbedded relational database used as the primary session archive + sqlite-vecSQLite extension for vector similarity search | Avoids coupling with main sessionA single conversation or interaction with an AI coding agent, typically one run/launch DB |
| RRFReciprocal Rank Fusion merge | k=60 constant | Standard RRFReciprocal Rank Fusion parameter for hybrid search |
| Build strategy | Full scan then incremental | Balances completeness with efficiency |
Risks and Unknowns
- Embedding APIApplication Programming Interface cost for large datasets
- sqlite-vecSQLite extension for vector similarity search SQLiteEmbedded relational database used as the primary session archive extension availability on all platforms
Test Plan: Semantic Search
Scope
Tests cover embeddingA vector representation of text used for semantic search index building, generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) management (activation/retirement), pure semantic search, hybrid RRFReciprocal Rank Fusion search, configurable embeddingA vector representation of text used for semantic search endpoints, incremental index updates, and background scheduler.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Build embeddingA vector representation of text used for semantic search index from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages | Message content | Vector index created |
| TC-2 | Activate/retire embeddingA vector representation of text used for semantic search generations | Generation IDs | Correct generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) state |
| TC-3 | Pure semantic search returns similarity-ranked results | Query text | Results ordered by cosine similarity |
| TC-4 | Hybrid search merges FTS5Full-Text Search version 5 and semantic results | Query text | RRFReciprocal Rank Fusion-merged ranked results |
| TC-5 | Embedding encoder batches requests | Multiple messages | Batched APIApplication Programming Interface calls |
| TC-6 | Index staleness detection | Build version vs current data | Correct staleness verdict |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-7 | Full embeddingA vector representation of text used for semantic search build pipeline | Messages synced | Vectors stored and searchable |
| TC-8 | Index repair on corruption | Corrupted vector index | Index rebuilt |
Test Files
internal/vector/build_test.go- Embedding index build testsinternal/vector/search_test.go- Semantic/hybrid search testsinternal/vector/index_test.go- Index management testsinternal/vector/encoder_test.go- Embedding encoder testsinternal/vector/manager_test.go- Generation management testsinternal/vector/repair_test.go- Index repair testsinternal/vector/chunk_test.go- Chunking for embeddingA vector representation of text used for semantic search testsinternal/server/huma_routes_embeddings_test.go- Embedding APIApplication Programming Interface tests
Edge Cases and Failure Scenarios
| ID | Scenario | Expected Behavior |
|---|---|---|
| TC-9 | Embedding APIApplication Programming Interface endpoint unreachable | Build fails gracefully with error |
| TC-10 | Zero messages to index | No-op, no index created |
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall support building an embeddingA vector representation of text used for semantic search index from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messageAn individual turn in a session (user prompt, assistant response, tool result) content | TC-1, TC-7 |
| FR-2MustThe system shall support multiple embeddingA vector representation of text used for semantic search generations (different models/dimensions) | TC-2 |
| FR-3MustThe system shall support activating and retiring generations | TC-2 |
| FR-4MustThe system shall support pure semantic search (query → vector → similarity) | TC-3 |
| FR-5MustThe system shall support hybrid search (RRFReciprocal Rank Fusion merge of FTS5Full-Text Search version 5 + semantic results) | TC-4 |
| FR-6MustThe system shall support configurable embeddingA vector representation of text used for semantic search endpoints, APIApplication Programming Interface keys, and models | TC-5 |
requirements
- What are the recommended default embedding providers and models?
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: Analytics & Cost Tracking
Overview
The analytics and cost tracking feature provides dashboards and reports on sessionA single conversation or interaction with an AI coding agent, typically one run/launch activity, tool usage, token consumption, and associated costs. It includes activity heatmaps, velocity metrics, per-model cost breakdowns, sessionA single conversation or interaction with an AI coding agent, typically one run/launch health signals, and AI-generated insights.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Understand personal productivity patterns and AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) costs |
| Engineering manager | Team-wide usage visibility via PostgreSQL syncThe process of discovering new/changed session files and updating the database and analytics queries |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall compute and display activity heatmaps over configurable time ranges | Must | The system shall compute and display activity heatmaps over configurable time ranges |
| FR-2MustThe system shall track token usage per sessionA single conversation or interaction with an AI coding agent, typically one run/launch and per model | Must | The system shall track token usage per sessionA single conversation or interaction with an AI coding agent, typically one run/launch and per model |
| FR-3MustThe system shall compute cost estimates using a pricing catalog | Must | The system shall compute cost estimates using a pricing catalog |
| FR-4MustThe system shall provide daily cost summaries with per-model breakdowns | Must | The system shall provide daily cost summaries with per-model breakdowns |
| FR-5MustThe system shall compute sessionA single conversation or interaction with an AI coding agent, typically one run/launch health signals (outcome, tool health, context pressure) | Must | The system shall compute sessionA single conversation or interaction with an AI coding agent, typically one run/launch health signals (outcome, tool health, context pressure) |
| FR-6MustThe system shall provide a sessionA single conversation or interaction with an AI coding agent, typically one run/launch stats endpoint with distribution metrics | Must | The system shall provide a sessionA single conversation or interaction with an AI coding agent, typically one run/launch stats endpoint with distribution metrics |
| FR-7MustThe system shall support timezone-aware date bucketing | Must | The system shall support timezone-aware date bucketing |
| FR-8ShouldThe system shall provide AI-generated insights from sessionA single conversation or interaction with an AI coding agent, typically one run/launch data | Should | The system shall provide AI-generated insights from sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| FR-9ShouldThe system shall support automated sessionA single conversation or interaction with an AI coding agent, typically one run/launch classification (human vs automated) | Should | The system shall support automated sessionA single conversation or interaction with an AI coding agent, typically one run/launch classification (human vs automated) |
| FR-10ShouldThe system shall compute velocity metrics (sessions/day, messages/sessionA single conversation or interaction with an AI coding agent, typically one run/launch) | Should | The system shall compute velocity metrics (sessions/day, messages/sessionA single conversation or interaction with an AI coding agent, typically one run/launch) |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustAnalytics queries should complete in under 2 seconds | Must | Performance | Analytics queries should complete in under 2 seconds |
| NFR-2ShouldCost tracking prices are within 5% of actual provider billing | Should | Accuracy | Cost tracking prices are within 5% of actual provider billing |
Acceptance Criteria
- FR-1MustThe system shall compute and display activity heatmaps over configurable time ranges
- Given sessionA single conversation or interaction with an AI coding agent, typically one run/launch data exists
- When the analytics page is loaded
- Then activity heatmaps are displayed for the configured time range
- FR-4MustThe system shall provide daily cost summaries with per-model breakdowns
- Given sessions have token usage data
- When
agentsview usage dailyis run - Then a cost summary with model breakdown is printed
Open Questions
- How should prompt caching costs be modeled for accurate pricing?
Specification: Analytics & Cost Tracking
Overview
Analytics queries are implemented as SQL aggregations over the sessions and messages tables. Cost tracking uses a model pricing catalog with LiteLLMOpen-source LLM pricing catalog used for cost estimation integration for dynamic pricing and an offline fallback. Health signals are computed post-syncThe process of discovering new/changed session files and updating the database using heuristic rules on sessionA single conversation or interaction with an AI coding agent, typically one run/launch content.
Architecture
SQLiteEmbedded relational database used as the primary session archive DB → Analytics Queries → RESTRepresentational State Transfer APIApplication Programming Interface → Frontend Charts
↓
Pricing Catalog (LiteLLMOpen-source LLM pricing catalog used for cost estimation + offline)
↓
Cost Calculator (cache-aware)
↓
Usage/Tracking APIApplication Programming Interface
APIApplication Programming Interface Contracts
GET /api/v1/analytics/summary
Response: Summary statistics (sessions, messages, tokens, costs)
GET /api/v1/analytics/activity
Query Parameters: since, until, granularity, agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), projectA named group of sessions, typically corresponding to a code repository
Response: Activity time series
GET /api/v1/analytics/heatmap
Response: Heatmap data for the web UI
GET /api/v1/usage/summary
Response: Per-model cost breakdown, total spend, token counts
Data Models
Usage Stats
| Field | Type | Description |
|---|---|---|
| session_id | TEXT | Session identifier |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | TEXT | Agent type |
| model | TEXT | Model name |
| input_tokens | INTEGER | Input token count |
| output_tokens | INTEGER | Output token count |
| cache_creation_tokens | INTEGER | Cache creation tokens |
| cache_read_tokens | INTEGER | Cache read tokens |
| cost_usd | REAL | Estimated cost in USD |
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Pricing source | LiteLLMOpen-source LLM pricing catalog used for cost estimation catalog + offline fallback | Covers most models; works offline |
| Cost calculation | Cache-aware (creation + read tokens) | Reflects actual billing for providers with prompt caching |
| Timezone | Configurable per-query | Supports global users and teams |
| Session classification | ML classifier in DB | Distinguishes automated vs human-driven sessions |
Risks and Unknowns
- LiteLLMOpen-source LLM pricing catalog used for cost estimation APIApplication Programming Interface dependency for pricing updates
- ML classifier accuracy varies by agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) type
Test Plan: Analytics & Cost Tracking
Scope
Tests cover activity heatmaps, token usage tracking, cost estimation, daily cost summaries, sessionA single conversation or interaction with an AI coding agent, typically one run/launch health signals, sessionA single conversation or interaction with an AI coding agent, typically one run/launch stats, timezone-aware bucketing, AI-generated insights, sessionA single conversation or interaction with an AI coding agent, typically one run/launch classification, and velocity metrics.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Activity heatmap computed over time range | Session data + time range | Heatmap data points |
| TC-2 | Token usage tracked per sessionA single conversation or interaction with an AI coding agent, typically one run/launch and model | Parsed usage events | Correct token counts |
| TC-3 | Cost estimated from pricing catalog | Token counts + model | Cost in USD |
| TC-4 | Daily cost summary with per-model breakdown | Multiple sessions with usage | Grouped cost by day and model |
| TC-5 | Session health signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) computation | Session messages | Signal scores |
| TC-6 | Session stats with distribution metrics | Session data | Statistical distributions |
| TC-7 | Timezone-aware date bucketing | UTC timestamps + timezone | Bucketed by local date |
| TC-8 | Pricing match from LiteLLMOpen-source LLM pricing catalog used for cost estimation catalog | Model name + usage | Matched pricing |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-9 | Activity query integration | Sessions with activity data | Correct activity metrics |
| TC-10 | Usage rollup integration | Sessions with token data | Accurate cost rollups |
Test Files
internal/activity/activity_test.go- Activity aggregation testsinternal/activity/query_test.go- Activity query testsinternal/activity/sessions_test.go- Session activity testsinternal/activity/usage_test.go- Usage activity testsinternal/db/usage_test.go- Usage database queriesinternal/db/activity_test.go- Activity database queriesinternal/db/activityreport_test.go- Activity report testsinternal/db/analytics_test.go- Analytics query testsinternal/db/pricing_test.go- Pricing match testsinternal/db/stats_test.go- Stats query testsinternal/db/trends_test.go- Trends query testsinternal/db/usage_perf_test.go- Usage performance testsinternal/pricing/litellm_test.go- LiteLLMOpen-source LLM pricing catalog used for cost estimation pricing testsinternal/server/activity_test.go- HTTP activity handler testsinternal/server/activity_report_test.go- Activity report handler testsinternal/server/analytics_test.go- Analytics handler testsinternal/server/usage_test.go- Usage handler testsinternal/service/usage_test.go- Service layer usage tests
Edge Cases and Failure Scenarios
| ID | Scenario | Expected Behavior |
|---|---|---|
| TC-11 | No token data for sessionA single conversation or interaction with an AI coding agent, typically one run/launch | Cost shows as unavailable |
| TC-12 | Model not in pricing catalog | Unpriced model flagged |
| TC-13 | Empty sessionA single conversation or interaction with an AI coding agent, typically one run/launch data | Zero-filled results |
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall compute and display activity heatmaps over configurable time ranges | TC-1, TC-9 |
| FR-2MustThe system shall track token usage per sessionA single conversation or interaction with an AI coding agent, typically one run/launch and per model | TC-2, TC-10 |
| FR-3MustThe system shall compute cost estimates using a pricing catalog | TC-3, TC-8 |
| FR-4MustThe system shall provide daily cost summaries with per-model breakdowns | TC-4 |
| FR-5MustThe system shall compute sessionA single conversation or interaction with an AI coding agent, typically one run/launch health signals (outcome, tool health, context pressure) | TC-5 |
| FR-6MustThe system shall provide a sessionA single conversation or interaction with an AI coding agent, typically one run/launch stats endpoint with distribution metrics | TC-6 |
| FR-7MustThe system shall support timezone-aware date bucketing | TC-7 |
requirements
- How should prompt caching costs be modeled for accurate pricing?
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: PostgreSQL Sync
Overview
The PostgreSQL syncThe process of discovering new/changed session files and updating the database feature pushes sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from the local SQLiteEmbedded relational database used as the primary session archive archive to a shared PostgreSQL instance, enabling team-wide dashboards and read-only queries. It supports incremental push with fingerprint-based change detection, named targets for multiple PG instances, and read-only serving of the web UI from PostgreSQL.
Stakeholders
| Stakeholder | Interest |
|---|---|
| Engineering manager | Team-wide visibility into agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) usage |
| Team member | Access sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from PostgreSQL-connected tools |
| Operator | Configure multiple PG targets (work, archive) |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall push sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL incrementally | Must | The system shall push sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL incrementally |
| FR-2MustThe system shall support projectA named group of sessions, typically corresponding to a code repository filtering (include/exclude projects) | Must | The system shall support projectA named group of sessions, typically corresponding to a code repository filtering (include/exclude projects) |
| FR-3MustThe system shall use fingerprint-based change detection for incremental pushes | Must | The system shall use fingerprint-based change detection for incremental pushes |
| FR-4MustThe system shall support named PostgreSQL targets (multiple destinations) | Must | The system shall support named PostgreSQL targets (multiple destinations) |
| FR-5MustThe system shall serve the web UI read-only from PostgreSQL | Must | The system shall serve the web UI read-only from PostgreSQL |
| FR-6MustThe system shall support conflict detection based on owning machine | Must | The system shall support conflict detection based on owning machine |
| FR-7ShouldThe system shall provide a watch mode that auto-pushes on sessionA single conversation or interaction with an AI coding agent, typically one run/launch changes | Should | The system shall provide a watch mode that auto-pushes on sessionA single conversation or interaction with an AI coding agent, typically one run/launch changes |
| FR-8ShouldThe system shall support OS service management (systemd/launchd) for auto-push | Should | The system shall support OS service management (systemd/launchd) for auto-push |
| FR-9ShouldThe system shall support pricing syncThe process of discovering new/changed session files and updating the database from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL | Should | The system shall support pricing syncThe process of discovering new/changed session files and updating the database from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustPush must be idempotent; re-running produces the same state | Must | Consistency | Push must be idempotent; re-running produces the same state |
Acceptance Criteria
- FR-1MustThe system shall push sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL incrementally
- Given local SQLiteEmbedded relational database used as the primary session archive has sessionA single conversation or interaction with an AI coding agent, typically one run/launch data
- When
agentsview pg pushis run - Then sessionA single conversation or interaction with an AI coding agent, typically one run/launch data is replicated to PostgreSQL
- FR-7ShouldThe system shall provide a watch mode that auto-pushes on sessionA single conversation or interaction with an AI coding agent, typically one run/launch changes
- Given the watch daemon is running
- When new sessions are synced locally
- Then they are automatically pushed to PostgreSQL
Open Questions
None.
Specification: PostgreSQL Sync
Overview
The PostgreSQL integration has two modes: push syncThe process of discovering new/changed session files and updating the database and read-only serve. Push syncThe process of discovering new/changed session files and updating the database (internal/postgres/push.go) incrementally copies sessions from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL using a fingerprint-based diff. Read-only serve (internal/postgres/store.go) implements db.Store for PostgreSQL-backed queries. Named targets allow multiple PG destinations.
Architecture
SQLiteEmbedded relational database used as the primary session archive DB → Push Engine (fingerprint diff) → PostgreSQL
↓
Watch Daemon (fsnotifyGo library for filesystem event notification events)
↓
OS Service (systemd/launchd)
Data Models
Named PG Target Config
| Field | Type | Description |
|---|---|---|
| name | string | Target name (e.g. "work", "archive") |
| url | string | PostgreSQL DSNData Source Name |
| machine_name | string | Identifier for this machine |
| exclude_projects | []string | Projects to skip |
| schema | string | PG schema name |
APIApplication Programming Interface Contracts
POST /api/v1/push/pg
Request: {"target": "default"}
Response: Push status
Sequences
Incremental Push
1. Query SQLiteEmbedded relational database used as the primary session archive for sessions with fingerprint > last_push_fingerprint
2. Batch insert/update into PostgreSQL
3. Record max fingerprint for next push
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Change detection | Monotonic fingerprint column | O(1) incremental detection |
| Read serve | Implements db.Store interface |
Same APIApplication Programming Interface handlers work for SQLiteEmbedded relational database used as the primary session archive, PG, DuckDB |
| Conflict resolution | Machine-owner wins | Different machines may have different versions of same sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
Risks and Unknowns
- Schema drift between SQLiteEmbedded relational database used as the primary session archive and PostgreSQL on version upgrade
Test Plan: PostgreSQL Sync
Scope
Tests cover incremental push from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL, projectA named group of sessions, typically corresponding to a code repository filtering, fingerprint-based change detection, named targets, read-only serve, conflict detection, watch mode, OS service management, and pricing syncThe process of discovering new/changed session files and updating the database.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Push syncs sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL | SQLiteEmbedded relational database used as the primary session archive sessionA single conversation or interaction with an AI coding agent, typically one run/launch data | PostgreSQL has matching rows |
| TC-2 | Fingerprint change detection identifies new/modified | Session fingerprint state | Correct incremental set |
| TC-3 | Project filtering includes/excludes correctly | Include/exclude lists | Filtered push set |
| TC-4 | Named target resolution correct | Config with named targets | Correct target selected |
| TC-5 | Conflict detection by owning machine | Same sessionA single conversation or interaction with an AI coding agent, typically one run/launch from different machines | Conflict flagged |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-6 | Full push lifecycle | Real PostgreSQL instance | Data replicated correctly |
| TC-7 | Watch mode pushes on file change | Running watch daemon | Auto-push on new sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| TC-8 | Read-only serve from PostgreSQL | PostgreSQL with data | Web UI served from PG |
Test Files
internal/postgres/push_test.go- Push lifecycle testsinternal/postgres/push_pgtest_test.go- Push integration tests (PG)internal/postgres/push_fingerprint_test.go- Fingerprint testsinternal/postgres/push_window_test.go- Push window testsinternal/postgres/sync_test.go- Sync lifecycle testsinternal/postgres/schema_test.go- Schema management testsinternal/postgres/store_test.go- Store query testsinternal/postgres/sessions_test.go- Session query testsinternal/postgres/messages_test.go- Message query testsinternal/postgres/connect_test.go- Connection testsinternal/postgres/analytics_pgtest_test.go- Analytics PG testsinternal/postgres/activityreport_pgtest_test.go- Activity report PG testsinternal/postgres/curation_pgtest_test.go- Curation PG testsinternal/postgres/pricing_pgtest_test.go- Pricing PG tests
Edge Cases and Failure Scenarios
| ID | Scenario | Expected Behavior |
|---|---|---|
| TC-9 | PostgreSQL unreachable | Push fails with connect error |
| TC-10 | Schema version mismatch | Migration applied or error reported |
| TC-11 | Duplicate push (idempotency) | Same state, no duplicates |
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall push sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to PostgreSQL incrementally | TC-1, TC-6 |
| FR-2MustThe system shall support projectA named group of sessions, typically corresponding to a code repository filtering (include/exclude projects) | TC-3 |
| FR-3MustThe system shall use fingerprint-based change detection for incremental pushes | TC-2 |
| FR-4MustThe system shall support named PostgreSQL targets (multiple destinations) | TC-4 |
| FR-5MustThe system shall serve the web UI read-only from PostgreSQL | TC-8 |
| FR-6MustThe system shall support conflict detection based on owning machine | TC-5 |
| FR-7ShouldThe system shall provide a watch mode that auto-pushes on sessionA single conversation or interaction with an AI coding agent, typically one run/launch changes | TC-7 |
requirements
None.
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: DuckDB Mirror
Overview
The DuckDB mirror pushes sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from the local SQLiteEmbedded relational database used as the primary session archive archive to a DuckDB database file, enabling portable analytics and read-only local serving. It also supports the QuackDuckDB's remote protocol for network access to DuckDB files remote protocol for network access to DuckDB, allowing remote querying of the sessionA single conversation or interaction with an AI coding agent, typically one run/launch mirror.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Portable analytics file for data analysis |
| Power user | Remote access to sessionA single conversation or interaction with an AI coding agent, typically one run/launch data via QuackDuckDB's remote protocol for network access to DuckDB files protocol |
| Analyst | SQL-based querying of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data using DuckDB |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall push sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to DuckDB | Must | The system shall push sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to DuckDB |
| FR-2MustThe system shall support incremental push with fingerprint-based change detection | Must | The system shall support incremental push with fingerprint-based change detection |
| FR-3MustThe system shall serve the web UI read-only from DuckDB | Must | The system shall serve the web UI read-only from DuckDB |
| FR-4MustThe system shall expose the DuckDB mirror over the QuackDuckDB's remote protocol for network access to DuckDB files remote protocol | Must | The system shall expose the DuckDB mirror over the QuackDuckDB's remote protocol for network access to DuckDB files remote protocol |
| FR-5MustThe system shall support dual DuckDB drivers (CGOC Go interoperability and pure Go) | Must | The system shall support dual DuckDB drivers (CGOC Go interoperability and pure Go) |
| FR-6ShouldThe system shall support projectA named group of sessions, typically corresponding to a code repository filtering | Should | The system shall support projectA named group of sessions, typically corresponding to a code repository filtering |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustQuackDuckDB's remote protocol for network access to DuckDB files server binds to loopback by default and requires a token | Must | Security | QuackDuckDB's remote protocol for network access to DuckDB files server binds to loopback by default and requires a token |
Acceptance Criteria
- FR-1MustThe system shall push sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to DuckDB
- Given local SQLiteEmbedded relational database used as the primary session archive has sessionA single conversation or interaction with an AI coding agent, typically one run/launch data
- When
agentsview duckdb pushis run - Then sessionA single conversation or interaction with an AI coding agent, typically one run/launch data is replicated to DuckDB
- FR-4MustThe system shall expose the DuckDB mirror over the QuackDuckDB's remote protocol for network access to DuckDB files remote protocol
- Given the QuackDuckDB's remote protocol for network access to DuckDB files server is running
- When a remote client connects with a valid token
- Then the client can query the DuckDB mirror
Open Questions
None.
Specification: DuckDB Mirror
Overview
The DuckDB integration mirrors SQLiteEmbedded relational database used as the primary session archive data to DuckDB for portable analytics. It supports CGOC Go interoperability-based and pure Go (modernc) DuckDB drivers. The QuackDuckDB's remote protocol for network access to DuckDB files protocol exposes the DuckDB file over the network with token-based authentication. Read-only serve implements db.Store for DuckDB-backed queries.
Architecture
SQLiteEmbedded relational database used as the primary session archive DB → Push Engine (fingerprint diff) → DuckDB File
↓
QuackDuckDB's remote protocol for network access to DuckDB files Server (remote protocol)
↓
Remote DuckDB Clients
Data Models
DuckDB Config
| Field | Type | Description |
|---|---|---|
| path | string | Path to DuckDB file |
| quack_bind | string | QuackDuckDB's remote protocol for network access to DuckDB files server bind address |
| quack_token | string | QuackDuckDB's remote protocol for network access to DuckDB files authentication token |
Sequences
Push
1. Query SQLiteEmbedded relational database used as the primary session archive for sessions with fingerprint > last_push_fingerprint
2. Bulk insert into DuckDB using prepared statements
3. Record max fingerprint
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Driver | CGOC Go interoperability + modernc (pure Go) fallback | Platform compatibility |
| Protocol | QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB-native remote protocol |
| Authentication | Bearer token | Simple, secure enough for local network |
Risks and Unknowns
- DuckDB driver compatibility across platforms
- QuackDuckDB's remote protocol for network access to DuckDB files protocol maturity and feature support
Test Plan: DuckDB Mirror
Scope
Tests cover push from SQLiteEmbedded relational database used as the primary session archive to DuckDB, incremental push with fingerprinting, full rebuild, read-only serve, QuackDuckDB's remote protocol for network access to DuckDB files remote protocol, dual driver support (CGOC Go interoperability and pure Go), and projectA named group of sessions, typically corresponding to a code repository filtering.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Push syncs sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to DuckDB | SQLiteEmbedded relational database used as the primary session archive sessionA single conversation or interaction with an AI coding agent, typically one run/launch data | DuckDB has matching rows |
| TC-2 | Incremental push detects changed sessions | Modified sessionA single conversation or interaction with an AI coding agent, typically one run/launch fingerprints | Only changed data pushed |
| TC-3 | Full rebuild creates complete DuckDB mirror | All SQLiteEmbedded relational database used as the primary session archive data | Complete DuckDB mirror |
| TC-4 | Read-only serve returns query results | DuckDB mirror | Query results returned |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-5 | QuackDuckDB's remote protocol for network access to DuckDB files protocol remote query | QuackDuckDB's remote protocol for network access to DuckDB files server running | Remote query returns results |
| TC-6 | Mirror watch detects file changes | Watch daemon running | Auto-push on change |
| TC-7 | Store contract parity | SQLiteEmbedded relational database used as the primary session archive + DuckDB with same data | Identical query results |
Test Files
internal/duckdb/push_bounded_test.go- Bounded push testsinternal/duckdb/sync_test.go- Sync lifecycle testsinternal/duckdb/sync_fastpath_test.go- Fast path syncThe process of discovering new/changed session files and updating the database testsinternal/duckdb/rebuild_test.go- Full rebuild testsinternal/duckdb/probe_test.go- Mirror probe testsinternal/duckdb/smoke_test.go- Smoke testsinternal/duckdb/store_test.go- Store query testsinternal/duckdb/store_contract_test.go- Store contract testsinternal/duckdb/connect_test.go- Connection testsinternal/duckdb/quack_smoke_duckdbtest_test.go- QuackDuckDB's remote protocol for network access to DuckDB files protocol testsinternal/duckdb/quack_sql_test.go- QuackDuckDB's remote protocol for network access to DuckDB files SQL testsinternal/duckdb/quack_url_form_duckdbtest_test.go- QuackDuckDB's remote protocol for network access to DuckDB files URL testsinternal/duckdb/mirror_watch_test.go- Mirror watch tests
Edge Cases and Failure Scenarios
| ID | Scenario | Expected Behavior |
|---|---|---|
| TC-8 | DuckDB file locked | Push waits or reports error |
| TC-9 | Incompatible DuckDB driver version | Error reported |
| TC-10 | QuackDuckDB's remote protocol for network access to DuckDB files connection without token | Connection rejected |
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall push sessionA single conversation or interaction with an AI coding agent, typically one run/launch data from SQLiteEmbedded relational database used as the primary session archive to DuckDB | TC-1, TC-3 |
| FR-2MustThe system shall support incremental push with fingerprint-based change detection | TC-2 |
| FR-3MustThe system shall serve the web UI read-only from DuckDB | TC-4 |
| FR-4MustThe system shall expose the DuckDB mirror over the QuackDuckDB's remote protocol for network access to DuckDB files remote protocol | TC-5 |
| FR-5MustThe system shall support dual DuckDB drivers (CGOC Go interoperability and pure Go) | TC-6 |
requirements
None.
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: Admin & Operations
Overview
The admin and operations features cover secret scanning across sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, remote syncSyncing sessions from other machines via SSH or HTTP from other machines, export/import capabilities, AI-generated insights from sessionA single conversation or interaction with an AI coding agent, typically one run/launch data, and self-update functionality. These features support day-to-day administration and data portability.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Export sessions, import from other tools, scan for leaked secrets |
| Operator | Remote syncThe process of discovering new/changed session files and updating the database from multiple machines, monitor for security issues |
| Security-conscious user | Detect accidentally committed APIApplication Programming Interface keys and tokens in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall scan sessionA single conversation or interaction with an AI coding agent, typically one run/launch content for secrets (APIApplication Programming Interface keys, tokens, passwords) using rule-based detection | Must | The system shall scan sessionA single conversation or interaction with an AI coding agent, typically one run/launch content for secrets (APIApplication Programming Interface keys, tokens, passwords) using rule-based detection |
| FR-2MustThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch export as HTML | Must | The system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch export as HTML |
| FR-3MustThe system shall support publishing sessions to GitHub Gists | Must | The system shall support publishing sessions to GitHub Gists |
| FR-4MustThe system shall support importing from Claude.ai and ChatGPT exports | Must | The system shall support importing from Claude.ai and ChatGPT exports |
| FR-5MustThe system shall support self-update via GitHub release check | Must | The system shall support self-update via GitHub release check |
| FR-6MustThe system shall support remote syncSyncing sessions from other machines via SSH or HTTP from other machines via SSHSecure Shell | Must | The system shall support remote syncSyncing sessions from other machines via SSH or HTTP from other machines via SSHSecure Shell |
| FR-7ShouldThe system shall support AI-generated insights (daily activity summaries, agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) analysis) | Should | The system shall support AI-generated insights (daily activity summaries, agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) analysis) |
| FR-8ShouldThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch health grades (A-F) | Should | The system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch health grades (A-F) |
| FR-9ShouldThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch starring and pinning | Should | The system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch starring and pinning |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustSecrets scan results are stored locally and not sent externally | Must | Security | Secrets scan results are stored locally and not sent externally |
Acceptance Criteria
- FR-1MustThe system shall scan sessionA single conversation or interaction with an AI coding agent, typically one run/launch content for secrets (APIApplication Programming Interface keys, tokens, passwords) using rule-based detection
- Given sessionA single conversation or interaction with an AI coding agent, typically one run/launch content contains an APIApplication Programming Interface key pattern
- When a secret scan is run
- Then the finding is recorded and flagged in the UI
- FR-3MustThe system shall support publishing sessions to GitHub Gists
- Given a sessionA single conversation or interaction with an AI coding agent, typically one run/launch
- When "Publish to Gist" is selected
- Then the sessionA single conversation or interaction with an AI coding agent, typically one run/launch is published as a GitHub Gist
- FR-6MustThe system shall support remote syncSyncing sessions from other machines via SSH or HTTP from other machines via SSHSecure Shell
- Given a remote host is configured
- When
agentsview syncThe process of discovering new/changed session files and updating the database --host <host>is run - Then sessions from the remote host are synced
Open Questions
- What secret patterns should be included in the default ruleset?
Specification: Admin & Operations
Overview
Admin features are implemented across several internal packages: internal/secrets/ for secret scanning, internal/export/ for sessionA single conversation or interaction with an AI coding agent, typically one run/launch export, internal/importer/ for importing from other tools, internal/insightAn AI-generated summary or analysis of session data/ for AI-generated insights, internal/update/ for self-updates, internal/signals/ for health signals, and internal/remotesync/ for remote syncSyncing sessions from other machines via SSH or HTTP.
Architecture
Session Content
├→ Secret Scanner (rule-based patterns)
├→ Exporter (HTML, Gist)
├→ Insight Generator (AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.))
└→ Health Signal Computer (heuristic rules)
Data Models
Secret Finding
| Field | Type | Description |
|---|---|---|
| id | TEXT | Finding identifier |
| session_id | TEXT | Source sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| type | TEXT | Secret type (api_key, token, password) |
| match | TEXT | Redacted matched content |
| line | INTEGER | Line number |
| severity | TEXT | Severity level |
Insight
| Field | Type | Description |
|---|---|---|
| id | TEXT | Insight identifier |
| type | TEXT | daily, agent_analysis, canned |
| title | TEXT | Insight title |
| content | TEXT | Generated insightAn AI-generated summary or analysis of session data body |
| model | TEXT | Model used for generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) |
| created_at | TEXT | Generation timestamp |
APIApplication Programming Interface Contracts
GET /api/v1/secrets
Response: List of secret findings with sessionA single conversation or interaction with an AI coding agent, typically one run/launch context
POST /api/v1/secrets/scan
Response: SSEServer-Sent Events stream of scan progress
POST /api/v1/insights/generate
Response: SSEServer-Sent Events stream of generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) progress
GET /api/v1/sessions/{id}/export
Response: HTML export of the sessionA single conversation or interaction with an AI coding agent, typically one run/launch
Sequences
Secret Scanning
1. Walk all sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages
2. Apply regex patterns for each secret type
3. Redact matched content
4. Store findings in secrets table
5. Notify UI via SSEServer-Sent Events
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| Insight generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | Uses configured agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | Leverages existing LLM infrastructure |
| Secret detection | Rule-based regex | Fast, no ML dependency |
| Export format | HTML with embedded styling | Self-contained, portable |
| Remote syncThe process of discovering new/changed session files and updating the database | SSHSecure Shell transport | Ubiquitous, no additional infrastructure |
Risks and Unknowns
- Insight generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) cost and latency
- False positives in secret scanning
Test Plan: Admin & Operations
Scope
Tests cover secret scanning, sessionA single conversation or interaction with an AI coding agent, typically one run/launch export (HTML, GitHub Gist), Claude.ai/ChatGPT import, self-update, remote syncSyncing sessions from other machines via SSH or HTTP via SSHSecure Shell, AI-generated insights, sessionA single conversation or interaction with an AI coding agent, typically one run/launch health grades, and starring/pinning.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Secret scanning detects APIApplication Programming Interface key patterns | Session content with secrets | Findings recorded with redacted matches |
| TC-2 | Session export produces valid HTML | Session data | HTML output |
| TC-3 | Import from Claude.ai/ChatGPT | Export archive | Sessions imported into SQLiteEmbedded relational database used as the primary session archive |
| TC-4 | Self-update check | Current version + remote | Update available/not available |
| TC-5 | Remote syncThe process of discovering new/changed session files and updating the database via SSHSecure Shell | SSHSecure Shell host config | Sessions synced from remote |
| TC-6 | Secret ruleset version handling | Ruleset version | Correct rule application |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-7 | Full remote syncSyncing sessions from other machines via SSH or HTTP lifecycle | SSHSecure Shell configured | Sessions transferred and indexed |
| TC-8 | Canonical JSON export/import | Session data | Round-trip fidelity |
| TC-9 | Insight generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) from sessionA single conversation or interaction with an AI coding agent, typically one run/launch data | Sessions with messages | Structured insights produced |
Test Files
internal/secrets/secrets_test.go- Secret scanning testsinternal/secrets/rules_test.go- Ruleset testsinternal/secrets/bench_test.go- Scanning performance benchmarksinternal/export/canonical_json_test.go- Canonical JSON export/importinternal/export/pricing_test.go- Pricing export testsinternal/export/project_identity_test.go- Project identity exportinternal/importer/importer_test.go- Import testsinternal/importer/assets_test.go- Import asset testsinternal/importer/zip_test.go- Zip import testsinternal/update/update_test.go- Self-update testsinternal/remotesync/archive_test.go- Remote syncThe process of discovering new/changed session files and updating the database archive testsinternal/remotesync/http_test.go- Remote syncThe process of discovering new/changed session files and updating the database HTTP testsinternal/remotesync/manifest_test.go- Manifest testsinternal/remotesync/mirror_test.go- Mirror testsinternal/remotesync/resolve_test.go- Host resolution testsinternal/remotesync/cleanup_registry_test.go- Cleanup registry testsinternal/remotesync/failure_test.go- Failure handling testsinternal/remotesync/import_test.go- Import testsinternal/remotesync/migration_test.go- Migration testsinternal/ssh/ssh_test.go- SSHSecure Shell connection testsinternal/ssh/classify_test.go- SSHSecure Shell classification testsinternal/ssh/extract_test.go- SSHSecure Shell path extraction testsinternal/ssh/resolve_test.go- SSHSecure Shell resolution testsinternal/ssh/transfer_test.go- SSHSecure Shell transfer testsinternal/insightAn AI-generated summary or analysis of session data/generate_test.go- Insight generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) testsinternal/insightAn AI-generated summary or analysis of session data/prompt_test.go- Insight prompt testsinternal/insightAn AI-generated summary or analysis of session data/summary_test.go- Insight summary testsinternal/db/secret_findings_test.go- Secret findings DB tests
Edge Cases and Failure Scenarios
| ID | Scenario | Expected Behavior |
|---|---|---|
| TC-10 | SSHSecure Shell host unreachable | Sync fails with connect error |
| TC-11 | No new version available | Update reports up-to-date |
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall scan sessionA single conversation or interaction with an AI coding agent, typically one run/launch content for secrets (APIApplication Programming Interface keys, tokens, passwords) using rule-based detection | TC-1, TC-6 |
| FR-2MustThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch export as HTML | TC-2 |
| FR-4MustThe system shall support importing from Claude.ai and ChatGPT exports | TC-3, TC-8 |
| FR-5MustThe system shall support self-update via GitHub release check | TC-4 |
| FR-6MustThe system shall support remote syncSyncing sessions from other machines via SSH or HTTP from other machines via SSHSecure Shell | TC-5, TC-7 |
| FR-7ShouldThe system shall support AI-generated insights (daily activity summaries, agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) analysis) | TC-9 |
requirements
- What secret patterns should be included in the default ruleset?
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: MCPModel Context Protocol Server
Overview
The MCPModel Context Protocol (Model Context Protocol) server exposes agentsview's sessionA single conversation or interaction with an AI coding agent, typically one run/launch data as read-only tools for AI coding assistants. It supports stdio and StreamableHTTP transports, providing tools for searching sessions, listing sessions, viewing sessionA single conversation or interaction with an AI coding agent, typically one run/launch details and messages, content search, and usage summaries.
Stakeholders
| Stakeholder | Interest |
|---|---|
| AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) user | Query sessionA single conversation or interaction with an AI coding agent, typically one run/launch data directly from within an AI coding assistant |
| Developer | Integrate agentsview data into MCPModel Context Protocol-compatible tools |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall provide an MCPModel Context Protocol server with stdio transport | Must | The system shall provide an MCPModel Context Protocol server with stdio transport |
| FR-2MustThe system shall provide an MCPModel Context Protocol server with StreamableHTTP transport | Must | The system shall provide an MCPModel Context Protocol server with StreamableHTTP transport |
| FR-3MustThe system shall expose a `search_sessions` tool for FTS5Full-Text Search version 5 search | Must | The system shall expose a search_sessions tool for FTS5Full-Text Search version 5 search |
| FR-4MustThe system shall expose a `list_sessions` tool for sessionA single conversation or interaction with an AI coding agent, typically one run/launch listing | Must | The system shall expose a list_sessions tool for sessionA single conversation or interaction with an AI coding agent, typically one run/launch listing |
| FR-5MustThe system shall expose a `get_session_overview` tool for sessionA single conversation or interaction with an AI coding agent, typically one run/launch details | Must | The system shall expose a get_session_overview tool for sessionA single conversation or interaction with an AI coding agent, typically one run/launch details |
| FR-6MustThe system shall expose a `get_messages` tool for messageAn individual turn in a session (user prompt, assistant response, tool result) retrieval | Must | The system shall expose a get_messages tool for messageAn individual turn in a session (user prompt, assistant response, tool result) retrieval |
| FR-7MustThe system shall expose a `search_content` tool for content search | Must | The system shall expose a search_content tool for content search |
| FR-8MustThe system shall expose a `get_usage_summary` tool for usage data | Must | The system shall expose a get_usage_summary tool for usage data |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustMCPModel Context Protocol server operates on existing auth; no additional credentials required | Must | Security | MCPModel Context Protocol server operates on existing auth; no additional credentials required |
Acceptance Criteria
- FR-1MustThe system shall provide an MCPModel Context Protocol server with stdio transport
- Given agentsview is running
- When
agentsview mcpis run - Then an MCPModel Context Protocol server starts on stdio, providing the defined tools
Open Questions
None.
Specification: MCPModel Context Protocol Server
Overview
The MCPModel Context Protocol server (internal/mcp/) implements the Model Context Protocol using the Go MCPModel Context Protocol SDK. It registers six read-only tools that wrap the existing sessionA single conversation or interaction with an AI coding agent, typically one run/launch service interface. The server supports two transport modes: stdio for direct subprocess invocation and StreamableHTTP for RESTRepresentational State Transfer-based MCPModel Context Protocol clients.
Architecture
MCPModel Context Protocol Client (AI Assistant) ↔ MCPModel Context Protocol Server (agentsview)
↓
Internal/Service Backend
↓
SQLiteEmbedded relational database used as the primary session archive / PG / DuckDB
APIApplication Programming Interface Contracts
Tools
| Tool | Description | Parameters |
|---|---|---|
| search_sessions | Full-text search across sessions | query, agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), projectA named group of sessions, typically corresponding to a code repository, limit |
| list_sessions | List sessions with filters | agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), projectA named group of sessions, typically corresponding to a code repository, limit, offset |
| get_session_overview | Get sessionA single conversation or interaction with an AI coding agent, typically one run/launch summary | session_id |
| get_messages | Get messages for a sessionA single conversation or interaction with an AI coding agent, typically one run/launch | session_id, limit, offset |
| search_content | Search within messageAn individual turn in a session (user prompt, assistant response, tool result) content | query, session_id, mode |
| get_usage_summary | Get usage/cost summary | since, until, agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) |
Sequences
MCPModel Context Protocol Tool Invocation
Client → JSON-RPC Request → MCPModel Context Protocol Server → Service Backend → Response
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| SDK | Go MCPModel Context Protocol SDK (modelcontextprotocol/go-sdk) | Official SDK, spec-compliant |
| Transports | stdio + StreamableHTTP | Covers local AI tool integration and remote clients |
| Architecture | Thin wrapper over service layer | Reuses existing query logic, no data duplication |
Risks and Unknowns
- MCPModel Context Protocol protocol version compatibility as the spec evolves
Test Plan: MCPModel Context Protocol Server
Scope
Tests cover MCPModel Context Protocol server startup with stdio and StreamableHTTP transports, tool definitions for search_sessions, list_sessions, get_session_overview, get_messages, search_content, get_usage_summary, and tool invocation.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | MCPModel Context Protocol server starts with stdio transport | Server config | Server listening on stdio |
| TC-2 | MCPModel Context Protocol server starts with StreamableHTTP transport | Server config | HTTP endpoint available |
| TC-3 | Tool definitions are correctly shaped | Tool definitions | Valid MCPModel Context Protocol tool shapes |
| TC-4 | search_sessions tool | Search query | Ranked sessionA single conversation or interaction with an AI coding agent, typically one run/launch results |
| TC-5 | list_sessions tool | Filter params | Session list |
| TC-6 | get_session_overview tool | Session ID | Session details |
| TC-7 | get_messages tool | Session ID | Message list |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-8 | Full MCPModel Context Protocol tool invocation cycle | Server running | Tools execute and return |
Test Files
internal/mcp/server_test.go- Server lifecycle testsinternal/mcp/shape_test.go- Tool shape definition testsinternal/mcp/tools_test.go- Tool invocation tests
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall provide an MCPModel Context Protocol server with stdio transport | TC-1 |
| FR-2MustThe system shall provide an MCPModel Context Protocol server with StreamableHTTP transport | TC-2 |
| FR-3MustThe system shall expose a `search_sessions` tool for FTS5Full-Text Search version 5 search | TC-3, TC-4 |
| FR-4MustThe system shall expose a `list_sessions` tool for sessionA single conversation or interaction with an AI coding agent, typically one run/launch listing | TC-3, TC-5 |
| FR-5MustThe system shall expose a `get_session_overview` tool for sessionA single conversation or interaction with an AI coding agent, typically one run/launch details | TC-3, TC-6 |
| FR-6MustThe system shall expose a `get_messages` tool for messageAn individual turn in a session (user prompt, assistant response, tool result) retrieval | TC-3, TC-7 |
| FR-7MustThe system shall expose a `search_content` tool for content search | TC-3, TC-4 |
| FR-8MustThe system shall expose a `get_usage_summary` tool for usage data | TC-3 |
requirements
None.
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |
Pipeline Progress ?⚪ not started not yet begun✏️ draft initial version🔍 in review under review🚧 in progress actively worked on⛔ blocked waiting on dependency✅ done completed⏭️ skipped not applicable
Requirements: RESTRepresentational State Transfer APIApplication Programming Interface & Web Frontend
Overview
The RESTRepresentational State Transfer APIApplication Programming Interface and web frontend provide the primary user interface for agentsview. The APIApplication Programming Interface is built with the HumaOpenAPI 3.1 REST framework for Go framework (OpenAPI 3.1) and exposes endpoints for sessions, messages, search, analytics, usage, and administration. The frontend is a Svelte 5Frontend framework using runes for reactivity SPASingle Page Application embedded in the Go binary, featuring sessionA single conversation or interaction with an AI coding agent, typically one run/launch browsing, analytics dashboard, usage tracking, and settings.
Stakeholders
| Stakeholder | Interest |
|---|---|
| End user | Browse, search, and interact with sessionA single conversation or interaction with an AI coding agent, typically one run/launch data through a web UI |
| APIApplication Programming Interface consumer | Integrate with agentsview via RESTRepresentational State Transfer APIApplication Programming Interface |
| Developer | Build on top of agentsview's OpenAPI schema |
Functional Requirements
| ID | Priority | Requirement |
|---|---|---|
| FR-1MustThe system shall provide a RESTRepresentational State Transfer APIApplication Programming Interface with OpenAPI 3.1 specification | Must | The system shall provide a RESTRepresentational State Transfer APIApplication Programming Interface with OpenAPI 3.1 specification |
| FR-2MustThe system shall serve a Svelte 5Frontend framework using runes for reactivity SPASingle Page Application embedded in the Go binary | Must | The system shall serve a Svelte 5Frontend framework using runes for reactivity SPASingle Page Application embedded in the Go binary |
| FR-3MustThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch listing with rich filtering and cursor pagination | Must | The system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch listing with rich filtering and cursor pagination |
| FR-4MustThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch detail view with full messageAn individual turn in a session (user prompt, assistant response, tool result) history | Must | The system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch detail view with full messageAn individual turn in a session (user prompt, assistant response, tool result) history |
| FR-5MustThe system shall provide an analytics dashboard with charts and metrics | Must | The system shall provide an analytics dashboard with charts and metrics |
| FR-6MustThe system shall provide a usage/cost tracking page | Must | The system shall provide a usage/cost tracking page |
| FR-7MustThe system shall provide an activity/concurrency report page | Must | The system shall provide an activity/concurrency report page |
| FR-8MustThe system shall provide a full-text search page | Must | The system shall provide a full-text search page |
| FR-9MustThe system shall provide SSEServer-Sent Events for live updates | Must | The system shall provide SSEServer-Sent Events for live updates |
| FR-10MustThe system shall support keyboard navigation (j/k, Cmd+K search, ? shortcuts) | Must | The system shall support keyboard navigation (j/k, Cmd+K search, ? shortcuts) |
| FR-11MustThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch starring and trash/restore | Must | The system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch starring and trash/restore |
| FR-12MustThe system shall support messageAn individual turn in a session (user prompt, assistant response, tool result) pinning | Must | The system shall support messageAn individual turn in a session (user prompt, assistant response, tool result) pinning |
| FR-13ShouldThe system shall provide a recent edits feed showing files changed by agents | Should | The system shall provide a recent edits feed showing files changed by agents |
| FR-14ShouldThe system shall support settings management via the web UI | Should | The system shall support settings management via the web UI |
Non-Functional Requirements
| ID | Priority | Category | Requirement |
|---|---|---|---|
| NFR-1MustAPIApplication Programming Interface rejects requests with unrecognized Host headers (DNS rebinding protection) | Must | Security | APIApplication Programming Interface rejects requests with unrecognized Host headers (DNS rebinding protection) |
| NFR-2MustServer binds to 127.0.0.1 by default | Must | Security | Server binds to 127.0.0.1 by default |
| NFR-3ShouldFrontend bundle is embedded for single-binary deployment | Should | Performance | Frontend bundle is embedded for single-binary deployment |
Acceptance Criteria
- FR-2MustThe system shall serve a Svelte 5Frontend framework using runes for reactivity SPASingle Page Application embedded in the Go binary
- Given the server is running
- When a browser navigates to http://127.0.0.1:8080
- Then the Svelte SPASingle Page Application is served and interactive
- FR-10MustThe system shall support keyboard navigation (j/k, Cmd+K search, ? shortcuts)
- Given the sessionA single conversation or interaction with an AI coding agent, typically one run/launch list is displayed
- When j/k keys are pressed
- Then the selection moves up/down in the list
Open Questions
- Should there be a mobile-responsive layout or is desktop-only sufficient?
Specification: RESTRepresentational State Transfer APIApplication Programming Interface & Web Frontend
Overview
The RESTRepresentational State Transfer APIApplication Programming Interface is registered in internal/server/huma_route_groups.go using the HumaOpenAPI 3.1 REST framework for Go v2 framework, which automatically generates an OpenAPI 3.1 schema. The frontend is a Svelte 5Frontend framework using runes for reactivity SPASingle Page Application with runes-based reactivity, built with Vite+ and TypeScript. The SPASingle Page Application is embedded into the Go binary during build for single-binary deployment.
Architecture
Browser → HTTP Server (HumaOpenAPI 3.1 REST framework for Go + Chi) → Service Backend → SQLiteEmbedded relational database used as the primary session archive/PG/DuckDB
↓
Embedded SPASingle Page Application (frontend/dist/)
↓
SSEServer-Sent Events Broadcaster (live updates)
Frontend Component Tree
App.svelte
├── ThreeColumnLayout
│ ├── AppHeader (search, nav)
│ ├── SessionList (sidebar)
│ │ └── SessionItem
│ ├── MessageList (content)
│ │ ├── MessageContent
│ │ ├── ToolBlock / CallGroup / ParallelGroup
│ │ ├── ThinkingBlock / CodeBlock / MermaidBlock
│ │ └── ActivityLane
│ └── SessionVitals (right panel)
├── AnalyticsPage
├── UsagePage
├── ActivityPage
├── TrendsPage
├── InsightsPage
├── PinnedPage
├── TrashPage
├── RecentEditsPage
└── SettingsPage
APIApplication Programming Interface Contracts
Core Routes (OpenAPI 3.1)
| Method | Path | Description |
|---|---|---|
| GET | /api/v1/sessions | List sessions |
| GET | /api/v1/sessions/{id} | Get sessionA single conversation or interaction with an AI coding agent, typically one run/launch detail |
| GET | /api/v1/sessions/{id}/messages | List messages |
| GET | /api/v1/search | Full-text search |
| GET | /api/v1/analytics/summary | Analytics summary |
| GET | /api/v1/usage/summary | Usage summary |
| GET | /events | SSEServer-Sent Events events |
Sequences
SPASingle Page Application Load
1. Browser requests /
2. Server serves index.html with embedded SPASingle Page Application
3. SPASingle Page Application loads and calls GET /api/v1/sessions for initial data
4. SPASingle Page Application opens SSEServer-Sent Events connection to /events for live updates
Technical Decisions
| Decision | Choice | Rationale |
|---|---|---|
| APIApplication Programming Interface framework | HumaOpenAPI 3.1 REST framework for Go v2 | OpenAPI 3.1 native, code-first |
| Frontend framework | Svelte 5Frontend framework using runes for reactivity (runes) | Lightweight, fast, reactive |
| Embedding | Build-time copy of frontend/dist | Single binary deployment |
| SSEServer-Sent Events | /events endpoint with Broadcaster | Real-time updates without polling |
| Routing | Client-side store-based router | SPASingle Page Application navigation without page reloads |
| i18n | Paraglidei18n framework used in the Svelte frontend for message catalogs JS + inlang | Type-safe messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
Risks and Unknowns
- SPASingle Page Application bundle size growth over time
- Browser compatibility with Svelte 5Frontend framework using runes for reactivity runes
Test Plan: RESTRepresentational State Transfer APIApplication Programming Interface & Web Frontend
Scope
Tests cover RESTRepresentational State Transfer APIApplication Programming Interface endpoints, Svelte SPASingle Page Application rendering, sessionA single conversation or interaction with an AI coding agent, typically one run/launch listing/filtering/pagination, sessionA single conversation or interaction with an AI coding agent, typically one run/launch detail view, analytics dashboard, usage tracking, activity reports, full-text search UI, SSEServer-Sent Events live updates, keyboard navigation, starring/trash, messageAn individual turn in a session (user prompt, assistant response, tool result) pinning, recent edits feed, and settings management.
Unit Tests
| ID | Description | Input | Expected Output |
|---|---|---|---|
| TC-1 | Session list endpoint with cursor pagination | Filter params + cursor | Paginated sessionA single conversation or interaction with an AI coding agent, typically one run/launch list |
| TC-2 | Session detail returns full messageAn individual turn in a session (user prompt, assistant response, tool result) history | Session ID | Complete messageAn individual turn in a session (user prompt, assistant response, tool result) tree |
| TC-3 | SSEServer-Sent Events stream sends live updates | Sync event | Event received by client |
| TC-4 | Frontend sessionA single conversation or interaction with an AI coding agent, typically one run/launch list renders correctly | Session data array | Rendered list items |
| TC-5 | Frontend messageAn individual turn in a session (user prompt, assistant response, tool result) content renders correctly | Message data | Rendered messages |
| TC-6 | Frontend analytics dashboard renders | Analytics data | Charts and metrics displayed |
| TC-7 | Frontend usage page renders | Usage data | Cost breakdown displayed |
| TC-8 | Keyboard navigation works | j/k keypresses | Selection moves |
Integration Tests
| ID | Description | Preconditions | Expected Outcome |
|---|---|---|---|
| TC-9 | Frontend-backend full integration | Server running | SPASingle Page Application loads and queries APIApplication Programming Interface |
| TC-10 | E2E sessionA single conversation or interaction with an AI coding agent, typically one run/launch browsing | Test data | Full browse flow works |
| TC-11 | E2E search flow | Indexed sessions | Search results displayed |
Test Files
internal/server/sessions_test.go- Session APIApplication Programming Interface handler testsinternal/server/session_mgmt_test.go- Session management testsinternal/server/session_usage_test.go- Session usage APIApplication Programming Interface testsinternal/server/session_stats_test.go- Session stats APIApplication Programming Interface testsinternal/server/search_test.go- Search APIApplication Programming Interface testsinternal/server/analytics_test.go- Analytics APIApplication Programming Interface testsinternal/server/activity_test.go- Activity APIApplication Programming Interface testsinternal/server/usage_test.go- Usage APIApplication Programming Interface testsinternal/server/server_test.go- Server lifecycle testsinternal/server/auth_test.go- Auth middleware testsinternal/server/middleware_test.go- Middleware testsinternal/server/export_test.go- Export handler testsinternal/server/starred_test.go- Starred sessionA single conversation or interaction with an AI coding agent, typically one run/launch testsinternal/server/pins_test.go- Pinned messageAn individual turn in a session (user prompt, assistant response, tool result) testsinternal/server/spa_test.go- SPASingle Page Application serving testsinternal/server/broadcaster_test.go- SSEServer-Sent Events broadcaster testsinternal/server/huma_routes_sessions_test.go- Session route testsfrontend/src/lib/api/client.test.ts- APIApplication Programming Interface client testsfrontend/src/lib/api/runtime.test.ts- Runtime testsfrontend/src/lib/stores/sessions.test.ts- Session store testsfrontend/src/lib/stores/search.test.ts- Search store testsfrontend/src/lib/stores/analytics.test.ts- Analytics store testsfrontend/src/lib/stores/usage.test.ts- Usage store testsfrontend/src/lib/stores/activity.test.ts- Activity store testsfrontend/src/lib/components/sidebar/SessionList.test.ts- Session list componentfrontend/src/lib/components/content/MessageContent.test.ts- Message content componentfrontend/src/lib/components/analytics/AnalyticsPage.test.ts- Analytics page componentfrontend/src/lib/components/usage/UsagePage.test.ts- Usage page componentfrontend/e2e/sessionA single conversation or interaction with an AI coding agent, typically one run/launch-list.spec.ts- Session list e2efrontend/e2e/usage.spec.ts- Usage page e2efrontend/e2e/navigation.spec.ts- Navigation e2e
Coverage Matrix
| Requirement | Test Cases |
|---|---|
| FR-1MustThe system shall provide a RESTRepresentational State Transfer APIApplication Programming Interface with OpenAPI 3.1 specification | TC-1 |
| FR-2MustThe system shall serve a Svelte 5Frontend framework using runes for reactivity SPASingle Page Application embedded in the Go binary | TC-9 |
| FR-3MustThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch listing with rich filtering and cursor pagination | TC-1 |
| FR-4MustThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch detail view with full messageAn individual turn in a session (user prompt, assistant response, tool result) history | TC-2 |
| FR-5MustThe system shall provide an analytics dashboard with charts and metrics | TC-6 |
| FR-6MustThe system shall provide a usage/cost tracking page | TC-7 |
| FR-8MustThe system shall provide a full-text search page | TC-11 |
| FR-9MustThe system shall provide SSEServer-Sent Events for live updates | TC-3 |
| FR-10MustThe system shall support keyboard navigation (j/k, Cmd+K search, ? shortcuts) | TC-8 |
| FR-11MustThe system shall support sessionA single conversation or interaction with an AI coding agent, typically one run/launch starring and trash/restore | TC-1 |
| FR-12MustThe system shall support messageAn individual turn in a session (user prompt, assistant response, tool result) pinning | TC-2 |
requirements
- Should there be a mobile-responsive layout or is desktop-only sufficient?
Vocabulary
Domain Terms
| Term | Definition |
|---|---|
| sessionA single conversation or interaction with an AI coding agent, typically one run/launch | A single conversation or interaction with an AI coding agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.), typically one run/launch |
| messageAn individual turn in a session (user prompt, assistant response, tool result) | An individual turn in a sessionA single conversation or interaction with an AI coding agent, typically one run/launch (user prompt, assistant response, tool result) |
| tool callAn invocation of a tool/function by the AI agent during a session | An invocation of a tool/function by the AI agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.) | An AI coding tool that produces sessionA single conversation or interaction with an AI coding agent, typically one run/launch files (Claude, Codex, Cursor, etc.) |
| parserCode that reads an agent-specific session file format and extracts structured data | Code that reads an agentAn AI coding tool that produces session files (Claude, Codex, Cursor, etc.)-specific sessionA single conversation or interaction with an AI coding agent, typically one run/launch file format and extracts structured data |
| projectA named group of sessions, typically corresponding to a code repository | A named group of sessions, typically corresponding to a code repository |
| project identityThe git remote + worktree mapping used to disambiguate projects | The git remote + worktree mapping used to disambiguate projects |
| syncThe process of discovering new/changed session files and updating the database | The process of discovering new/changed sessionA single conversation or interaction with an AI coding agent, typically one run/launch files and updating the database |
| parse diffThe difference between two parser runs, used to detect incremental changes | The difference between two parserCode that reads an agent-specific session file format and extracts structured data runs, used to detect incremental changes |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive Full-Text Search version 5, used for messageAn individual turn in a session (user prompt, assistant response, tool result) content search |
| embeddingA vector representation of text used for semantic search | A vector representation of text used for semantic search |
| generationA version of the semantic search embedding index (a set of embeddings with a specific model and dimensions) | A version of the semantic search embeddingA vector representation of text used for semantic search index (a set of embeddings with a specific model and dimensions) |
| signalA health or outcome indicator computed from session content (success/failure, tool health, context pressure) | A health or outcome indicator computed from sessionA single conversation or interaction with an AI coding agent, typically one run/launch content (success/failure, tool health, context pressure) |
| insightAn AI-generated summary or analysis of session data | An AI-generated summary or analysis of sessionA single conversation or interaction with an AI coding agent, typically one run/launch data |
| pull quoteA notable excerpt extracted from session messages, used in reports | A notable excerpt extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch messages, used in reports |
| recall entryA distilled, reusable fact extracted from session history with evidence links | A distilled, reusable fact extracted from sessionA single conversation or interaction with an AI coding agent, typically one run/launch history with evidence links |
| secret findingDetected credentials or secrets in session content, stored with redacted matches | Detected credentials or secrets in sessionA single conversation or interaction with an AI coding agent, typically one run/launch content, stored with redacted matches |
| usage eventA record of token consumption during a session or portion of a session | A record of token consumption during a sessionA single conversation or interaction with an AI coding agent, typically one run/launch or portion of a sessionA single conversation or interaction with an AI coding agent, typically one run/launch |
| remote syncSyncing sessions from other machines via SSH or HTTP | Syncing sessions from other machines via SSHSecure Shell or HTTP |
| QuackDuckDB's remote protocol for network access to DuckDB files | DuckDB's remote protocol for network access to DuckDB files |
Technical Terms
| Term | Definition |
|---|---|
| SQLiteEmbedded relational database used as the primary session archive | Embedded relational database used as the primary sessionA single conversation or interaction with an AI coding agent, typically one run/launch archive |
| FTS5Full-Text Search version 5 | SQLiteEmbedded relational database used as the primary session archive virtual table module for full-text indexing and search |
| SSEServer-Sent Events | Server-Sent Events, used for real-time UI updates |
| HumaOpenAPI 3.1 REST framework for Go | OpenAPI 3.1 RESTRepresentational State Transfer framework for Go |
| Svelte 5Frontend framework using runes for reactivity | Frontend framework using runes for reactivity |
| pgxPostgreSQL driver for Go | PostgreSQL driver for Go |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion, a method for merging FTS5Full-Text Search version 5 and semantic search results |
| fsnotifyGo library for filesystem event notification | Go library for filesystem event notification |
| cobraCLI framework for Go | CLICommand Line Interface framework for Go |
| testifyGo testing library with assertions and mocking | Go testing library with assertions and mocking |
| sqlite-vecSQLite extension for vector similarity search | SQLiteEmbedded relational database used as the primary session archive extension for vector similarity search |
| MCPModel Context Protocol | Model Context Protocol, a protocol for AI tools to expose capabilities |
| TauriDesktop application framework wrapping web UIs | Desktop application framework wrapping web UIs |
| golangci-lintGo linter aggregator | Go linter aggregator |
| NilAwayGo nil pointer analysis tool | Go nil pointer analysis tool |
| LiteLLMOpen-source LLM pricing catalog used for cost estimation | Open-source LLM pricing catalog used for cost estimation |
| Paraglidei18n framework used in the Svelte frontend for message catalogs | i18n framework used in the Svelte frontend for messageAn individual turn in a session (user prompt, assistant response, tool result) catalogs |
| kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend | Shared UI component library (@kenn-io/kit-uiShared UI component library (@kenn-io/kit-ui) used in the frontend) used in the frontend |
| TOMLTom's Obvious Minimal Language, used for configuration files | Tom's Obvious Minimal Language, used for configuration files |
| CGOC Go interoperability | Go's mechanism for calling C code, required for the sqlite3 driver |
| CWDCurrent Working Directory, used for sync path filtering | Current Working Directory, used for syncThe process of discovering new/changed session files and updating the database path filtering |
Acronyms and Abbreviations
| Abbreviation | Expansion |
|---|---|
| SSEServer-Sent Events | Server-Sent Events |
| FTSFull-Text Search | Full-Text Search |
| FTS5Full-Text Search version 5 | Full-Text Search version 5 |
| RRFReciprocal Rank Fusion | Reciprocal Rank Fusion |
| MCPModel Context Protocol | Model Context Protocol |
| SPASingle Page Application | Single Page Application |
| CLICommand Line Interface | Command Line Interface |
| APIApplication Programming Interface | Application Programming Interface |
| RESTRepresentational State Transfer | Representational State Transfer |
| CORSCross-Origin Resource Sharing | Cross-Origin Resource Sharing |
| CSPContent Security Policy | Content Security Policy |
| DSNData Source Name | Data Source Name |
| DDLData Definition Language | Data Definition Language |
| CRUDCreate, Read, Update, Delete | Create, Read, Update, Delete |
| S3Simple Storage Service (Amazon S3-compatible) | Simple Storage Service (Amazon S3Simple Storage Service (Amazon S3-compatible)-compatible) |
| SSHSecure Shell | Secure Shell |
| SSEServer-Sent Events | Server-Sent Events |
| IDEIntegrated Development Environment | Integrated Development Environment |
| TUITerminal User Interface | Terminal User Interface |
| DMGApple Disk Image | Apple Disk Image |
| CI/CDContinuous Integration / Continuous Deployment | Continuous Integration / Continuous Deployment |
| WALWrite-Ahead Logging | Write-Ahead Logging |
| ADRArchitecture Decision Record | Architecture Decision Record |
| LOTRLLM Organized Tree of Retrievals (from internal/insight) | LLM Organized Tree of Retrievals (from internal/insightAn AI-generated summary or analysis of session data) |
| OTelOpenTelemetry | OpenTelemetry |
| CGOC Go interoperability | C Go interoperability |
| DSNData Source Name | Data Source Name |