refactor: remove QMD semantic indexing — keep keyword search only
Remove the entire QMD search engine (semantic indexing, hybrid search, vector embeddings) and replace with a simple walkdir-based keyword search. QMD never worked reliably in production, added bundle bloat, and showed "Index failed" in the status bar. What changed: - Rust: deleted indexing.rs, rewrote search.rs as pure keyword search - Frontend: removed useIndexing hook, IndexingBadge, hybrid search - Menu: removed "Reindex Vault" command from palette and menu bar - Config: removed qmd from bundle resources and build script - Deleted tools/qmd/ (QMD CLI tool, MCP server, tests) - Updated docs (ARCHITECTURE, ABSTRACTIONS, GETTING-STARTED) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Test
@@ -490,42 +490,29 @@ The Inspector panel (`src/components/Inspector.tsx`) is composed of sub-panels:
|
||||
|
||||
`useClosedTabHistory` hook (`src/hooks/useClosedTabHistory.ts`) provides a LIFO stack for closed tab entries, used by `useTabManagement` to support Cmd+Shift+T reopen. Each entry stores the note's path, tab index, and full `VaultEntry`. The stack is in-memory only (resets on restart), capped at 20 entries, and deduplicates by path.
|
||||
|
||||
## Search & Indexing
|
||||
## Search
|
||||
|
||||
### Search Modes
|
||||
### Search
|
||||
|
||||
Keyword-based search scans all vault `.md` files using `walkdir`:
|
||||
|
||||
```typescript
|
||||
type SearchMode = 'keyword' | 'semantic' | 'hybrid'
|
||||
|
||||
interface SearchResult {
|
||||
title: string
|
||||
path: string
|
||||
snippet: string
|
||||
score: number
|
||||
}
|
||||
|
||||
interface SearchResponse {
|
||||
results: SearchResult[]
|
||||
elapsedMs: number
|
||||
}
|
||||
```
|
||||
|
||||
### Search Integration
|
||||
|
||||
`SearchPanel` component provides the search UI:
|
||||
- Mode selector (keyword/semantic/hybrid)
|
||||
- Real-time results as user types
|
||||
- Real-time results as user types (300ms debounce)
|
||||
- Click result to open note in editor
|
||||
- Shows relevance score and snippet
|
||||
|
||||
### Indexing
|
||||
|
||||
Managed by `useIndexing` hook:
|
||||
- Checks index status on vault load
|
||||
- Two-phase indexing: scanning (parse files) → embedding (generate vectors)
|
||||
- Progress streamed via Tauri events
|
||||
- Incremental updates after git sync
|
||||
- Metadata persisted in `.laputa-index.json`
|
||||
No indexing step required — search runs directly against the filesystem.
|
||||
|
||||
## Vault Management
|
||||
|
||||
|
||||
@@ -79,7 +79,7 @@ flowchart LR
|
||||
| Frontmatter parsing | gray_matter | 0.2 |
|
||||
| AI (in-app chat) | Anthropic Claude API (Haiku 3.5 default) | - |
|
||||
| AI (agent panel) | Claude CLI subprocess (streaming NDJSON) | - |
|
||||
| Search | qmd (keyword + semantic + hybrid) | - |
|
||||
| Search | Keyword (walkdir-based file scan) | - |
|
||||
| MCP | @modelcontextprotocol/sdk | 1.0 |
|
||||
| Tests | Vitest (unit), Playwright (E2E/smoke), cargo test (Rust) | - |
|
||||
| Package manager | pnpm | - |
|
||||
@@ -113,7 +113,7 @@ flowchart TD
|
||||
GIT["git/"]
|
||||
GH["github/"]
|
||||
THEME["theme/"]
|
||||
SEARCH["search.rs + indexing.rs"]
|
||||
SEARCH["search.rs"]
|
||||
CLI["claude_cli.rs"]
|
||||
end
|
||||
|
||||
@@ -121,7 +121,6 @@ flowchart TD
|
||||
ANTH["Anthropic API\n(Claude chat)"]
|
||||
CCLI["Claude CLI\n(agent subprocess)"]
|
||||
MCP["MCP Server\n(ws://9710, 9711)"]
|
||||
QMD["qmd\n(search engine)"]
|
||||
GHAPI["GitHub API\n(OAuth, repos, clone)"]
|
||||
end
|
||||
|
||||
@@ -359,53 +358,16 @@ flowchart LR
|
||||
|
||||
The `WsBridgeChild` state wrapper in `lib.rs` ensures the bridge process is killed on app exit via `RunEvent::Exit` handler.
|
||||
|
||||
## Search & Indexing
|
||||
## Search
|
||||
|
||||
### Search Engine
|
||||
Search is keyword-based, using `walkdir` to scan all `.md` files in the vault directory. No external binary or indexing step required.
|
||||
|
||||
Search uses the external `qmd` binary (semantic search engine) with three modes:
|
||||
- Matches query against file titles and content (case-insensitive)
|
||||
- Scores results: title matches ranked higher than content-only matches
|
||||
- Extracts contextual snippets around the first match
|
||||
- Skips trashed and hidden files
|
||||
|
||||
| Mode | Command | Description |
|
||||
|------|---------|-------------|
|
||||
| `keyword` | `qmd search` | Term matching (default) |
|
||||
| `semantic` | `qmd vsearch` | Vector similarity search |
|
||||
| `hybrid` | `qmd query` | Combined keyword + semantic |
|
||||
|
||||
### Indexing Flow
|
||||
|
||||
```mermaid
|
||||
flowchart TD
|
||||
A([Vault opened]) --> B[check_index_status]
|
||||
B --> C{Index status?}
|
||||
C -->|Fresh| D[run_incremental_update\ngit diff since last commit]
|
||||
C -->|Stale / Missing| E
|
||||
|
||||
subgraph E[Full Indexing — start_indexing]
|
||||
E1["Phase 1: qmd update\n(scan all .md files)"]
|
||||
E2["Phase 2: qmd embed\n(generate vector embeddings)"]
|
||||
E1 --> E2
|
||||
end
|
||||
|
||||
E --> F[Save .laputa-index.json\nlast_indexed_commit + timestamp]
|
||||
D --> G([Search ready])
|
||||
F --> G
|
||||
|
||||
E2 -.->|failure is non-fatal| G
|
||||
G --> H{Search mode}
|
||||
H -->|keyword| I[qmd search]
|
||||
H -->|semantic| J[qmd vsearch]
|
||||
H -->|hybrid| K[qmd query]
|
||||
```
|
||||
|
||||
Embedding failure is non-fatal — keyword search still works.
|
||||
|
||||
### qmd Binary Resolution
|
||||
|
||||
1. Bundled macOS app resource: `<app>/Contents/Resources/qmd/qmd`
|
||||
2. Dev mode: `CARGO_MANIFEST_DIR/resources/qmd/qmd`
|
||||
3. System locations: `~/.bun/bin/qmd`, `/usr/local/bin/qmd`, `/opt/homebrew/bin/qmd`
|
||||
4. PATH lookup via `which qmd`
|
||||
5. Auto-install via `bun install -g qmd` if missing
|
||||
The `search_vault` Tauri command runs the scan in a blocking Tokio task and returns results sorted by relevance score.
|
||||
|
||||
## Vault Cache System
|
||||
|
||||
@@ -533,7 +495,6 @@ sequenceDiagram
|
||||
VL->>T: invoke('get_modified_files')
|
||||
VL->>T: useMcpStatus — register if needed
|
||||
VL->>T: useThemeManager — load active theme
|
||||
VL->>T: useIndexing — incremental update if stale
|
||||
VL-->>A: entries ready
|
||||
end
|
||||
|
||||
@@ -619,8 +580,7 @@ The vault backend (`src-tauri/src/vault/`) is split into focused submodules:
|
||||
| `git/` | Git operations (`commit.rs`, `status.rs`, `history.rs`, `conflict.rs`, `remote.rs`, `pulse.rs`) |
|
||||
| `github/` | GitHub OAuth + API (`auth.rs`, `api.rs`, `clone.rs`) |
|
||||
| `theme/` | Theme management (`mod.rs`, `create.rs`, `defaults.rs`, `seed.rs`) |
|
||||
| `search.rs` | qmd search integration (keyword/semantic/hybrid) |
|
||||
| `indexing.rs` | qmd indexing with progress streaming |
|
||||
| `search.rs` | Keyword search — walkdir-based vault file scan |
|
||||
| `claude_cli.rs` | Claude CLI subprocess spawning + NDJSON stream parsing |
|
||||
| `ai_chat.rs` | Direct Anthropic API client (non-streaming, for Tauri builds) |
|
||||
| `mcp.rs` | MCP server spawning + config registration |
|
||||
@@ -689,14 +649,11 @@ The vault backend (`src-tauri/src/vault/`) is split into focused submodules:
|
||||
| `github_create_repo` | Create new repo |
|
||||
| `clone_repo` | Clone repo with token auth |
|
||||
|
||||
### Search & Indexing
|
||||
### Search
|
||||
|
||||
| Command | Description |
|
||||
|---------|-------------|
|
||||
| `search_vault` | Search via qmd (keyword/semantic/hybrid) |
|
||||
| `get_index_status` | Check qmd index state |
|
||||
| `start_indexing` | Full index with progress streaming |
|
||||
| `trigger_incremental_index` | Incremental index update |
|
||||
| `search_vault` | Keyword search across vault files |
|
||||
|
||||
### Theme
|
||||
|
||||
@@ -772,7 +729,7 @@ No Redux or global context. State lives in the root `App.tsx` and custom hooks:
|
||||
| `useAIChat` | `messages`, `isStreaming` | AI chat conversation |
|
||||
| `useAiAgent` | `messages`, `status`, tool actions | AI agent conversation |
|
||||
| `useAutoSync` | Sync interval, pull/push state | Git auto-sync |
|
||||
| `useIndexing` | Index status, progress | Search indexing |
|
||||
| `useUnifiedSearch` | Query, results, loading state | Keyword search |
|
||||
| `useSettings` | App settings (API keys, GitHub token) | Persistent settings |
|
||||
| `useVaultConfig` | Per-vault UI preferences | Vault-specific config |
|
||||
|
||||
|
||||
@@ -7,7 +7,6 @@ How to navigate the codebase, run the app, and find what you need.
|
||||
- **Node.js** 18+ and **pnpm**
|
||||
- **Rust** 1.77.2+ (for the Tauri backend)
|
||||
- **git** CLI (required by the git integration features)
|
||||
- **qmd** (optional — for search indexing; auto-installed if missing)
|
||||
|
||||
## Quick Start
|
||||
|
||||
@@ -97,7 +96,7 @@ laputa-app/
|
||||
│ │ ├── useEditorSave.ts # Auto-save with debounce
|
||||
│ │ ├── useTheme.ts # Flatten theme.json → CSS vars
|
||||
│ │ ├── useThemeManager.ts # Vault theme lifecycle
|
||||
│ │ ├── useIndexing.ts # Search indexing management
|
||||
│ │ ├── useUnifiedSearch.ts # Keyword search
|
||||
│ │ ├── useNoteSearch.ts # Note search
|
||||
│ │ ├── useCommandRegistry.ts # Command palette registry
|
||||
│ │ ├── useAppCommands.ts # App-level commands
|
||||
@@ -158,8 +157,7 @@ laputa-app/
|
||||
│ │ │ ├── mod.rs, auth.rs, api.rs, clone.rs
|
||||
│ │ ├── theme/ # Theme module
|
||||
│ │ │ ├── mod.rs, create.rs, defaults.rs, seed.rs
|
||||
│ │ ├── search.rs # qmd search integration
|
||||
│ │ ├── indexing.rs # qmd indexing + progress streaming
|
||||
│ │ ├── search.rs # Keyword search (walkdir-based)
|
||||
│ │ ├── claude_cli.rs # Claude CLI subprocess management
|
||||
│ │ ├── ai_chat.rs # Direct Anthropic API client
|
||||
│ │ ├── mcp.rs # MCP server lifecycle + registration
|
||||
@@ -220,7 +218,7 @@ laputa-app/
|
||||
| `src-tauri/src/frontmatter/ops.rs` | YAML manipulation — how properties are updated/deleted in files. |
|
||||
| `src-tauri/src/git/` | All git operations (commit, pull, push, conflicts, pulse). |
|
||||
| `src-tauri/src/github/` | GitHub OAuth device flow + repo clone/create. |
|
||||
| `src-tauri/src/search.rs` | qmd search integration (keyword/semantic/hybrid). |
|
||||
| `src-tauri/src/search.rs` | Keyword search — scans vault files with walkdir. |
|
||||
| `src-tauri/src/claude_cli.rs` | Claude CLI subprocess spawning + NDJSON stream parsing. |
|
||||
|
||||
### Editor
|
||||
|
||||
Reference in New Issue
Block a user