fix(vault): prevent gitignored folder scan freezes
This commit is contained in:
@@ -363,7 +363,7 @@ All Notes starts from Markdown notes and excludes Markdown files under `attachme
|
||||
|
||||
The folder tree hides the legacy `type/` directory, since those type documents already appear through the Types sidebar section. Default vault folders such as `attachments/` and `views/` remain visible alongside user-created folders under the synthetic vault-root row.
|
||||
|
||||
Command-facing vault content is filtered through `vault::filter_gitignored_entries`, `vault::filter_gitignored_folders`, and `vault::filter_gitignored_paths` when the app setting `hide_gitignored_files` is enabled. The cache still stores the complete scan; `list_vault`, `reload_vault`, `list_vault_folders`, and search apply the visibility filter at the boundary before React consumes entries. The filter batches paths through `git check-ignore --no-index --stdin`, so negated and specific `.gitignore` patterns follow Git semantics as closely as the app can reasonably support.
|
||||
Command-facing vault content is filtered through `vault::filter_gitignored_entries`, `vault::filter_gitignored_folders`, and `vault::filter_gitignored_paths` when the app setting `hide_gitignored_files` is enabled. The cache still stores the complete scan; `list_vault`, `reload_vault`, `list_vault_folders`, and search apply the visibility filter at the boundary before React consumes entries. The filter batches paths through `git check-ignore --no-index --stdin`, drains stdout while stdin is still being written, and short-circuits root `.gitignore` detection before walking for nested ignore files, so large ignored folder sets cannot deadlock the native UI while preserving Git semantics as closely as the app can reasonably support.
|
||||
|
||||
A `vault_health_check` command detects stray files in non-protected subfolders and filename-title mismatches. On vault load, a migration banner offers to flatten stray files to the root via `flatten_vault`.
|
||||
|
||||
|
||||
@@ -88,7 +88,7 @@ flowchart LR
|
||||
3. **No orphan state updates**: Never call `updateEntry()` before the corresponding `handleUpdateFrontmatter()` or `handleDeleteProperty()` has resolved. The three functions in `useEntryActions` (`handleCustomizeType`, `handleRenameSection`, `handleToggleTypeVisibility`) follow this rule — disk write first, then state update.
|
||||
4. **Recovery via reload**: If state ever diverges from disk (crash, external edit, race condition), `Reload Vault` (Cmd+K → "Reload Vault") invalidates the cache and does a full filesystem rescan via the `reload_vault` Tauri command, replacing all React state. The `reload_vault_entry` command can re-read a single file.
|
||||
5. **Cache is disposable**: The `reload_vault` command deletes the cache file before rescanning, guaranteeing fresh data. The cache never contains data that doesn't exist on the filesystem.
|
||||
6. **Visibility filters are command-boundary concerns**: Gitignored-content visibility is applied after scanning/caching, before entries, folders, or search results reach React. The cache remains complete so toggling the setting can show ignored content again without rebuilding a different cache shape.
|
||||
6. **Visibility filters are command-boundary concerns**: Gitignored-content visibility is applied after scanning/caching, before entries, folders, or search results reach React. The cache remains complete so toggling the setting can show ignored content again without rebuilding a different cache shape. Large folder filtering runs on the blocking Tokio pool and drains `git check-ignore` output while feeding stdin so broad ignore matches cannot freeze the native UI thread.
|
||||
|
||||
#### External Change Detection
|
||||
|
||||
@@ -644,7 +644,7 @@ The vault backend (`src-tauri/src/vault/`) is split into focused submodules:
|
||||
| `parsing.rs` | Text processing: snippet extraction, markdown stripping, ISO date parsing, `extract_title` (H1 → legacy frontmatter → filename), `slug_to_title` |
|
||||
| `title_sync.rs` | Legacy filename → `title` frontmatter sync helper; no longer used by the normal note-open flow |
|
||||
| `cache.rs` | Git-based incremental vault caching (`scan_vault_cached`), git helpers |
|
||||
| `ignored.rs` | Gitignored-content visibility filtering via batched `git check-ignore` |
|
||||
| `ignored.rs` | Gitignored-content visibility filtering via batched, pipe-safe `git check-ignore` |
|
||||
| `filename_rules.rs` | Cross-platform validation for note filenames, folder names, and custom view filenames |
|
||||
| `rename.rs` | `rename_note` / `rename_note_filename` / `move_note_to_folder` — stage crash-safe file moves, update `title` frontmatter when needed, recover unfinished rename transactions, and report backlink rewrite failures |
|
||||
| `image.rs` | `save_image` / `copy_image_to_vault` — save editor image attachments with sanitized filenames |
|
||||
@@ -684,6 +684,7 @@ The vault backend (`src-tauri/src/vault/`) is split into focused submodules:
|
||||
| `rename_note` | Crash-safe note rename + `title` frontmatter update + cross-vault wikilinks + failed backlink counts |
|
||||
| `move_note_to_folder` | Crash-safe folder move that preserves the filename, reloads the moved note, and rewrites path-based wikilinks |
|
||||
| `create_vault_folder` | Create a folder relative to the active vault root |
|
||||
| `list_vault_folders` | Build the folder tree on the blocking Tokio pool, then apply Gitignored-content visibility → `Vec<FolderNode>` |
|
||||
| `rename_vault_folder` | Rename a folder relative to the active vault root and return old/new relative paths |
|
||||
| `delete_vault_folder` | Permanently delete a folder subtree relative to the active vault root |
|
||||
| `sync_note_title` | Legacy helper: rewrite `title` frontmatter from filename → `bool` (modified); not used by the normal note-open flow |
|
||||
|
||||
@@ -271,8 +271,12 @@ pub fn list_vault(path: PathBuf) -> Result<Vec<VaultEntry>, String> {
|
||||
}
|
||||
|
||||
#[tauri::command]
|
||||
pub fn list_vault_folders(path: PathBuf) -> Result<Vec<FolderNode>, String> {
|
||||
with_expanded_vault_root(path.as_path(), scan_visible_vault_folders)
|
||||
pub async fn list_vault_folders(path: PathBuf) -> Result<Vec<FolderNode>, String> {
|
||||
tokio::task::spawn_blocking(move || {
|
||||
with_expanded_vault_root(path.as_path(), scan_visible_vault_folders)
|
||||
})
|
||||
.await
|
||||
.map_err(|e| format!("Task panicked: {e}"))?
|
||||
}
|
||||
|
||||
#[cfg(test)]
|
||||
@@ -348,8 +352,8 @@ mod tests {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn folder_and_listing_commands_use_expanded_vault_root() {
|
||||
#[tokio::test]
|
||||
async fn folder_and_listing_commands_use_expanded_vault_root() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
let root = vault_root(&dir);
|
||||
fs::write(dir.path().join("root.md"), "# Root\n").unwrap();
|
||||
@@ -364,7 +368,7 @@ mod tests {
|
||||
assert!(entries.iter().any(|entry| entry.filename == "root.md"));
|
||||
assert!(entries.iter().any(|entry| entry.filename == "project.md"));
|
||||
|
||||
let folders = list_vault_folders(root).unwrap();
|
||||
let folders = list_vault_folders(root).await.unwrap();
|
||||
assert!(folders.iter().any(|folder| folder.name == "Projects"));
|
||||
}
|
||||
|
||||
|
||||
@@ -23,6 +23,10 @@ fn should_descend_for_gitignore(entry: &DirEntry) -> bool {
|
||||
}
|
||||
|
||||
fn has_gitignore_file(vault_path: &Path) -> bool {
|
||||
if vault_path.join(".gitignore").is_file() {
|
||||
return true;
|
||||
}
|
||||
|
||||
WalkDir::new(vault_path)
|
||||
.follow_links(false)
|
||||
.into_iter()
|
||||
@@ -43,14 +47,17 @@ fn run_git_check_ignore(vault_path: &Path, relative_paths: &[String]) -> Option<
|
||||
.spawn()
|
||||
.ok()?;
|
||||
|
||||
{
|
||||
let stdin = child.stdin.as_mut()?;
|
||||
for path in relative_paths {
|
||||
writeln!(stdin, "{path}").ok()?;
|
||||
let mut stdin = child.stdin.take()?;
|
||||
let paths = relative_paths.to_vec();
|
||||
let writer = std::thread::spawn(move || -> std::io::Result<()> {
|
||||
for path in paths {
|
||||
writeln!(stdin, "{path}")?;
|
||||
}
|
||||
}
|
||||
Ok(())
|
||||
});
|
||||
|
||||
let output = child.wait_with_output().ok()?;
|
||||
writer.join().ok()?.ok()?;
|
||||
if output.status.success() || output.status.code() == Some(1) {
|
||||
return Some(String::from_utf8_lossy(&output.stdout).to_string());
|
||||
}
|
||||
@@ -186,6 +193,8 @@ pub fn filter_gitignored_folders(
|
||||
mod tests {
|
||||
use super::*;
|
||||
use std::fs;
|
||||
use std::sync::mpsc;
|
||||
use std::time::Duration;
|
||||
use tempfile::TempDir;
|
||||
|
||||
fn write_file(root: &Path, relative: &str, content: &str) {
|
||||
@@ -288,6 +297,34 @@ mod tests {
|
||||
assert_eq!(filtered[0].path, "notes");
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn filters_large_ignored_folder_sets_without_blocking_on_git_stdout() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
init_git_repo(dir.path());
|
||||
write_file(dir.path(), ".gitignore", "generated/\n");
|
||||
|
||||
let folders = (0..6_000)
|
||||
.map(|index| FolderNode {
|
||||
name: format!("package-{index}"),
|
||||
path: format!("generated/package-{index}"),
|
||||
children: vec![],
|
||||
})
|
||||
.collect::<Vec<_>>();
|
||||
let vault_path = dir.path().to_path_buf();
|
||||
let (sender, receiver) = mpsc::channel();
|
||||
|
||||
std::thread::spawn(move || {
|
||||
let filtered = filter_gitignored_folders(vault_path.as_path(), folders, true);
|
||||
let _ = sender.send(filtered);
|
||||
drop(dir);
|
||||
});
|
||||
|
||||
let filtered = receiver
|
||||
.recv_timeout(Duration::from_secs(5))
|
||||
.expect("large gitignored folder filtering should not block on child stdout");
|
||||
assert!(filtered.is_empty());
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn has_no_effect_without_gitignore_file() {
|
||||
let dir = TempDir::new().unwrap();
|
||||
|
||||
Reference in New Issue
Block a user