Skip to content

feat(files): move workspace indexing and search to native for saf/files#2529

Open
bajrangCoder wants to merge 4 commits into
mainfrom
feat/native-workspace-index
Open

feat(files): move workspace indexing and search to native for saf/files#2529
bajrangCoder wants to merge 4 commits into
mainfrom
feat/native-workspace-index

Conversation

@bajrangCoder

@bajrangCoder bajrangCoder commented Jul 23, 2026

Copy link
Copy Markdown
Member

Summary

Moves file indexing and project-wide search for SAF and file:// workspaces to the Android native layer.

This prevents large workspaces, such as Termux home, from creating a complete file tree in the WebView.

Changes

  • Added a native SQLite workspace index.
  • Added paginated native filename queries.
  • Added the new asynchronous fileIndex plugin API.
  • Migrated Quick Open to native queries.
  • Migrated project-wide search to native root-based search.
  • Batched native search results to reduce Cordova bridge callbacks.
  • Removed progress callbacks from individual file reads.
  • Skips files and directories that return permission errors.
  • Fixed project search after closing one folder and opening another.
  • Fixed folder exclusion matching, including node_modules.
  • Prevented recursive indexing of Termux ~/storage.
  • Kept FTP, SFTP, and custom storage providers on the JavaScript fallback.

Breaking changes

acode.require("fileList") is deprecated.

It now contains files from non-native providers only. Plugins using it for SAF or file:// workspaces must migrate to:

const fileIndex = acode.require("fileIndex");
const result = await fileIndex.query({
  roots: [workspaceUrl],
  text: "filename",
  limit: 200,
});

Differences from fileList:

  • fileIndex is asynchronous.
  • Results are flat metadata records instead of Tree objects.
  • Large results use cursor pagination.
  • Native search can emit batched search-results events.

The native index database version is also updated. Existing generated indexes will be rebuilt automatically.

Compatibility

  • SAF and file://: native index and search.
  • FTP and SFTP: existing JavaScript fallback.
  • Low-level sdcard.workspaceSearch() keeps single-result events unless batchResults: true is provided.

@bajrangCoder
bajrangCoder marked this pull request as ready for review July 23, 2026 14:11
@greptile-apps

greptile-apps Bot commented Jul 23, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR moves workspace file indexing and project-wide search for SAF and file:// roots into the Android native layer (SQLite + Cordova), preventing large workspaces such as the Termux home directory from materialising a full in-memory file tree in the WebView.

  • New fileIndex module (src/lib/fileIndex.js) exposes scan, update, query, search, get, markDirty, clear, and whenReady as a clean async API wrapping five new Cordova bridge calls.
  • fileList.js now delegates append, remove, and rename to the native index for SAF/file:// roots, while keeping the JavaScript Tree path for FTP, SFTP, and custom providers.
  • searchInFiles now passes workspace root URLs to the native search rather than an explicit file list; results are batched (search-results) and drained through requestAnimationFrame slices to keep the UI responsive.

Confidence Score: 5/5

Safe to merge; the native indexing path is well-guarded by availability checks and falls back gracefully for non-native providers.

The architectural split between native and JS providers is cleanly implemented. The Java Job.cancel()/setSignal() protocol is correct under concurrent access thanks to volatile fields and a post-assignment if (cancelled) guard. SQL inputs are fully parameterised. The rAF drain pipeline in searchInFiles carries proper version guards so stale batches are discarded safely. All observations are style-level with no impact on correctness.

Files Needing Attention: WorkspaceIndex.java is the most complex file; the batch-clear loop and the Termux ~/storage exclusion regex are worth a second read, but neither introduces a defect.

Important Files Changed

Filename Overview
src/lib/fileIndex.js New module: clean async API wrapping the five Cordova bridge calls; scan-ID lifecycle, pending-scan map, and subscriber pattern are correct.
src/plugins/sdcard/src/android/WorkspaceIndex.java Large refactor: adds workspaceUpdate/workspaceQuery, batch search results, CancellationSignal support, and shouldSkipDirectory; Job.cancel() volatile protocol is correct, minor style nits around batch-clear.
src/lib/fileList.js Correctly delegates append/remove/rename to fileIndex for native roots; non-native path unchanged; whenReady now includes native scan promises.
src/sidebarApps/searchInFiles/index.js Replaces file-list-based native search with root-based native search; rAF drain pipeline for batched results is logically correct with proper version guards.
src/palettes/findFile/index.js Migrated Quick Open to dynamic-mode hints with native index query; deduplication via seen Set prevents duplicates across editor files, JS tree, and native results.
src/components/inputhints/index.js Adds dynamic option to regenerate hints on each input change; version guard prevents stale async results from replacing newer ones; pages reset in setHints is a correct fix.
src/lib/openFolder.js Order of FileList.remove/addedFolder.splice correctly swapped so findNativeRoot can still locate the folder during removal; complex CASE paste logic replaced with helpers.
src/components/fileTree/index.js appendEntry now guards against duplicate URLs and inserts at the correct sorted position instead of full re-render.
src/plugins/sdcard/www/plugin.js Adds workspaceUpdate and workspaceQuery Cordova bridge shims; straightforward.
src/plugins/sdcard/src/android/SDcard.java Adds two new case branches for workspace update and workspace query; pattern is consistent with existing actions.

Sequence Diagram

sequenceDiagram
    participant UI as WebView (JS)
    participant FI as fileIndex.js
    participant CB as Cordova Bridge
    participant WI as WorkspaceIndex.java
    participant DB as SQLite DB

    Note over UI,DB: Workspace open (SAF / file://)
    UI->>FI: scan(root)
    FI->>CB: "sdcard.workspaceScan({id, rootUrl, ...})"
    CB->>WI: scan(options, callback)
    WI->>DB: beginTransaction / INSERT files
    WI-->>CB: event type done
    CB-->>FI: resolve(event)
    FI-->>UI: Promise resolves

    Note over UI,DB: Quick Open (findFile palette)
    UI->>FI: "query({roots, text, limit})"
    FI->>CB: sdcard.workspaceQuery(options)
    CB->>WI: query(options, callback)
    WI->>DB: SELECT files WHERE name LIKE ?
    DB-->>WI: Cursor (paginated)
    WI-->>CB: "{entries[], cursor, hasMore}"
    CB-->>FI: resolve(result)
    FI-->>UI: entries[]

    Note over UI,DB: Project-wide search
    UI->>CB: "sdcard.workspaceSearch({roots, batchResults:true})"
    CB->>WI: search(options, callback)
    WI->>DB: mergeIndexedFiles(roots)
    WI->>WI: searchInContent per file
    WI-->>CB: event type search-results data:[...]
    CB-->>UI: enqueueNativeSearchResults(batch, version)
    UI->>UI: "rAF drain -> appendSearchResult x4/frame"
    WI-->>CB: event type done-searching
    CB-->>UI: finishSearchTask(version)
Loading

Reviews (2): Last reviewed commit: "Merge branch 'main' into feat/native-wor..." | Re-trigger Greptile

- incrementally update native workspace indexes
- stop nested folder creation from refreshing every workspace
- serialize index writes and fix scan cancellation ownership
- keep FileTree state synchronized after copy and move operations
- insert new sidebar entries without rebuilding expanded directories
@bajrangCoder

This comment was marked as outdated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

Status: Backlog

Development

Successfully merging this pull request may close these issues.

1 participant