Skip to content

fix(scan): reduce memory use during directory scans - #1748

Open
John-David Dalton (jdalton) wants to merge 1 commit into
v1.xfrom
fix/stream-filter-glob-dedup
Open

John-David Dalton (jdalton) wants to merge 1 commit into
v1.xfrom
fix/stream-filter-glob-dedup

Conversation

@jdalton

@jdalton John-David Dalton (jdalton) commented Nov 21, 2026 •

Copy link
Copy Markdown
Contributor

Large directory scans no longer retain every matched path just to deduplicate before the supported-file filter runs. Filtered scans now turn off fast-glob uniqueness and deduplicate supported results as they stream, preserving results for overlapping patterns.

@cursor

cursor Bot commented Nov 21, 2026 •

Copy link
Copy Markdown

PR Summary

Low Risk
Localized change to glob streaming behavior with regression tests; no auth, security, or API surface changes.

Overview
Large monorepo scans that pass a supported-file filter (e.g. manifest discovery via globWithGitIgnore) no longer ask fast-glob to deduplicate every match before filtering. When a filter is set, unique is turned off and deduplication runs only on paths that survive the filter, using a streaming seen set.

This keeps overlapping patterns (such as **/*.json plus src/**/*) from inflating memory with unsupported files that would be dropped anyway. A focused test asserts filtered results stay deduplicated across overlapping globs.

Reviewed by Cursor Bugbot for commit fa8be3f. Configure here.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Development

Successfully merging this pull request may close these issues.

1 participant