Skip to content

Check existing alerts under --strict-blocking when no manifest changed - #462

Draft
David Larsen (dc-larsen) wants to merge 6 commits into
mainfrom
fix/strict-blocking-no-manifest-change
Draft

David Larsen (dc-larsen) wants to merge 6 commits into
mainfrom
fix/strict-blocking-no-manifest-change

Conversation

@dc-larsen

@dc-larsen David Larsen (dc-larsen) commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

With --strict-blocking and without --ignore-commit-files, a pull request whose changes touch no supported manifest passes even when the repository has existing blocking alerts. The CLI falls back to a full scan, logs "--strict-blocking is only supported in diff mode", and exits 0. Most pull requests change code only, so on a default setup --strict-blocking did nothing for most of them. #453 listed this as a known gap. Found while verifying 2.13.0 against a customer report.

Root Cause

Without --ignore-commit-files, main_code reads the changed files to decide whether to compare. When none is a supported manifest it sets force_api_mode, creates a full scan of the target path, and switches blocking off because a full scan has no baseline (#101 added this; before it, such commits skipped the scan entirely). For a normal run that is reasonable: a change that can't alter dependencies has no new alerts to block on or comment about. --strict-blocking exists to fail on alerts the change didn't introduce, so the fallback skipped exactly what the flag is for. The same fallback also skipped the PR/MR comment flow under --scm github/gitlab.

Fix

Under --strict-blocking, a change with no supported manifest now sets force_diff_mode, the same path --ignore-commit-files takes. The run compares every manifest in the target path against the baseline, loads existing alerts from the new full scan through Core.add_existing_alerts_from_full_scan, fails on existing Block alerts, and lists them in the console and the PR/MR comment exactly as on any other comparison. If existing alerts can't be read, the existing infrastructure-error handling applies (3, --exit-code-on-api-error, 0 under --disable-blocking).

The "compares every manifest" log line is printed after the SCM event check, and only when the run goes on to compare. Skipped pull request events (closed, labeled, ...) and SCM branch pipelines don't print it.

I chose this over reading existing alerts from the fallback full scan. That full scan has no baseline, so "existing" would mean "every alert in the scan", the comment flow would still be skipped, and it would need a second blocking path. Reusing the comparison keeps one code path and produces the same output a user already gets with --ignore-commit-files.

Unchanged:

  • Runs without --strict-blocking keep the full-scan fallback.
  • SCM branch (non-PR/MR) pipelines still create a full scan and don't block, as documented in Scan type by pipeline.
  • Changes that touch a manifest already compared; nothing changes there.

Upgrade note: without --scm there is no event type, so this applies to every --strict-blocking run with no manifest change, not only pull requests. Default-branch pushes in Bitbucket, Azure Pipelines, Jenkins and other pipelines that don't set --scm, and target paths that aren't git repositories, now compare against the head scan and fail on existing Block alerts where they used to pass. --ignore-commit-files already behaved this way. The CHANGELOG has an "Upgrade notes" entry and the CLI reference says so.

--reach: the baseline is the head scan of the same type, so a --reach PR scan needs a default-branch scan made with --reach. Otherwise the baseline is empty and every alert is labeled new. The exit code is the same under --strict-blocking, but the comment says the change introduced them. This predates this PR (--ignore-commit-files already hit it), but code-only strict PRs now reach it on default flags, so the CLI reference notes it.

Cost: for --strict-blocking runs on code-only changes, the CLI now does what it already did with --ignore-commit-files or a manifest change. It resolves the baseline, waits for the diff-scan comparison, and streams the new full scan once for existing alerts. Manifest upload is the same as before, since the fallback also scanned the whole target path. Measured end to end, those runs went from about 1 s to 13-14 s (about 5.5 s comparison wait plus about 6 s existing-alert stream), matching the --ignore-commit-files runs on the same commits.

Docs: the --strict-blocking help text, the CLI reference (flag table, Strict Blocking notes, File Selection Behavior, Scanning Modes) and ci-cd.md no longer say the flag is diff-only on no-manifest changes.

2.13.1 and 2.13.2 were released from main on 2026-11-20, so this branch merges main at v2.13.2 and bumps to 2.13.3 in pyproject.toml, socketsecurity/__init__.py and uv.lock, with its CHANGELOG entries under a new ## 2.13.3 heading. #459, #460, #461, #462, #463, #464 and #465 all make the same bump, so the version files merge cleanly between them, and once one merges the rest ride main's unreleased 2.13.3 (Version Check accepts a version equal to main's while it is above the latest PyPI release). Merging a second one conflicts only in CHANGELOG.md: keep one ## 2.13.3 heading and combine the bullets under its subsections (Upgrade notes, Changed, Fixed, Documentation, in that order). There are no code conflicts between them. If main releases 2.13.3 before this merges, bump to the next patch again.

Testing

  • New tests/core/test_strict_blocking_no_manifest_change.py (26 tests) drives socketcli.cli() with a README-only change across --enable-diff, default API mode, --files without a manifest, and a non-git target: existing Block alert exits 1, no Block alert exits 0, --disable-blocking exits 0, unreadable existing alerts use the infrastructure code (default, remapped, nonblocking), no fallback warning under the flag, manifest changes still compare, non-strict runs keep the full-scan fallback, a code-only GitHub pull request posts the comment with the existing alerts, a GitLab merge request does the same, GitHub and GitLab push pipelines still create a full scan and don't block even when that scan carries a Block alert, an API-mode default-branch push compares (with make_default_branch) and fails, and the comparison log line appears only when a comparison runs (not on a skipped closed event or a branch pipeline). 16 of the 26 fail on main.
  • With Scan reopened pull requests instead of passing them unscanned #460 merged locally (not pushed), a live pull_request closed / labeled run with --strict-blocking on a code-only change: before this fix the log said every manifest would be compared and then "No scan ran; exiting 0"; now it prints only the "No scan ran" warning. Exit 0 both ways, and the combined suite passes (1036).
  • uv run pytest -q -o addopts=: 1027 passed (1001 on main). ruff check ., ruff format --check on the new test file, uv lock --locked --offline, hatch build and twine check pass.
  • Live, against a test org, stock 2.13.0 vs this branch (editable install, same socketdev 3.6.2), on four projects: a small npm project with a multi-CVE Block package, a second npm project, a PyPI requirements.txt project and a shallow clone of a public React app (1,199 packages). --enable-diff --strict-blocking on a README-only commit: 2.13.0 exits 0 with the fallback warning on all four. This branch exits 1 on all four (run twice each), with the same existing-alert counts and the same rendered comment (scan IDs aside) as --ignore-commit-files on the same commit. Manifest-changing commits, non-strict runs, --ignore-commit-files runs, and --disable-blocking behaved the same before and after.
  • Re-run after the follow-up commit, three projects (small npm, PyPI, React app), each case twice: code-only --strict-blocking (default API mode and --enable-diff) exit 1 with 3 / 1 / 9 existing Block alerts, the same counts as --ignore-commit-files --strict-blocking; non-strict exit 0; --strict-blocking --disable-blocking exit 0. Stock 2.13.0 exits 0 with the fallback warning. API-mode default-branch push with --default-branch --strict-blocking on a README-only commit: 2.13.0 exit 0, this branch exit 1 (twice).

Public Changelog

--strict-blocking now fails on existing blocking alerts when a change touches no dependency manifest. Previously such changes got a full scan with blocking turned off and passed. Without --scm, this includes default-branch pushes and non-git target paths, which can now fail on existing blocking alerts after upgrading.

Without --ignore-commit-files, the CLI reads the changed files to decide
whether to compare. When none is a supported manifest it sets
force_api_mode, creates a full scan of the target path, and turns blocking
off because a full scan has no baseline. That fallback exists so a change
that cannot alter dependencies doesn't post a comment or block on alerts
it didn't introduce. Under --strict-blocking it also skipped the check the
flag exists for: a code-only pull request passed with a warning that
--strict-blocking needs diff mode, even when the repository had existing
blocking alerts.

With --strict-blocking, a change that touches no manifest now runs the
comparison over every manifest in the target path, the same way
--ignore-commit-files does. Existing alerts are loaded from the new full
scan and fail the run, and PR/MR comments list them. Failures reading them
keep the infrastructure-error code. Runs without --strict-blocking keep
the full-scan fallback, and SCM branch pipelines still create a full scan.
@lelia lelia changed the title Check existing alerts under --strict-blocking when no manifest changed Check existing alerts under --strict-blocking when no manifest changed Nov 20, 2026
@lelia lelia self-assigned this Nov 20, 2026
The no-manifest message was logged when the mode was chosen, before the
SCM event type was known. Skipped pull request events (closed, for example)
printed that every manifest would be compared and then exited without a
scan, and SCM branch pipelines printed it before creating a full scan. Log
it after the event check, only when the run goes on to compare.

Document the wider reach of the change: without --scm, default-branch
pushes and non-git target paths now compare under --strict-blocking and can
fail on existing blocking alerts. Note that --reach pull request scans need
a default-branch scan made with --reach, and reword the API Mode line.

Add tests for GitLab merge request and branch pipelines, an API-mode
default-branch push, and the log message. The branch pipeline tests now
return a blocking alert from the full scan, so they fail if blocking were
left on.
…o-manifest-change

# Conflicts:
#	CHANGELOG.md
…o-manifest-change

# Conflicts:
#	CHANGELOG.md
2.13.2 was released from main, so this change needs the next patch version.

This branch was successfully deployed

1 active deployment
socket-firewall — 009a605e Deployed Nov 20, 2026 by dc-larsen via python-sfw-smoke-enterprise #665
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants