Skip to content

List every existing advisory under --strict-blocking - #459

Draft
David Larsen (dc-larsen) wants to merge 7 commits into
mainfrom
fix/strict-blocking-keep-each-existing-advisory
Draft

David Larsen (dc-larsen) wants to merge 7 commits into
mainfrom
fix/strict-blocking-keep-each-existing-advisory

Conversation

@dc-larsen

@dc-larsen David Larsen (dc-larsen) commented Nov 20, 2026 •

Copy link
Copy Markdown
Contributor

Blocked on socketdev 3.6.4 being published (SocketDev/socket-sdk-python#134). socketdev 3.6.3 was released from the SDK's main on 2026-11-20 without the dedupe fix, so #134 now ships as 3.6.4 and this PR pins socketdev==3.6.4, which isn't on PyPI yet. Until it is, python-tests, ruff, dependency-audit, the e2e jobs and the Socket Firewall smoke job (and with it dependency-review-gate) fail at install. After 3.6.4 is published: run uv lock and push. Don't lock against 3.6.3: the new tests fail on it. If 2.13.3 has shipped by then, bump pyproject.toml, socketsecurity/__init__.py and the socketsecurity entry in uv.lock to the next patch and move this PR's changelog lines under that heading.

Follow-up to #453. Under --strict-blocking, the Existing alerts table and the EXISTING blocking issues count show one CVE per package and severity. In a customer report, handlebars@4.0.11 has 6 Critical CVEs with a Block action in the raw full scan, and the CLI listed 1. The whole repro scan had 8 existing blockers; the CLI reported 3 ("Loaded 49 alerts" vs 69 raw).

The same bug can pass a run that should fail. If the first-listed CVE of a group is resolved or triaged to monitor/ignore, the Block CVEs behind it are dropped. Reproduced below: vm2@3.9.11 with 34 Block CVEs, stock 2.13.0 exits 0.

Root Cause

Core.add_existing_alerts_from_full_scan reads alerts on unchanged packages through Core.get_sbom_data → sdk.fullscans.stream(). socketdev 3.6.2 runs Dedupe on that stream, which keys alerts on (type, severity, category, file, start, end) and keeps the first alert's action. CVE alerts have no file, so all same-severity advisories on a package collapse into one before the CLI sees them. #453 fixed the CLI's own key (Issue.consolidation_key() includes the advisory ID), but the data was already gone. In diff mode, new-alert rows aren't affected because added packages come from the diff-scan artifacts, not the stream.

The same stream also feeds full-scan mode when --enable-json, --enable-sarif or --enable-gitlab-security is set (Core.create_full_scan_with_report_url → get_sbom_data), and existing alerts in diff mode for --enable-gitlab-security and --legal-format fossa. Those outputs now get one entry per advisory as well. Live example, PyPI project (Django 2.2, Pillow 6.2.0 and others), full-scan mode with --enable-json: new_alerts went from 5 alerts / 3 Block to 22 / 20 Block. The changelog has a bullet for this.

Fix

  • Pin socketdev==3.6.4. That release keys merged alerts on the advisory ID and keeps the most severe action (error > warn > monitor > ignore).

  • No CLI logic change. get_unchanged_alerts already filters to error/warn before it dedupes on consolidation_key(), so it handles the full alert list correctly once it gets it.

  • New tests in tests/core/test_existing_alerts.py run the real FullScans.stream() over a canned NDJSON response (the existing tests stub stream, so they never exercised the SDK merge):

    • six same-severity handlebars CVEs → 6 existing alerts, "Loaded 6 alerts", run fails, PR comment lists all 6 GHSA IDs
    • first CVE ignore or monitor, second error → the Block CVE is kept and the run fails

    They fail on socketdev 3.6.2 (3 failed; the triaged case returns no existing alerts) and pass with the Doug/fix reachability engine cwd #134 branch (3.6.4).

  • Changelog: a ### Changed bullet for the SDK bump and three ### Fixed bullets under ## 2.13.3.

Version: 2.13.1 and 2.13.2 were released from main on 2026-11-20, so this branch merges main at v2.13.2 and bumps to 2.13.3 in pyproject.toml, socketsecurity/__init__.py and uv.lock, with its CHANGELOG entries under a new ## 2.13.3 heading. In uv.lock only the socketsecurity version line changes; its socketdev entry stays at 3.6.3 until 3.6.4 is published, which is why uv lock --locked fails. #459, #460, #461, #462, #463, #464 and #465 all make the same bump, so the version files merge cleanly between them, and once one merges the rest ride main's unreleased 2.13.3 (Version Check accepts a version equal to main's while it is above the latest PyPI release). Merging a second one conflicts only in CHANGELOG.md: keep one ## 2.13.3 heading and combine the bullets under its subsections (Upgrade notes, Changed, Fixed, Documentation, in that order). There are no code conflicts between them. If main releases 2.13.3 before this merges, bump to the next patch again.

Tests

  • pytest with socketdev 3.6.4 installed from the SDK branch, API key variables unset: 1004 passed (main: 1001 passed). ruff check passes.
  • hatch build and twine check dist/* pass.

End to end: stock 2.13.0 + socketdev 3.6.2 vs this branch + the #134 SDK branch, against a test org. Each project has a default-branch baseline and a PR branch with no dependency change, run with --strict-blocking --ignore-commit-files --integration api --scm api. "Raw" is counted from GET /orgs/{org}/full-scans/{id} for the same scan: distinct (package, alert type, advisory) with action error. Patched runs were repeated twice with identical results.

Project Build Exit Alerts loaded Existing blockers (CLI) Blockers (raw API) Existing rows in PR comment
npm: lodash 4.17.11 + handlebars 4.0.11 stock 1 49 3 8 3
patched x2 1 68 8 8 8 (6 handlebars rows, each with its own GHSA)
npm: minimist, node-forge, jsonwebtoken, axios, ejs, underscore (old versions) stock 1 61 4 4 4
patched x2 1 117 4 4 4
PyPI: Django 2.2, Jinja2 2.10, requests 2.19.0, PyYAML 5.3, urllib3 1.24.1, Pillow 6.2.0 stock 1 56 3 20 5
patched x2 1 189 20 20 22 (20 Block + 2 Warn)
Public MERN app, 1,545 packages stock 1 2,744 17 22 17
patched x2 1 2,879 22 22 22
npm: vm2 3.9.11, first-listed Critical CVE resolved (ignore) stock x2 0 19 0 34 none
patched x2 1 73 34 34 34
  • In the second npm project every package has at most one Critical CVE, so the blocker count doesn't change. The extra alerts loaded are same-severity High and Medium CVEs that were collapsed.
  • The vm2 case used a repo-scoped alert resolution on the first-listed Critical CVE, deleted after the test. A legacy triage rule wasn't enough there because the org's alert policy outranks it.
  • New-alerts path unchanged: adding vm2 on a PR branch gives the same 35 new-alert rows with both builds (with and without --strict-blocking). A non-strict run on the unchanged branch exits 0 with both. Adding a package to the MERN app changes only the existing count (17 → 22).
  • With 35 new rows plus 8 existing, the comment's size budget hides 4 existing rows ("4 more existing alert(s) are not shown"). That's the size cap from Fail on existing alerts under --strict-blocking, streamline exit code behavior #453, not this change; the header still says 8.
  • One row per advisory fills that size budget sooner, so a capped comment can name fewer distinct packages than before. In a combined run with 35 new vm2 rows and 10 existing blockers, the stock comment's existing table covered 5 packages; with this change it shows 4 rows covering 2 packages, then "6 more existing alert(s) are not shown" and the report link. The counts in the header and log and the exit code stay correct. A possible follow-up is to list each package once before its extra advisories.

Public Changelog

--strict-blocking lists and counts every CVE on an unchanged package, and a triaged CVE no longer hides a Block CVE of the same severity on that package. JSON, SARIF and GitLab security reports from a full scan also list every advisory on a package.

Existing alerts for --strict-blocking are read from the new full scan
through socketdev's fullscans.stream(). socketdev 3.6.2 merged alerts that
share a type and severity on a package, so only the first CVE of each
severity reached the CLI. The Existing alerts table and the EXISTING
blocking issues count undercounted (3 instead of 8 in the repro), and when
the first CVE of a group was resolved or triaged to monitor or ignore, the
Block CVEs behind it were dropped and the run passed.

socketdev 3.6.3 keys merged alerts on the advisory ID and keeps the most
severe action. Pin it. The CLI's own consolidation already keeps the
advisory ID (Issue.consolidation_key) and filters on error/warn before it
dedupes, so no CLI logic changes.

The new tests run the real FullScans.stream() over a canned response, so
they fail if the pinned SDK collapses advisories again.

Bumps the version to 2.13.1. uv.lock needs `uv lock` once socketdev 3.6.3
is published.
@lelia lelia self-assigned this Nov 20, 2026
@lelia lelia changed the title List every existing advisory under --strict-blocking List every existing advisory under --strict-blocking Nov 20, 2026
Full-scan mode with --enable-json, --enable-sarif or --enable-gitlab-security
reads alerts through the same SDK stream, so those reports now list every
advisory on a package too. Diff mode's existing alerts in the GitLab and
FOSSA-format output change the same way.
socketdev 3.6.3 was released without the advisory dedupe fix, which now
ships as 3.6.4. uv.lock still needs `uv lock` once 3.6.4 is published.
…eep-each-existing-advisory

# Conflicts:
#	CHANGELOG.md
#	pyproject.toml
…eep-each-existing-advisory

# Conflicts:
#	CHANGELOG.md
2.13.2 was released from main, so this change needs the next patch version.

This branch had an error being deployed

1 failed deployment
socket-firewall — 79177a5b Deployed Nov 20, 2026 by dc-larsen via python-sfw-smoke-enterprise #668
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants