Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,21 @@
# Changelog

## 2.13.3

### Fixed

- With `--scm github`, `pull_request` runs with the `reopened` or
`ready_for_review` action now scan the pull request. They exited `0` without
scanning, so a reopened pull request could pass a required check on a commit an
earlier run failed.
- Skipped GitHub events and actions, and skipped GitLab pipeline sources, log a
warning that no scan ran. `pull_request_target` gets its own message, and a
missing event name or action says it could not be determined. Exit codes are
unchanged.
- `docs/ci-cd.md` explains that the example GitHub workflow's `push` trigger also
runs the job on a pull request's head commit as a full scan that exits `0`, and
how to keep that run from satisfying a required check.

## 2.13.2

### Added
Expand Down
47 changes: 45 additions & 2 deletions docs/ci-cd.md
Original file line number Diff line number Diff line change
Expand Up @@ -84,8 +84,9 @@ they also hide findings and usage errors. Even with exit `0`, a setup/API failur
does not mean a clean scan; check the logs for a completed report.

Findings block only when a [diff comparison runs](#scan-type-by-pipeline).
SCM event filters can also skip unsupported events with exit `0`; confirm a report
was created when validating a new workflow.
SCM event filters can also skip unsupported events with exit `0` and a warning
(see [Skipped events](#skipped-events)); confirm a report was created when
validating a new workflow.
`--strict-blocking` also checks existing blocking alerts; an incomplete comparison
or incomplete existing-alert data is an infrastructure failure. See the
[2.13.0 migration table](../README.md#upgrading-to-2130) when updating existing CI rules.
Expand Down Expand Up @@ -520,6 +521,48 @@ runs the comparison without the PR comment adapter.
`--generate-license` and `--legal-format fossa` work on both paths; a full scan
fetches the package list for them.

### Skipped events

With `--scm github`, `pull_request` runs scan on the `opened`, `synchronize`,
`reopened` and `ready_for_review` actions. Other actions (`closed`, `labeled`,
`edited`, ...) and events the CLI does not handle (`pull_request_target`,
`merge_group`, `schedule`, ...) are skipped: no scan runs, the CLI logs a warning
and exits `0`. `issue_comment` runs the ignore-command flow, not a scan. With
`--scm gitlab`, pipeline sources other than `web`, `merge_request_event`, `push`,
`api`, `pipeline` and the comment flow are skipped the same way.

GitHub doesn't document which run a required check uses when a job with the same
name runs more than once on a commit, so a skipped run that passes can satisfy the
check in place of an earlier failure. Subscribe the Socket job only to events and
actions the CLI scans.

- `merge_group`: if the required check also runs on merge queue events, the job
passes there without scanning. The `pull_request` run is the gate.
- `pull_request_target` is skipped because it runs against the base branch, so it
can't stand in for `pull_request` on fork pull requests. Fork pull requests on
`pull_request` get no repository secrets, so the CLI has no API token and exits
`3` (or the `--exit-code-on-api-error` value). Decide how fork pull requests get
a token before making the check required.

### Push and pull request runs on the same commit

The [example workflow](../workflows/github-actions.yml) runs one `socket-security`
job on `push` to every branch and on `pull_request`. A new commit on a pull
request's branch triggers both events on the same head commit. The `pull_request`
run compares against the baseline and can fail. The `push` run is a full scan,
which never blocks and exits `0`. Both runs report a check with the same name, so
a required check can resolve against the passing `push` run.

If the Socket job is a required check, do one of these:

- Restrict `push` to the default branch (`branches: [main]`). Default-branch scans
still keep the baseline current; feature branches are scanned only through their
pull requests.
- Run the pull request scan in its own job (for example `socket-pr` with
`if: github.event_name == 'pull_request'`, and
`if: github.event_name != 'pull_request'` on the existing job), and require that
job's check.

## Pull request and Dashboard association

The CLI sends the resolved pull request number with each full scan and attaches
Expand Down
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ build-backend = "hatchling.build"

[project]
name = "socketsecurity"
version = "2.13.2"
version = "2.13.3"
requires-python = ">= 3.11"
license = {"file" = "LICENSE"}
dependencies = [
Expand Down
2 changes: 1 addition & 1 deletion socketsecurity/__init__.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
__author__ = 'socket.dev'
__version__ = '2.13.2'
__version__ = '2.13.3'
USER_AGENT_NAME = 'socket-python-cli'
USER_AGENT = f'{USER_AGENT_NAME}/{__version__}'
60 changes: 54 additions & 6 deletions socketsecurity/core/scm/github.py
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,18 @@
from socketsecurity.exit_codes import InfrastructureError
from socketsecurity.socketcli import CliClient

# pull_request actions that can leave a PR head without a current Socket result:
# new commits (opened, synchronize) and a PR coming back after it was closed or
# drafted (reopened, ready_for_review). A skipped run exits 0, and GitHub doesn't
# define which run a required check uses when the same job runs more than once on a
# commit, so a skipped run on one of these could satisfy the check in place of an
# earlier failure. Other actions (closed, labeled, edited, ...) are skipped with a
# warning.
PULL_REQUEST_SCAN_ACTIONS = ("opened", "synchronize", "reopened", "ready_for_review")
HANDLED_EVENTS = (
"push, workflow_dispatch and pull_request (scan); issue_comment (ignore commands)"
)


@dataclass
class GithubConfig:
Expand Down Expand Up @@ -175,19 +187,55 @@ def __init__(
raise InfrastructureError("Unable to get Github API Token")

def check_event_type(self) -> str:
if self.config.event_name.lower() in ["push", "workflow_dispatch"]:
event_name = self.config.event_name.lower()
if event_name in ["push", "workflow_dispatch"]:
if not self.config.pr_number:
return "main"
return "diff"
elif self.config.event_name.lower() == "pull_request":
if self.config.event_action and self.config.event_action.lower() in ['opened', 'synchronize']:
elif event_name == "pull_request":
action = (self.config.event_action or "").lower()
if action in PULL_REQUEST_SCAN_ACTIONS:
return "diff"
log.info(f"Pull Request Action {self.config.event_action} is not a supported type")
if not action:
log.warning(
"Could not determine the pull_request action: EVENT_ACTION is not set "
"and GITHUB_EVENT_PATH has no 'action'. No scan ran; exiting 0. Set "
"EVENT_ACTION (or GITHUB_EVENT_PATH) when running with --scm github "
"outside GitHub Actions."
)
sys.exit(0)
log.warning(
f"Pull request action '{self.config.event_action}' is not scanned "
f"(scanned actions: {', '.join(PULL_REQUEST_SCAN_ACTIONS)}). No scan ran; "
"exiting 0. If this job is a required check, its success here does not "
"reflect a scan: remove this action from the workflow's pull_request types."
)
sys.exit(0)
elif self.config.event_name.lower() == "issue_comment":
elif event_name == "issue_comment":
return "comment"
elif event_name == "pull_request_target":
# Not treated as pull_request on purpose: GITHUB_SHA and GITHUB_REF point at
# the base branch and the default checkout is the base branch, so a "PR scan"
# here would compare the base against itself and pass.
log.warning(
"GitHub event 'pull_request_target' is not scanned: it runs against the "
"base branch, not the pull request head. No scan ran; exiting 0. Trigger "
"the Socket job on pull_request instead."
)
sys.exit(0)

log.error(f"Unknown event type {self.config.event_name}")
if not event_name:
log.warning(
"Could not determine the GitHub event: GITHUB_EVENT_NAME is not set. "
"No scan ran; exiting 0. Set GITHUB_EVENT_NAME when running with "
"--scm github outside GitHub Actions."
)
sys.exit(0)
log.warning(
f"GitHub event '{self.config.event_name}' is not scanned (handled events: "
f"{HANDLED_EVENTS}). No scan ran; exiting 0. If this job is a required "
"check, its success here does not reflect a scan."
)
sys.exit(0)

def post_comment(self, body: str) -> None:
Expand Down
14 changes: 13 additions & 1 deletion socketsecurity/core/scm/gitlab.py
Original file line number Diff line number Diff line change
Expand Up @@ -213,8 +213,20 @@ def check_event_type(self) -> str:
return "diff"
elif pipeline_source == "issue_comment":
return "comment"
elif not pipeline_source:
log.warning(
"Could not determine the GitLab pipeline source: CI_PIPELINE_SOURCE is "
"not set. No scan ran; exiting 0. Set CI_PIPELINE_SOURCE when running "
"with --scm gitlab outside GitLab CI."
)
sys.exit(0)
else:
log.error(f"Unknown event type {pipeline_source}")
log.warning(
f"GitLab pipeline source '{self.config.pipeline_source}' is not scanned "
"(handled sources: web, merge_request_event, push, api and pipeline "
"(scan); issue_comment (ignore commands)). No scan ran; exiting 0. If "
"this job gates merges, its success here does not reflect a scan."
)
sys.exit(0)

def post_comment(self, body: str) -> None:
Expand Down
12 changes: 9 additions & 3 deletions tests/core/test_cli_exit_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -275,14 +275,20 @@ def test_branch_full_scan_does_not_gate_on_findings(cli_runtime, monkeypatch, ou

@pytest.mark.parametrize("scm, event, action", [
("github", "pull_request", "closed"), ("github", "unsupported", ""),
("gitlab", "unsupported", ""),
("github", "pull_request_target", "opened"), ("gitlab", "unsupported", ""),
])
def test_scm_event_filters_skip_without_scanning(cli_runtime, monkeypatch, scm, event, action):
def test_scm_event_filters_skip_without_scanning(cli_runtime, monkeypatch, caplog, scm, event, action):
monkeypatch.setenv("GH_API_TOKEN", "test")
monkeypatch.setenv("GITLAB_TOKEN", "test")
monkeypatch.setenv("GITHUB_EVENT_NAME", event)
monkeypatch.setenv("EVENT_ACTION", action)
monkeypatch.setenv("CI_PIPELINE_SOURCE", event)

assert cli_runtime.run(["--scm", scm]) == 0
with caplog.at_level("INFO", logger="socketdev"):
assert cli_runtime.run(["--scm", scm]) == 0
cli_runtime.scan.assert_not_called()
# A skipped run passes, so it has to say so where a required-check owner looks.
assert any(
record.levelname == "WARNING" and "No scan ran" in record.getMessage()
for record in caplog.records
)
Loading