Repository navigation
feat: Add per-metric windows to lock free metrics batcher - #8327
alexander-alderman-webb wants to merge 37 commits into
Conversation
Coverage Results 📊✅ 59490 passed | ⏭️ 3035 skipped | Total: 62525 | Pass Rate: 95.15% | Execution Time: 239m 36s 📊 Comparison with Base Branch
All tests are passing successfully. ✅ Patch coverage is 88.89% (112 of 126 changed executable lines covered; target 80%). Changed files with executable lines (2)
Coverage diff@@ Coverage Diff @@
## master #8327 +/-##
==========================================
- Coverage 90.22% 90.01% -0.21%
==========================================
Files 203 201 -2
Tracked lines 27349 22351 -4998
Branches 10252 7826 -2426
==========================================
- Hits 24675 20119 -4556
+ Misses 2674 2232 -442
+ Partials 1557 1292 -265Generated by Coverage Action |
…window-metrics-batcher-2
…window-metrics-batcher-2 # Conflicts: # tests/test_metrics.py
|
I just inlined |
sentrivana
left a comment
There was a problem hiding this comment.
Two things:
- Retiring/harvesting might have a race, see comment on the diff
- I like the lock-free design with the buckets by thread, it's cool. But it also means that memory usage scales with how many threads are active at a time, which could potentially be a lot. Might need some testing to see if that's an issue.
| window = self._retired.popleft() | ||
| except IndexError: | ||
| break | ||
| sampled_out += self._harvest(window, items) |
There was a problem hiding this comment.
Wondering if we should decouple the harvesting from the flush somehow, or at least introduce some small delay in between. A recently retired window might probably still have a thread writing to it, if the write was initiated before it was retired, so we might be losing some metrics there.
There was a problem hiding this comment.
I've added another 0.05 seconds delay so there's at least a guaranteed buffer since I couldn't come up with a good synchronization mechanism to strictly not lose any metrics.
See 5e08286
Yes great point, currently each reservoir can hold up to I just realized that we dropped unsampled metrics when we reach We could add a mechanism where metrics are pruned when the |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes and found 1 potential issue.
There are 2 total unresolved issues (including 1 from previous review).
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 008f662. Configure here.

Description
Keeps window-sampled metrics in a thread-local reservoir, and performs uniform sampling once the window is no longer the active window. Segmenting metrics per thread while the window is being modified enables lock-free operations (only a thread touches its state).
Conceptually, each window has thread state, and each thread state has a reservoir of metrics.
There can be multiple consumers if
sentry_sdk.flush()runs concurrently with the background flusher. This is expected and handled byIndexError.Issues
Reminders
uv run ruff.feat:,fix:,ref:,meta:)