Skip to content

[Tracer] Updating Statsd in RuntimeEventListener #7512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Sep 15, 2025
Merged

[Tracer] Updating Statsd in RuntimeEventListener #7512

merged 1 commit into from
Sep 15, 2025

Conversation

@link04
Copy link
Contributor

@link04 link04 commented Sep 13, 2025 •
edited by lucaspimentel
Loading

Summary of changes

Fixed runtime metrics reinitialization issue by added the option to UpdateStatsd to IRuntimeMetricsListener and implementing it in related classes.

Reason for change

Customer helped us notice runtime metrics system was experiencing issues where:

  • Metrics were missing after upgrade
  • The Statsd object wasn't being properly updated during reinitialization
  • Runtime metrics listeners were holding onto stale Statsd references

Implementation details

  • Added UpdateStatsd method to IRuntimeMetricsListener.cs interface
  • Modified RuntimeEventListener.cs and related classes to properly handle reinitialization scenarios
  • Updated RuntimeMetricsWriter.cs to use newly created statsd object
  • Changed _statsd field from readonly to mutable in all runtime metrics listener implementations

Test coverage

Added test UpdateStatsdOnReinitialization to confirm the metrics are being sent where expected.

Other details

Fixes Jira APMS-17172

@dd-trace-dotnet-ci-bot
Copy link

Execution-Time Benchmarks Report ⏱️

Execution-time results for samples comparing the following branches/commits:

Execution-time benchmarks measure the whole time it takes to execute a program. And are intended to measure the one-off costs. Cases where the execution time results for the PR are worse than latest master results are shown in red. The following thresholds were used for comparing the execution times:

  • Welch test with statistical test for significance of 5%
  • Only results indicating a difference greater than 5% and 5 ms are considered.

Note that these results are based on a single point-in-time result for each branch. For full results, see the dashboard.

Graphs show the p99 interval based on the mean and StdDev of the test run, as well as the mean value of the run (shown as a diamond below the graph).

gantt
    title Execution time (ms) FakeDbCommand (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7512) - mean (72ms)  : 71, 74
     .   : milestone, 72,
    master - mean (72ms)  : 71, 73
     .   : milestone, 72,

    section Baseline
    This PR (7512) - mean (69ms)  : 64, 74
     .   : milestone, 69,
    master - mean (69ms)  : 65, 73
     .   : milestone, 69,

    section CallTarget+Inlining+NGEN
    This PR (7512) - mean (1,004ms)  : 980, 1027
     .   : milestone, 1004,
    master - mean (1,001ms)  : 974, 1028
     .   : milestone, 1001,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7512) - mean (107ms)  : 106, 108
     .   : milestone, 107,
    master - mean (107ms)  : 105, 109
     .   : milestone, 107,

    section Baseline
    This PR (7512) - mean (107ms)  : 105, 109
     .   : milestone, 107,
    master - mean (106ms)  : 104, 108
     .   : milestone, 106,

    section CallTarget+Inlining+NGEN
    This PR (7512) - mean (707ms)  : 683, 731
     .   : milestone, 707,
    master - mean (708ms)  : 690, 726
     .   : milestone, 708,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7512) - mean (95ms)  : 93, 96
     .   : milestone, 95,
    master - mean (94ms)  : 93, 95
     .   : milestone, 94,

    section Baseline
    This PR (7512) - mean (94ms)  : 92, 96
     .   : milestone, 94,
    master - mean (94ms)  : 92, 96
     .   : milestone, 94,

    section CallTarget+Inlining+NGEN
    This PR (7512) - mean (668ms)  : 647, 689
     .   : milestone, 668,
    master - mean (666ms)  : 651, 681
     .   : milestone, 666,
Loading 
gantt
    title Execution time (ms) FakeDbCommand (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7512) - mean (93ms)  : 92, 95
     .   : milestone, 93,
    master - mean (93ms)  : 92, 94
     .   : milestone, 93,

    section Baseline
    This PR (7512) - mean (93ms)  : 91, 95
     .   : milestone, 93,
    master - mean (92ms)  : 90, 94
     .   : milestone, 92,

    section CallTarget+Inlining+NGEN
    This PR (7512) - mean (593ms)  : 583, 603
     .   : milestone, 593,
    master - mean (600ms)  : 588, 611
     .   : milestone, 600,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Framework 4.8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7512) - mean (197ms)  : 194, 199
     .   : milestone, 197,
    master - mean (196ms)  : 193, 198
     .   : milestone, 196,

    section Baseline
    This PR (7512) - mean (193ms)  : 188, 199
     .   : milestone, 193,
    master - mean (193ms)  : 188, 199
     .   : milestone, 193,

    section CallTarget+Inlining+NGEN
    This PR (7512) - mean (1,114ms)  : 1082, 1146
     .   : milestone, 1114,
    master - mean (1,112ms)  : 1080, 1143
     .   : milestone, 1112,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET Core 3.1) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7512) - mean (277ms)  : 273, 282
     .   : milestone, 277,
    master - mean (277ms)  : 272, 282
     .   : milestone, 277,

    section Baseline
    This PR (7512) - mean (277ms)  : 271, 283
     .   : milestone, 277,
    master - mean (277ms)  : 271, 283
     .   : milestone, 277,

    section CallTarget+Inlining+NGEN
    This PR (7512) - mean (898ms)  : 873, 923
     .   : milestone, 898,
    master - mean (898ms)  : 864, 932
     .   : milestone, 898,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 6) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7512) - mean (271ms)  : 266, 276
     .   : milestone, 271,
    master - mean (271ms)  : 265, 276
     .   : milestone, 271,

    section Baseline
    This PR (7512) - mean (271ms)  : 265, 276
     .   : milestone, 271,
    master - mean (269ms)  : 264, 275
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7512) - mean (882ms)  : 846, 918
     .   : milestone, 882,
    master - mean (878ms)  : 838, 918
     .   : milestone, 878,
Loading 
gantt
    title Execution time (ms) HttpMessageHandler (.NET 8) 
    dateFormat  X
    axisFormat %s
    todayMarker off
    section Bailout
    This PR (7512) - mean (269ms)  : 263, 274
     .   : milestone, 269,
    master - mean (268ms)  : 262, 273
     .   : milestone, 268,

    section Baseline
    This PR (7512) - mean (268ms)  : 262, 275
     .   : milestone, 268,
    master - mean (269ms)  : 263, 274
     .   : milestone, 269,

    section CallTarget+Inlining+NGEN
    This PR (7512) - mean (789ms)  : 767, 812
     .   : milestone, 789,
    master - mean (794ms)  : 771, 818
     .   : milestone, 794,
Loading

@link04 link04 marked this pull request as ready for review September 15, 2025 05:38
@link04 link04 requested a review from a team as a code owner September 15, 2025 05:38
andrewlock
andrewlock approved these changes Sep 15, 2025
View reviewed changes
Copy link
Member

@andrewlock andrewlock left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work 👍

bouwkast
bouwkast approved these changes Sep 15, 2025
View reviewed changes
Copy link
Collaborator

@bouwkast bouwkast left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@link04 link04 merged commit 65cf9f5 into master Sep 15, 2025
158 checks passed
@link04 link04 deleted the maximo/updating-statsd-in-listeners branch September 15, 2025 13:35
@github-actions github-actions bot added this to the vNext-v3 milestone Sep 15, 2025
link04 added a commit that referenced this pull request Sep 15, 2025
@link04
[Tracer] Updating Statsd in RuntimeEventListener (#7512)
a39e5fb 
## Summary of changes
Fixed runtime metrics reinitialization issue by added the option to
`UpdateStatsd` to IRuntimeMetricsListener and implementing it in related
classes.

## Reason for change
Customer helped us noriced runtime metrics system was experiencing
issues where:
- Metrics were missing after upgrade
- The Statsd object wasn't being properly updated during
reinitialization
- Runtime metrics listeners were holding onto stale Statsd references

## Implementation details
- Added `UpdateStatsd` method to `IRuntimeMetricsListener.cs` interface
- Modified `RuntimeEventListener.cs `and related classes to properly
handle reinitialization scenarios
- Updated `RuntimeMetricsWriter.cs` to use newly created _statsd_ object
- Changed `_statsd` field from readonly to mutable in all runtime
metrics listener implementations

## Test coverage
Added test `UpdateStatsdOnReinitialization` to confirm the metrics are
being sent where expected.

## Other details
Fixes Jira
[APMS-17172](https://datadoghq.atlassian.net/browse/APMS-17172)


<!--  ⚠️ Note:

Where possible, please obtain 2 approvals prior to merging. Unless
CODEOWNERS specifies otherwise, for external teams it is typically best
to have one review from a team member, and one review from apm-dotnet.
Trivial changes do not require 2 reviews.

MergeQueue is NOT enabled in this repository. If you have write access
to the repo, the PR has 1-2 approvals (see above), and all of the
required checks have passed, you can use the Squash and Merge button to
merge the PR. If you don't have write access, or you need help, reach
out in the #apm-dotnet channel in Slack.
-->


[APMS-17172]:
https://datadoghq.atlassian.net/browse/APMS-17172?atlOrigin=eyJpIjoiNWRkNTljNzYxNjVmNDY3MDlhMDU5Y2ZhYzA5YTRkZjUiLCJwIjoiZ2l0aHViLWNvbS1KU1cifQ
This was referenced Sep 15, 2025
Closed
Merged
link04 added a commit that referenced this pull request Sep 15, 2025
@link04 @tonyredondo
Maximo/hotfix patch 3.26.2 (#7518)
aec1d6c 
## Summary of changes
Trying to release fixes for telemetry and Statsd:

- #7503
- #7507
- #7512

---------

Co-authored-by: Ganesh Jangir <ganesh.jangir@datadoghq.com>
Co-authored-by: Tony Redondo <tony.redondo@datadoghq.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@bouwkast bouwkast bouwkast approved these changes

@andrewlock andrewlock andrewlock approved these changes

Assignees

No one assigned

Labels

area:runtime-metrics identified-by:customer type:bug

Projects

None yet

Milestone

3.27.0

Development

Successfully merging this pull request may close these issues.

3 participants

@link04 @bouwkast @andrewlock