| Wargo | |
| Jun 16 2025, 9:33 PM |
| F62383895: image.png | |
| Jun 19 2025, 1:42 AM |
| F62376697: Zrzut ekranu 2025-06-18 152617.png | |
| Jun 18 2025, 1:31 PM |
Some users, including me, noticed some notifications are sending multiple times and they are repeated every hour.
For me it is mention notification from 5 "selected" actions made two months ago. These notifications are created for about week.
Example edit, with notification ID:
https://pl.wikipedia.org/w/index.php?title=Dyskusja:Nothofagus_discoidea&oldid=prev&diff=76391030&markasread=29692500&markasreadwiki=plwiki
Same old issue: T257766
| Status | Subtype | Assigned | Task | ||
|---|---|---|---|---|---|
| Resolved | BUG REPORT | Atieno | T397103 Constant repeating notifications | ||
| Open | Atieno | T397520 Echo: Add tests for revert detection |
includes/MediaWikiEventIngress/PageEventIngress.php if ( $editResult == null ) {
I guess this can be left if there are situations when editResult object is not given, but the null edit itself needs to be [also] checked as previous: $editResult->isNullEdit().
This might be related to T393467: Echo: Port onPageSaveComplete to PageUpdated event?
Pulling into our sprint to take a look! @Wargo -- do you have any additional information about the types or frequency of notifications that you're seeing repeat? We need to understand the impact; if it's all notifications, we should probably roll back the change.
@Atieno -- since these are your chances, could you please take the lead on investigation? Tagging @matmarex and @pmiazga as Echo experts, as well.
About frequency: one notification per hour about one change (revert) in April (https://pl.wikipedia.org/w/index.php?diff=76393193&oldid=76387960&title=Dyskusja_wikipedysty%3ADownhands). The first notification on my account was sent on June 13th about 7 pm CET.
Mention. But only for 5 (for me) constant pages (edits); older actions do not trigger them. Every hour or null edit.
I can reproduce this locally – ping another user in an edit summary, and then every time you null-edit the page, they get another notification.
Tested with git bisect, it's caused by rECHO8dfef39cd0c5: DomainEvents: Port onPageSaveComplete hook to PageUpdatedEvent (https://gerrit.wikimedia.org/r/c/mediawiki/extensions/Echo/+/1148316).
This seems correct to me.
Comparing side-by-side the old code and new code, we seem to have swapped $editResult->isNullEdit() for $editResult == null, which is not the same. But the new interface allows $editResult to be null, while the old one didn't; and it may not be clear from the name of the new interface that it passes through null edits; so maybe these are actually two unrelated changes that just look cofusing.
You can actually see the increased rate of notifications on this dashboard: https://grafana.wikimedia.org/d/onyD7cOMk/echo-extension-notification-baseline-track?orgId=1&from=now-14d&to=now&timezone=utc&viewPanel=panel-2 if you select "mention_summary" in the list on the side (but in the default view it's completely drowned out by more common events):
If I'm reading this right (and it's not too easy to read out the numbers), we've gone from sending about 20 summary mention notifications per hour (across all Wikimedia projects), to about 500 per hour. We should probably backport the fix tomorrow. @Atieno I'll let you write the patch, since you're already working on this (unless you're not working tomorrow, I know it's a holiday in lots of countries).
I clicked through all notification types on that dashboard, and luckily this seems to be the only type that is affected. Some other types have more robust checks to avoid sending notifications for edits that were already processed.
The event name PageRevisionUpdated or newly renamed PageLatestRevisionChanged doesn't make it too obvious that it's fired for null edits, but it's noted in the event's documentation. We probably shouldn't rename it again :) but this behavior is something to keep in mind for the future migrations (@Atieno @daniel).
By the way, I noticed that we also changed $isRevert = $editResult->getRevertMethod() === EditResult::REVERT_UNDO || $editResult->getRevertMethod() === EditResult::REVERT_ROLLBACK; to $isRevert = $event->isRevert();, which is not exactly the same either. As a result, manual reverts (i.e. viewing an older version of the page, then clicking edit and save; not using the undo or rollback buttons) will now be treated as reverts by this code.
This doesn't affect the revert notifications themselves, since they are sent elsewhere, but it will affect some other notification types that check whether an edit is a revert before sending notifications: at a glance, this will affect the summary mention notifications too (they are not sent for reverts), possibly fixing T373750 but making T340883 a bit worse; and page linked notifications (also not sent for reverts), apparently fixing T148949.
The change itself is probably okay (I think the way it worked was just a historical accident, and not intentional), but someone should check that these tasks are actually affected (I just read the code, haven't tested), maybe write unit tests for them.
hey @matmarex I'm working today....Maybe might be for later fix but considering adding testcases that would catch these
Change #1161507 had a related patch set uploaded (by Atieno; author: Atieno):
[mediawiki/extensions/Echo@master] DomainEvents: Constant repeating notifications
Change #1161589 had a related patch set uploaded (by Bartosz Dziewoński; author: Atieno):
[mediawiki/extensions/Echo@wmf/1.45.0-wmf.6] DomainEvents: Constant repeating notifications
Change #1161507 merged by jenkins-bot:
[mediawiki/extensions/Echo@master] DomainEvents: Constant repeating notifications
Change #1161589 merged by jenkins-bot:
[mediawiki/extensions/Echo@wmf/1.45.0-wmf.6] DomainEvents: Constant repeating notifications
Mentioned in SAL (#wikimedia-operations) [2025-06-19T20:26:26Z] <kharlan@deploy1003> Started scap sync-world: Backport for [[gerrit:1161588|PageChangeEmissionTest: order move events by kind. (T397087)]], [[gerrit:1161589|DomainEvents: Constant repeating notifications (T397103)]]
Mentioned in SAL (#wikimedia-operations) [2025-06-19T20:28:32Z] <kharlan@deploy1003> kharlan, matmarex: Backport for [[gerrit:1161588|PageChangeEmissionTest: order move events by kind. (T397087)]], [[gerrit:1161589|DomainEvents: Constant repeating notifications (T397103)]] synced to the testservers (see https://wikitech.wikimedia.org/wiki/Mwdebug). Changes can now be verified there.
Mentioned in SAL (#wikimedia-operations) [2025-06-19T20:36:35Z] <kharlan@deploy1003> Finished scap sync-world: Backport for [[gerrit:1161588|PageChangeEmissionTest: order move events by kind. (T397087)]], [[gerrit:1161589|DomainEvents: Constant repeating notifications (T397103)]] (duration: 10m 08s)