Closing this task and moving to the next quarter task :)

@Jdlrobson-WMF Thank you for this! Question: does userInGroup allow for exclusion of groups rather than inclusion? In some cases, we will want to exclude Temp users from registered users surveys (e.g. asking questions about a feature only available to registered editors, or comparability to past surveys like the Safety Survey; alternatively, as long as we can identify within the quicksurvey response log that a user is a temp user that can help us to treat their responses as distinct).

Central Notice survey review:

• Received a CN request from a community member working with researchers from the University of Zurich on a survey on AI and science communication. I will provide feedback on their materials next week.

Weekly updates:

• The survey was undeployed and closed on Monday the 22nd
• I'm about 95% through the survey data cleaning and recoding, and will consult with @YLiou_WMF next week for weighting strategy.
• Stakeholders are encouraged to reach out if there are any survey questions that are high priority in the next 1-2 weeks, as I can prioritize those analyses if needed.

Weekly updates:

• We're ready to close the survey on Monday :)
• I've started working on the R code for cleaning and question recodes, and will update it with the full dataset once we're done with response collection.

@DDeSouza We're ready to close this survey on Monday or at your earliest convenience after, thank you so much! However, rather than fully undeploying it, we should reduce the coverage back to 0, so we can use this to test some ideas we have about missing QS initation and response events.

Qualtrics:

• @Bethany is now the official Qualtrics account manager, though we will continue to work together on onboarding as new tasks come up through the transition. @TAndic will keep a backup admin account.

QuickSurveys:

• We met across teams a couple weeks ago to start scoping QuickSurvey improvements with Contributor Metrics. The Research team (@TAndic with @YLiou_WMF for input) will consult for the analytics-side requirements (working through scoping here [internal]).

Weekly updates:

• The survey launched on Tuesday the 9th and is collecting data!
• An issue with ad blockers affecting event logs is causing a proportion of responses to not have a corresponding quicksurvey event (which we use for weights); @YLiou_WMF and I anticipated this from the QA process and are brainstorming about how to account for this.

We're ready to increase the coverage and start collecting responses! (as just patched by @DDeSouza)

Weekly updates:

• The survey was announced on the English Wikipedia Village Pump on the 3rd and has had no comments so far (will check again on Monday)
• QA on the survey flow (on-wiki QuickSurvey, link to survey, parse QuickSurvey token to LimeSurvey) appears to be working well. I will pull eventlogging data for the tests on Monday to make sure everything looks okay on that side.
• We're scheduled to deploy (that is, increase QuickSurvey coverage) and start collecting data on Tuesday the 9th!

Thank you so much @DDeSouza! I'll work on checking all the survey flow (linking, logs, QS token passing, etc) components in the meantime :)

Weekly updates:

• Research page and questionnaire published, resources page drafted (but may need a bit more work to make it digestible and engaging)
• Privacy statement request submitted and in progress
• Finalized KR language
• Next steps:
  • QuickSurvey configuration and message component setup
  • Reach out to movement comms
  • Final tweaks to LimeSurvey before activation (add QS variable, finalize and check skip patterns, add privacy statement link when it's ready, and end-of-survey page pointing to resources)

Weekly updates:

• A draft of the survey is available for feedback and testing - I hope to have this finalized next week after everyone has had a chance to review. Thanks to all who have taken it and given feedback already!
• An initial codebook has been made for the survey, with strategic mapping of the questions to KRs; this will later become the survey dataset codebook.
• Next steps:
  • Begin QuickSurvey configuration and message setup
  • Make a metawiki Research: page for the survey
    • Make a resources page for respondents, showing them where to find the tools, pages, and spaces we asked them about
  • Reach out to movement comms for communications guidance
  • Get a survey privacy statement from Legal

Weekly updates:

• Reader foundational research (T398379): (from @MRaishWMF while @YLiou_WMF is away!)
  • continued developing a framework that describes use cases in terms of on-site behavior using examples from recent data collection to illustrate it
  • presented early findings to some stakeholders for feedback on project direction
  • continued working to develop the direction of phase 2 (diary study)
• Retraining recommendation (T399726):
  • Build new evaluation dataset
  • Compare different retraining period lengths (3 months, vs 1 year)
• Research to ML code migration (T398974):
  • Initial discussion for scoping with Özge. The proposal will be based on two components: a project in ml-pipelines for shared code, and the usage of the Command api for airflow configuration.
• Understanding mentorship experiences (T398250):
  • Programmed the first draft of the newcomers survey in LimeSurvey (feel welcome to test it and give feedback on the ticket!)

• Qualtrics SMTP issue resolved in record time thanks to SRE (T400882)! We can test the daily distribution for the diary study next week.

• @MRaishWMF and I discovered that the SMTP relay for Qualtrics isn't functioning (needed for a Readers Diary study T393802), which means survey emails can't be sent automatically through the platform. A possible cause is a new range of IPs added by Qualtrics which needs to be updated on our side, though it could be something else. I'll reach out to SRE to see if we can fix this.

Weekly updates:

• Met with stakeholders to discuss the survey and where it can support related projects
• Shared a survey question brainstorming document for teams to add potential questions; stakeholder teams will input to these with a deadline of July 24th.
• Drafted a research brief (internal) to be updated after survey question brainstorming
• Note: I will be out of office for the next two weeks, so there will be no updates during this time :)

Weekly updates:

• Scoped population viability for editors who registered in the last 6 months, have made 50+ edits (and made an edit in the last month, to approximate likelihood of being sampled on-wiki) with major thanks to @CMyrick-WMF and @Hghani for helping with the query.
  • Finding: Enwiki has a viable population of approximately 2000 qualified editors in any given month; other Wikipedias may have too small of populations and other research approaches may need to be considered if we want to expand to other languages (for example, the estimate for eswiki is around 300 qualified editors).
• Scheduled meetings with potential stakeholders and staff doing related research for next week.
• Began brainstorming potential survey questions (will need input here from relevant teams).

• Moderator Tools Support: Nuke tool follow-up survey deployed last week, and is ready for undeployment.
• Moderator Motivations: Metawiki page with an overview of motivations and potential research avenues posted, and literature has been added to the WMFResearch Zotero library. This research will be presented in a Contributors Strategy meeting in early Q1.
• Metrics on Patrolling Work: Data pipelines have been developed to generate the dataset, which has been made accessible via a Superset dashboard, as specified here. The analysis with the dashboard has revealed moderation gaps, which have been shared with stakeholders and documented in the revised report with feedback from the stakeholders. The dataset is expected to be used to provide data on the retention rate of patrollers using the FlaggedRevs and reverting editors at T396493 as a comparable moderator retention rate metric to inform targets. Furthermore, additional opportunities to leverage this data have been identified at T398071. As a consequence, the hypothesis is supported.

Updates & task closure:

• Metawiki page has been posted, which covers the theoretical bases of motivations and sustained activity, the use of serious games for onboarding to editing, and spaces for further research on motivations.
  • A shorter overview slide deck for staff has been prepared and shared for feedback; this will be presented to the Contributor Strategy group in early Q1.
• Literature has been added to the WMFResearch Zotero library, inclusive of literature which did not make it into the final report (for example, moderator identities, which tend to be retrospective and arise later on a motivational scale).
• We aim to pursue relatedness & mentorship as an underexplored aspect of editing motivations in Q1.

• Moderator Tools Support: Patrollers survey stage two deployed, now on larger projects (English, Commons, Wikidata, French, German).
• Moderator Motivations: Shared overview of motivations slides to moderator research group, discussed ongoing research proposals with @Isaac and now brainstorming more potential mentorship, collaboration & relatedness research avenues (as a subset of motivations)
• Centralizing Contributions:
  • We have updated our public Meta-Wiki page with our key findings and results
  • Stakeholder share-out is scheduled for later today
  • We can consider this research completed
• Metrics on Patrolling Work: This week's efforts primarily focused on interactions with the Moderator Tools and Product Analytics teams. During a joint meeting, the dataset and dashboard were presented, and feedback was gathered (notes). Following the session, both teams engaged further to request additional details, particularly in relation to defining baselines for the projected increase in moderation actions for FY 25/26 WE 1.3 KR (T396493). In parallel, while awaiting input from stakeholders regarding specific product interventions to be tracked using this data, report for this project started to be drafted.
• Peacock Support: Collaborating with @achou to analyze data from languages where we don't have ground-truth to establish an evaluation methodology for those cases.

Hi all, I received a message from a user that the survey was expired - I've gone ahead and unexpired it just so editors aren't confused. Unexpiration was done at ~13:31 UTC (in case you need this time for comparisons to event data).

• Moderator Tools Support: We completed collection on the patroller tools baseline survey, but analysis will be delayed due to deprioritization by moderator tools team. Work is underway to complete the nuke follow-up survey (T396039) by the end of Q4
• Moderator Motivations: Presented progress to the team on and incorporated feedback and ideas. Prepared overview of motivations (theory and applications) for stakeholders to share on Monday.
• Centralizing Contributions: We have finished our final report and sent it to stakeholders for review.
  • Key takeaways:
    • Participants unanimously desire a centralized hub to access, tailored to their interests or most common contribution types
    • Participants also want a centralized hub to discover new moderation activities and relevant wiki policies that they can learn about
    • It is easier for editors to understand why they should perform moderation actions, if they are described as an extension of editing, rather than as a separate category of activities entirely
      • Explicitly calling such activities “moderation” is more likely to confuse users, as opposed to labelling them as “additional editing actions”
    • The moderation activities of almost all study participants is limited to the articles on their watchlist or contribution history
    • Users use off-wiki channels to find on-wiki help articles, suggesting a need for better discoverability for help articles and policy pages
    • Users typically found personalized impact metrics (e.g. pageviews on pages they edited or created) more meaningful than site-wide ones (e.g. number of pages in the Articles for Deletion category)
• Metrics on Patrolling Work: This week’s work focused on building a Superset dashboard to deliver the dataset, which also included modifications to the existing notebooks, e.g., moving the parsing of edit summaries (to extract links to the project namespace) into the data collection phase. The dashboard will be reviewed next week in meetings with colleagues from the Product Analytics and Moderator Tools teams.
• Peacock Support: Helping to define a procedure to analyze feasibility of using the peacock detection on model in languages were we have limited labeled data.

• Moderator Motivations: Proposal submitted for WikiConfNA; Presenting current findings and proposed avenues for further research to Applied Sciences meeting on Monday for feedback (special thanks to @diego for providing feedback this week)
• Metrics on Patrolling Work: (Shorter week for @Pablo) Updated the data collection notebook to incorporate revision tags and the comment of the reverting revision for those who are reverted. The data exploration notebook has been updated to now include the answers to the three questions using the October 2024 dataset. Progress with the data has been reviewed with @Isaac during our 1:1 meeting. In addition, two other meetings have been scheduled with stakeholders as the dataset is expected to be used on an Product Analytics + Moderators Tools teams's effort to measure current moderator activity to inform a Key Result target for WE 1.3 FY 25/26.
• Peacock Support: The team is trying to address the community concerns. Some editors pointed out that tone-check tool might facilitate the work to spammers/pay-editors. Our contributions from research to discussion are:
  • Acknowledge that this risk is real, and we have shared this as a concern.
  • Suggest some mitigation strategies:
    • Add a tag if the revision triggered the "peacock alarm" before submitting. Because the main paradigm shift were are introducing here is that we are not logging the editing process, and a tag could partially mitigate this problem.
    • We are writing a more complete model description, so editors can understand how the model was trained.

• Moderator Motivations: Onboarded @MRaishWMF to current progress and next steps; Continued literature review write-up (core sections: broader theories of motivation; existing literature on moderation, motivations and self identification; gaps in knowledge and applicability within the Wikimedia ecosystem); draft for WikiConNA proposal ready for feedback
• Centralizing Contributions: Finished all participant sessions!
• Metrics on Patrolling Work: Continued development on the data collection notebook. For the October 2024 snapshot, metadata now includes information on addition and removal of article maintenance templates. The data exploration notebook has been also expanded accordingly. First, it show the distribution of revisions per wiki based on the number of links to Wikipedia project namespaces found in edit summaries. This metric will be used to approximate the question: How many editors provided feedback when rolling back an edit?. In addition, the notebook already includes data addressing the question: How many editors have added a messagebox to an article?.

• Moderator Tools Support: No new updates this week (conference attendance).
• Moderator Motivations: No major updates, literature write-up/synthesis draft in progress.
• Centralizing Contributions: We're at 3 sessions complete, 3 more scheduled; if all goes well we will have hit our target session count. We've since learned that our user testing platform for this research, Userlytics, does not permit moderated sessions without the use of a webcam. We have devised a workaround with our existing suite of software for testing, but this will be useful to note for future research held on that platform.
• Metrics on Patrolling Work: Advanced the notebook to create the dataset of edits in March 2025. The dataset has been expanded to include detailed information on reverting editors and predicted revert risk scores. The notebook has been re-run to generate an analogous dataset for edits in October 2024, for which metadata on article maintenance templates will be available (see task T384600). A exploration notebook is already available providing feedback to the proposed question How many editors have reverted at least Y edits in 30 days?.
• Peacock Support: No major updates this week.

• Moderator Tools Support: Survey is collecting data!
• Moderator Motivations: No major updates, focused on literature on extrinsic motivations in online communities.
• Centralizing Contributions: We've started our first participant sessions, so data collection has officially begun! We have three other sessions scheduled so far, on track to hit our target of 6-8 total sessions by May 23.
• Metrics on Patrolling Work: Started working on a notebook to create the dataset of edits in March 2025 with metadata, including mediawiki_history fields and patrolling information and status (prevented, delete, reverted, reviewed, edited_over, autopatrolled).
• Peacock Support: ML-team is using annotool (developed by @MunizaA) to gather evaluation data.

No major updates this week.

• Moderator Tools Support: Still awaiting deployment
• Moderator Motivations: No major updates
• Centralizing Contributions: Using the Design Research Participant Database, we've done our first round of recruitment. We just need to finish a run-through of the testing protocol on our Userlytics platform, and hope to send screeners and start scheduling sessions by end of week.
• Metrics on Patrolling Work: The hypothesis in support of Moderation and the Centralizing Contributions project has been formulated (already posted in Asana). @Isaac J and @Pablo met to discuss details on the deliverable: a dataset of all edits for a month, including metadata on each edit and of its patrolling review process. In addition, a coordination meeting was held with the Moderators Tool team to align on recent and next steps.
• Peacock Support: No updates this week (conference attendance)

• Moderator Tools Support: No major updates this week
• Moderator Motivations: @TAndic continued lit review and existing process explorations (note: I will be OOO next week)
• Centralizing Contributions: @cwylo continued research brief scoping discussions
• Metrics on Patrolling Work: @Pablo initiated scoping for Q4 research
• Peacock Support: @diego risk assessment and mitigation plans

Updates:

• No major updates; Continued with literature review (focused on existing moderation schemas/"ideal types"), looking at existing workflows, informal conversations with staff experienced in moderation

Updates (last 2 weeks):

• @TAndic and @MRaishWMF met to kick off project and brainstorm scope
• Began scoping literature review starting with identifying relevant citations in Ren et al. 2023
• Began exploring current/past workflows and moderation projects (eg. Recent Changes, CVU project) to start to understand moderation better
• Met with wider Research team members working on moderator research (T387462 / @cwylo , @Pablo ); ~6 of us will check in regularly this quarter for resource sharing and dividing scope on related projects (adding @dchen for relevant recent watchlist work as well)

Closing task as criteria are met.

• Alongside translation proceedure and LimeSurvey Codex theme alignment, basic LimeSurvey tutorials are now complete (wikifying on office without UploadWizard for 100+ images is a bit more difficult :); this final step to ensure visibility can be finished this week (office draft for moving once finalized here))

As a reminder, decision briefs and tasks identified that need further scoping will be addressed in Q4.

Update:

• We decided on three small improvements that will be impactful for either the Research team or wider Foundation which can be implemented this quarter:
  • Update the procedure for survey translations within the Research team (@Bethany) [Status: complete T382626]
  • Update the LimeSurvey survey template to align with the design Codex (@DDeSouza) [Status: V1 complete, needs a full survey pause to make into the default template as this will change all current and past surveys using the default template - @TAndic to coordinate this]
  • Develop tutorials for basic LimeSurvey onboarding and reference (and, as use-cases arise, document more technical implementations) (@TAndic) [Status: in progress, V1 estimated March 31]
• Some broader proposals based on the Areas to Address require decision briefs or further scoping and alignment, which will be done in Q4.

Report posted on metawiki, closing this task :)

• Metawiki report is posted
  • There are a few smaller tasks (link updating, informing respondents who wanted to be updated on the results, and community and staff announcements) to do next week, but I believe we can consider this task resolved

Closing this task as the core work is done on this wave of the survey. We will regroup in January for reporting on metawiki with comms support, and have agreed that there will at most be one more wave of safety surveys this FY (to be determined).

Closing this task as Q2 is over :) No major updates for the last two weeks.

End of year stakeholder meeting and activity complete (slides, internal), closing this task as the core work is done but we will have some tail for follow-up and public reporting in January.

Weekly updates:

• We have scheduled a stakeholder meeting for December 19th to review the archetypes and have requested async feedback; currently working on incorporating existing feedback

Weekly updates:

• Data analyzed and reported to T&S for internal review; met last week to wrap up for this quarter and will meet with Nina next week to discuss results and evaluation strategy further. Agreed that we will do metawiki reporting in January with comms support.

Weekly updates:

• Pushed LimeSurvey account creation for staff from Design :) (will assist with prepping them for CN request in January)
• Helped Movement Comms with LimeSurvey skip pattern how-to
• Discussed LimeSurvey branding with Bethany for next quarter

Weekly updates:

• Back to work on the metawiki report! Goal for posting: 19 December

@DDeSouza we're also ready to undeploy the enwiki survey tomorrow, ideally at the last backport window (but either of the last two is fine)

Hi @DDeSouza ! The remaining wikis are ready to deploy at 0 coverage for testing either tomorrow or next Monday (your choice :))

Weekly updates:

• Data cleaning and analysis for EN, ES, FA, FR complete, still need to do JA and PT. Cross-team check in next week.

Weekly updates:

• LimeSurvey renewal is paid (Bug last week paying for it due to unknown IP/offsite hotel internet)
• Updated CentralNotice useage guidelines have been posted, including for survey requests (https://meta.wikimedia.org/wiki/CentralNotice/Usage_guidelines#Special_Kind_of_Banners)
• CN: frwiki researchers reached out with their survey draft for a demographic survey of fr Wikipedians; requested a document draft for comments and recommended they schedule office hours with me as this is a more complex scenario (external academic researcher, demographic survey, intent to release the dataset)
• CN: WM Chile affiliate member reached out about potential regional wikidata survey in 2025, pointed them to recent WMDE wikidata survey and recommended office hours as they plan their project
• Met briefly with staff working on analyzing wiki for human rights survey, shared resources for regional categorization and how to explore data on LimeSurvey

Apologies for the delay here @XiaoXiao-WMF -- based on conversations at our offsite we're happy to keep the new schema without changes (that is, including isTemp() into isAnon()=FALSE *but* we'd like to be able to distinguish isTemp (temp editors) from isNamed (registered users) in the datasets we get back. This can be useful for understanding unregistered editors and if they differ from other editors. I'm unsure if this will be available in eventlogging data. Do you know how we might figure this out?

@Jdrewniak pinging that we'll be deploying a series of QuickSurveys starting soon (possibly today, more likely next week), and going through mid-December (we should be done by the 19th), re: T344393#9375484 grafana alerts that web team receives, in case you'd like to turn them off :))

@DDeSouza here is the setup for deploying English Wikipedia at 0 coverage (using last year's configuration with minor changes). We'd like to test first at 0 coverage to make sure the QS token is parsing over correctly, and then after that test, increase coverage for enwiki to .04.

Update: Privacy statement complete (https://foundation.wikimedia.org/wiki/Legal:Global_Readers_Survey_Privacy_Statement) - it will be the same as last year and can be reused (evergreen) - NOTE: check in with privacy@ if there are any changes to the survey the following wave (ie. 2025 and beyond), specifically pointing out differences.

Weekly updates:

• Archetype drafts complete! Looking forward to feedback at the offsite to help clarify interpretation and making final tweaks

Weekly updates:

• Weights pulled and calculated
• Survey data pulled, cleaning in progress

Weekly updates:

• LimeSurvey renewal approved by finance, renewal scheduled for next Thursday
• Assisted Wikimania scholarships with activating the survey
• Figured out some LimeSurvey workarounds with Caroline (how to add a write-in text box to an array, tested and works)

Weekly updates:

• Survey has been undeployed, data collection complete :)
• Minor snag with trying replicate weights data pull, meeting with KCV next week to figure it out
• Up next: data cleaning, weighting, and analysis

Weekly updates:

• Continued work with Sneha and Mike on design side of archetypes cards, made descriptive data table for editor archetypes for options to include on cards; next week we will finalize the draft cards for the offsite unconference session

Weekly updates:

• No major updates, assisted Wiki4HumanRights survey with activating survey, still awaiting draft from CentralNotice requesters

Weekly updates:

• No major updates this week, we have moved the metawiki report date back to December to work around other priority projects

@kostajh We have sufficient responses for the surveys, and are ready to undeploy!

Weekly updates:

• Onboarded Committee Support staff to LimeSurvey
• Assisted Wikimania staff organizers in copying scholarship form from last year and giving new COT account permissions for collaboration
• Second office hours session with research grantees, @YLiou_WMF joined the discussion as well!
• Central Notice request for a demographic survey of frwiki editors conducted by external researcher in collaboration with an affiliate member; they will send their survey draft via email, and I have informed them that this is a more complex case to consider (external researcher with the intent to publish the dataset); I can assess better once I see the survey draft but may need some team or Foundation support to consider options.

Weekly updates:

• Pulled data thus far to inspect it, looks good! Started writing R code for recombining datasets (as each question counts as a separate "event" in this circumstance, so one respondent answering both questions gets 2 lines).
• Allowing one more weekend to try to get more data points for the 2nd question, but we should be ready to undeploy early next week (aiming for Tuesday).

Weekly updates:

• Continuing work with designing the archetypes cards, had two working sessions this week to talk through data and how to best visually organize it; aim is for Sneha to have templates for us next week so we can begin inputting data

Thank you @hnowlan ! Everything is working.

Hello SRE! I have been issued a new computer as part of a transition to req holder and need to update my SSH key for access.

Weekly updates:

• No updates this week as other projects are at peak activity

Weekly updates:

• No major updates, added LimeSurvey account for new colleague in Committee Support, will have an onboarding session for surveys and tools next week

Weekly updates:

• Work is progressing, we've shared a first run of analyses and breakouts with Sneha (Design) to start collaborating on archetypes storytelling and have two coworking sessions for next week
• Sneha suggested we attempt comparisons between the active editor sample and non-Wikimedian sample, and I've combined the datasets based on 19 paired motivation variables; with the first pass at analyzing this today we're seeing indications that Wikimedians are unique in their motivations compared to other online contributors

Weekly updates:

• Surveys are live and collecting data as of Thursday! Now we wait a week or two :) (I'll keep checking the data to see when we can undeploy.)

@kostajh I believe we're ready to increase coverage!

Weekly updates:

• Worked with TSP to fix bugs in the survey end messages and other message components (T376517)
• I will take a look at the test data over the weekend (surveys currently deployed at 0% coverage) but we should be ready to deploy on Tuesday
• T&S will inform admins of the survey on Monday

Weekly updates:

• External archetypes data collection is done for EN, ES, and FR (awaiting AR as it's a separate sample and platform, but we can move forward with the data we have), Mike and I co-worked for data exploration this week
• Motivation factors and clusters made from the editors data (dataset "finalized"), working in parallel with Mike on analysis and we should be actively collaborating again with Design starting next week

Thank you for this wonderful fix @kostajh !
Confirming it is working on every surveyed wiki, except jawiki -- I can't quite tell why. The only difference I can think of is that the jawiki end message was made after the HTML linking fix was committed? But I can't tell whether or why that would make a difference. I attempted moving around spaces outside and inside the linking brackets, in case spaces were an issue in parsing the links, and it doesn't seem to have an effect.

Glad we tested. It appears linking doesn't work in the end message (tested two styles of wiki linking):

So, it seems the messages exist on every one of the wikis we're surveying, and I think I found it in the code search! But it's untranslated for most and, more importantly, the actual preferences change seems to only be available on enwiki and eswiki (when clicking the link in the mediawiki message) :