Skip to content

tests_for misses non-name-matching test callers in src/ package layouts (callers_of finds them) #720

Description

@scottconverse

Summary

query tests_for misses test callers whose names don't reference the target function, but only in src/ package layouts. callers_of finds all of them correctly in the same graph. In a flat layout, tests_for behaves correctly, which made this fun to minimize.

Version: code-review-graph==2.3.7 (pip; MCP serverInfo reports internal component 3.4.4). Observed on Windows 11 / Python 3.12; the trigger appears to be layout, not OS.

Minimal reproduction

repro/
  src/mypkg/__init__.py        (empty)
  src/mypkg/runner.py
  tests/test_runner.py

src/mypkg/runner.py:

def render_thing(code: str) -> str:
    return code.upper()

tests/test_runner.py:

from mypkg.runner import render_thing


def test_render_thing_basic():
    assert render_thing("a") == "A"


def test_pipeline_uses_uppercase():
    assert render_thing("bc") == "BC"
git init
code-review-graph build
code-review-graph query tests_for  "src/mypkg/runner.py::render_thing"
code-review-graph query callers_of "src/mypkg/runner.py::render_thing"

Observed

  • tests_for -> "result_count": 1 - only test_render_thing_basic (the test whose name references the function)
  • callers_of -> "result_count": 2 - both tests, with CALLS edges at confidence 1.0 EXTRACTED

Expected: tests_for returns both tests - test_pipeline_uses_uppercase calls the target directly.

Control (does NOT reproduce)

The identical two tests against a flat-layout module (frob.py at repo root, from frob import frobnicate) give tests_for -> 2 results, matching callers_of. Adding a same-named function in a second module (ambiguity) also does not degrade the flat case.

Real-world impact

On a real src/-layout codebase (~150 files, ~27k edges), a central function with 20 direct test callers: callers_of found all 20 (verified against a manual grep of every call site - zero false positives, including a module-attribute call), while tests_for returned 1 - again the single test whose name contains the function name. So in src layouts, tests_for effectively degrades to name matching, silently under-reporting by whatever the naming convention hides.

Workaround

Derive dependent tests from callers_of filtered to "is_test": true - that path is accurate in every case I tested. Happy to provide more detail or test a fix.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions