Skip to content

Detailed guide on truncation option in ModelSettings #1494

@abdul-kabir-jawed

Description

@abdul-kabir-jawed

OpenAI Agents SDK Truncation Parameter Documentation

The truncation parameter in the OpenAI Agents SDK is currently documented as:

truncation: Literal["auto", "disabled"] | None = None
"""The truncation strategy to use when calling the model."""

This single-line description is too vague for safe production use. Developers working with long histories, large inputs, or streaming runs cannot reliably predict or handle truncation without knowing the exact behavior of each option.

Why This Matters for Developers

Production Reliability Risk

In "auto" mode, silent removal of earlier context can cause critical loss of information and unexpected behavior.

Unexpected Failures in "disabled" Mode

If the total tokens exceed the model's limit, "disabled" will cause a context_length_exceeded error and fail the run entirely — not explained in current docs.

Debugging Difficulty

Without knowing exactly which parts of the input/history get dropped, developers cannot easily diagnose output discrepancies.

Safety & Compliance Concerns

In regulated or safety-critical workflows, silent truncation without documented rules can create compliance risks.

Specific Gaps in Current Docs

  • "auto" behavior is not explained (priority order for truncation, earliest input_history dropped first, how tool outputs are handled).
  • "disabled" behavior is not explained (guaranteed error if over context limit, no truncation).
  • Effect of None is unclear (does it default to "auto"?).
  • No examples of safe handling for each mode.

Proposed Documentation Additions

Explain "auto" Behavior

State that oldest messages in input_history are truncated first until the prompt fits the model's context window.
Note that tool outputs and intermediate steps may also be truncated.

Explain "disabled" Behavior

Clarify that the SDK will perform no truncation and will raise context_length_exceeded if the prompt is too long.

Clarify None

Explicitly state whether None defaults to "auto" or other behavior.

Add Safe Usage Examples

# Example: Prevent silent loss of context
runner.run_sync(
    input="Large input...",
    truncation="auto",  # Will remove earliest history first
)

# Example: Strict mode - fail instead of truncate
runner.run_sync(
    input="Critical input...",
    truncation="disabled",  # Will raise if too long
)

Impact of Fix

This change will:

  • Prevent misunderstandings of truncation behavior.
  • Help developers choose the right mode for their use case.
  • Reduce unexpected errors and debugging complexity.
  • Improve production reliability for long or complex runs.

Offer to Contribute

I'm happy to draft a PR adding detailed documentation for truncation, including examples, behavior descriptions, and edge cases, once maintainers confirm style and placement.

Metadata

Metadata

Assignees

No one assigned

    Labels

    documentationImprovements or additions to documentation

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions