Skip to content

Clarify and Control Conversational History for LLM Token Optimization #702

@borso271

Description

@borso271

It's unclear how the ADK Runner uses session.events and session.state to construct the contents[] array for LLM prompts. This makes it difficult to manage and predict token consumption from conversational history, leading to potentially high costs and exceeding context limits as conversations grow.

We need to understand if/how the ADK inherently manages this token growth from history, or how developers are expected to.

Describe the solution you'd like:

  • Documentation: Clearly document the default mechanism by which the ADK Runner incorporates session.events and session.state into the LLM prompt's contents[], especially regarding history length and token count.
  • Control/Configuration: Provide clear configuration options or developer hooks to manage how much and what parts of the session history are included in LLM prompts, enabling strategies like summarization or windowing to optimize token usage.

Metadata

Metadata

Assignees

Labels

bot triaged[Bot] This issue is triaged by ADK botservices[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions