-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Open
Labels
bot triaged[Bot] This issue is triaged by ADK bot[Bot] This issue is triaged by ADK botservices[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc
Milestone
Description
It's unclear how the ADK Runner uses session.events and session.state to construct the contents[] array for LLM prompts. This makes it difficult to manage and predict token consumption from conversational history, leading to potentially high costs and exceeding context limits as conversations grow.
We need to understand if/how the ADK inherently manages this token growth from history, or how developers are expected to.
Describe the solution you'd like:
- Documentation: Clearly document the default mechanism by which the ADK Runner incorporates session.events and session.state into the LLM prompt's contents[], especially regarding history length and token count.
- Control/Configuration: Provide clear configuration options or developer hooks to manage how much and what parts of the session history are included in LLM prompts, enabling strategies like summarization or windowing to optimize token usage.
Metadata
Metadata
Assignees
Labels
bot triaged[Bot] This issue is triaged by ADK bot[Bot] This issue is triaged by ADK botservices[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc