-
Notifications
You must be signed in to change notification settings - Fork 1.7k
Closed as not planned
Closed as not planned
DatabaseSessionService has race condition with
create_all()
when running multiple instances/replicas#1561Bug
Copy link
Labels
bot triaged[Bot] This issue is triaged by ADK bot[Bot] This issue is triaged by ADK botrequest clarification[Status] The maintainer need clarification or more information from the author[Status] The maintainer need clarification or more information from the authorservices[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc
Description
Description
When running multiple instances of an application using DatabaseSessionService
(e.g., multiple Kubernetes pods, multiple containers, or multiple processes), a race condition occurs during table creation that causes instances to crash with a PostgreSQL constraint violation error.
Environment
- ADK Version: 1.3.0
- Python Version: 3.13
- Database: PostgreSQL 17
- Deployment: Kubernetes with 2+ replicas / Multiple Docker containers / Multiple processes
Error Message
sqlalchemy.exc.IntegrityError: (psycopg.errors.UniqueViolation) duplicate key value violates unique constraint "pg_type_typname_nsp_index"
DETAIL: Key (typname, typnamespace)=(sessions, 16385) already exists.
[SQL:
CREATE TABLE sessions (
app_name VARCHAR(128) NOT NULL,
user_id VARCHAR(128) NOT NULL,
id VARCHAR(128) NOT NULL,
state JSONB NOT NULL,
create_time TIMESTAMP WITHOUT TIME ZONE NOT NULL,
update_time TIMESTAMP WITHOUT TIME ZONE NOT NULL,
PRIMARY KEY (app_name, user_id, id)
)
]
Root Cause
The issue occurs in database_session_service.py
at line 343:
class DatabaseSessionService(BaseSessionService):
def __init__(self, db_url: str, **kwargs: Any):
# ...
# Line 343 - This creates tables without checking for concurrent access
Base.metadata.create_all(self.db_engine)
When multiple instances start simultaneously:
- Instance A checks if tables exist (they don't)
- Instance B checks if tables exist (they don't)
- Instance A starts creating tables
- Instance B starts creating tables
- PostgreSQL throws a constraint violation on its internal
pg_type
catalog
Steps to Reproduce
- Create an application using DatabaseSessionService:
from google.adk.sessions import DatabaseSessionService
session_service = DatabaseSessionService(
db_url="postgresql://user:pass@host/db",
pool_size=10
)
- Deploy with multiple replicas (Kubernetes example):
apiVersion: apps/v1
kind: Deployment
spec:
replicas: 2 # Multiple replicas trigger the race condition
template:
spec:
containers:
- name: app
image: myapp:latest
- Both pods will attempt to start simultaneously, and one will crash with the pg_type error
Expected Behavior
DatabaseSessionService should handle concurrent initialization gracefully without crashes. Multiple instances should be able to start simultaneously without race conditions.
Suggested Fixes
- Consider using
checkfirst=True
parameter.
Base.metadata.create_all(self.db_engine, checkfirst=True)
Metadata
Metadata
Assignees
Labels
bot triaged[Bot] This issue is triaged by ADK bot[Bot] This issue is triaged by ADK botrequest clarification[Status] The maintainer need clarification or more information from the author[Status] The maintainer need clarification or more information from the authorservices[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc[Component] This issue is related to runtime services, e.g. sessions, memory, artifacts, etc