Skip to content

AI Chat now pointing to MCP server #860

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 11 commits into
base: dev
Choose a base branch
from
Open

AI Chat now pointing to MCP server #860

wants to merge 11 commits into from

Conversation

madster456
Copy link
Collaborator

@madster456 madster456 commented Aug 22, 2025

Updates the AI Chat to point to MCP server rather than to llms.txt. A lot more accurate. I also tested 2.5-pro vs 2.5-flash. It seems that 2.5-flash was still the better option, as it provided slightly more context than 2.5-pro did.


Important

AI Chat now uses MCP server for more accurate responses and integrates tool-assisted documentation retrieval with several enhancements and bug fixes.

  • Behavior:
    • AI Chat now points to MCP server instead of llms.txt for more accurate responses in route.ts.
    • Integrates tool-assisted documentation retrieval in ai-chat.tsx.
    • Per-session tracking with auto-reset after inactivity in ai-chat.tsx.
  • Enhancements:
    • Upgraded AI model to gemini-2.5-flash in route.ts for faster responses.
    • New editable chat input with placeholder and improved submit flow in ai-chat.tsx.
    • Auto-scroll and "Thinking" indicator added in ai-chat.tsx.
    • Inline display of tool results in assistant replies in ai-chat.tsx.
  • Bug Fixes:
    • Doc links now correctly resolve to the docs subdomain in message-formatter.tsx.
  • Chores:
    • Updated @modelcontextprotocol/sdk and ai dependencies in package.json.

This description was created by Ellipsis for d091b6e. You can customize this summary. It will automatically update as commits are pushed.


Summary by CodeRabbit

  • New Features

    • Tool-assisted documentation retrieval in chat; per-session tracking with auto-reset after inactivity.
    • Optional Discord telemetry for chat messages and AI responses.
  • Enhancements

    • Upgraded AI model for more concise responses.
    • New editable chat input, improved submit flow, auto-scroll, “Thinking” indicator, and inline display of tool results.
  • Bug Fixes

    • Doc links now resolve to the correct docs subdomain.
  • Chores

    • Bumped documentation site dependencies.

Copy link

vercel bot commented Aug 22, 2025

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Preview Comments Updated (UTC)
stack-backend Ready Ready Preview Comment Aug 23, 2025 11:40am
stack-dashboard Ready Ready Preview Comment Aug 23, 2025 11:40am
stack-demo Ready Ready Preview Comment Aug 23, 2025 11:40am
stack-docs Ready Ready Preview Comment Aug 23, 2025 11:40am

Copy link
Contributor

coderabbitai bot commented Aug 22, 2025

Note

Other AI code review bot(s) detected

CodeRabbit has detected other AI code review bot(s) in this pull request and will avoid duplicating their findings in the review comments. This may lead to a less comprehensive review.

Walkthrough

Integrates MCP tooling into the chat API and switches the model to gemini-2.5-flash; updates system prompt and streaming args. Chat UI adds tool-call rendering, contentEditable input, session tracking, Discord logging, and removes inline docs caching. Adds link normalization and bumps docs dependencies.

Changes

Cohort / File(s) Summary
Dependency bumps
docs/package.json
Update @modelcontextprotocol/sdk ^1.12.0^1.17.2 and ai ^4.3.16^4.3.17.
Chat API: MCP, prompt & model
docs/src/app/api/chat/route.ts
Create MCP client via experimental_createMCPClient with StreamableHTTPClientTransporthttps://mcp.stack-auth.com/api/internal/mcp; fetch tools, pass tools and maxSteps: 50 to streamText; change model to gemini-2.5-flash; replace inline docsContent with tool-driven doc retrieval; minor error logging and system prompt updates.
Chat UI: input, tool calls, telemetry
docs/src/components/chat/ai-chat.tsx
Add ToolCallDisplay for tool invocations; replace textarea with contentEditable input synced to state; remove docsContent caching; add sessionId/sessionData in localStorage with 1h expiry; auto-scroll anchor; Thinking loader UI; send per-message and per-AI Discord payloads asynchronously; render tool outputs before assistant text; other UI adjustments.
Message formatting: link normalization
docs/src/components/chat/message-formatter.tsx
Normalize stackauth.com/docs/ (and //stackauth.com/...) to docs.stack-auth.com/docs/ for link hrefs.

Sequence Diagram(s)

sequenceDiagram
  autonumber
  participant U as User
  participant UI as Docs Chat UI
  participant API as POST /api/chat
  participant MCP as MCP (stack-auth)
  participant LLM as Gemini 2.5 Flash
  participant D as Discord Webhook

  U->>UI: send message / starter
  UI->>D: sendToDiscord(user msg + session metadata) (async)
  UI->>API: POST messages
  API->>MCP: createMCPClient + tools()
  API->>LLM: streamText(prompt + tools, maxSteps=50)
  alt toolInvocation requested
    LLM->>API: toolInvocation
    API->>MCP: invoke tool
    MCP-->>API: tool result
    API-->>LLM: tool result
  end
  LLM-->>API: streamed assistant tokens (incl. tool invocations)
  API-->>UI: streamed response
  UI->>D: sendAIResponseToDiscord(assistant msg + model + session) (async)
  UI-->>U: Render tool outputs + assistant text
Loading
sequenceDiagram
  autonumber
  participant UI as Docs Chat UI
  participant LS as localStorage
  Note over UI,LS: Session lifecycle
  UI->>LS: load sessionId/sessionData
  alt missing/expired (>1h)
    UI->>LS: generate new sessionId, reset counters
  else active
    UI->>LS: update message/time counters
  end
  Note over UI: contentEditable sync & auto-scroll on messages
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~60 minutes

Possibly related PRs

Suggested reviewers

  • N2D4

Poem

Thump-thump I hop and fetch the docs, with whiskers twitching bright,
I call the MCP, I nudge the stream, and tumble through the byte.
Links tidy, packets scurry, Discord gets a cheerful ping,
Gemini hums, tools answer — oh what joy this rabbit brings! 🐇✨

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment
  • Commit unit tests in branch aichat_mcp

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@madster456 madster456 changed the title new packages, updated packages AI Chat now pointing to MCP server Aug 22, 2025
Copy link

recurseml bot commented Aug 22, 2025

Review by RecurseML

🔍 Review performed on 301398f..ba9e631

Severity Location Issue
Medium docs/src/app/api/chat/route.ts:28 Hardcoded URL string instead of using URL builder pattern
Medium docs/src/app/api/chat/route.ts:25 Missing OAuth flow type specification in MCP client variable name
Medium docs/src/app/api/chat/route.ts:26 Async operation not wrapped with runAsynchronously
Medium docs/src/app/api/chat/route.ts:31 Async operation not wrapped with runAsynchronously
✅ Files analyzed, no issues (2)

docs/src/components/chat/ai-chat.tsx
docs/src/components/chat/message-formatter.tsx

⏭️ Files skipped (low suspicion) (2)

docs/package.json
pnpm-lock.yaml

Need help? Join our Discord

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Greptile Summary

This PR modernizes the AI chat system in the documentation site by migrating from a static llms.txt approach to a dynamic Model Context Protocol (MCP) server integration. The changes span across four key files:

Core Architecture Change: The chat API route (docs/src/app/api/chat/route.ts) now creates an MCP client that connects to https://mcp.stack-auth.com/api/internal/mcp instead of accepting pre-loaded documentation content. This allows the AI to dynamically fetch relevant documentation in real-time rather than working with static content.

Enhanced UI Components: The AI chat component (docs/src/components/chat/ai-chat.tsx) has been significantly refactored to support tool call rendering through a new ToolCallDisplay component. The input system was upgraded from a basic textarea to a more sophisticated contentEditable div with custom event handlers. The component also removes ~60 lines of documentation caching logic that is no longer needed with the MCP approach.

URL Correction Logic: A targeted fix in the message formatter (docs/src/components/chat/message-formatter.tsx) addresses AI-generated links that incorrectly reference 'stackauth.com/docs/' instead of the proper 'docs.stack-auth.com/docs/' domain.

Dependency Updates: The package.json updates @modelcontextprotocol/sdk from ^1.12.0 to ^1.17.2 and ai from ^4.3.16 to ^4.3.17 to support the new MCP functionality.

This architectural shift represents a move toward modern AI assistant patterns where tools provide just-in-time information retrieval rather than loading all content upfront. The PR description indicates that this approach provides "a lot more accurate" responses and that gemini-2.5-flash was chosen over 2.5-pro for better contextual performance.

Confidence score: 3/5

  • This PR requires careful review due to potential runtime issues and architectural changes
  • Score reflects concerns about error handling, SmartRouteHandler violations, and the significant architectural shift
  • Pay close attention to the API route implementation and MCP client initialization

4 files reviewed, 4 comments

Edit Code Review Bot Settings | Greptile

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (3)
docs/src/components/chat/message-formatter.tsx (1)

240-258: Sanitize link protocols to prevent javascript: and other unsafe hrefs.

Assistant/tool content can include arbitrary links. Rendering them directly allows javascript: or other unsafe schemes to execute in user context even with target="_blank". Only allow http(s), root-relative, hash, or mailto. Also handle missing/empty URLs by not rendering an anchor.

Apply this diff to harden the link rendering:

 case 'link': {
-      // Fix incorrect domain links
-      let fixedUrl = node.url || '';
-      if (fixedUrl.includes('stackauth.com/docs/')) {
-        fixedUrl = fixedUrl.replace('stackauth.com/docs/', 'docs.stack-auth.com/docs/');
-      }
-      if (fixedUrl.includes('//stackauth.com/docs/')) {
-        fixedUrl = fixedUrl.replace('//stackauth.com/docs/', '//docs.stack-auth.com/docs/');
-      }
-
-      return (
-        <a
-          key={index}
-          href={fixedUrl}
+      // Normalize and sanitize link targets
+      const rawUrl = node.url ?? '';
+      if (!rawUrl) {
+        return <>{node.children?.map(renderNode)}</>;
+      }
+      // Normalize stackauth docs domain
+      let fixedUrl = rawUrl.replace(
+        /(^https?:\/\/)?(\/\/)?(www\.)?stackauth\.com\/docs\//i,
+        'https://docs.stack-auth.com/docs/'
+      );
+      // Allow only safe protocols: http(s), root-relative, hash, or mailto
+      const isSafe =
+        /^(https?:\/\/)/i.test(fixedUrl) ||
+        fixedUrl.startsWith('/') ||
+        fixedUrl.startsWith('#') ||
+        fixedUrl.toLowerCase().startsWith('mailto:');
+      const safeHref = isSafe ? fixedUrl : '#';
+
+      return (
+        <a
+          key={index}
+          href={safeHref}
           target="_blank"
           rel="noopener noreferrer"
           className="inline-flex items-center gap-1 px-1.5 py-0.5 bg-blue-100 hover:bg-blue-200 text-blue-700 dark:bg-blue-900/40 dark:hover:bg-blue-900/60 dark:text-blue-300 rounded text-xs font-medium transition-all duration-150 hover:scale-[1.02]"
         >
           {node.children?.map(renderNode)}
docs/src/components/chat/ai-chat.tsx (2)

218-225: Model metadata mismatch with backend.

You send model: 'gemini-2.0-flash' to Discord while the API uses gemini-2.5-flash. Keep these consistent for observability.

Apply this diff:

       const context = {
         response: response,
         metadata: {
           sessionId: sessionId,
-          model: 'gemini-2.0-flash',
+          model: 'gemini-2.5-flash',
         }
       };

278-295: Programmatic submission passes a fake FormEvent; use a real form submit.

Calling handleChatSubmit({} as React.FormEvent) risks runtime errors if handleSubmit calls preventDefault. Prefer an actual <form onSubmit={...}> and make the send button type="submit". Also submit on Enter by requesting form submission.

Apply this diff to the input block:

-      <div className="px-3 pb-3">
-        <div className="border-input bg-background cursor-text rounded-3xl border px-3 py-2 shadow-xs">
+      <div className="px-3 pb-3">
+        <form onSubmit={handleChatSubmit}>
+        <div className="border-input bg-background cursor-text rounded-3xl border px-3 py-2 shadow-xs">
           <div className="flex items-center gap-2">
             <div className="flex-1 flex items-center">
               <div
                 ref={editableRef}
                 contentEditable
                 suppressContentEditableWarning={true}
                 className="text-primary w-full resize-none border-none bg-transparent shadow-none outline-none focus-visible:ring-0 focus-visible:ring-offset-0 text-sm empty:before:content-[attr(data-placeholder)] empty:before:text-fd-muted-foreground"
                 style={{ lineHeight: "1.4", minHeight: "20px" }}
                 onInput={(e) => {
                   const value = e.currentTarget.textContent || "";
                   handleInputChange({
                     target: { value },
                   } as React.ChangeEvent<HTMLInputElement>);
 
                   // Clean up the div if it's empty to show placeholder
                   if (!value.trim()) {
                     e.currentTarget.innerHTML = "";
                   }
                 }}
                 onKeyDown={(e) => {
                   if (e.key === "Enter" && !e.shiftKey) {
                     e.preventDefault();
-                    // eslint-disable-next-line no-restricted-syntax
-                    handleChatSubmit({} as React.FormEvent).catch(error => {
-                      console.error('Chat submit error:', error);
-                    });
+                    // Trigger form submission
+                    (e.currentTarget.closest('form') as HTMLFormElement | null)?.requestSubmit();
                   }
                 }}
                 onPaste={(e) => {
                   e.preventDefault();
                   const text = e.clipboardData.getData("text/plain");
                   e.currentTarget.textContent =
                     (e.currentTarget.textContent || "") + text;
                   const value = e.currentTarget.textContent;
                   handleInputChange({
                     target: { value },
                   } as React.ChangeEvent<HTMLInputElement>);
                 }}
                 data-placeholder="Ask about Stack Auth..."
               />
             </div>
             <button
               disabled={!input.trim() || isLoading}
-              onClick={() => {
-                // eslint-disable-next-line no-restricted-syntax
-                handleChatSubmit({} as React.FormEvent).catch(error => {
-                  console.error('Chat submit error:', error);
-                });
-              }}
+              type="submit"
               className="h-8 w-8 rounded-full p-0 shrink-0 bg-fd-primary text-fd-primary-foreground hover:bg-fd-primary/90 disabled:opacity-50 disabled:cursor-not-allowed flex items-center justify-center"
             >
               <Send className="w-4 h-4" />
             </button>
           </div>
         </div>
+        </form>
       </div>

If you prefer to keep it non-form-based, we can switch to append({ role: 'user', content: input }) from useChat (if available in your version).

Also applies to: 476-485, 499-509

🧹 Nitpick comments (5)
docs/src/components/chat/message-formatter.tsx (1)

241-249: Consolidate duplicate replacements and handle more URL variants.

The two includes(...).replace(...) calls can be collapsed into one regex to handle http://, https://, //, and www. variants in a single pass. The diff above already does this using a case-insensitive regex.

docs/src/app/api/chat/route.ts (2)

15-15: Avoid double logging of errors.

getErrorMessage logs, and the catch below logs again. Keep logging in one place to prevent noisy logs.

Apply this diff to remove the early console log:

 function getErrorMessage(error: unknown): string {
-  console.log('Error in chat API:', error);
   if (error instanceof Error) {
     return error.message;
   }
   return String(error);
 }

34-101: System prompt: enforce docs domain consistency and rate considerations.

The prompt mandates fetching docs for “every question.” This can increase latency and server load. Consider softening to “when a docs reference would materially improve the answer” and keep the canonical domain https://docs.stack-auth.com (already aligned with MessageFormatter).

docs/src/components/chat/ai-chat.tsx (2)

203-209: Auto-scroll to latest message looks good; consider scroll-behavior guard.

This pattern is fine. If you notice jank on rapid streams, debounce the scroll or only scroll when the user is near the bottom.

Also applies to: 450-452


241-275: Webhook payload contains PII-ish fields; confirm policy and user consent.

You send userAgent, pathname, timestamps, and session identifiers to Discord. Ensure this aligns with your privacy policy and explicitly inform users.

If needed, I can add:

  • a small UI notice/toggle for telemetry consent,
  • redaction of userAgent,
  • hashing of sessionId.

Say the word and I’ll patch it.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 301398f and ba9e631.

⛔ Files ignored due to path filters (1)
  • pnpm-lock.yaml is excluded by !**/pnpm-lock.yaml
📒 Files selected for processing (4)
  • docs/package.json (2 hunks)
  • docs/src/app/api/chat/route.ts (2 hunks)
  • docs/src/components/chat/ai-chat.tsx (7 hunks)
  • docs/src/components/chat/message-formatter.tsx (1 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: docker
  • GitHub Check: restart-dev-and-test
  • GitHub Check: lint_and_build (latest)
  • GitHub Check: docker
  • GitHub Check: setup-tests
  • GitHub Check: all-good
  • GitHub Check: build (22.x)
  • GitHub Check: build (22.x)
  • GitHub Check: Security Check
🔇 Additional comments (6)
docs/package.json (1)

25-25: Please manually verify dependency compatibility and clean build

Upgrading @modelcontextprotocol/sdk to ^1.17.2 and ai to ^4.3.17 should align with the new MCP client usage and useChat API, but our CI install/typecheck failed with missing modules and errors. To ensure nothing is broken, please:

  • Install dependencies with your project’s package manager (pnpm or npm) and confirm no peer-dependency warnings:
    pnpm install --ignore-scripts     # or npm install --ignore-scripts
  • Verify that @ai-sdk/react@^1.2.12, ai@^4.3.17, and @modelcontextprotocol/sdk@^1.17.2 resolve correctly:
    pnpm ls ai @ai-sdk/react @modelcontextprotocol/sdk    # or npm ls ai @ai-sdk/react @modelcontextprotocol/sdk
  • Typecheck the docs app to ensure no runtime or type errors:
    pnpm run -w docs typecheck        # or npm run -w docs typecheck
    # fallback:
    npx tsc -p docs/tsconfig.json --noEmit

Once these steps pass without warnings or errors, the dependency bump can be considered safe.

Also applies to: line 36.

docs/src/app/api/chat/route.ts (2)

105-109: Pass tools directly; avoid spreading in case of array return type.

If stackAuthMcp.tools() returns an object keyed by tool name, spreading is unnecessary. If it returns an array, spreading produces numeric keys and breaks tool resolution. Pass tools as-is.

Apply this diff:

-      tools: {
-        ...tools,
-      },
+      tools,

If TS complains, assert the correct type or transform the array to a name-keyed object:

const toolsRecord = Array.isArray(tools)
  ? Object.fromEntries(tools.map((t: any) => [t.name, t]))
  : tools;

23-31: Guard JSON parsing and ensure MCP client cleanup

Please wrap the request body parse and MCP client initialization in a try/catch, default messages to an empty array, and—if the client exposes a close() method—shut it down in a finally block. Verify whether createMCPClient supports explicit disposal and adjust the finally accordingly.

• File: docs/src/app/api/chat/route.ts
– Surround const { messages } = await request.json() and the MCP client creation in a try/catch.
– Replace direct destructuring with a safe parse that falls back to {} and validates messages as an array.
– After returning the response, add a finally block to invoke (stackAuthMcp as any)?.close?.().

Suggested diff:

 export async function POST(request: Request) {
-  const { messages } = await request.json();
-
-  // Create MCP client for Stack Auth documentation
-  const stackAuthMcp = await createMCPClient({
-    transport: new StreamableHTTPClientTransport(
-      new URL('https://mcp.stack-auth.com/api/internal/mcp')
-    ),
-  });
-  const tools = await stackAuthMcp.tools();
+  let stackAuthMcp;
+  try {
+    const body = await request.json().catch(() => ({}));
+    const messages = Array.isArray(body?.messages) ? body.messages : [];
+
+    // Create MCP client for Stack Auth documentation
+    stackAuthMcp = await createMCPClient({
+      transport: new StreamableHTTPClientTransport(
+        new URL('https://mcp.stack-auth.com/api/internal/mcp')
+      ),
+    });
+    const tools = await stackAuthMcp.tools();
+
+    // …rest of your handler using `messages` and `tools`
+  } catch (err) {
+    return new Response('Invalid request or MCP error', { status: 400 });
+  } finally {
+    try { (stackAuthMcp as any)?.close?.(); } catch {}
+  }
 }
docs/src/components/chat/ai-chat.tsx (3)

81-83: Refs addition LGTM.

Using refs for the contentEditable and end-of-messages anchor is appropriate for sync and scrolling.


427-441: “Thinking” banner UX looks good.

Clear and unobtrusive. Nice touch with subtle animation and brand icon.


413-419: Tool invocation rendering is a solid UX addition.

Showing the fetched doc title and a deep link improves trust and traceability.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/src/components/chat/ai-chat.tsx (1)

214-221: Model name mismatch with server; update Discord metadata.
The server uses gemini-2.5-flash, but you report gemini-2.0-flash to Discord.

       const context = {
         response: response,
         metadata: {
           sessionId: sessionId,
-          model: 'gemini-2.0-flash',
+          model: 'gemini-2.5-flash',
         }
       };
♻️ Duplicate comments (1)
docs/src/app/api/chat/route.ts (1)

25-46: Good: Robust MCP bootstrap with 503 fallback.
This resolves the earlier “missing error handling” concern for MCP initialization. Returning a 503 with a user-facing explanation is a solid choice here.

🧹 Nitpick comments (12)
docs/src/app/api/chat/route.ts (6)

15-15: Use console.error (and avoid duplicate logging) in error helper.
getErrorMessage currently logs with console.log and you also log again in the catch block below; this can double-log. Prefer console.error here or remove logging from this helper.

 function getErrorMessage(error: unknown): string {
-  console.log('Error in chat API:', error);
+  console.error('Error in chat API:', error);
   if (error instanceof Error) {
     return error.message;
   }
   return String(error);
 }

25-33: Make MCP base URL configurable.
Hardcoding https://mcp.stack-auth.com makes environment switches harder. Read from an env var with a sensible default.

-    const stackAuthMcp = await createMCPClient({
-      transport: new StreamableHTTPClientTransport(
-        new URL('/api/internal/mcp', 'https://mcp.stack-auth.com')
-      ),
-    });
+    const mcpBase = process.env.MCP_HTTP_BASE ?? 'https://mcp.stack-auth.com';
+    const stackAuthMcp = await createMCPClient({
+      transport: new StreamableHTTPClientTransport(
+        new URL('/api/internal/mcp', mcpBase)
+      ),
+    });

25-34: Type tools explicitly (and consider lightweight caching).
Typing avoids “{} is not assignable to Tools” friction and helps IDEs. Also, fetching tool descriptors on every request can be avoided with a short TTL cache.

-  let tools = {};
+  import type { Tool } from 'ai';
+  let tools: Record<string, Tool> = {};
   try {
     const stackAuthMcp = await createMCPClient({
       transport: new StreamableHTTPClientTransport(
         new URL('/api/internal/mcp', 'https://mcp.stack-auth.com')
       ),
     });
-    tools = await stackAuthMcp.tools();
+    // Optional: cache across requests (module-scope) with a TTL
+    tools = await stackAuthMcp.tools();

If you’d like, I can provide a small module-scope TTL cache snippet.


49-116: Prompt copy is strong; minor brand/UX nits.

  • “As an AI…” phrasing can read robotic; consider omitting to keep brand voice tight.
  • “ALWAYS use the available tools” can dead-end if tools are intentionally pruned; consider “default to” tools with allowance to answer from memory when appropriate.

120-128: Propagate request abort; align streaming with client disconnects.
Pass the request.signal to streamText so upstream calls are cancelled if the client disconnects.

   try {
-    const result = streamText({
+    const result = streamText({
       model: google('gemini-2.5-flash'),
       tools: {
         ...tools,
       },
       maxSteps: 50,
       system: systemPrompt,
       messages,
       temperature: 0.1,
+      abortSignal: (request as any).signal ?? undefined,
     });

Note: on the edge runtime Request has a signal; casting keeps TS happy in Node runtimes too.


9-11: Optional: fail fast if GOOGLE_AI_API_KEY is missing.
This prevents a harder-to-diagnose error later when calling the model.

 const google = createGoogleGenerativeAI({
-  apiKey: process.env.GOOGLE_AI_API_KEY,
+  apiKey: process.env.GOOGLE_AI_API_KEY ?? '',
 });
+if (!process.env.GOOGLE_AI_API_KEY) {
+  console.warn('GOOGLE_AI_API_KEY is not set; chat route will fail at runtime.');
+}

If you prefer, short-circuit in POST with a 500 and actionable message.

docs/src/components/chat/ai-chat.tsx (6)

55-63: Optional: build doc link via URL for extra safety.
This avoids subtle path concatenation bugs and normalizes // and missing slashes.

-            href={`https://docs.stack-auth.com${docId.startsWith('/') ? docId : `/${docId}`}`}
+            href={new URL(docId?.startsWith('/') ? docId : `/${docId ?? ''}`, 'https://docs.stack-auth.com').toString()}

199-205: Auto-scroll only when user is near the bottom.
Current behavior can yank the view while the user scrolls history. Track scroll position and only scroll if the user hasn’t scrolled up.

I can provide a small hook (useAutoscroll) that observes scrollTop and clientHeight to decide when to scroll.


315-317: Simplify: call handleSubmit() directly; no need to fake an event.
useChat’s handleSubmit accepts an optional event; passing {} is unnecessary.

-  const handleSubmitSafely = () => {
-    runAsynchronously(() => handleChatSubmit({} as React.FormEvent));
-  };
+  const handleSubmitSafely = () => {
+    runAsynchronously(() => handleChatSubmit(undefined as unknown as React.FormEvent));
+  };

Or even cleaner, change handleChatSubmit to call handleSubmit() with no args and update the onClick/onKeyDown sites accordingly:

-    handleSubmit(e);
+    handleSubmit();

452-491: Accessibility: add ARIA to contentEditable input.
contentEditable lacks native input semantics. Add role, aria-multiline, and an aria-label for screen readers.

               <div
                 ref={editableRef}
                 contentEditable
                 suppressContentEditableWarning={true}
+                role="textbox"
+                aria-multiline="true"
+                aria-label="Ask about Stack Auth"
                 className="text-primary w-full resize-none border-none bg-transparent shadow-none outline-none focus-visible:ring-0 focus-visible:ring-offset-0 text-sm empty:before:content-[attr(data-placeholder)] empty:before:text-fd-muted-foreground"

411-416: Harden against unexpected toolInvocations shape.
Guard with Array.isArray to avoid rendering errors if a provider changes the shape.

-                    {message.toolInvocations?.map((toolCall, index) => (
+                    {Array.isArray((message as any).toolInvocations) &&
+                     (message as any).toolInvocations.map((toolCall: any, index: number) => (
                       <ToolCallDisplay key={index} toolCall={toolCall} />
                     ))}

261-270: Optional: handle non-2xx Discord responses.
A failing webhook currently logs only on network error; surface 4xx/5xx too.

-      await fetch('/api/discord-webhook', {
+      const resp = await fetch('/api/discord-webhook', {
         method: 'POST',
         headers: {
           'Content-Type': 'application/json',
         },
         body: JSON.stringify(context),
       });
+      if (!resp.ok) {
+        console.error('Discord webhook responded with', resp.status);
+      }

Same for sendAIResponseToDiscord.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between ba9e631 and daf328d.

📒 Files selected for processing (2)
  • docs/src/app/api/chat/route.ts (2 hunks)
  • docs/src/components/chat/ai-chat.tsx (9 hunks)
🧰 Additional context used
🧠 Learnings (1)
📚 Learning: 2025-08-04T22:25:51.260Z
Learnt from: CR
PR: stack-auth/stack-auth#0
File: CLAUDE.md:0-0
Timestamp: 2025-08-04T22:25:51.260Z
Learning: Applies to apps/backend/src/app/api/latest/**/* : The project uses a custom route handler system in the backend for consistent API responses

Applied to files:

  • docs/src/app/api/chat/route.ts
🧬 Code graph analysis (2)
docs/src/components/chat/ai-chat.tsx (3)
docs/src/components/icons.tsx (2)
  • FileText (207-219)
  • ExternalLink (98-108)
packages/stack-shared/src/utils/promises.tsx (1)
  • runAsynchronously (343-366)
docs/src/components/chat/message-formatter.tsx (1)
  • MessageFormatter (297-347)
docs/src/app/api/chat/route.ts (2)
docs/src/app/api/discord-webhook/response/route.ts (1)
  • POST (3-18)
docs/src/app/api/discord-webhook/route.ts (1)
  • POST (4-20)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: all-good
  • GitHub Check: docker
  • GitHub Check: docker
  • GitHub Check: build (22.x)
  • GitHub Check: build (22.x)
  • GitHub Check: lint_and_build (latest)
  • GitHub Check: restart-dev-and-test
  • GitHub Check: setup-tests
  • GitHub Check: Security Check
🔇 Additional comments (1)
docs/src/app/api/chat/route.ts (1)

2-3: MCP + AI imports look correct and consistent with the dependency bumps.
The switch to StreamableHTTPClientTransport and experimental_createMCPClient aligns with the PR’s goal.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
docs/src/components/chat/ai-chat.tsx (1)

274-288: Don’t cast {} as a FormEvent; call handleSubmit with a real event or make it optional.

handleSubmit can call preventDefault; passing {} risks a runtime error. Make the parameter optional and synthesize a real event when needed; update the safe wrapper accordingly.

Apply:

-  const handleChatSubmit = async (e: React.FormEvent) => {
+  const handleChatSubmit = async (e?: React.FormEvent) => {
     if (!input.trim()) return;
@@
-    handleSubmit(e);
+    await handleSubmit(
+      e ?? (new Event('submit') as unknown as React.FormEvent)
+    );
   };
-  const handleSubmitSafely = () => {
-    runAsynchronously(() => handleChatSubmit({} as React.FormEvent));
-  };
+  const handleSubmitSafely = () => {
+    runAsynchronously(() => handleChatSubmit());
+  };

Also applies to: 315-317

♻️ Duplicate comments (1)
docs/src/components/chat/ai-chat.tsx (1)

26-69: ToolCallDisplay overall: nice, and prior console.log concerns are resolved.

Console logging was removed, optional chaining added, and link rendering is clean. With the chaining and URL fix above, this looks solid.

🧹 Nitpick comments (7)
docs/src/components/chat/ai-chat.tsx (7)

412-416: Prefer stable keys over array index for tool invocations.

Index keys can cause reconciliation issues on interleaved tool calls; a composite key is available.

Apply:

-                    {message.toolInvocations?.map((toolCall, index) => (
-                      <ToolCallDisplay key={index} toolCall={toolCall} />
-                    ))}
+                    {message.toolInvocations?.map((toolCall, index) => (
+                      <ToolCallDisplay
+                        key={`${toolCall.toolName}:${toolCall.args?.id ?? index}`}
+                        toolCall={toolCall}
+                      />
+                    ))}

206-212: Avoid caret jumps when syncing contentEditable with state.

Overwriting textContent while the element is focused will move the caret to the end. Guard against clobbering during active edits.

Apply:

-  useEffect(() => {
-    if (editableRef.current && editableRef.current.textContent !== input) {
-      editableRef.current.textContent = input;
-    }
-  }, [input]);
+  useEffect(() => {
+    const el = editableRef.current;
+    if (!el) return;
+    if (document.activeElement === el) return; // don't clobber caret while typing
+    if (el.textContent !== input) {
+      el.textContent = input;
+    }
+  }, [input]);

195-196: Telemetry model value may be stale; avoid hard-coding.

You’re logging the AI response to Discord here. The helper currently hard-codes model: 'gemini-2.0-flash'. Given this PR switches to MCP and, per PR notes, Gemini 2.5 may be in use, this can mislead analytics. Either omit the model, or plumb the actual model name from the backend response if available.

Suggested change (in sendAIResponseToDiscord body):

-          sessionId: sessionId,
-          model: 'gemini-2.0-flash',
+          sessionId: sessionId,
+          // model: supplied by server; omit here to avoid drift

237-271: Confirm consent before sending user messages + metadata to Discord.

This ships user prompts, userAgent, and path to /api/discord-webhook. Ensure this is disclosed and gated (e.g., env flag or an in-UI opt-in).

Lightweight pattern:

  • Gate behind NEXT_PUBLIC_ENABLE_AI_TELEMETRY === 'true'.
  • Provide a toggle in the drawer footer and remember via localStorage.
  • Strip userAgent unless strictly necessary.

I can open a follow-up PR to add the flag and toggle if you’d like.


426-435: A11y: Announce the “Thinking” state to screen readers.

Add role="status" and aria-live="polite" to the container so assistive tech is notified of progress.

Apply:

-              <div className="rounded-lg bg-fd-muted border border-fd-border p-3">
+              <div
+                className="rounded-lg bg-fd-muted border border-fd-border p-3"
+                role="status"
+                aria-live="polite"
+              >

199-205: Auto-scroll only on assistant messages to avoid interrupting the user.

Current effect scrolls on any message change, including when the user is typing or editing past messages. Consider scrolling when the last message is from the assistant.

Example:

-  useEffect(() => {
-    if (messagesEndRef.current) {
-      messagesEndRef.current.scrollIntoView({ behavior: 'smooth' });
-    }
-  }, [messages]);
+  useEffect(() => {
+    const last = messages[messages.length - 1];
+    if (last?.role !== 'assistant') return;
+    messagesEndRef.current?.scrollIntoView({ behavior: 'smooth' });
+  }, [messages]);

291-307: Hoist static starterPrompts outside the component.

It’s re-created every render. Minor, but easy to avoid.

Move the array to module scope and reference it inside the component.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between daf328d and d091b6e.

📒 Files selected for processing (1)
  • docs/src/components/chat/ai-chat.tsx (9 hunks)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (9)
  • GitHub Check: build (22.x)
  • GitHub Check: build (22.x)
  • GitHub Check: restart-dev-and-test
  • GitHub Check: setup-tests
  • GitHub Check: lint_and_build (latest)
  • GitHub Check: all-good
  • GitHub Check: docker
  • GitHub Check: docker
  • GitHub Check: Security Check

Comment on lines +40 to +45
const titleMatch = toolCall.result?.content[0]?.text.match(/Title:\s*(.*)/);
if (titleMatch?.[1]) {
docTitle = titleMatch[1].trim();
} else {
docTitle = 'No Title Found';
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Fix unsafe optional chaining on tool result to prevent runtime crash.

toolCall.result?.content[0] will throw if content is undefined because the index access happens before optional chaining on .text. Also .match can be undefined. Guard all hops.

Apply:

-    const titleMatch = toolCall.result?.content[0]?.text.match(/Title:\s*(.*)/);
-    if (titleMatch?.[1]) {
-      docTitle = titleMatch[1].trim();
-    } else {
-      docTitle = 'No Title Found';
-    }
+    const rawText = toolCall.result?.content?.[0]?.text;
+    const titleMatch = rawText?.match?.(/Title:\s*(.*)/);
+    if (titleMatch?.[1]) {
+      docTitle = titleMatch[1].trim();
+    } else {
+      docTitle = 'No Title Found';
+    }
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
const titleMatch = toolCall.result?.content[0]?.text.match(/Title:\s*(.*)/);
if (titleMatch?.[1]) {
docTitle = titleMatch[1].trim();
} else {
docTitle = 'No Title Found';
}
// Replace the unsafe match call with a guarded extraction of the raw text
const rawText = toolCall.result?.content?.[0]?.text;
const titleMatch = rawText?.match?.(/Title:\s*(.*)/);
if (titleMatch?.[1]) {
docTitle = titleMatch[1].trim();
} else {
docTitle = 'No Title Found';
}
🤖 Prompt for AI Agents
In docs/src/components/chat/ai-chat.tsx around lines 40 to 45, the current
expression accesses content[0] and calls .match without guarding that content
exists, is an array, and that text is a string, which can throw; change the
logic to first verify toolCall.result exists, that toolCall.result.content is an
array with at least one entry, and that content[0].text is a string before
calling .match, then check the match result for a captured group before
assigning docTitle, otherwise set the default 'No Title Found'.

Comment on lines +55 to +63
href={`https://docs.stack-auth.com/${encodeURIComponent(docId.replace(/^\//, ''))}`}
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-1 text-blue-600 dark:text-blue-400 hover:text-blue-800 dark:hover:text-blue-200 transition-colors"
>
<ExternalLink className="w-3 h-3" />
<span>Open</span>
</a>
)}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Broken docs link: use encodeURI on a normalized path instead of encodeURIComponent.

encodeURIComponent encodes slashes, producing URLs like https://docs.stack-auth.com/docs%2Fintro which 404. Preserve path separators and still escape unsafe chars.

Apply:

-            href={`https://docs.stack-auth.com/${encodeURIComponent(docId.replace(/^\//, ''))}`}
+            href={`https://docs.stack-auth.com${encodeURI(
+              (String(docId).startsWith('/') ? String(docId) : `/${String(docId)}`)
+            )}`}
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
href={`https://docs.stack-auth.com/${encodeURIComponent(docId.replace(/^\//, ''))}`}
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-1 text-blue-600 dark:text-blue-400 hover:text-blue-800 dark:hover:text-blue-200 transition-colors"
>
<ExternalLink className="w-3 h-3" />
<span>Open</span>
</a>
)}
href={`https://docs.stack-auth.com${encodeURI(
(String(docId).startsWith('/') ? String(docId) : `/${String(docId)}`)
)}`}
target="_blank"
rel="noopener noreferrer"
className="flex items-center gap-1 text-blue-600 dark:text-blue-400 hover:text-blue-800 dark:hover:text-blue-200 transition-colors"
🤖 Prompt for AI Agents
In docs/src/components/chat/ai-chat.tsx around lines 55 to 63, the href
currently uses encodeURIComponent which encodes slashes and breaks doc paths;
normalize the docId by stripping a leading slash, then use encodeURI (not
encodeURIComponent) on that normalized path so path separators are preserved
while unsafe characters are escaped, and update the href construction to use the
encoded result.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants