How to Build AI Apps with Multiple LLMs in Retool

If you want to build AI apps with multiple LLMs in Retool, you don't have to lock into a single provider. Retool supports integrations with models from OpenAI, Anthropic, and Google, and tools like OpenRouter let you access 200+ models behind a single API — with intelligent request routing baked in. Here's a practical breakdown of how to pull it all together, based on a hands-on session the Retool team ran with builder KeananKoppenhaver.

Why Use Multiple LLMs in a Single Retool App?

Different models have different strengths. GPT-4 might be your go-to for complex reasoning, while Claude handles long-context documents better, and Gemini excels at multimodal tasks. Hardcoding one model into your app means leaving performance on the table. A multi-model architecture lets you route each task to the model best suited for it — and swap models without rewriting your app logic.

The practical upside: you can build a comparison interface that runs the same prompt against multiple models simultaneously, then let users or downstream logic decide which output wins. The Retool team demoed exactly this with a concurrent LLM chat app — more on that below.

How to Connect Retool to Multiple LLM Providers

Getting Retool talking to more than one LLM provider is straightforward once you know the pattern. Here's the general approach:

• OpenAI: Use Retool's native OpenAI resource or hit the REST API directly with your API key stored as a Retool Secret.
• Anthropic (Claude): Connect via REST query using the Anthropic Messages API endpoint. Store your key in Retool Secrets and pass it as a header: x-api-key: {{anthropic_api_key}}.
• Google (Gemini): Use the Vertex AI REST endpoint or Google's Generative Language API. Authenticate via a service account or API key stored securely.
• OpenRouter: Point a single REST resource at https://openrouter.ai/api/v1/chat/completions and specify the model in the request body. One resource, 200+ models.

For enterprise deployments, route all model requests through Amazon Bedrock or an internal proxy. This keeps API keys off the client, enforces rate limits centrally, and gives your security team a single audit surface. Set the proxy URL as your base URL in the Retool REST resource and forward requests from there.

Using OpenRouter for Intelligent Model Routing

OpenRouter is the fastest way to get access to a wide range of models without managing multiple API keys and resources. The request format mirrors the OpenAI API, so if you've already built against gpt-4o, switching to a different model is as simple as changing the model field in your query body:

• "model": "openai/gpt-4o"
• "model": "anthropic/claude-3-5-sonnet"
• "model": "google/gemini-pro-1.5"

In Retool, you can expose this as a select component that lets users pick a model at runtime. Wire the component's value into the query body using {{modelSelector.value}} and you've built a model-agnostic interface in minutes.

Building a Concurrent LLM Comparison App in Retool

One of the most useful things you can ship is an app that runs the same prompt against multiple models at once — great for evaluating output quality or building internal AI tooling where teams need to compare responses. The Retool session demoed a working version of this, and the JSON export will be shared publicly in the community thread.

The core pattern:

• Create one REST query per model (or parameterize a single query by model name).
• Trigger all queries in parallel using a JS Query with Promise.all(): Promise.all([openaiQuery.trigger(), claudeQuery.trigger(), geminiQuery.trigger()]).
• Bind each query's .data to a separate column or card in a Container or Table component.
• Let users rate or select the best response to build an internal evaluation dataset over time.

Querying Your Retool Database with Natural Language

A common question from the session: can you use an LLM to query data that already lives in a RetoolDB Postgres database? Yes — and it works well. The pattern is called text-to-SQL:

• Send the user's natural language question to an LLM along with your database schema as context in the system prompt.
• The LLM returns a SQL query. For example, "How many visits did Technician A make last week?" becomes SELECT COUNT(*) FROM visits WHERE technician = 'techA' AND visit_date >= NOW() - INTERVAL '7 days'.
• Pass that generated SQL into a standard RetoolDB query using {{llmQuery.data.sql}} as the query string.
• Display the result in a Table or Stat component as normal.

This unlocks natural language reporting for non-technical users — things like revenue by client, close rates by teammate, or visit counts by technician — without building a custom filter UI for every possible question.

Keeping LLM Apps Secure in Retool

A few non-negotiables before you ship an LLM-powered internal tool:

• Store all API keys in Retool Secrets, never hardcoded in query bodies or JavaScript.
• Use Amazon Bedrock or an internal proxy as a gateway if your org has data residency or compliance requirements.
• Sanitize any user input before it hits the LLM prompt to prevent prompt injection — especially if users can freeform-type into the prompt.
• Scope Retool permissions so only the right user groups can trigger queries that write data based on LLM output.

Multi-model AI apps in Retool are genuinely within reach for any team already comfortable building internal tools. The session recording and example app JSON are available via the Retool community thread — worth grabbing as a starting point before building from scratch.