Skip to content

LLM Model Policy

Status: accepted policy for the native AI runtime.

ADR: docs/decisions/0017-native-two-provider-ai-runtime.md

Runtime Boundary

Customer-facing AI must use TradeMate's native compatibility layer. The layer is responsible for provider selection, model routing, deadlines, retries, structured-output validation, usage capture, cost estimates, audit records, and Phoenix trace correlation.

Supported runtime providers:

  • anthropic
  • openai

Retired legacy providers:

  • google
  • deepseek

Google and DeepSeek key rows may exist temporarily as historical/deletable records, but they are not valid for new key creation, runtime selection, or customer-facing agent execution.

The runtime has no network dependency on a proxy service. There is no deployment runtime switch and no request-level legacy fallback. If the native runtime has a production issue, operators must roll back the application deployment rather than routing individual paid BYOK requests through a second runtime.

BYOK Rules

  • Customer AI requires a paid tier and a validated per-user BYOK key.
  • Anthropic is the default primary provider when both supported providers have valid keys.
  • Users may select OpenAI as their primary provider.
  • A single valid provider key must support the full strategy-analysis workflow.
  • No provider key may be read from service environment variables as a customer fallback.
  • Raw provider keys must never be written to audit logs, telemetry attributes, exception messages, graph state payloads, persisted agent decisions, database metadata, or browser responses.
  • Decrypted keys may exist only in request-local sensitive context at the API or worker boundary.
  • Provider SDK usage is allowed only inside packages/agents provider adapters.
  • Provider Agent SDKs are not allowed for orchestration, tool policy, evidence analysis, or trading decisions.

Model Roles

Model role Anthropic default OpenAI default Use
worker claude-sonnet-5 gpt-5.6-terra Evidence specialists, extraction, classification, reflection, and routine analysis.
premium claude-opus-4-8 gpt-5.6-sol Final synthesis, independent verification, high-impact manual review, and hard cases.

max, OpenAI ultra, and provider-managed multi-agent modes are excluded from automated production workflows. XHigh reasoning is reserved for explicit deep-review operations.

Reasoning Profiles

TradeMate task Model role Reasoning profile
Simple extraction and classification worker Low
Technical evidence worker Medium
Fundamental evidence worker High
News evidence worker Medium
Market/regime evidence worker Medium
Risk-context evidence worker High
Final strategy synthesis premium High
Independent verification premium High
Schema repair worker Low
Reflection and portfolio narrative worker Medium
High-impact manual deep review premium XHigh

Reasoning profiles are normalized TradeMate concepts. Each provider adapter maps them to the provider's supported controls.

Verification And Fallback

  • Cross-provider verification is enabled only when both supported provider keys are valid and the user has allowed verifier calls.
  • The UI must clearly disclose that cross-provider verification creates a second BYOK call.
  • With one provider, synthesis may include same-provider self-critique, but it must be labeled internally as non-independent verification.
  • Provider disagreement produces abstain or manual_review; it must never be resolved by majority vote.
  • Automatic provider fallback is off by default.
  • If the user enables fallback, it is allowed only for definitive no-charge failures such as local rejection, invalid credentials, or an already-open circuit.
  • Fallback is not allowed after ambiguous timeouts where the first provider may already have processed the request.
  • Request-level fallback to a retired legacy runtime is not allowed.

Budget Rules

  • Free workflows use deterministic logic or explicit unavailable states.
  • Customer AI endpoints, pipelines, and webhooks require Pro/Elite BYOK context before any model call.
  • Every model attempt records provider, requested model, reported model, route kind, reasoning profile, token usage where available, latency, status, prompt/schema/evidence hashes, and estimated BYOK cost.
  • The runtime must enforce per-user/provider concurrency, per-run model-call budgets, per-run verifier/repair limits, and whole-run deadlines.
  • LLM output cannot override deterministic risk limits, portfolio permissions, broker mode, consent, or order-safety gates.

Trace Expectations

SigNoz owns API, worker, queue, and infrastructure telemetry. Phoenix/OpenInference owns LLM and agent debugging.

Safe Phoenix attributes:

  • Provider and requested/reported model.
  • Reasoning profile and route kind.
  • Attempt, latency, status, and normalized error code.
  • Token/cache/reasoning usage and estimated cost.
  • Provider request ID.
  • Prompt, schema, and evidence hashes.
  • Citation count, validation result, disagreement, and abstention flags.

Phoenix must not capture API keys, authorization headers, raw prompts, full evidence packets, full filings, full news bodies, full model outputs, brokerage account IDs, or unnecessary user-private identifiers.