# Awareness LLM Augmentation Layer — Quick Integration Guide This guide explains how to place Awareness between a user interface and a base LLM. Awareness evaluates generated responses against its certified corpus and returns a response only when the routing, retrieval, evaluation, and safety gates pass. ```text UI or application → Awareness → configured base-model provider ↓ retrieval / evaluation / policy gates ``` ## 1. Decide which runtime connection you need There are two separate connections: 1. **Base model → Awareness runtime**: configured by the Awareness operator in GCP. The provider key is stored in Secret Manager and is never given to the UI or application caller. 2. **UI/application → Awareness**: configured by each client with a tenant API key. Clients use the OpenAI-compatible Awareness endpoint. Do not configure a client to call the provider directly; that bypasses Awareness evaluation. ## 2. Supported base-model providers The deployed adapter supports these provider shapes: | Provider | Runtime settings | Notes | |---|---|---| | Sarvam | `AWARENESS_MODEL_PROVIDER=sarvam`, `AWARENESS_MODEL_BASE_URL=https://api.sarvam.ai/v1`, `AWARENESS_MODEL_NAME=sarvam-105b`, `AWARENESS_LLM_API_KEY` | Production-certified deployment in this project. The provider key is a Secret Manager binding. | | vLLM | `AWARENESS_MODEL_PROVIDER=vllm`, base URL, model name | Requires the configured OpenAI-compatible completions endpoint and the provider tokenizer endpoint. | | Ollama | `AWARENESS_MODEL_PROVIDER=ollama`, base URL, model name | Test mode only; requires `AWARENESS_TEST_MODE=true` and an explicit context cap of 8,192 tokens. | | Soket | `AWARENESS_MODEL_PROVIDER=`, client endpoint, exact model name | Soket's OpenAI-compatible `/v1/chat/completions` endpoint and HuggingFace tokenizer must be certified before production use. | An arbitrary LLM is not automatically supported merely because it is an LLM. It must expose the adapter’s required OpenAI-compatible contract, or a new adapter must be certified and deployed first. ## 3. Configure the base model in GCP Production configuration is applied to the Cloud Run service, not to Open WebUI. Use Secret Manager for the provider key and endpoint where appropriate: ```bash gcloud run services update awareness-service \ --region=asia-south1 \ --project=bluepatterns-awareness \ --set-env-vars="AWARENESS_ENVIRONMENT=production,AWARENESS_MODEL_PROVIDER=sarvam,AWARENESS_MODEL_NAME=sarvam-105b,AWARENESS_ENABLE_GENERATION=true,AWARENESS_KMS_KEY_NAME=projects/bluepatterns-awareness/locations/asia-south1/keyRings/awareness-keyring/cryptoKeys/awareness-kek" \ --set-secrets="AWARENESS_DATABASE_URL=awareness-prod-db-url:latest,AWARENESS_DATABASE_PASSWORD=awareness-prod-db-password:latest,AWARENESS_MODEL_BASE_URL=awareness-llm-endpoint:latest,AWARENESS_LLM_API_KEY=awareness-llm-api-key:latest" \ --network=awareness-vpc \ --subnet=awareness-asia-south1 \ --vpc-egress=private-ranges-only ``` The runtime service account must have Secret Manager accessor permission on the referenced secrets and KMS encrypt/decrypt permission on the configured key. Never place a provider key in source control, a Dockerfile, Open WebUI, or a client request. ## 4. Client/API connection Use the HTTPS load-balancer hostname as the client base URL: ```text https://awareness.bluepatterns.ai/v1 ``` Use a tenant Awareness API key, supplied as `Authorization: Bearer `. The service also accepts `X-API-Key: ` for service-to-service clients. ### OpenAI SDK ```python from openai import OpenAI client = OpenAI( base_url="https://awareness.bluepatterns.ai/v1", api_key="", ) result = client.chat.completions.create( model="sarvam-105b", messages=[{"role": "user", "content": ""}], stream=False, ) print(result.choices[0].message.content) ``` ### cURL ```bash curl -sS https://awareness.bluepatterns.ai/v1/chat/completions \ -H "Authorization: Bearer $AWARENESS_TENANT_API_KEY" \ -H "Content-Type: application/json" \ -d '{"model":"sarvam-105b","messages":[{"role":"user","content":""}],"stream":false}' ``` The direct Cloud Run `run.app` URL is IAM-protected and is intended for operator/service testing. Normal UI traffic should use the HTTPS load balancer. ## 5. Open WebUI configuration Create an OpenAI-compatible connection with: ```text API Base URL: https://awareness.bluepatterns.ai/v1 API Key: tenant Awareness API key Model: sarvam-105b ``` If Open WebUI only displays the model returned by `GET /v1/models`, add `sarvam-105b` as a custom model until the production model-list endpoint is made environment-aware. ## 6. Required smoke tests ### Positive certified-domain test The request must return HTTP 200, a non-empty answer, and evidence-grounded content for a certified legal benchmark such as Section 99-B. ### Negative fail-closed test Send an uncertified medical or unsupported-domain question. The response must not contain a generated medical/legal answer; it must return the documented fail-closed message. ### Operational checks - Confirm the latest Cloud Run revision is Ready. - Confirm the managed TLS certificate is `ACTIVE`. - Confirm the load-balancer hostname resolves to the reserved global IP. - Confirm Cloud Armor is attached to the backend service. - Confirm provider secrets are not present in logs or client-side configuration. ## 7. Integration boundaries and limitations - Awareness currently exposes `/v1/chat/completions` and `/v1/models` for UI compatibility, plus the native `/awareness/v1/augment` trace-oriented API. - Streaming is OpenAI-compatible, but the current implementation emits the evaluated answer as a response chunk; it is not provider-token streaming. - Only certified domains and task types can reach generation. - A new provider requires an adapter, token-budget strategy, timeout policy, response parser, evaluator certification, and deployment test. ## Open decisions requiring confirmation Before another team follows this guide, confirm: 1. Is `awareness.bluepatterns.ai` the permanent production hostname, or should the guide use a different approved domain? 2. Should the `/v1/models` endpoint advertise `sarvam-105b` in production instead of the current test-oriented model identifier? 3. Which additional providers (if any) must be certified beyond Sarvam, vLLM, and test-only Ollama? 4. Should Open WebUI use one shared tenant key or separate keys per workspace, user, or deployment? ## Client handoff: evaluating a client-owned base model If a client wants to evaluate its own model through Awareness, the client must provide the following runtime details through an approved secret-sharing channel. Do not put credentials in this Markdown file, email, tickets, or chat. ### Information required from the client 1. Provider protocol: OpenAI-compatible chat completions, OpenAI-compatible text completions, or a documented non-standard protocol. 2. Model identifier exactly as accepted by the provider. 3. Private or public base URL, including whether `/v1` is already part of the URL. 4. Authentication method and secret name/value delivery method. Awareness must receive the provider credential through Secret Manager or an equivalent secret store. 5. Tokenization endpoint or tokenizer package/version. Production exact-token budgeting cannot be certified without a reliable tokenizer. A conservative fallback is permitted only for explicitly approved test mode. 6. Context-window and maximum-output-token limits, including any provider rate limits, timeout limits, and regional/network restrictions. 7. Whether streaming is required and the provider's streaming response format. 8. Approved evaluation domain, corpus version, jurisdiction, and benchmark questions. The client must not request generation for an uncertified domain. ### What Awareness configures For an already-supported provider, the operator registers a provider-specific runtime configuration equivalent to: ```text AWARENESS_MODEL_PROVIDER= AWARENESS_MODEL_BASE_URL= AWARENESS_MODEL_NAME= AWARENESS_MODEL_CONTEXT_WINDOW= AWARENESS_ENABLE_GENERATION=true ``` The provider key is bound separately from Secret Manager as `AWARENESS_LLM_API_KEY`. The client never needs the Awareness service's provider credential. For a provider not listed above, the client must supply API documentation and a test credential so an adapter can be implemented, security-reviewed, and certified before production generation is enabled. ### Client test procedure 1. The client points its test UI or SDK to the Awareness base URL, not the model provider URL. 2. Awareness issues a tenant API key for that client environment. 3. The client runs the supplied certified benchmark and records request IDs, latency, status, and delivered/blocked decisions. 4. Awareness verifies retrieval, evaluator decisions, retry behavior, and fail-closed behavior against the selected corpus version. 5. Production enablement occurs only after the benchmark, security review, tokenizer budget, and operational gates pass. ### Client-side UI settings ```text API Base URL: https://awareness.bluepatterns.ai/v1 API Key: client tenant Awareness key Model: ``` The client UI must not call the provider directly. If the UI cannot configure an OpenAI-compatible endpoint or custom model identifier, the client must provide its UI's integration documentation before onboarding. ## Soket intake alignment and corrections The Soket evaluation intake has been reviewed against the deployed service. The following corrections are required when applying it: - Use the actual Awareness settings `AWARENESS_MODEL_BASE_URL`, `AWARENESS_MODEL_NAME`, and `AWARENESS_MODEL_PROVIDER`. The intake names `AWARENESS_LLM_ENDPOINT` and `AWARENESS_LLM_MODEL` are not runtime settings in this service. - Do not set `--allow-unauthenticated` on the production Cloud Run service. The approved path is the HTTPS load balancer with Cloud Run IAM protection and a tenant Awareness API key. If Soket needs a fast evaluation surface, create a separate staging service or use the authenticated load-balancer endpoint; do not weaken production ingress. - A Cloud Run `run.app` URL alone is not sufficient for Soket's Bearer API-key client because Cloud Run IAM is evaluated before the Awareness API key. Use the approved HTTPS edge, or provide a controlled identity-aware proxy. Never share a Google service-account private key with Soket. - Soket's HuggingFace tokenizer may be loaded only if its package/weights are pinned into the image or made available through an approved private artifact path. Runtime downloads from the public HuggingFace Hub are not assumed. If exact loading cannot be proven, label the evaluation as approximate and do not certify production token budgets. - Soket's exact model identifier is still required. Do not use `awareness-legal-test` as a production model name; it is a test-oriented identifier currently returned by `/v1/models`. - The intake's statement that a BNS 2023 corpus is already configured must be verified against the production `knowledge_base` contents and certification record. The Section 99-B synthetic benchmark does not by itself certify the full BNS 2023 corpus. - The current service emits an evaluated answer as SSE chunks. This is OpenAI-compatible but is not provider-token passthrough. Soket must approve this behavior for evaluation, or a true streaming passthrough design must be implemented and certified. - The intake's example SQL must be adapted to the deployed schema and tenancy workflow. Do not execute it verbatim until the current PostgreSQL migration, roles, expiry columns, and tenant quota fields have been verified. ### Required client acceptance evidence The client should return: - positive benchmark responses and their request IDs; - expected fail-closed responses for uncertified or unsafe prompts; - observed p50/p95 latency and timeout rates; - confirmation that provider keys are absent from browser requests and logs; - confirmation of the model version, tokenizer version, context cap, and provider rate-limit tier used during testing. ## Production client-service pattern The recommended service model is **one logical Awareness control plane with a separate security and policy boundary for every client tenant**. This is the usual pattern for managed AI platforms and prevents one client's keys, prompts, cache entries, traces, quotas, or corpus policy from crossing into another tenant. ### Recommended secure defaults - Create a unique tenant ID and separate API key for each client environment (development, staging, and production should not share keys). - Store only a hash of the Awareness API key in the database; display the raw key once at issuance and require rotation after suspected exposure. - Route clients through the HTTPS load balancer and Cloud Armor. Do not give clients the direct Cloud Run `run.app` URL. - Use tenant-scoped rate limits, quotas, cache namespaces, trace access, and encryption context. Never use a global cache key for client prompts or outputs. - Keep provider credentials server-side in Secret Manager. The client sends requests to Awareness; Awareness calls the client's model endpoint. - Prefer private connectivity to a client model (VPN, VPC peering, or an approved private service endpoint). If public HTTPS is required, restrict egress to the provider host, require TLS, and validate certificates. - Disable prompt, evidence, and rejected-output retention by default unless the client has approved a documented retention period and legal basis. - Redact secrets, authorization headers, and unnecessary personal data from logs. Audit key issuance, rotation, policy changes, model changes, and administrative access. - Certify each provider/model/tokenizer combination before enabling it for production traffic. A model that merely responds to an HTTP request is not automatically certified. - Roll out a new client model or policy in staging, then canary a small traffic percentage before full production enablement. Keep a tested rollback target. ### Client onboarding sequence 1. Create the tenant and environment-specific API key. 2. Collect the client's provider contract, model ID, endpoint, tokenizer, context limits, rate limits, region, and data-processing requirements. 3. Validate network reachability from the Awareness runtime without exposing provider credentials to the client UI. 4. Register or certify the provider adapter and model configuration. 5. Load and approve the client's corpus, jurisdiction, policy version, and benchmark set. 6. Run positive, negative, malformed-response, timeout, quota, and retry tests in staging. 7. Review security, privacy, retention, and incident contacts with the client. 8. Enable production generation only after the acceptance record is signed. ### Explicit failure contract Awareness must fail closed when certification, verified evidence, safety evaluation, token budgeting, authentication, or provider availability fails. It must never silently bypass evaluation and call the client's model directly. Clients should treat a blocked decision as a valid policy outcome, not as a provider answer. ### Choices the client must explicitly approve The onboarding record must capture the selected value for: - public versus private model connectivity; - prompt/evidence/output retention and deletion period; - per-tenant quota and rate-limit tier; - streaming versus evaluated-response delivery; - allowed jurisdictions and policy versions; - cross-region processing and data residency; - incident notification contacts and key-rotation owner.