for DoD Agency spaces at multiple Government locations. You will provide all Agency SIPR token administration, CAC resets... of personnel, visit requests, etc. to inform the collection of robust performance metrics. Ensure personnel are completing...
company — token consumption, adoption, leaderboards, and the impact metrics that feed executive reporting. Data Platform... scans, environment promotion, and rollback. Stand up and own logging, metrics, tracing, alerting, and cost monitoring...
: ReAct Self-Reflection Hierarchical Delegation Strong understanding of: Token Utilization Cost Optimization State Management... AI Tracing LLM Performance Metrics For applications and inquiries, contact: [email protected]...
, experiment troubleshooting, latency, token usage, cost, and failure analysis. Production Support Incident triage, root-cause... latency, tool-call latency, token usage, error rate, trace ingestion health, cost trends, and failed user journeys. Incident...
: safe auth flows, token handling guidance, secure storage, and protection against common web vulnerabilities. Collaborate... patterns (OIDC awareness a plus). Observability for the front end: client-side logging/metrics, error reporting, and debugging...
. Implements and governs authentication/authorization using OIDC/OAuth2, token validation, scopes/claims, and service-to-service... security patterns. Establishes robust observability practices (structured logging, metrics, tracing), incident response...
summarization patterns, and semantic search configurations. FinOps & Compliance: Optimize token usage, manage cloud costs, ensure... stacks, and metrics achieved within your most recent project history. Avoid generic filler descriptions or company-level...
solutions Instrument workflows with tracing, logging, and metrics to ensure observability and reliability Define and monitor... preferred Familiarity with prompt engineering, token economics, and LLM evaluation pipelines preferred Experience building...
). - Cost-aware testing (token usage, rate limits, fallbacks). - Resilience tests: retries, circuit breakers, model timeouts... and backend teams. - Ownership mindset: building reusable test harnesses, improving quality metrics, preventing regressions...
-to-end observability for agentic workflows including latency, throughput, token usage, and LLM quality metrics Implement LLM... integrations Monitor and optimize token economics, LLM provider costs, and cost-per-query performance Implement MCP server...