to standardize Gemini usage across F5 teams. Optimize prompt engineering, token efficiency, grounding strategies, and structured... evaluation pipelines and AI quality metrics. Familiarity with structured output enforcement (JSON schemas, function calling...
limits, token budgeting) Prompt engineering (few-shot, structured output, JSON mode) PydanticOutputParser / structured LLM... retriever abstractions RAG evaluation metrics: o Faithfulness o Relevance o NDCG o MRR Trace-level RAG evaluation...
, and implement OpenAI-compatible API endpoints with token streaming Develop and extend the plugin lifecycle system: packaging..., scalable environments Experience with the monitoring, logging, metrics, and operational support for production systems...
-based agent design (40+tool environments) Azure OpenAI / OpenAI APIs (GPT-4o, deployment mgmt, rate limits, token... retriever abstractions RAG evaluation metrics: o Faithfulness o Relevance o NDCG o MRR Trace-level RAG evaluation...
to standardize Gemini usage across F5 teams. Optimize prompt engineering, token efficiency, grounding strategies, and structured... evaluation pipelines and AI quality metrics. Familiarity with structured output enforcement (JSON schemas, function calling...
to standardize Gemini usage across F5 teams. Optimize prompt engineering, token efficiency, grounding strategies, and structured... evaluation pipelines and AI quality metrics. Familiarity with structured output enforcement (JSON schemas, function calling...
(Terraform/CloudFormation/Bicep). o Ensure observability (logs, metrics, traces with OpenTelemetry), SLIs/SLOs, and error... budgets. Security & Compliance o Apply OWASP Top 10 controls, secure coding, and token-based auth (OAuth2/OIDC/JWT...
, and iteration to improve reliability. Implement model telemetry and operational monitoring: latency, token usage, quality signals... understanding of ML fundamentals (supervised/unsupervised learning, embeddings, evaluation metrics) and how they relate to LLM...
Frameworks Build and maintain observability pipelines using tools like Arize AI (tracing, quality metrics, dataset drift.../hallucination tracking, embedding monitoring). Define and enforce platform-wide standards for: Tracing LLM calls Token usage...
, responsible, and non-toxic outputs. Optimize for cost & performance: Instrument, analyze, and optimize unit economics (token...), retrieval strategies, and indexing pipelines. Observability: Built robust tracing/metrics/logging for AI systems;familiarity...