Ai Red Teaming Expert

Build and scale the "Safety & Trust" engine for our agentic AI ecosystem. You will be the technical lead responsible for ensuring our AWS Bedrock-based agents meet the highest standards of FCA Operational Resilience, DORA, and EU AI Act (Art. 15) compliance.

Core Responsibilities

• Automated Red Teaming: Implement adversarial testing (Garak, Pyrit, AgentDojo) directly into CI/CD pipelines with automated release gating.

• Centralised Eval Platform: Operate a firm-wide service to measure success rates, uncertainty, hallucination, and bias across all non-deterministic systems.

• Secure Architecture: Map OWASP LLM Top 10 and agentic threats to technical controls; manage AWS Bedrock Guardrails and Knowledge Bases.

• AI Supply Chain: Own the AI-BOM, ensuring supply chain integrity, signed artifacts, and drift monitoring.

• Regulatory Evidence: Produce the technical documentation and robust testing evidence required for EU AI Act Article 15.

Technical Requirements

• AWS Bedrock Expert: Hands-on experience with Bedrock Agents, Knowledge Bases, and model lifecycle management.

• AI/ML Depth: Strong grasp of FMs, RAG, tool-use, and the failure modes of agentic workflows.

• Security & Compliance: Deep knowledge of NIST AI RMF, OWASP LLM Top 10, and UK/EU financial regulations (FCA/DORA).

• Testing Automation: Proven ability to build measurement frameworks for drift, memorization, and adversarial robustness.

Qualifications

• Significant experience in UK Financial Services.

• Expertise in automated adversarial testing and evaluation at scale.

• Ability to bridge the gap between complex AI engineering and rigid regulatory requirements.

Randstad Technologies is acting as an Employment Business in relation to this vacancy.