LastWeek

LastThreeDays

Today

Aoristou

FullTime

Hybrid

OnSite

The Director, AI Quality Engineering Lead ensures that AI systems across the enterprise are safe, fair, reliable, compliant, and observable from pre‑release through continuous operations. This role designs and operates enterprise evaluation frameworks (quality, bias/fairness, robustness, privacy), leads red‑teaming and guardrail development, encodes policy and regulatory requirements as automated checks, and drives metric reporting and post‑incident learning across AI programs. The lead partners closely with Enterprise Architecture, Compliance, Legal/Privacy, Security, Data Governance, and product/engineering teams to enforce quality and safety gates and enable responsible scale. (Aligned to Corporate Policy and Responsible AI principles.) Reporting into the Head of Applied AI Engineering, you will manage a team of QA Engineers and AI Engineers. This role blends deep technical leadership, product engineering strategy, and direct enablement of enterprise use cases and services. You will partner closely with business, legal and technical stakeholders to design and deliver robust, scalable solutions that demonstrate the value of AI across our enterprise in a safe and compliant way. This role is execution-oriented with strong influence on building AI solutions responsibly.Evaluation and Test Strategy<ul><li>Define and run evaluation frameworks for LLMs and multi-modal systems covering performance, fairness, bias, robustness, safety, and privacy.</li><li>Build automatic + human-in-the-loop evaluation pipelines with reproducible scoring, statistical significance, and error analysis.</li><li>Establish SLOs/SLIs for quality (e.g., grounded accuracy, hallucination rate, privacy leak rate, harmful-content rate).</li></ul>Red-Teaming &amp; Safety Guardrails<ul><li>Design and run red-teaming against prompt injection, jailbreaks, sensitive data leakage, tool-abuse, and retrieval abuse.</li><li>Develop guardrail solutions (classification, content safety, PII/PHI detection, grounding checks, allow/deny lists, rate limiting, output filtering) to mitigate hallucinations, leakage, and compliance violations.</li><li>Maintain attack libraries and scenario banks; simulate adversarial behavior and monitor adaptive attacks over time.</li></ul>Pre‑Release Gating &amp; Continuous Assurance<ul><li>Operate pre‑release evaluation gates with baselines, regression checks, canary/A–B testing, and rollback criteria.</li><li>Run continuous evaluation in production with synthetic &amp; real traffic (shadow/ghost testing) and drift, bias, and toxicity monitoring.</li></ul>Stakeholder Leadership &amp; Culture<ul><li>Coach product teams on responsible AI practices and establish quality ownership norms.</li><li>Build a community of practice across Engineering, Security, Data Governance, and Risk—standardizing test data, benchmarks, and review rituals.</li><li>Mentor engineers building evaluation, red-team, and guardrail capabilities.</li></ul>BASIC QUALIFICATIONS<ul><li>7+ years in ML/AI/Platform/Quality roles, including 3+ years evaluating production systems.</li><li>Advanced degree in Computer Science, Engineering or related discipline.</li><li>Proven design &amp; operation of evaluation harnesses and policy-as-code checks in CI/CD.</li><li>Hands-on Python and expertise with evaluation and safety tooling such as: OpenAI Evals, Ragas/LangSmith, DeepEval, Azure AI Content Safety / OpenAI Moderation, Presidio, Guardrails.ai, Nemo, MLFlow, Grafana, Kibana, Azure monitor</li><li>Cloud &amp; MLOps: Azure/AWS, CI/CD (GitHub Actions/Azure DevOps, feature stores, vector DBs (Azure AI Search, OpenSearch, Pinecone), testing in pipelines.</li><li>Excellent communication with executives and engineers</li><li>Experience leading an engineering team.</li></ul>PREFERRED QUALIFICATIONS<ul><li>Led an AI safety/quality function or enterprise red-team program.</li><li>Prior work with AI based applications, tool use/orchestration, and enterprise data governance.</li><li>Regulatory literacy (e.g., GDPR, EU AI Act readiness), and experience collaborating with Compliance/Risk, Security, and Legal.</li><li>Strong grasp of red-team techniques (prompt/indirect injection, jailbreaks, training data extraction, membership inference, model poisoning, tool abuse).</li><li>Experience with retrieval grounding checks, memory governance, and multi-tenant safety controls.</li></ul>

Director, AI Quality Engineering Lead 4946565

Your TasksAs a Cyber Security Specialist (m/f/d) you are an important part of our Offensive Security Team. You will be responsible to conduct targeted security analyses, penetration tests or red team engagements (adversary simulations) and attack our customers' systems and networks. The following activities are within your area of responsibilities:<ul><li>Understanding of current cyber threats and the ability to identify and exploit vulnerabilities.</li><li>Gaining knowledge of attack techniques (TTPs) from advanced threat actors and implementing the latest attack methods, as well as automating test cases.</li><li>Presenting technically complex attacks and vulnerabilities in a comprehensible way and providing our customers with solutions to protect against cyber-attacks.</li><li>Actively shaping the development and evolution of the DTS Red team, continuously sharing knowledge with your colleagues to improve the quality of our services.</li><li>Researching topics and working on the latest concepts such as evasion techniques, malware execution or lateral movement for successful Red and Purple team assignments.</li></ul>Your Profile<ul><li>Experience with Penetration Testing, Red Team Assessments or similar Offensive Security related work.</li><li>Good knowledge on networks, applications, Active Directory and operating systems (Microsoft and/or Linux).</li><li>Keen interest in Offensive Security tools and control frameworks.</li><li>Creative, good analytical skills and an attacker-like mindset.</li><li>Familiarity with the basics of a programming/scripting language.</li><li>Excellent communication skills with an ability to explain complex security issues to technical and non-technical audiences.</li></ul>We Offer<ul><li>30 days of vacation&nbsp;(regardless of the length of service)</li><li>Private health insurance for our employees and their dependents (spouse/children)</li><li>In-depth, comprehensive training (approx. 1-3 months) in Germany at our expenses included one monthly flight back to Greece during the whole training period</li><li>Go for EAT - Ticket restaurant card</li><li>Modern hybrid way of working, characterized by flexible working arrangements</li><li>Team-oriented working environment and a broad variety of tasks in an international and diverse team, based on trust and transparency</li><li>Continuous opportunity for personal and professional development and learning</li><li>Personal company hardware</li><li>Free snacks and beverages</li></ul>

Job Ads Quality Controller Μακεδονία, Information Technology / It

director, ai quality engineering lead 4946565

Pfizer Hellas

cyber security specialist – offensive security / red teaming (m/f/d) - thessaloniki

DTS Systeme GmbH

Also hiring nearby:

Jobily.gr

For Employers

For Job Seekers

Company