What you'll do
- Build model evaluation and benchmarking frameworks for LLM, SLM, and agentic AI use cases
- Recommend model selection based on accuracy, latency, cost, safety, explainability, and operational constraints
- Conduct fine-tuning, LoRA / PEFT, prompt optimization, and retrieval optimization experiments for priority business use cases
- Perform distillation and model compression proof points for smaller or more efficient models
- Develop evaluation datasets, test harnesses, golden-answer sets, and regression testing routines for AI applications
- Create observability dashboards and quality feedback loops covering model performance, hallucination risk, drift, cost, and user feedback
- Design target-state patterns for open-weight or self-hosted LLM adoption including architecture, governance, and operational readiness
Key requirements
- Python programming skills and ML/LLM engineering experience
- Expertise in model evaluation, benchmarking, model selection, and regression testing
- Proficiency in fine-tuning, LoRA / PEFT, prompt optimization, and retrieval optimization techniques
- Knowledge of model compression, distillation, and related optimization methods
- Experience with observability tools such as MLflow, W&B, LangSmith, or Langfuse
- Familiarity with LLM, SLM, human evaluation workflows, and related AI model governance
- Computer Science / Engineering / Artificial Intelligence / Data Science / Machine Learningpreferred
Benefits
- Competitive salary and benefits including life/health insurance, performance bonuses, flexible working, share purchase plan, parental leave, continuous training, career coaching, diverse and inclusive culture, and corporate citizenship initiatives.
About the job
At Accenture Newra AI Hub, we are not just building technology, we are redefining how banking operates. As part of our strategic collaboration with Piraeus, Newra is designed to responsibly embed AI at the core of its business, moving beyond experimentation to real-world impact at scale. Built to make a real difference, Newra reflects our belief that AI creates value only when it genuinely improves people’s lives.
You will work on advanced AI solutions that span the full spectrum of the bank -from core banking systems to customer experience- simplifying complexity, automating critical processes and delivering measurable results where they matter most.
Joining Newra means becoming part of a high-performing team of innovators at the beginning of a major reinvention. This is a space for people who approach AI with depth, discipline and purpose. You will collaborate across disciplines, develop future-proof skills, and help turn technology into real-world transformation.
What you 'll build
You'll help shape how the bank evaluates, adapts, and optimizes LLM and SLM capabilities for enterprise use cases. You'll work on model selection, benchmarking, fine-tuning patterns, prompt and retrieval optimization, and cost-performance improvements across AI solutions, building the evaluation rigor and optimization patterns that improve reliability, safety, latency, and measurable business impact.
Responsibilities:
Model evaluation and benchmarking frameworks for LLM, SLM, and agentic AI use cases
Model selection recommendations based on accuracy, latency, cost, safety, explainability, and operational constraints
Fine-tuning, LoRA / PEFT, prompt optimization, and retrieval optimization experiments for priority business use cases
Distillation and model compression proof points where smaller or more efficient models can deliver sufficient performance
Evaluation datasets, test harnesses, golden-answer sets, and regression testing routines for AI applications
Observability dashboards and quality feedback loops covering model performance, hallucination risk, drift, cost, and user feedback
Target-state patterns for open-weight or self-hosted LLM adoption, including architecture, governance, and operational readiness consideration
Qualification
What we need
B.Sc. M.Sc. or equivalent experience in CS, Engineering, AI, Data Science, Machine Learning, or related field
Strong Python skills, hands-on experience in ML/LLM engineering & production-grade AI experimentation
Good understanding of LLMs, SLMs, open-weight models, model families, context windows, token economics, latency, accuracy, and cost trade-offs
Experience with model evaluation, benchmarking, test sets, quality metrics, regression testing, and human evaluation workflows
Hands-on exposure to fine-tuning, LoRA / PEFT, prompt optimization, retrieval optimization, or model adaptation techniques
Familiarity with RAG, GraphRAG, embeddings, vector search, retrieval quality improvement
Experience with MLflow, W&B, LangSmith, Langfuse, or similar experiment tracking and observability tools
Comfortable working with engineering teams to translate model insights into production-ready AI patterns
Nice to have:
Experience with Hugging Face, PyTorch, transformers, quantization, model compression, or distillation
Awareness of self-hosted / open-weight LLM architecture, deployment, monitoring, and governance considerations
Experience with Azure AI Foundry, Azure ML, Databricks, or enterprise AI platforms
Understanding of regulated environments, model risk, Responsible AI, data privacy, and banking compliance requirements
What's in it for you
Competitive salary and benefits, including but not limited to: life/health insurance, performance based bonuses, monthly vouchers, company car (depending on management level), flexible work arrangements, employee share purchase plan, parental leave and various corporate discounts
Continuous training & development through global platforms & local academy. At Accenture, we believe in bringing the best to our clients through continuous learning & improvement – from basic skills to industry-specific content – available to all our people
Career coaching and mentorship to help you manage your career and develop professionally
Ongoing strength and skill-based evaluation process
Various opportunities to develop your career across a spectrum of clients, industries and projects
Diverse and inclusive culture
Opportunities to get involved in corporate citizenship initiatives, from volunteering to doing charity work
Under our Brain Regain initiative, extra relocation benefits may apply


