big data engineer

As a Big Data Engineer at our client’s, you will design, develop, and optimize Scala/Spark applications for big data processing at large scale.

Responsibilities

• Implement analytical algorithms in Spark
• Use Spark to transform and process network data
• Optimize and test Spark applications
• Deploy and monitor applications in customers’ environments
• Assist other teams in the design and development of Spark applications

Qualifications Required

• At least 3 years of experience developing Spark applications in PySpark and/or Scala
• Excellent skills in deploying and monitoring distributed applications
• Strong experience with Unix systems
• Extensive experience with Python
• Experience with workflow engines i.e. airflow;
• Strong skills in teamwork, communication, and analytical thinking;
• Knowledge of version control tools;
• Fluency in English, both oral and written.

Preferred

• Hands on experience working on Cloud Infrastructure (AWS)
• Good understanding of computer networks
• Experience using infrastructure-as-code tools (e.g., Terraform)
• Hands on Experience with Databases

Education

• BSc or MSc in Computer Science, Information Systems Management or a related field is required

Benefits

• Competitive compensation package
• Remote-first company culture and modern working environment
• Selective conference and training expenses covered
• Private Insurance