Internship: Failure prediction of cluster jobs

Description de l'offre

If you could change one thing about travel, what would it be? At Amadeus, you can make that happen!

Travel makes the world a better place and we are fully dedicated to improving it and making it even more rewarding. We are one of the world’s top 15 software companies: we provide technology solutions and services within the travel industry.

Do you have ideas on how to improve travel for everyone? Do you find the idea of working in a diverse, multicultural environment exciting? Are you ready to make an impact across the world? Great, then join us! Let’s shape the future of travel together. #shapethefutureoftravel@Amadeus

Overview

Currently, most cloud failures are detected only when they indeed happen. Some prior studies on large-scale system reliability have focused on finding correlations between resource consumption and failure behaviour of applications. Evidences suggestthat fault-free nodes in MapReduce have similar behaviors , and that a deviation from this behaviour indicates a failure. Others studieshave found that often faults are anticipated by unstable performance behaviors, thus enabling failure prediction techniques. Besides these algorithms, other investigationsfind that most of the task failures in clouds result from out of memory exceptions.

In this internship, we want to build a prediction tool for cloud systems that makes use of the usage measures and logs, to predict job and task failures. We are looking for ambitious and enthusiastic candidates with a strong interest in machine learning applications, and a solid theoretical background.

Project goals

The goals of this internship are summarized as follows:

· Analysis of system failure behaviour
· Collect relevant data(resource indicator, logs) from the cluster
· Anomalies detection in the cluster behavior
· Predict failures based on resource usage and logs
· Root cause analysis automation with AI

Candidate skills

· Research oriented mindset
· Very good understanding of machine learning algorithms and their applications
· Linux
· Scala/Spark
· Python/Scikit-learn or R
· HDFS

Our diversity commitment: equality, diversity and inclusion are part of who we are. We’re committed to equal opportunities and treatment regardless of age, ethnicity, gender, beliefs, sexual orientation or disability.

Any duplication and display of partial or full content of our job advertisement on any support, such as brochures, websites, mail, emails, this list is not exhaustive, is strictly forbidden without prior formal Amadeus’ authorisation.

Recruitment agencies: Amadeus does not accept agency resumes. Please do not forward resumes to our jobs alias, Amadeus employees or any other company location. Amadeus is not responsible for any fees related to unsolicited resumes.