Data Engineer, Lighthouse

Job description

The role involves:
• Building robust batch and streaming data pipelines for production-grade data products/platforms
• Creating web services or APIs to connect analytical stacks to application layers
• Building and maintaining both cloud and on-premise data infrastructure'

The ideal candidate should possess:
• Bachelors or Masters in Computer Science, Computer Engineering, Information Systems, or similar IT related courses
• 2 - 8 years of experience in ETL, data pipeline building, and data warehousing
• Proficient in designing efficient and robust data integration workflows using Ab Initio, Informatica , Microsoft or IBM products
• Advanced understanding of database principles, security, and administration
• Familiar with SQL, dimensional modelling, data warehousing, and data integration
• Familiar with system architecture design and analysis
• Competent in software engineering and agile development
• Proficient with data visualisation tools and languages such as Tableau and D3.js
• Familiar with statistical programming languages, distributed computing concepts, frameworks, and products
• Knowledge and interest in data mining, machine learning, natural language processing, or information retrieval
• Prior exposure to NoSQL or NewSQL databases
• Familiarity with Hadoop, Spark, Airflow
• Knowledge of languages like C++ and Scala
• Able to implement and maintain components of big data technologies for both exploratory and production data science platforms
• Strong problem solving skills, curiosity and passion for technology
• Strong co-ordination and time management skills to handle complex projects

Personal data collected will be used for recruitment purposes only.