Who you are
As a Data Engineer in Analytics and Insights team, you will apply your deep expertise in Data integration and transformation, prepare the data for computing the insights for the business teams to take decisions across the organization. Also automate the test processes and accelerate the overall development cycle.
Job Role and Responsibilities:
• Perform data analysis on large-scale datasets
• Build dashboards to Communicate insights across Digital platforms
• Data capturing and Transformation
• Find elegant solutions to complex problems in delivering data and analytics
• Hadoop development and implementation.
• Loading from disparate data sets.
• Pre-processing using Hive and Pig.
• Designing, building, installing, configuring and supporting Hadoop.
• Translate complex functional and technical requirements into detailed design.
• Perform analysis of vast data stores and uncover insights.
• Maintain security and data privacy.
• Create scalable and high-performance web services for data tracking.
• High-speed querying.
• Managing and deploying HBase.
• Being a part of a POC effort to help build new Hadoop clusters.
• Rich and hands on experience with Hadoop distributed frameworks, handling large amount of big data using Spark and Hadoop Ecosystems
• Experience with Spark, Scala, and Sqoop are required
• Strong knowledge in designing and developing RDBMS and SQL queries are required
Auto req ID
Role ( Job Role )
State / Province
Primary job category
Software Development & Support
(0063) IBM India Private Limited
Is this role a commissionable/sales incentive based position?
IBM Business Group
City / Township / Village
Being You @ IBM
IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
Required Technical and Professional Expertise
· Overall 6 + years of experience
· Proven experience and detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.
· Experience in relational database concepts with a solid knowledge of SQL.
· Strong knowledge of various data warehousing methodologies and data modelling concepts. Hands on modeling experience is highly desired
· Knowledge of professional software engineering practices & best practices for the full software development lifecycle, including coding standards, code reviews, source control management, build processes, testing, and operations
· Experience and knowledge in unit testing, automated testing
· Experience in agile development methodologies
Preferred Technical and Professional Experience
· Experience in Data engineering work around Data Integration / ETL
· Experience in Hadoop ecosystem, i.e. Hadoop, HBase, Hive, Scala, spark, Sqoop, Flume, Kafka and Python
· Experience in software development process and Trouble shooting critical issues.
· Experience in Design and architecture review. Proficiency in data analysis and strong SQL skills.
· Prove experience in data warehousing, ETL tools, MPP database systems
· Experience working in HIVE & Impala & creating custom UDFs and custom input/output formats /serdes
· Ability to acquire, compute, store and process various types of datasets in Hadoop platform. Experience with Scala.
Secondary Job Category
Other Software Development & Support
Early Professional Track
Industry Early Professional
For additional information about location requirements, please discuss with the recruiter following submission of your application.
Software Developers at IBM are the backbone of our strategic initiatives to design, code, test, and provide industry-leading solutions that make the world run today - planes and trains take off on time, bank transactions complete in the blink of an eye and the world remains safe because of the work our software developers do. Whether you are working on projects internally or for a client, software development is critical to the success of IBM and our clients worldwide.
At IBM, you will use the latest software development tools, techniques and approaches and work with leading minds in the industry to build solutions you can be proud of.
Your Life @ IBM
What matters to you when you're looking for your next career challenge?
Maybe you want to get involved in work that really changes the world? What about somewhere with incredible and diverse career and development opportunities – where you can truly discover your passion? Are you looking for a culture of openness, collaboration and trust – where everyone has a voice? What about all of these? If so, then IBM could be your next career challenge. Join us, not to do something better, but to attempt things you never thought possible.
Impact. Inclusion. Infinite Experiences. Do your best work ever.
About Business Unit
At IBM Cognitive Applications, we build open applications that unlock the power of data for clients, partners, and developers. Running on top of IBM's unique Hybrid, Multi-cloud and AI infrastructures, these applications work across horizontal domains and bring our technology to life for end users. Cognitive Applications unit includes: Watson Customer Engagement, Watson IoT, Watson Media and Weather, Talent & Collaboration, Digital Growth & Commerce, and IBM Developer teams.
IBM's greatest invention is the IBMer. We believe that progress is made through progressive thinking, progressive leadership, progressive policy and progressive action. IBMers believe that the application of intelligence, reason and science can improve business, society and the human condition. Restlessly reinventing since 1911, we are the largest technology and consulting employer in the world, with more than 380,000 IBMers serving clients in 170 countries.