Big Data Engineer (Back end developer, Spark/Kubernettes)
Amsterdam (Montgomery) IT development
Job description
Do you have a passion for learning and applying modern software engineering practices to help solve business problems or bring new insight? If so, we are looking for energetic team members to join our agile squad in 51 Astor Pl, New York, NY 10003.
Job Duties:
We are looking for someone to develop a modern, cloud based solution for a strategic initiative that enables IBM to more effectively and efficiently make business decisions based using your solution. Your team's responsibility will be to create the working foundation of infrastructure and architecture design of the large scale, big data solution. This will include the development of the prototype through to a full operational system that many different teams (including yours) perform development within.
The key experiences required include Apache SPARK, Apache Kafka, Python, and Parquet formats. Also, significant experience using ETL (extract, transform, and load tools) not mentioned are needed. Experience with Scala is also a nice to have.
You will also be responsible for selecting and using the DevOps tools for continuous integration, builds, and monitoring of the solutions. You will join and exciting team, in an open landscaped, collaborative environment using agile delivery methods. Any experience in converting legacy ETL into the newer technologies is highly valued as well.
Above all, we are looking for applicants who will thrive in an open, energetic, flexible, fun-spirited, collaborative environment and desire creative freedom and an opportunity to work on high performing teams!
No remote opportunities exist
Will consider relocation for candidates with the right skills/experience
Must have the ability to work in the US without current/future need for IBM sponsorship
CIODEM19
Auto req ID
184368BR
Required Education
High School Diploma/GED
Role ( Job Role )
Software Developer
State / Province
NEW YORK
Primary job category
Software Development & Support
Company
(0147) International Business Machines Corporation
Contract type
Regular
Employment Type
Full-Time
ERBP
Yes
Is this role a commissionable/sales incentive based position?
No
Travel Required
No Travel
IBM Business Group
F&O
Preferred Education
Associate's Degree/College Diploma
City / Township / Village
NEW YORK
EO Statement
IBM is committed to creating a diverse environment and is proud to be an equal opportunity employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, gender, gender identity or expression, sexual orientation, national origin, genetics, disability, age, or veteran status. IBM is also committed to compliance with all fair employment practices regarding citizenship and immigration status.
Required Technical and Professional Expertise
· 1+ years professional experience with Python and using Big Data technologies such as Apache Spark, Kafka, HDFS, Hive & Hadoop.
· 1+ years professional experience using Parquet formats
· 1+ years professional experience using ETL (extract, transform, and load) tools.
· 1+ years of Cloud base architecture (for example: Bluemix or AWS development)
· 1+ years experience using both relational and No-SQL databases.
· 1+ years of experience in DevOps (for example: Jenkins, Travis CI, UrbanCode Deploy, Nagios)
Skill-keywords
Node, Angular, full stack, java, Agile
CIO - develop applications
applications for price and cost
build and manage - internal business
Country/Region
United States
Preferred Technical and Professional Experience
· 2+ years experience with Docker and Kubernettes
· 2+ years Experience in Test Driven Development (TDD)
· 2+ years experience in Automated Testing
· 2+ years experience with Data Science / Data Analyst and their associated tools.
Secondary Job Category
Other Software Development & Support
Eligibility Requirements
· None
Position Type
Professional
Early Professional Track
Not Applicable - Professional Hire
New Collar Role
Yes