Level:
Experienced Professional
We are looking for experienced Software Engineers to be part of our Data Engineering team. You will be focused on designing and implementing solutions on Apache Spark, Kafka, ELK, Talend, and Tableau. As a Data Engineer, you will be exposed to AWS, Azure, GCP, and Cloudera solutions and thus, the ideal candidate should have a strong foundation in big data technology and a passion to learn new technologies.
Main duties and responsibilities:
- Design and develop data pipelines utilizing PySpark and Hadoop Frameworks.
- Read, extract, transform, stage, and load data to multiple targets including Databases, Data Lakes, and Data Warehouses using any of the ff: Talend, Informatica, DataStage, Pentaho, etc.
- Migrate existing data processing from standalone or legacy technology scripts to PySpark framework processing.
Qualifications:
- Bachelor’s degree in Computer Science/IT/Computing or equivalent
- Preferably with 2-3 years experience in Data Engineering
- At least 1 yr of experience in ETL project implementation using PySpark
- At least 1 yr of experience in Linux environment
- At least 3 years of experience in Python Development
- At least 3 years of experience in Hadoop and Big Data Ecosystem