As a data engineer you will be working on our massive (PBs) data pipelines, making sure the data is clean, whole, and accessible. Your team's goal is to create amazing ground breaking tools to make the data scientists more productive and agile.
If you love working on complicated network pipelines, you understand the importance of reliable data and have felt the pain of big data inconsistencies,and you're the type who thinks of great solutions and want to bring them to life, come work with us.
" B.Sc / M.Sc in Computer Science / Software Engineering / Electrical engineering or similar
" +5 years of proven experience as a Data/Backend Engineer
" +3 years of experience skills with Python
" Experience with designing and building ETL processes and data pipelines
" Experience with Big Data technologies and solutions (For example: Spark, Presto, Hive, Splunk, Hadoop, BigQuery, MapReduce, flink)
" Experience with Cloud Environment, advantage to AWS (S3, Athena, Ecs, Lambda, Emr, Glue, etc)
" Experience working with a high scale of data (we process 50 TB a day)
" Good Analytical skills in SQL.
" Familiarity with Airflow, Pandas library, NodeJs, PySpark
" Familiarity with data science and machine learning tools
" Knowledge of Linux operating systems
" Good knowledge in containerizing