Data Engineer at Addison Group (Washington, DC)

Top Skills:

• PySpark Experience
• Someone who has some kind of software engineering experience.
• Someone who has written applications that have been deployed to production and checked into a Git version control system.
• Worked with a Data Scientist or Machine learning engineer team before

What you will do:

• Work weekly with motivated engineers and stakeholders to enable their teams with asynchronous messaging on a Kakfa pipeline that you help build and maintain.
• Create and measure an anomaly detection system for business metrics to drastically improve the company’s mean time to discovery for important events.
• Own the construction of frameworks to help our machine learning team deploy models and jobs into production.
• Give monthly lightning talks to the wider engineering team and company about the cultural changes you are trying to drive using data.
• Collaborate frequently with analysts, data scientists, and engineers to solve their problems and build new features.
• Help the company and the engineering team adapt to a data-streaming mentality.
• Work with analytics and machine learning teams to troubleshoot notebooks and jobs running Apache Spark.
• Attend and run book clubs, online course discussions, and other continuous learning events on the data engineering team and beyond

Required:

• 4+ years overall software engineering experience.
• 2+ years experience working as a Data Engineer or equivalent role.
• Strong experience working with structured and unstructured data.
• Experience working with data lake technologies like Hive, Parquet, and S3.
• Experience with data warehouse technologies like Redshift.
• Experience with streaming technologies like Apache Kafka and Spark.
• Experience maintaining or designing ETL processes in Airflow, AWS DMS, etc.
• Experience partnering with stakeholders to understand their requirements and provide data which can help them observe patterns and spot anomalies.
• Strong written and verbal communication skills.

Preferred:

• You are very experienced in Python and Scala/Java.
• You have designed or architected production systems from the ground up.
• You have networking/DevOps experience.
• You have some experience building web applications in PHP or NodeJS.