Data Engineer at Addison Group (Washington, DC)

Top Skills:

  • PySpark Experience
  • Someone who has some kind of software engineering experience.
  • Someone who has written applications that have been deployed to production and checked into a Git version control system.
  • Worked with a Data Scientist or Machine learning engineer team before

What you will do:

  • Work weekly with motivated engineers and stakeholders to enable their teams with asynchronous messaging on a Kakfa pipeline that you help build and maintain.
  • Create and measure an anomaly detection system for business metrics to drastically improve the company’s mean time to discovery for important events.
  • Own the construction of frameworks to help our machine learning team deploy models and jobs into production.
  • Give monthly lightning talks to the wider engineering team and company about the cultural changes you are trying to drive using data.
  • Collaborate frequently with analysts, data scientists, and engineers to solve their problems and build new features.
  • Help the company and the engineering team adapt to a data-streaming mentality.
  • Work with analytics and machine learning teams to troubleshoot notebooks and jobs running Apache Spark.
  • Attend and run book clubs, online course discussions, and other continuous learning events on the data engineering team and beyond

Required:

  • 4+ years overall software engineering experience.
  • 2+ years experience working as a Data Engineer or equivalent role.
  • Strong experience working with structured and unstructured data.
  • Experience working with data lake technologies like Hive, Parquet, and S3.
  • Experience with data warehouse technologies like Redshift.
  • Experience with streaming technologies like Apache Kafka and Spark.
  • Experience maintaining or designing ETL processes in Airflow, AWS DMS, etc.
  • Experience partnering with stakeholders to understand their requirements and provide data which can help them observe patterns and spot anomalies.
  • Strong written and verbal communication skills.

Preferred:

  • You are very experienced in Python and Scala/Java.
  • You have designed or architected production systems from the ground up.
  • You have networking/DevOps experience.
  • You have some experience building web applications in PHP or NodeJS.

Please let the company know you found this position via aijobsdb.com so we can keep providing you with quality jobs.