Data Engineer

Summary

Join Shift5, a rapidly growing cybersecurity and data analytics company, as a Data Engineer. You will be responsible for designing, building, and maintaining data pipelines and products. This role requires strong programming skills, experience with big data technologies, and a deep understanding of data modeling. You will collaborate with various teams to create scalable and reliable data solutions. The ideal candidate will have experience with AWS cloud services and a proven track record of delivering complex data applications. Shift5 offers a competitive salary and benefits package, including a bonus program, equity, comprehensive health insurance, and unlimited paid time off.

Requirements

• 5+ years software/data engineering
• 3+ years experience with a major programming language (GoLang, Java, Python)
• 3+ years experience with of relational database experience (PostgreSQL, MySQL, Oracle, etc.)
• 3+ years experience with Big Data (Hadoop, Spark) and Data Modeling
• 2+ years experience with containerization and cloud services: Docker, Kubernetes, cloud monitoring tools
• Proven experience building complex data applications from scratch including requirements gathering, design, planning, and implementation
• Familiar with modern data stack components including data ingestion, transformation, and orchestration
• Expertise with batch and streaming data pipelines using technologies such as Apache Airflow or Benthos
• Demonstrated delivery of scalable, observable, supportable, reliable data processing applications
• Deep understanding of software engineering practices and concepts
• Experience with end-to-end feature design and ownership
• Comfortable working in a fast-paced environment
• Ability to efficiently multitask and accommodate change of priorities on demand

Responsibilities

• Interpret requirements and design specifications and take ownership of building features from the ground up
• Write clean, well-documented, scalable, extensible, testable code to ensure application quality and maintainability
• Work cross-functionally across Shift5 engineering teams to build applications that integrate cleanly across other Shift5 componentry
• Design, build, and maintain data processing, storage, and integration pipelines
• Build robust and scalable data products for various stakeholders including data scientists, transportation engineers, and executives that drive insights and decision making
• Create solutions that utilize AWS cloud services that are efficient, reliable, cost-effective, dynamically scalable, and observable
• Collaborate with data scientists and engineers to create semantic data models to represent complex vehicular systems
• Perform analysis on complex data sets to create data ontologies, verify data quality/integrity, and ensure accuracy of data after passing through data pipelines
• Operate in a team-based environment with engineers, product managers, program managers and designers to conceive, implement and shape major features

Preferred Qualifications

• Active US Government TS clearance (and willingness to maintain) or an ability to obtain one
• MS in Computer Science, Cybersecurity, Cyber Intelligence, or equivalent
• Experience and understanding of data lakes/warehouses (e.g. Snowflake, Databricks, Redshift)
• Proficiency with CI/CD, source control, design reviews, and integrating observable practices
• Strong verbal and written skills
• Excellent analytical and problem-solving abilities
• Ability to work effectively in a team and with cross-functional groups

Benefits

• Base Salary: $150,000-$200,000
• Bonus program and equity in a fast-growing startup
• Competitive medical, dental, and vision coverage for employees and their families
• Health Savings Account with annual employer contributions
• 401k with employer contributions
• Employer-paid Life and Disability Insurance
• Uncapped paid time off policy
• Flexible work & remote work policy
• Tax-deferred public transit benefits with Metro SmartBenefits (DC/MD/VA)