Data Scientist

Summary

Join Agile Defense, a company committed to adaptive innovation in national security, and contribute to critical missions. As a Data Engineer, you will design, build, and maintain efficient data pipelines, develop and optimize SQL queries, and implement data quality checks. You will collaborate with cross-functional teams and work with various technologies, including Databricks, AWS, and more. The role requires 3-5 years of experience in an Agile environment and expertise in data engineering. Agile Defense offers a competitive benefits package and a positive work culture.

Requirements

• 3-5 Years of Experience working in an Agile and Scrum software development environment performing the duties of a Data Engineer
• Experience with Databricks notebooks, job schedules, SQL query tools
• Experience managing multi-hope data processing pipelines (185 separate nodes in data pipeline)
• Experience with scripting in  SQL, pyspark, and bash shell
• Experience with postgres database management
• Experience with FlaskAPI
• Experience with AWS Cloudflare, S3, EC2, EKS, ALB
• Experience with Docker containerization
• Experience with Apigee
• Experience with Jenkins, GitHub and separate code environments

Responsibilities

• Strong oral and written communication skills
• Able to communicate effectively with Product Owners (POs) and stakeholders to understand requirements/scope and be able to translate that into user stories
• Able to effectively communicate and coordinate between stakeholders, POs, and team members including designers, business analysts, developers, and Scrum Masters across other development teams
• Work to become a subject matter expert for the products that are supported
• Able to assist with testing/communication/de-bugging with external integration partner system teams
• After team onboarding, work independently to continue to learn the system and products by reviewing resources and documentation, testing, and engaging with design, development, Product Owner and stakeholders in great detail
• Design, build, and maintain efficient ETL/ELT data pipelines
• Develop and optimize SQL queries for large-scale data processing
• Implement data quality checks and monitoring systems
• Create and maintain database schemas and data models
• Collaborate with cross-functional teams to integrate data from multiple sources
• Optimize database performance through indexing, query analysis, and partitioning
• Automate data workflows using CI/CD pipelines

Preferred Qualifications

• Familiar with and experience using tools like Jira, Confluence, Mural, and Teams to communicate requirements, design, processes, and content
• Experience working in an Agile software development environment and familiar with Scrum structure and ceremonies
• Knowledge of cloud platforms (AWS, Azure, etc.)
• Experience with Jenkins or similar CI/CD tools
• Familiarity with data visualization tools
• USCIS or DHS Experience