Summary
Join Acquia as a Senior Data Engineer to design, build, and maintain data pipelines that power various analytics and AI-driven products across the company.
Requirements
- 5+ years of experience in data engineering or data platform development
- Experience with data engineering tools and frameworks (e.g., Meltano, dbt, Dagster)
- Strong hands-on experience with data lakes (e.g., S3), and data warehouse tool - Snowflake
- Strong hands-on experience with cloud platforms (AWS, GCP, or Azure)
- Proficiency in SQL, Python, and ETL/ELT design patterns
- Understanding of real-time analytics and vector databases
- Strong knowledge of data governance practices and cataloging tools like DataHub
- Experience working with cross-functional teams and translating business needs into technical requirements
- Proven expertise in building enterprise-level data platforms
- Experience with real-time data processing and analytics tools
Responsibilities
- Lead the design, development, and optimization of data pipelines that enable the collection, transformation, and storage of data from multiple sources
- Build and manage data ingestion processes using tools like Meltano and orchestrate workflows with Dagster, ensuring scalability and real-time data processing
- Implement data storage solutions leveraging Snowflake for data warehousing and S3 for data lakes, optimizing for cost, performance, and security
- Work with the data transformation layer using dbt and ensure seamless integration with orchestration layers to maintain real-time and batch processing efficiency
- Establish data governance frameworks and data catalogs (e.g., DataHub) to ensure data quality, discoverability, and security across the platform
- Collaborate with stakeholders across the organization, including data scientists, product managers, and business analysts, to translate business needs into technical solutions
- Implement real-time analytics solutions by building and maintaining a vector database and real-time analytics database to support machine learning models and predictive analytics
- Design and implement a semantic layer (e.g., cube.js) to facilitate user-facing analytics while integrating AI services and vector databases for advanced analytics
Preferred Qualifications
- Excellent communication skills with the ability to explain technical concepts to non-technical stakeholders
- Familiarity with dashboarding and reporting tools like Domo
- Experience with AI/ML-driven platforms and integrating AI services for advanced analytics
Benefits
- Competitive healthcare coverage
- Wellness programs
- Take it when you need it time off
- Parental leave
- Recognition programs