Staff Data Engineer

Summary

Join Oportun's Enterprise Data and Technology (EDT) team as a Senior Data Engineer. You will be a key member responsible for designing, developing, and maintaining sophisticated software and data platforms. Leverage your technical expertise to solve business problems and contribute to architectural decisions. Mentor junior engineers and collaborate with cross-functional teams to deliver high-quality, scalable solutions. Lead large initiatives, from requirements gathering to final product delivery. This role offers the opportunity to make a significant impact on Oportun's mission of providing affordable credit.

Requirements

• Bachelor's or Master's degree in Computer Science, Data Science, or a related field
• 10+ years of experience in data engineering, with a focus on data architecture, ETL, and database management
• Ability to identify opportunities for optimization and efficiency gains within the data pipelines/architecture/infrastructure
• Work like a technical thought leader of the team and provide necessary guidance and mentorship to senior team members
• Ability to take ownership of critical projects and initiatives, providing project leadership, and ensuring successful delivery through effective project management and communication
• Involve in rigorous code and data quality reviews, offering valuable feedback to maintain best practices, quality, performance, and maintainability
• Lead the team in designing and building complex end-to-end data pipelines
• Proficiency in programming languages like Python/PySpark and Java/Scala
• Expertise in big data technologies such as Hadoop, Spark, Kafka, etc
• In-depth knowledge of SQL and experience with various database technologies (e.g., PostgreSQL, MySQL, NoSQL databases)
• Guide the team in orchestrating and designing job schedules using the CICD tools like Jenkins and Airflow
• Ability to work in an Agile environment (Scrum, Lean, Kanban, etc)
• Familiarity with cloud platforms (e.g., AWS, Azure, GCP) and their data services (e.g., AWS Redshift, S3, Azure SQL Data Warehouse)
• Strong leadership, problem-solving, and decision-making skills
• Excellent communication and collaboration abilities

Responsibilities

• Provide technical leadership and mentorship to the data engineering team, promoting best practices, code quality, and professional growth
• Act as a subject matter expert in data engineering, staying updated with industry trends and advancements, and sharing knowledge with the team
• Lead the design and implementation of complex data architectures, ensuring scalability, reliability, and performance to meet business objectives and analytical needs
• Collaborate with stakeholders while demonstrating strong business context to define data architecture standards and guidelines
• Design and develop efficient and scalable data pipelines, ensuring data ingestion, processing, and transformation align with business requirements
• Optimize data pipelines for performance, resilience, and cost-effectiveness
• Oversee the design and management of databases, data warehouses, and data lakes, optimizing for performance, security, and scalability
• Implement and optimize ETL processes to maintain data integrity and efficiency in data storage solutions
• Lead and manage complex data engineering projects, ensuring successful project delivery within specified timelines and budget
• Collaborate with cross-functional teams to align project goals and priorities, providing technical expertise to meet project requirements
• Establish and enforce data quality standards, governance policies, and compliance with relevant regulations, ensuring data accuracy, consistency, and security
• Implement data quality monitoring and improvement processes
• Collaborate with various stakeholders to understand their data needs and provide data solutions that drive business value
• Communicate effectively with both technical and non-technical stakeholders, translating technical concepts into actionable insights
• Lead the end-to-end delivery of complex software applications, ensuring adherence to best practices and high overall quality standards
• Create frameworks and abstractions that lessen the workload of other engineers. Your documentation and tutorials enable other teams to leverage your work independently. You identify and resolve improvements in test coverage for end-to-end business flows across multiple systems (and teams). You introduce new tools, techniques, and coding patterns to other engineers, some of which may be developed by yourself, and drive this change to other teams
• Drive technical decision-making, contribute to, and sometimes lead, architectural discussions, and propose innovative solutions to address complex technical and/or systemic architectural challenges. You collaborate across teams and build systems to address architectural gaps
• Take ownership of critical projects and initiatives, providing project leadership, and ensuring successful delivery through effective project management and communication. You engage with stakeholders across the group, understanding their needs and working through the complexity and conflicting goals
• Have the ability to provide a clear and concise explanation of business strategy, technical concepts, designs or implementation to a non-technical audience. You collaborate closely with product managers, designers, and fellow engineers to gather and translate business requirements into efficient and scalable software solutions. You have a track record of leading cross-team efforts that deliver meaningful results across engineering, product, and/or design. You work through technical, organizational and scheduling complexity to align teams and deliver results in a positive way for the people and teams involved. Your interactions build trust and speed at scale into our organization
• Be the technical thought leader of the team and provide necessary guidance and mentorship to senior team members whenever required
• Play the key technical counterpart to an engineering manager with a larger team; you can provide effective feedback to engineering managers at all levels to identify and help address gaps in their teams
• Successfully advocate for technologies and processes that make multiple teams more efficient
• Conduct rigorous code reviews, offering valuable feedback to maintain code quality, performance, and maintainability
• Continuously identify opportunities for process improvements and efficiency gains within the software development lifecycle
• Stay informed about emerging technologies, industry trends, and best practices, and proactively introduce relevant advancements to enhance our products and services