Aws Glue Developer

Summary

Join our team as an AWS Glue Developer and leverage your expertise in data integration, ETL processes, and cloud-based data solutions. You will design, develop, and maintain data pipelines using AWS Glue, ensuring efficient data processing and integration. Collaborate with data engineers, data scientists, and stakeholders to deliver scalable and reliable data solutions. This role requires strong experience with AWS services, programming languages like Python and Scala, and big data technologies. You will be responsible for data processing, cataloging, monitoring, troubleshooting, and documentation. We offer a competitive salary and benefits package, including flexible working hours and remote work options.

Requirements

• Hold a Bachelor’s degree in Computer Science, Information Technology, or a related field
• Possess proven experience as an AWS Glue Developer or in a similar role
• Demonstrate strong experience with AWS services, particularly AWS Glue, S3, Redshift, RDS, Lambda, and IAM
• Have hands-on experience with ETL processes, data integration, and data pipeline development
• Show proficiency in programming languages such as Python, Scala, or Java
• Have experience with big data technologies like Apache Spark, Hadoop, or similar
• Show familiarity with SQL and database management systems (e.g., MySQL, PostgreSQL, etc.)
• Possess a strong understanding of data modeling, data architecture, and data governance
• Demonstrate excellent problem-solving skills and attention to detail
• Have the ability to work independently and as part of a team
• Possess strong communication and collaboration skills

Responsibilities

• Design, develop, and maintain ETL pipelines using AWS Glue to extract, transform, and load data from various sources (e.g., S3, RDS, Redshift, etc.)
• Implement data integration solutions to support business requirements and data analytics needs
• Optimize and tune AWS Glue jobs for performance and cost-efficiency
• Develop and maintain data processing workflows using AWS Glue, PySpark, and other relevant technologies
• Handle large-scale data processing tasks, ensuring data quality and integrity
• Implement data transformation logic using Python, Scala, or SQL
• Manage and maintain the AWS Glue Data Catalog, including metadata management and schema definitions
• Ensure data is properly cataloged and accessible for querying and analysis
• Work closely with data engineers, data scientists, and business analysts to understand data requirements and deliver solutions
• Collaborate with cross-functional teams to integrate data pipelines with other systems and applications
• Monitor AWS Glue jobs and data pipelines for performance, errors, and data quality issues
• Troubleshoot and resolve issues related to data processing and integration
• Document data pipelines, workflows, and processes for future reference and knowledge sharing
• Maintain up-to-date documentation on data architecture, ETL processes, and AWS Glue configurations
• Ensure data security and compliance with company policies and industry regulations
• Implement data encryption, access controls, and other security measures as needed

Preferred Qualifications

• Have experience with CI/CD pipelines and version control systems (e.g., Git, Jenkins, etc.)
• Possess knowledge of data visualization tools (e.g., Tableau, Power BI, etc.)
• Show familiarity with machine learning and data science workflows
• Have experience with containerization technologies (e.g., Docker, Kubernetes)
• Hold AWS Certified Data Analytics – Specialty certification
• Hold AWS Certified Solutions Architect – Associate/Professional certification
• Hold other relevant AWS certifications

Benefits

• Competitive salary and benefits package
• Opportunities for professional growth and development
• A collaborative and innovative work environment
• Flexible working hours and remote work options
• Access to cutting-edge technologies and tools