Senior Software Engineer

Summary

Join Competera, a company building a platform for optimal pricing decisions using AI, as a Senior Software Engineer (Scraping). You will own Data Delivery engineering growth and direction, designing and implementing scalable web scraping services using Python and related libraries. Lead the development team in overcoming anti-scraping measures, mentor junior engineers, and automate scraper development processes. You will monitor system performance, design microservices architecture using DDD principles, and stay updated on web scraping advancements. This role requires 5+ years of software engineering experience with a strong web scraping background.

Requirements

• 5+ years of experience as a Software Engineer with a strong track record in web scraping
• Proven expertise in Python and relevant libraries like Scrapy, Selenium, Pyppeteer, Playwright
• In-depth understanding of web protocols and how modern web works (SSL, proxies, cookies, headers, browsers)
• Experience with anti-scraping detection and bypass techniques
• Experience with Kafka, Redis, and microservices architecture
• Familiarity with Domain-Driven Design (DDD) principles
• Eng - interm +

Responsibilities

• Design and implement scalable and efficient services for a web scraping system using Python and relevant libraries
• Lead the development team in overcoming anti-scraping measures such as CAPTCHAs, rate limiting, and dynamic content loading
• Mentor and guide junior engineers in the team, fostering their growth and development
• Automate existing scraper development processes using tools and frameworks to improve efficiency and maintainability
• Monitor system performance and proactively identify and address any issues
• Design and implement microservices architecture for modularity and scalability
• Apply Domain-Driven Design (DDD) principles to ensure clean and maintainable code
• Stay up-to-date with the latest advancements and best practices in web scraping

Preferred Qualifications

• Experience with cloud platforms like AWS or GCP
• Experience with data quality and cleaning techniques
• Knowledge of machine learning and data analysis techniques
• Ability to lead and mentor junior engineers in overcoming technical challenges
• Proactive approach to identifying and solving problems, seeking out new opportunities
• Ability to work independently and as part of a team in a fast-paced environment
• Ability to learn new technologies and adapt to changing web scraping landscapes

Benefits

• Remote-first ideology: freedom to operate from the home office or any suitable coworking
• Flexible working hours (we start from 8 to 11 am) and no time tracking systems on
• Regular performance and compensation reviews
• Recurrent 1-1s and measurable OKRs
• In-depth onboarding with a clear success track
• Competera covers 70% of your training/course fee
• 20 vacation days, 15 days off, and up to one week of paid Christmas holidays
• 20 business days of sick leave
• Partial medical insurance coverage
• We reimburse the cost of coworking