Principal Data Acquisition Engineer

Principal-level role on a project combining OSINT and modern data analytics for defense-tech, leading a Data Acquisition Team to scale data collection from dozens to hundreds of open-source donors using AI-assisted tooling.

• Create AI-assisted and agentic tools that turn a new source into ready-to-use parsers
• Scale the system from tens to hundreds of data donors
• Develop self-healing mechanisms that detect source changes and failures, adapt, and recover without manual intervention
• Own the entire acquisition architecture: shared scraper/parser frameworks, proxy and stealth strategy, scheduling, and data delivery to ADLS
• Ensure resilience against anti-bot mechanisms, support source migrations, and maintain system reliability in production
• Maintain stability of the existing parser fleet and donors during gradual rearchitecture
• Mentor engineers, define technical standards, and promote an AI-first approach in the team

• Experience at Principal or Staff level in production data collection or web scraping at large scale
• Experience defining technical direction, not just implementing ready-made solutions
• Strong Senior Engineer ready to move to next level also considered
• Experience creating platforms and frameworks used by other engineers
• Deep knowledge of Python
• Knowledge of JavaScript sufficient for reverse engineering websites
• Practical experience with JavaScript-heavy sites, headless browsers, proxies, rate limiting, anti-bot systems
• Practical application of LLM or agentic systems for code generation, error correction, or autonomous workflows
• Experience with Docker, Kubernetes, and cloud environments (Azure preferred)
• Understanding of cloud object storage and streaming systems
• Regular use of Claude Code, Codex or other agentic coding tools
• Systematic approach to testing, quality, and code maintainability
• High level of autonomy, technical leadership, and ability to strengthen the team
• Fluent written and spoken English (main communication language)