AWS Data Engineer (DataBricks+DataLake)

H2B Group is a consulting company that helps clients automate business processes with AI and supports IT specialists in career development through B2B model. They are seeking an AWS Data Engineer with Databricks and Data Lake experience for an international corporate client based in Canada/USA.

• Building batch and near real-time data pipelines
• Designing and managing data governance, security, and compliance models
• Migrating database schemas, procedures, functions and validating data
• Implementing CI/CD for data pipelines and infrastructure (IaC with Terraform)
• Performing data quality checks, monitoring, and alerting (SLA/SLO)
• Working cross-team with application teams and architects
• Managing migration risk and communicating status, blockers, and technical decisions

• 4+ years in data engineering
• 2+ years hands-on in Databricks
• 2+ years hands-on in AWS
• Experience in at least one large data or platform migration
• Very good knowledge of Spark and PySpark
• Experience with Delta Lake: ACID, partitioning, Z-Ordering, optimize, vacuum
• Knowledge of Databricks Workflows, Jobs, Unity Catalog
• Strong knowledge of S3 as storage layer
• Experience with IAM, KMS, VPC, PrivateLink, Secrets Manager
• Integration with Lambda, SNS/SQS, Glue
• Designing dev/test/prod environments and account/permission separation
• Very good SQL, ideally also experience with SQL Server and PostgreSQL
• Ability to convert T-SQL logic to Databricks/PG approach
• Experience in schema migration, procedures, functions, data validation
• Reconciliation after migration: quality checks, record compliance, difference reporting
• CI/CD for data pipelines and infrastructure
• Infrastructure as Code, preferably Terraform
• Data testing: unit, integration, data quality checks
• Monitoring and alerting: pipeline SLA/SLO, incident handling, runbooks
• Cross-team collaboration with application teams and architects
• Good migration risk management per client
• Clear communication of status, blockers, and technical decisions
• Product thinking: not just delivering code but a stable migration path
• Data governance: designing permission model at catalog, schema, table, column level
• Experience with data lineage, audit, and access policies
• Understanding of compliance requirements and least privilege principle
• Working with sensitive data: masking, tokenization, access control

• Flexibility – choose projects matching your skills and interests
• Transparency – clear cooperation rules, fully transparent compensation and conditions
• Speed of action – quickly find or change projects that match your competencies
• Development opportunities – work on innovative projects, develop key skills