Databrics Developer (42279)

About the Position

We are looking for a Databricks Developer to design, develop, and optimize data pipelines for large-scale data. The role will involve optimizing Spark jobs, managing Delta Lake, and dealing with terabyte-scale datasets. You will also manage governance with Unity Catalog and optimize clusters for maximum efficiency.

Responsibilities6

• Designing, developing, and optimizing data pipelines in Databricks for large-scale data
• Optimizing the performance of Spark jobs
• Managing data in Delta Lake including handling tables of terabyte size
• Managing access rights and data governance using Unity Catalog
• Optimizing cluster settings for efficient resource utilization and scalability
• Collaborating with the data team to design a robust and reliable Lakehouse architecture

Requirements4

• At least 1 year of experience with Unity Catalog including management, governance, and access rights
• At least 2 years of experience with Delta Lake including debugging and optimizing large data tables
• At least 2 years of experience optimizing Spark pipelines on large-scale data projects
• Experience tuning cluster configuration in Databricks including autoscaling and resource utilization strategies