Britain's Premier Job Portal
Description
& Summary:We are seekinghands-on Databricks Data Engineer with 5–7 years of experience who can design, build, andoptimizescalable data pipelines on theDatabricks Lakehouse . The ideal candidate is strong inSQL, Python, andPySpark , understandsDelta Lake inside out, and is experienced withproduction-grade ETL/ELT ,orchestration ,cost/performance optimization , anddata governance (e.g., Unity Catalog).
Candidate should have hands-on knowledge of AzureADF andADLS .
Responsibilities:
You will work on:
Build reliable, scalablebatch and streaming pipelines usingPySpark andDatabricks Workflows/Jobs .
ImplementDelta Lake best practices: schema enforcement/evolution, ACID transactions, time travel, OPTIMIZE/ZORDER, and VACUUM.
DevelopDelta Live Tables (DLT) orStructure...