Data Engineer (Databrick + Pyspark)
Capco is seeking a senior data engineer to design, build, and maintain scalable ETL/ELT pipelines in a hybrid role based in Pune. The candidate will work extensively with PySpark, Databricks, and Apache Airflow to process large-scale datasets within cloud-based environments. The role requires deep expertise in Spark internals, complex SQL, and managing data workflows on platforms like EMR Serverless. This position focuses on performance optimization and ensuring the reliability of enterprise-grade data solutions.
50k new jobs listed every day. Install TAL to find more jobs like this.
Experience
5-9 years
Function
Engineering
Work mode
Hybrid, India
Company
Tier 2
What you will work on
Capco is seeking a senior data engineer to design, build, and maintain scalable ETL/ELT pipelines in a hybrid role based in Pune. The candidate will work extensively with PySpark, Databricks, and Apache Airflow to process large-scale datasets within cloud-based environments. The role requires deep expertise in Spark internals, complex SQL, and managing data workflows on platforms like EMR Serverless. This position focuses on performance optimization and ensuring the reliability of enterprise-grade data solutions.
TAL's take
Solid tier-2 global consultancy with a clearly defined technical scope and specific skill requirements.
Crisp JD with clearly defined responsibilities, required skills, and expected tech stack.
Salaries at Capco
20.4 LPA average
Based on 7 Grapevine salary entries for Capco.
Engineering
12 - 14 years
27 LPA average
Range: 27 - 27 LPA
Design
4 - 6 years | L1
12 LPA average
Range: 12 - 12 LPA
Other roles
4 - 6 years | L1
13 LPA average
Range: 13 - 13 LPA
Sales
8 - 10 years | M3
26 LPA average
Range: 26 - 26 LPA
Must haves
- Strong hands-on experience with PySpark and Apache Spark internals
- Experience with Databricks
- Proficiency in Apache Airflow
- Solid experience building ETL/ELT pipelines at scale
- Strong SQL skills and Data Warehouse experience
- Experience running Spark workloads on EMR Serverless
- Hands-on experience with cloud data lakes like S3
Tools and skills
Nice to have: delta lake, apache iceberg, spark structured streaming, kafka, ci/cd.
About the company
Capco is a large, established global consultancy and technology services firm.
Posts mentioning Capco
Need advice to select the right company
I want to know about the company @Capco. Current domain: Data Engg YOE: 4.7 CCTC: 16.4 Offer: 23(fixed)+1 JB Post: Sr Consultant 2 grade M2. I want to know about the company culture,WLB,job security,bench policy.Also,I am a bit confused.The role offered is for a consultant,but the questions were more oriented toward arch. There were no questions on PySpark.The manager said the work is on ETL migration for Barclays using AWS Glue and Spark,but no related questions.Should I take the offer or continue rounds with EPAM,UST,EY?
Meta Launches Edits, A Powerful Video Editing App
- Meta introduces 'Edits,' a new video editing app, following CapCut's removal due to the TikTok ban. - The app will provide creators with advanced tools, inspiration tabs, and performance insights on Instagram. - Edits will be available on iOS next month and Android later, aiming to fill the gap left by CapCut. Source: [TechCrunch](https://techcrunch.com/2025/01/19/meta-announces-a-new-video-editing-app-called-edits-amidst-tiktok-and-capcut-ban/)
Wipro to lay-off middle level (on-site)
They want to improve margins Source - Economic times