Technical Lead – Databricks
Technical Lead - Databricks
About Us:
Kenexai delivers smart, data-driven solutions that empower businesses across industries. Our mission is to combine deep, domain-specific expertise with cutting-edge technology to drive meaningful impact.
With a trusted team, consistent quality, and a growing global presence, we remain committed to delivering excellence while staying true to our core values: innovation, integrity, and client success.
Be part of a team that’s not just building solutions—but shaping the future with intelligence.
Experience: 8 - 12 Years
Department: Data Engineer/Analytics
Job Type: Full Time
Job Location: Ahmedabad/Indore
Roles and Responsibility:
- Lead the design and implementation of end-to-end data platforms, lakehouse architectures and enterprise data engineering solutions using Databricks.
- Architect batch and streaming ingestion, transformation and serving pipelines using Apache Spark, PySpark, Spark SQL and Delta Lake.
- Design and implement medallion architecture patterns (Bronze, Silver and Gold) and data models aligned with business and analytical needs.
- Establish best practices for Delta tables, schema evolution, partitioning, optimisation, file sizing, caching and workload performance.
- Work with Unity Catalog and related capabilities to implement data governance, access controls, lineage and secure data sharing.
- Design and optimise workflows using Databricks Workflows, Jobs, Delta Live Tables / Lakeflow capabilities and orchestration integrations as appropriate.
- Lead migration, modernisation and integration of data workloads across on-premises and cloud environments.
- Drive data quality, observability, testing, CI/CD, infrastructure automation and operational support practices.
- Review code and architecture, resolve complex Spark and pipeline issues, and guide performance and cost optimisation.
- Mentor data engineers, plan technical work, estimate effort and ensure engineering quality and delivery commitments.
- Collaborate with AI/ML, BI, cloud and application teams to provide trusted, well-governed data products for analytics and AI use cases.
- Support solution proposals, technical workshops, client discussions and platform capability development.
Skill Requirement:
- Strong experience in data engineering, including design and delivery of large-scale data pipelines.
- Hands-on expertise in Databricks and Apache Spark, with strong PySpark and Spark SQL skills.
- Strong Python and SQL programming skills.
- Practical experience with Delta Lake and lakehouse architecture, including medallion patterns.
- Experience with data modelling, ETL/ELT, batch and incremental processing, and data quality frameworks.
- Understanding of performance tuning, Spark execution, partitioning, joins, shuffles and optimisation techniques.
- Experience with at least one cloud ecosystem (Azure, AWS or GCP) and its data services.
- Experience leading technical delivery, mentoring engineers and reviewing code and architecture.
- Strong communication skills and the ability to translate business and data requirements into technical solutions.
Good to Have:
- Experience with Unity Catalog, Delta Sharing, Databricks SQL and governance/lineage capabilities.
- Exposure to Delta Live Tables / Lakeflow Declarative Pipelines, Workflows, Auto Loader and Structured Streaming.
- Experience with Databricks Asset Bundles, Terraform, CI/CD and Git-based deployment workflows.
- Knowledge of MLflow, Feature Store, Mosaic AI or integration patterns supporting AI/ML workloads.
- Experience with Snowflake, dbt, Kafka, Airflow, cloud object storage or other modern data stack components.
- Relevant Databricks certifications such as Data Engineer Professional or Data Engineer Associate.
Why Kenexai?
- Be part of building something from the ground up in a high-growth, high-impact domain
- Work alongside passionate experts in AI, data, and industry consulting
- Competitive base + uncapped commission structure tied directly to performance
- Remote-first flexibility with real ownership and career growth potential
Walk-In Drive in Indore on 5th-6th December
Perks and Benefits:
- Employees are entitled to flexible working hours to support work-life balance.
- The company operates on a 5-day work week.
- A healthy, inclusive, and collaborative work environment is maintained.
- The company organizes Fun Fridays and festive celebrations to foster team spirit.
- Employees have access to opportunities for continuous learning and career growth.
- An annual company trip is organized for team building and relaxation.
- Comprehensive medical insurance benefits are provided to employees.
- Performance-based bonuses and annual salary revisions are offered.
- A hybrid working model is available, allowing a mix of in-office and remote work as per company policy.
In Our Customers’ Words
Real pleasure consulting with Kenexai to set up our company’s entire data warehouse and dashboards on AWS. I will definitely be reaching out to them for future work to be done. Our project was effective and 100% achieved what I planned to do in the beginning, in a shorter time frame and with less effort than I expected.
Hans
United States
CCR Data perform complex data migrations, we needed and extra pair of hands to restore an Oracle database and transfer the data to a Microsoft SQL database ready for our migration analysts to do their stuff. We would not hesitate in recommending or using Kenexai again and would be happy to outsource bigger projects to them in the future.
Henry Sykes
Director - CCR Data
I have used RA on numerous occasions over the past 2 years, specifically with Nitesh Solanki for the delivery on PDI ETL jobs. I am very happy with him and the high level of quality work he has provided. He seems to be available all the time and works extremely hard to deliver high quality solutions.
Mark Scriven
Technical Director - Value Ad