Tektend LLC is seeking an experienced Databricks Spark Engineer to join our high-impact data engineering practice in Dallas, TX on a hybrid schedule. In this role, you will design, build, and optimize enterprise-scale distributed data pipelines using Databricks, Apache Spark, and Delta Lake. You will collaborate closely with analytics teams, data scientists, and solution architects to deliver reliable, high-performance data infrastructure supporting mission-critical analytics and machine learning initiatives.
Key Responsibilities:
- Architect, develop, and maintain scalable, robust ETL/ELT pipelines using PySpark and Databricks.
- Design, implement, and optimize Delta Lake architectures for high-throughput batch and streaming workloads.
- Analyze, tune, and optimize Spark job performance, resource utilization, and query execution plans.
- Collaborate with cross-functional data teams to translate complex business requirements into resilient technical architectures.
- Implement automated data quality checks, observability, and robust CI/CD deployment pipelines.
Requirements:
- 5+ years of hands-on data engineering experience with strong proficiency in Python and SQL.
- 3+ years of dedicated experience developing large-scale distributed applications using Apache Spark and Databricks.
- Demonstrated expertise working with Delta Lake, lakehouse paradigms, and structured streaming.
- Strong background in cloud platforms (AWS, Azure, or GCP) and infrastructure orchestration.
- Bachelor's degree in Computer Science, Information Systems, or equivalent practical experience.
