Senior Databricks Data Engineer (AWS)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Senior Databricks Data Engineer (AWS) based in the United States.

This role offers the opportunity to contribute to a large-scale federal data modernization program focused on building secure, scalable, and high-performance cloud data platforms. You will work at the heart of enterprise data engineering, designing and optimizing data pipelines that power analytics, reporting, and machine learning use cases. The environment is highly collaborative, bringing together cloud engineers, architects, and cybersecurity specialists to deliver trusted data solutions. You will be responsible for leveraging modern Databricks Lakehouse capabilities on AWS to enable governed, production-grade data products. The position emphasizes performance, reliability, and data governance at scale within a regulated federal context. This is a remote role within the United States supporting mission-critical data infrastructure.

• Design, develop, and maintain scalable ETL/ELT pipelines and data workflows using Databricks and Amazon Web Services (AWS), ensuring reliability and performance at enterprise scale.
• Implement and optimize Lakehouse components such as Delta Lake, Unity Catalog, Auto Loader, Delta Live Tables, Databricks SQL, and Delta Sharing to enable governed data access and analytics.
• Build and maintain data models and governed data products supporting analytics, reporting, and machine learning workloads across the organization.
• Develop medallion architecture pipelines and enforce enterprise data quality, lineage, metadata management, and governance standards.
• Optimize data processing performance through partitioning strategies, query tuning, caching, and workload optimization techniques.
• Collaborate with cross-functional teams including cloud engineers, security specialists, and business stakeholders to deliver secure, production-ready data solutions.
• Support architecture reviews, platform modernization initiatives, and continuous improvement efforts across the enterprise data ecosystem.

Requirements:

• Bachelor’s degree in Computer Science, Information Systems, Engineering, Mathematics, or a related field.
• Minimum 8 years of experience in cloud data engineering, data architecture, or platform development roles.
• At least 3 years of hands-on experience working with Databricks in production environments.
• Strong expertise in building and maintaining large-scale ETL/ELT pipelines and cloud-native data architectures.
• Deep experience with AWS services such as S3, Glue, Athena, Lambda, Redshift, IAM, and CloudWatch.
• Strong programming skills in SQL and Python for data engineering and transformation tasks.
• Proven experience implementing data governance, security controls, and high-quality data management practices.
• Experience working with data lakes, data warehouses, and modern analytics platforms.
• Familiarity with performance tuning, troubleshooting, and optimization in distributed data environments.
• Preferred certifications: Databricks Data Engineer (Associate or Professional), AWS Data Analytics Specialty.
• Exposure to regulated or federal environments, including familiarity with compliance frameworks such as FedRAMP, FISMA, or NIST, is a plus.

Benefits:

• Competitive compensation aligned with senior-level data engineering roles in regulated environments.
• Fully remote position within the United States.
• Opportunity to contribute to a high-impact federal data modernization initiative.
• Work with cutting-edge Lakehouse and cloud-native data technologies.
• Strong focus on professional growth, certifications, and continuous learning.
• Exposure to large-scale enterprise data platforms and advanced analytics use cases.