Hiring: Databricks Engineer
Company: Confidential (Federal Prime Contractor)
Location: United States / Remote
Employment Type: Full-Time / Contract
Seniority Level: Mid-Senior level
Estimated Salary: $120,000 - $160,000 per year (Competitive federal contracting rates)
Industry: Information Technology / Government Consulting
Company Overview
Our client is a leading digital transformation partner for United States federal agencies, specializing in the modernization of large-scale data ecosystems. By leveraging advanced cloud-native architectures, the firm enables government entities to transition from siloed legacy frameworks to integrated, high-performance data intelligence platforms.
With a focus on security, scalability, and actionable insights, the organization supports mission-critical projects that demand the highest standards of data integrity and computational efficiency. As a key player in the federal technology space, the company provides an environment where engineers tackle complex data challenges while contributing to the public interest through cutting-edge technology.
Key Responsibilities
ETL Pipeline Engineering: Design, develop, and maintain robust Extract, Transform, Load (ETL) pipelines using the Databricks Lakehouse Platform to ensure seamless data flow and high reliability.
Legacy System Migration: Lead the technical migration of existing legacy data pipelines—including those currently operating on Azure Synapse Analytics—into optimized, unified Databricks environments.
Performance Tuning: Optimize Spark jobs and SQL queries to improve processing speeds and reduce computational overhead within the cloud environment.
Data Architecture Support: Collaborate with data architects to implement structured, semi-structured, and unstructured data storage solutions that adhere to modern Lakehouse standards.
Security & Compliance: Ensure all data engineering activities and infrastructure configurations meet strict federal security requirements and data governance protocols.
Experience & Qualifications
Databricks Expertise: Strong professional experience building and operationalizing data pipelines specifically within the Databricks environment.
Data Engineering Background: Proven history of managing large datasets and complex data transformations in a cloud-native or hybrid-cloud setting.
Migration Experience: Hands-on experience transitioning enterprise-level data workloads from legacy platforms (e.g., Azure Synapse, Hadoop, or on-premise SQL) to Spark-based architectures.
Education: Bachelor’s or Master’s degree in Computer Science, Data Engineering, Information Technology, or a related quantitative field.
Technical Skills Required
- Advanced proficiency in Apache Spark (PySpark or Scala) for large-scale data processing.
- Expert-level knowledge of SQL for complex data manipulation and analysis.
- Solid understanding of the Medallion Architecture (Bronze, Silver, Gold layers) within Delta Lake.
- Familiarity with Azure or AWS cloud infrastructure and integrated security services.
- Experience with CI/CD pipelines and version control (e.g., Git, Azure DevOps) for data engineering workflows.
Clearance Requirement
Public Trust Clearance: This position requires the candidate to be eligible for and maintain a Federal Public Trust clearance. Applicants must typically be U.S. citizens or permanent residents to meet federal background investigation standards.
Apply Now
If you are a high-performing Data Engineer with specialized Databricks skills and a commitment to federal service excellence, we encourage you to join our specialized engineering team. Submit your resume today to begin the technical evaluation and clearance verification process.