Preferred: Experience with cloud environments (AWS, Azure, and/or Google Cloud) and common platform services (storage, compute...-paced, ambiguous delivery environments Data engineering experience with Spark, Airflow/dbt, streaming, data modeling or ML...
degree required. Proficiency in data engineering tools and languages, such as Python, SQL, and Spark... with cloud-based data solutions (e.g., AWS, Azure, Google Cloud). Familiarity with data warehousing, SQL, NoSQL databases...
on the Databricks Lakehouse Platform. Develop and optimize scalable data pipelines using Apache Spark and Databricks... engineering, or distributed systems. 3+ years of hands-on experience with Databricks. Strong expertise in Apache Spark (PySpark...
Familiarity with tools such as Kafka, Spark, Airflow, and cloud platforms (AWS or GCP) Experience working with messy, high-volume...
Familiarity with tools such as Kafka, Spark, Airflow, and cloud platforms (AWS or GCP) Experience working with messy, high-volume...
experience with Apache Spark Container experience with Docker, Podman, or containerd Git source control experience Desired... Skills Atlassian tools experience, Jira and Confluence AWS cloud services and infrastructure familiarity Compensation...
/upserts, schema evolution). Diagnose and resolve performance challenges in distributed Spark workloads (data skew, shuffle...-on experience with Databricks, Spark/PySpark, and distributed data processing at scale Deep understanding of Delta Lake and modern...
: Experience with Apache Spark, Kafka, Airflow, dbt, and Vector Databases (Pinecone, Milvus). Cloud Platforms: Hands on experience... like Apache Spark, Airflow, and Kafka to support AI and ML workloads. AI Ready Data Preparation: Transform messy, unstructured...
on defining data architecture and selecting appropriate cloud technologies to support data engineering solutions. ICA... language central to data engineering (e.g., Python, Powershell, R, Spark). Experience with ETL/ELT processes, optimizing...
, and hybrid cloud infrastructure. These solutions are powered by engineering for business advantage, transforming mission-critical... immigration sponsorship may be available Preferred qualifications Experience with cloud environments (AWS, Azure...