Delta Lake, Unity Catalog, Workflows, and Structured Streaming. The ideal candidate is hands-on with Spark (PySpark/Scala... PySpark/Scala code with modular, reusable libraries. Optimize Spark jobs (partitioning, Z-Order, file sizes, caching, AQE...
organization. The administrator will manage workspace configuration, cluster policies, user access, integrations with cloud... Required Experience administering Databricks workspaces in a cloud environment - AWS 8 Required Strong understanding of Databricks...
Title :: Data Ops Engineer (Azure Data Factory) Bellevue, WA Contract Qualifications 3 6+ years in data/cloud.../CAB practices;clear incident comms. Nice-to-Have (Engineer) Databricks or Spark;data quality frameworks;cost/FinOps...
Gen2, Notebooks, Spark) and legacy SSIS. Develop ingestion frameworks for flat files (CSV/Excel), APIs, SaaS platforms..., cloud feeds, and partner data. Implement medallion architecture (Bronze, Silver, Gold) using Lakehouse (Delta Lake...
). Experience with SQL, data visualization tools (Power BI, Tableau). Knowledge of cloud platforms (AWS, Azure, GCP) for ML... deployment. Familiarity with NLP, deep learning, and big data frameworks (Spark, Hadoop) is a plus. Desired Competencies...
such as Apache Hive, Apache Airflow, , Spark SQL , or AWS S3 and EC2 Knowledge of data visualization tools such as Tableau, Power BI..., or D3.js Experience with cloud-based data platforms Certification in data engineering or a related field...
and develop effective AI-driven solutions. Deploy AI solutions on cloud platforms such as AWS. Collaborate with technical...). Knowledge of big data technologies (Spark, Hadoop, Databricks). Background in NLP, computer vision, or other advanced...
is a plus Experience in Cloud native development in Azure, AWS or GCP is preferred. Experience in streaming technologies like Apache Kafka..., Spark is preferred. Experience with RDBMS like PostgreSQL, and/or MS SQL Experience in NoSQL databases like Cassandra...
, primarily on cloud platforms. 7+ years of hands-on experience in: Data acquisition framework design Spark-based ingestion... in Azure cloud architecture, Databricks, modern data ingestion frameworks, Building Canonical Data Models, and Medallion...
Technologies Knowledge of ETLELT SQL and big data technologies Kafka Spark Hadoop Infrastructure Experience with Linux cloud...