end-to-end data pipelines using Databricks (Spark, Delta Lake, notebooks, jobs, and workflows). - Develop and optimize... skills (Python, SQL, and Spark). - Experience building scalable and reliable data pipelines. - Advanced English level...
to have - Experience with Hadoop ecosystem (Hive, Impala, Hue, Spark, etc.)...
. - Strong skills in Python and SQL. - Experience building ETL/ELT pipelines with tools like Airflow, dbt, or Spark. - Hands...
and storage - Experience with Apache Airflow, contributing to streamlined workflow orchestration - Familiarity with Apache Spark... collaboration within an international team Nice to have - Familiarity with DataBricks and Spark, further enhancing your ability...
- Understanding of big data technologies like Hadoop or Spark - Certifications in data-related fields or technologies...
- Understanding of API design and integration principles - Familiarity with big data frameworks such as Hadoop or Spark...
We are seeking a Senior Data DevOps Engineer to join our team supporting the migration of Spark, Flink... scalable data workflows - Optimize Spark and Flink workflows and orchestrate jobs for efficient pipeline execution...
technologies such as Hadoop, Spark, or Kafka - Familiarity with AI/ML technologies and predictive analytics solutions - Knowledge...
through the use of tools like Apache Airflow and Apache Spark - Stay updated on industry advancements and integrate... with Apache Airflow and Apache Spark for optimizing workflows and data processing - Proficiency with CI/CD tools like Jenkins...
and have a strong background in AWS, Apache Spark, and Python, we invite you to join our team. Responsibilities - Design and implement data... Web Services, including AWS Lambda, S3, EC2, Athena and Redshift - Proficiency in Apache Spark, Python, and SQL, enabling...