. You know how to structure data pipelines for reliability, scalability and optimal performance (e.g. with Spark or Flink...
. A plus would be additional experience with Airflow/Mlflow/Docker/Flink Experience collaborating closely with multi-functional teams, driving...
. Familiarity with streaming technologies (ex. Kafka, Flink, Spark Streaming, Kinesis) and real-time or near real time use cases (ex...
Airflow and data processing frameworks such as PySpark Experience with data streaming technologies like Kafka, Kinesis, Flink...
connectors (Kafka, dbt, Spark, Flink, Beam, Fivetran), and integrations with leading visualization platforms (Grafana, Tableau... internals of at least one of the following technologies: Apache Spark, Apache Flink, Kafka Connect, or Apache Beam. Experience...
systems and event-driven architectures. Experience working with Flink or similar systems is a strong plus. Ability...
such as Apache Spark, Kafka, or Flink. Hands-on experience with cloud data platforms: AWS (Redshift, Glue, S3), GCP (BigQuery...
or real-time data ingestion (Kafka, Kinesis, Flink, Spark Streaming, or similar) into a warehouse or lakehouse environment...
need to build them, but you need to understand what they enable and how to prioritise investment (e.g. Kafka, Spark, Flink...
and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark), orchestrating... Processing: Dataflow (Apache Beam), with Flink/Spark where appropriate Orchestration: Airflow (Cloud Composer) Analytical store...