Senior Data Engineer - Data Platform (m/f/d)
, Apache Beam, Apache Flink, etc.) Having worked in a scale up environment at a company of similar size Interest in the climate...
, Apache Beam, Apache Flink, etc.) Having worked in a scale up environment at a company of similar size Interest in the climate...
need to build them, but you need to understand what they enable and how to prioritise investment (e.g. Kafka, Spark, Flink...
Dataflow, Apache Beam, Apache Flink, etc.) Having worked in a scale up environment at a company of similar size Interest...
) and modern data stacks (e.g., Spark, Flink) to manage complex, distributed systems. Proficiency in Python, Java, or Scala...
) and modern data stacks (e.g., Spark, Flink) to manage complex, distributed systems. Proficiency in Python, Java, or Scala...
and ClickHouse, building real-time pipelines on Pub/Sub and Kafka with Dataflow (and where it fits, Flink/Spark), orchestrating... Processing: Dataflow (Apache Beam), with Flink/Spark where appropriate Orchestration: Airflow (Cloud Composer) Analytical store...
, Rust, C++, C#) and ODBC driver;ship first-party connectors for Kafka Connect, Apache Spark, Apache Flink, dbt, and Airbyte... ClickHouse connectors for Kafka, Flink, Spark, and others, and the broader JVM ecosystem (data ingestion tools, BI platforms...
applications and streaming data processing (e.g. GCP Dataflow, Apache Beam, Apache Flink, etc.) Having worked in a scale up...
processing engine built on Apache Flink, together with the surrounding ecosystem that makes it production-ready for demanding... effectively in an asynchronous, remote environment. Preferred Qualifications: Contributions to Apache Flink or comparable...
engine performance and reliability Enhance integration with streaming engines (e.g., Flink) Engage with the open-source... and replication (Raft, Paxos, etc.) Storage engines (LSM trees, B-Trees, WAL, compaction) Streaming systems (Kafka, Pulsar, Flink...