using Spark, Delta Lake, and Databricks notebooks. Integration : Connect Databricks with Azure Data Lake, Synapse, Power BI..., and other services. Optimization : Tune Spark jobs for performance and cost efficiency. Collaboration : Work with data scientists...
data pipelines in enterprise environments. Strong proficiency in SQL and Python, including PySpark and Spark SQL...
, requests, pytest, logging). Read and modify existing Scala/Spark jobs as needed. Use shell scripting for automation.... Implement Spark Structured Streaming with stateful aggregations, watermarking, and checkpointing. Support finance use cases...
concepts. Experience with big data and distributed systems (Spark, Hadoop, etc.) Hands-on experience with cloud platforms (AWS...
large-scale data platform implementations end-to-end. Strong hands-on technical background in Spark, Scala, and SQL. Deep...
Understanding of data engineering principles : ETL, data lakes, streaming pipelines (Kafka, Spark) Demonstrate excellent...
appropriate open-ended questions that spark conversations resulting in uncovering financial needs of customers, prospects...
, OpenShift preferred. Object Storage. Execution Engines such as Spark or Trino. Table formats such as Iceberg or Hive...
purchases (with some exceptions). Paid Breaks: during 8-hour shifts. Responsibilities: Be the spark! Greet every customer...
. Required Skills - Hands-on experience with Databricks and core data engineering tools (SQL, Python, Spark;Hadoop familiarity...