Shaked Farkash

Senior Data Engineer

Israel

ShakedFarkash.

I build and operate production data platforms — lakehouse, streaming, services, and model serving.

  • 5+ years in production
  • 20+ pipelines running
  • Sole owner of EV-EDGE's data platform
Email meLinkedIn

The platform I own, end to end.

Batch and streaming in, trusted data and AI services out — observed at every stage, with alerts before anyone notices a problem.

  • Data
  • Telemetry
  • Alerts
EV-EDGE DATA PLATFORM · AWSEV chargerstelemetryERPdatabaseCRMdatabaseBatchGlue · PySparkStreamingKafka · FlinkS3 lakehouseraw → curatedRedshiftSCD2 historyServicesECS · FastAPIAIBedrock · RAGReal-timeload mgmtObservabilityCloudWatch · Datadog · anomaliesRow-count anomaly · sessionsFreshness check passedSchema drift · partner feedBackfill completed · tariffs
Architecture of EV-EDGE's data platform. Alert messages are illustrative.
  • Python
  • Spark
  • Kafka
  • Flink
  • Airflow
  • Parquet
  • Pandas
  • PostgreSQL
  • MySQL
  • MongoDB
  • Docker
  • Kubernetes
  • FastAPI
  • Datadog
  • Git
  • AWS Glue
  • Lambda
  • S3
  • Redshift
  • DynamoDB
  • ECS
  • Bedrock
  • CloudWatch
  • Iceberg
  • SCD Type 2
  • RAG
  • Text-to-SQL

Pipelines that are boring in the best way.

I'm a senior data engineer who builds and operates production data platforms end to end — ingestion, lakehouse, streaming, and the services built on top. At EV-EDGE I'm the sole owner of the company-wide AWS stack and of the data it produces.

I care about pipelines that are idempotent, resumable, and safe to backfill, and about data people can actually trust: quality, freshness, and clear serving contracts. Lately that includes production LLM and RAG services on top of the platform.

Hebrew (native) · English (professional)

A platform, not a pile of pipelines.

Classic ETL moves tables from A to B and calls it done. I build the platform other teams and systems build on — data served through clear contracts, APIs, streams and models, owned like a product.

Classic ETLPlatform
  • Jobs that move tablesData products with consumers and contracts
  • Done when the job succeedsDone when the consumer can trust it
  • Every pipeline from scratchA shared framework — new sources in days
  • Data lands in a warehouseData is served — APIs, real-time streams, AI
  • Users find the problemsObserved at every stage, alerts come first

Where I've built it.

  1. Senior Data Engineer at EV-EDGE

    2021 – 2024 · 2025 – Now

    Sole owner of the company-wide data platform: ingestion, lake, warehouse, streaming, orchestration, ECS-hosted services, and model serving.

    • Architected a unified AWS lakehouse from scratch — multi-source ingestion into a three-zone S3 lake (raw → processed → curated) and a Redshift warehouse layer.
    • Built and operate 20+ production ETL/ELT pipelines in PySpark (Glue), Lambda, and Python, with DynamoDB watermarks and UPSERT staging so every run is idempotent and safe to backfill.
    • Shipped a class-based Glue/PySpark framework that cut new-pipeline time from weeks to days.
    • Built Kafka + Flink streaming for real-time load management, allocating power across charging stations on the same grid.
    • Built production AI services on AWS Bedrock — RAG over schema embeddings and a FastAPI Text-to-SQL API other systems call.
    • PySpark
    • Glue
    • Kafka
    • Flink
    • Redshift
    • ECS
    • Bedrock

  2. Data Engineer at Migdal Insurance

    Jan – Oct 2025

    Enterprise data platform team — production pipelines, streaming, and lakehouse.

    • Designed and shipped production pipelines in Python, Spark, SQL, and AWS, and contributed to platform modernization.
    • Built Kafka streaming workflows for near-real-time ingestion, monitoring, and downstream delivery.
    • Developed ETL preparing structured and semi-structured data for LLM consumption.
    • Worked with Apache Iceberg for schema and partition evolution at lakehouse scale.
    • Spark
    • Kafka
    • Iceberg
    • AWS
    • Docker

Education

  1. Afeka Tel Aviv Academic College of Engineering

    In progress

    M.Sc. Industrial Engineering & Management — Information Systems

  2. Holon Institute of Technology (HIT)

    2021 – 2024

    B.Sc. Computer Science

Let's build something that flows.

Open to conversations about data platforms, pipelines, and new roles.

Email meLinkedIn