Spark Fault Tolerance via RDD Lineage

When a Spark executor fails, the partitions it held are lost. Instead of replicating data (like HDFS), Spark uses lineage: the recorded sequence of transformations that produced each partition.

The driver detects the failure, identifies the lost partitions, and recomputes them by replaying the transformations from the nearest persisted ancestor. This approach is cheaper than replication for most workloads because transformations are typically narrow (map, filter) and operate on local data.