FA RCU: Fault Aware Read-Copy-Update

Abstract

Deferred freeing is the fundamental technique used in Read-Copy-Update (RCU) synchronization technique where reclamation of resources is deferred until the completion of all active RCU read-side critical sections. We observe that faults inside an RCU read-side critical section can indefinitely block writers that are waiting for the completion of RCU readers and also lead to system failures by preventing the reclamation of deferred resources. We show that the impact of such faults in the Linux kernel is global; a fault in one subsystem can propagate and exhaust critical resources in other unrelated subsystems opening a window of opportunity for DoS-based attacks. For example, a fault in a filesystem can exhaust the process ulimit resulting in fork failures. Since, guaranteeing the absence of faults is practically impossible, it is imperative to harden RCU to tolerate faults. We first study the impact of mitigating lockup by termination of the faulty thread, as thread termination is standard approach used by Linux as recovery strategy. Whereas, another solution is stack based and do not require termination of faulty thread. We demonstrate the impact of faults in RCU read-side critical sections and present RCU recovery techniques that use novel approaches to detect and isolate effect of such faults. We also discuss system consistency once the fault is handled by our approaches. Timely recovery results in a usable system, preserving the user application state and increasing the system’s availability. Our evaluation in the Linux kernel shows that our solution can prevent resource exhaustion in the presence of faults with no additional overhead in the absence of faults.