SwitchFS: Exploiting Persistent Memory Bandwidth in Cluster via Learning-based I/O Path Selection

Abstract: Persistent Memory (PM) and Remote Direct Memory Access (RDMA) technologies have significantly improved the storage and network performance in data centers and spawned a slew of distributed file systems (DFS) designs. Existing DFSs often consider remote storage a performance constraint, assuming it delivers lower bandwidth and higher latency than local devices. However, the advancements in RDMA technology present an opportunity to narrow the performance disparity between local and remote access, empowering DFSs to harness both local and remote I/O capabilities, thereby attaining higher aggregated throughput. We propose SwitchFS, a new DFS architecture that exploits all the available PM bandwidth in a cluster with generative I/O path assignment policy. It dynamically steers client-side I/O requests to the local cache and the remote devices accessible through RDMA network. To determine the run-time I/O path adaptively, we introduce MELON, an I/O path selection policy based on a reinforcement learning model, designed to optimize expected I/O latency. Meanwhile, SwitchFS adopts a model-driven opportunistic replication mechanism at the server-side, which further improves I/O bandwidth utilization of remote devices across the cluster. Furthermore, we adopt the fine-grained concurrency control approach to improve scalability. Across synthetic and real-world application workloads, SwitchFS improves throughput by up to 5.9× over existing DFS designs while preserving low tail latency and incurring modest learning overhead.