title: Toward Millions of File System IOPS on Low-Cost, Commodity Hardware.
abstract: We describe a storage system that removes I/O bottlenecks to achieve more than one million IOPS based on a user-space file abstraction for arrays of commodity SSDs. The file abstraction refactors I/O scheduling and placement for extreme parallelism and non-uniform memory and I/O. The system includes a set-associative, parallel page cache in the user space. We redesign page caching to eliminate CPU overhead and lock-contention in non-uniform memory architecture machines. We evaluate our design on a 32 core NUMA machine with four, eight-core processors. Experiments show that our design delivers 1.23 million 512-byte read IOPS. The page cache realizes the scalable IOPS of Linux asynchronous I/O (AIO) and increases user-perceived I/O performance linearly with cache hit rates. The parallel, set-associative cache matches the cache hit rates of the global Linux page cache under real workloads.
authors.affiliation: Department of Computer Science, Johns Hopkins University.
authors.forename: Da
authors.initials: D
authors.lastname: Zheng
authors.affiliation: Department of Computer Science, Johns Hopkins University.
authors.forename: Randal
authors.initials: R
authors.lastname: Burns
authors.affiliation: Department of Physics and Astronomy, Johns Hopkins University.
authors.forename: Alexander S
authors.initials: AS
authors.lastname: Szalay
keywords: Data-intensive computing; Design; Performance; low cost; millions of IOPS; page cache optimization; solid-state storage devices