2021 19th USENIX Conference on File and Storage Technologies, February 23–25, 2021
We propose SpanDB, an LSM-tree-based KV store that adapts the popular RocksDB system to utilize selective deployment of high-speed SSDs. SpanDB allows users to host the bulk of their data on cheaper and larger SSDs, while relocating write-ahead logs (WAL) and the top levels of the LSM-tree to a much smaller and faster NVMe SSD. To better utilize this fast disk, SpanDB provides high-speed, parallel WAL writes via SPDK, and enables asynchronous request processing to mitigate inter-thread synchronization overhead and work efficiently with polling-based I/O.
SPDK on Microsoft Windows
MayaData has created the Windows Platform Development Kit (WPDK) to act as a foundational layer to make it easier to port the SPDK to Windows. Additionally, they are working with the SPDK community to make a few changes to the code base to support WPDK. Are you interested in porting SPDK to Windows? Please head to https://wpdk.github.io to get started. Community contributions are welcomed and needed!
OpenEBS Mayastor: Accelerating Container Attached Storage with Intel® Optane™ Technology and SPDK
The OpenEBS Mayastor storage engine uses Intel® Optane™ SSD DC P4800X Series and the SPDK NVMe-oF target to deliver high-performance, disaggregated storage to Kubernetes for high-performance workloads. MayaData benchmarked several workloads and demonstrated that the performance overhead of Mayastor was less than 10% when it used SPDK NVMe-oF target to export the Intel® Optane™ SSD over the network vs. locally accessing the SSD. Furthermore, MayaData provides a solution that can deliver over 1 million IOPS to a single container.
Intel & Nutanix Joint Innovation Lab
Blockstore and SPDK improved application performance and reduced latency in synthetic and real-world testing in the next-generation NX-8170 cluster with Intel® Optane™ SSDs and Intel® SSD DC P4510 NVMe SSDs. Intel and Nutanix benchmarked several workloads on a four-node Nutanix NX-8170-G7 cluster. The PostgreSQL DB benchmarks showed a 66% improvement in DB transactions and latency was reduced by 45% on the next-generation NX-8170 cluster vs the previous generation NX-3060 cluster with SATA SSDs.
DAOS: Revolutionizing High-Performance Storage with Intel® Optane™ Technology, SPDK and PMDK
DAOS is entirely in userspace, utilizing the PMDK and SPDK open source libraries, not only bypassing the performance limitations of I/O through the kernel, but also alleviating other issues such as jitter, ease of administration, etc. DAOS also breaks free from the performance limitations of the POSIX interface, relying on the more performant optimistic concurrency control rather than the pessimistic concurrency control of POSIX.
2020 USENIX Annual Technical Conference, July 15–17, 2020
In this paper, members of AliCloud and Shanghai Jiao Tong University together propose Spool, a reliable NVMe virtualization system. Spool is proposed based on the SPDK NVMe driver but focuses on the reliability of the virtualized storage system. The bedrock of SPDK is a userspace, polled-mode, asynchronous, lockless NVMe driver. Usermode drivers help improve the stability of the host operating system because they can only access the address space of the processes running them, and a buggy implementation does not cause system-wide problems. Spool has three key advantages including diagnosability of different failure types, consistency and correctness of the data and reduced restart time.
Radian Memory Systems® Announces Zoned Flash SSD with SPDK
Radian ported their host Block Translation Layer (BTL) for zoned Flash from the Linux® Kernel to SPDK. The company has posted a technical deep dive video that discusses this effort, which includes a comparative, system level analysis between the Linux® Kernel and SPDK implementations.
VAST Data Universal Storage 3.0 Webinar
VAST Data announced they will be using the SPDK the upcoming 3.2 release, because they observed “30-40% better latency resulting in 50-100% overall improvement in system IOPs when they bypass the kernel with SPDK for metadata read and write operations which allowed them to improve the system responsiveness”.
Nutanix .Next 2019 Conference in Copenhagen, Denmark
Nutanix launched a new generation of the Acropolis Operating System (AOS) which has a BlockStore that uses SPDK. Stephen Poitras, Principal Solutions Architect at Nutanix, talked about the work his team did to rearchitect the datapath to take advantage of emerging technologies such as NVMe, RDMA and Memory Class Storage devices like Intel® Optane™ SSDs. They purpose-built a BlockStore for the next generation of storage media that completely eliminated the filesystem overhead by using SPDK to access storage devices in user space. Using SPDK improves performance by driving down latency and increasing efficiency.
ASPLOS’20, March 16–20, 2020, Lausanne, Switzerland
In this paper, AliCloud proposes the design of a new high-density, multi-tenant, bare-metal Cloud called BM-Hive. BM-Hive features a hardware-software hybrid virtio I/O system that enables the guest to directly access the cloud network and storage services. In BM-Hive, all the I/O requests are handled in the user space with vhost-user protocol interfacing to Cloud infrastructure: the customized DPDK vSwitch and the SPDK Cloud storage.
Circuit Blvd - Kubernetes and Docker setup for SPDK
In this technote, Circuit Blvd presents an example of how to use Docker and Kubernetes to containerize and orchestrate an SPDK Vhost application that serves virtio block devices to application containers.
UCloud - End to End IO Path Optimization with SPDK
Cloud service provider UCloud described the good practice of integrating SPDK vhost and other modules in their end to end solution from VM to the cloud disk. UCloud used SPDK to optimize the virtualized IO path, proposed the features like hot upgrade and online migration solution based on SPDK vhost, and successfully coupled with the high-performance cloud disk (UCloud RSSD) to achieve good results which reach up to 1.2 million IOPS. UCloud also actively joins the SPDK community and summarizes that each version of SPDK brings many interesting features and they would like to explore and further apply to the performance improvement of cloud disk and other products.
May 6 2019 Intel® Chip Chat Episode 654
Dr. Felipe Franciosi from Nutanix, and Nate Marushak from Intel, describe how SPDK brings higher performance to today’s hyperconverged data center and holds promise for the future.
February 2019 17th USENIX Conference on File and Storage Technologies (FAST ‘19)
This page describes how uDepot, a key-value store, utilizes SPDK and Intel Optane SSD to achieve equivalent performance to its DRAM-based counterparts at a much lower cost.
November 2 2018 Linux Piter #4
At Linux Piter, Dr. Felipe Franciosi discussed the performance challenges of storage virtualization. He talked about how SPDK is being leveraged to minimize the software overhead to record lows and shared performance data that show that it is possible for virtual machines to access Intel® Optane™ SSDs in under 10 microseconds for 4K I/O requests through virtio-scsi.
The Nutanix Bible
Explains the pros/cons of User vs. Kernel Space.
As devices have become faster (e.g. NVMe, Intel® Optane™, pMEM), the kernel and device interaction has become a bottleneck. To eliminate these bottlenecks, a lot of vendors are moving things out of the kernel to user space with polling and seeing much better results. Great examples of this are the Storage Performance Development Kit (SPDK) and Data Plane Development Kit (DPDK). These projects are geared at maximizing the performance and reducing latency as much as possible, and have shown great success.
AliCloud - High Performance Local SSD
Through close collaborations between AliCloud and SPDK, AliCloud offers one model (local SSD) among different kinds of cloud services. The AliCloud local SSD model is for critical applications like OLTP and NoSQL databases and designed with NVMe SSD and advanced SPDK storage techniques to achieve the extra high performance on single node like 480K max random read/write IOPS, 200us lower latency, 4GB/s max sequential read bandwidth, 2.4GB/s max sequential write bandwidth…
September 18 2018 CloudTech
Intel, Mirantis and Mellanox are working together to enable NVMe-oF in Openstack. There 2 solutions to implement NVMe-oF with OpenStack: Kernel NVMe-oF target and SPDK NVMe-oF target. If compared with Kernel-based implementations, SPDK reduces NVMe-oF software overheads and yields high throughput and performance …
August 20 2018 Forbes
At FMS, SolarFlare showcased Intel® Optane™ SSDs with Solarflare XTremeScale™ 100GBE Ethernet adapters implementing the upcoming NVMe-oF TCP spec using SPDK. The NVMe-based Optane storage over TCP provides a low-cost high performance solid-state network that has latency as low as 15 microseconds.
August 2018 44th International Conference on Very Large Data Bases
PolarFS: An Ultra-low Latency and Failure Resilient Distributed File System for Shared Storage Cloud Database
This paper describes how PolarFS uses emerging techniques like RDMA, NVMe SSDs and SPDK to reduce the end-to-end latency. The write latency of PolarFS is close to a local file system on SSD.
June 11 2018 14th International Workshop on Data Management
In this paper, SPDK was used to optimize the I/O path in Deuterony (a Log-Structured data caching DB) and reduce the cost of I/O operations by a third.
May 15 2018 SPDK Summit
Over 170 attendees from 51 different companies attended the 2018 SPDK US Summit at Dolce Hayes Mansion, San Jose on May 15th and 16th.
March 20 2018 OCP Summit 2018
At the OCP Summit 2018, Cavium, Microsemi and Marvell demonstrated concurrent 100Gbps NVMe-oF RoCE and iWARP connectivity … showcases how storage IHVs can leverage PCIe P2P and SPDK technology …
February 6 2018 IBM Research Report
A research team at IBM used SPDK to optimize the performance of gets and puts operations in the uDepot KV-store. They showcased latency of 7.2 usecs and throughput of 6.17 million get operations/sec with Intel Optane SSDs and SPDK.
July 12 2017 DigiTimes
Intel has launched its new Intel Xeon scalable processors, providing … with Intel Optane SSDs and Storage Performance Development Kit (SPDK), …
July 12 2017 ETCIO.com
Bangalore: Intel India today launched its new Intel Xeon Scalable … with Intel Optane SSDs and Storage Performance Development Kit (SPDK), …