Clone and build the SPDK repository as per https://github.com/spdk/spdk
Clone the RocksDB repository from the SPDK GitHub fork into a separate directory. Make sure you check out the
Build RocksDB. Only the
db_bench benchmarking tool is integrated with BlobFS. (Note: add
DEBUG_LEVEL=0 for a release build.)
Create an NVMe section in the configuration file using SPDK's
Verify the configuration file has specified the correct NVMe SSD. If there are any NVMe SSDs you do not wish to use for RocksDB/SPDK testing, remove them from the configuration file.
Make sure you have at least 5GB of memory allocated for huge pages. By default, the SPDK
setup.sh script only allocates 2GB. The following will allocate 5GB of huge page memory (in addition to binding the NVMe devices to uio/vfio).
Create an empty SPDK blobfs for testing.
At this point, RocksDB is ready for testing with SPDK. Three
db_bench parameters are used to configure SPDK:
spdk- Defines the name of the SPDK configuration file. If omitted, RocksDB will use the default PosixEnv implementation instead of SpdkEnv. (Required)
spdk_bdev- Defines the name of the SPDK block device which contains the BlobFS to be used for testing. (Required)
spdk_cache_size- Defines the amount of userspace cache memory used by SPDK. Specified in terms of megabytes (MB). Default is 4096 (4GB). (Optional)
SPDK has a set of scripts which will run
db_bench against a variety of workloads and capture performance and profiling data. The primary script is
BlobFS provides a FUSE plug-in to mount an SPDK BlobFS as a kernel filesystem for inspection or debug purposes. The FUSE plug-in requires fuse3 and will be built automatically when fuse3 is detected on the system.
Note that the FUSE plug-in has some limitations - see the list below.