Clone and build the SPDK repository as per https://github.com/spdk/spdk
Clone the RocksDB repository from the SPDK GitHub fork into a separate directory. Make sure you check out the 6.15.fb
branch.
Build RocksDB. Only the db_bench
benchmarking tool is integrated with BlobFS.
Or you can also add DEBUG_LEVEL=0
for a release build (need to turn on USE_RTTI
).
Create an NVMe section in the configuration file using SPDK's gen_nvme.sh
script.
Verify the configuration file has specified the correct NVMe SSD. If there are any NVMe SSDs you do not wish to use for RocksDB/SPDK testing, remove them from the configuration file.
Make sure you have at least 5GB of memory allocated for huge pages. By default, the SPDK setup.sh
script only allocates 2GB. The following will allocate 5GB of huge page memory (in addition to binding the NVMe devices to uio/vfio).
Create an empty SPDK blobfs for testing.
At this point, RocksDB is ready for testing with SPDK. Three db_bench
parameters are used to configure SPDK:
spdk
- Defines the name of the SPDK configuration file. If omitted, RocksDB will use the default PosixEnv implementation instead of SpdkEnv. (Required)spdk_bdev
- Defines the name of the SPDK block device which contains the BlobFS to be used for testing. (Required)spdk_cache_size
- Defines the amount of userspace cache memory used by SPDK. Specified in terms of megabytes (MB). Default is 4096 (4GB). (Optional)SPDK has a set of scripts which will run db_bench
against a variety of workloads and capture performance and profiling data. The primary script is test/blobfs/rocksdb/rocksdb.sh
.
BlobFS provides a FUSE plug-in to mount an SPDK BlobFS as a kernel filesystem for inspection or debug purposes. The FUSE plug-in requires fuse3 and will be built automatically when fuse3 is detected on the system.
Note that the FUSE plug-in has some limitations - see the list below.