The SPDK repository is, first and foremost, a collection of high-performance storage-centric software libraries. With this in mind, much care has been taken to ensure that these libraries have consistent and robust naming and versioning conventions. The libraries themselves are also divided across two directories (
module) inside of the SPDK repository in a deliberate way to prevent mixing of SPDK event framework dependent code and lower level libraries. This document is aimed at explaining the structure, naming conventions, versioning scheme, and use cases of the libraries contained in these two directories.
The SPDK libraries are divided into two directories. The
lib directory contains the base libraries that compose SPDK. Some of these base libraries define plug-in systems. Instances of those plug-ins are called modules and are located in the
module directory. For example, the
spdk_sock library is contained in the
lib directory while the implementations of socket abstractions,
sock_uring are contained in the
The libraries in the
lib directory can be readily divided into four categories:
spdk_logcontains macro definitions that provide a consistent logging paradigm and
spdk_jsonis a general purpose JSON parsing library.
spdk_vhosteach define the storage protocols after which they are named.
spdk_bdevprovides a general block device abstraction layer,
spdk_lvolprovides a logical volume abstraction,
spdk_blobfsprovides a filesystem abstraction, and
spdk_ftlprovides a flash translation layer abstraction.
spdk_jsonrpc) and thread abstractions (see
spdk_thread). The most notable library in this category is the
spdk_env_dpdklibrary which provides a shim for the underlying Data Plane Development Kit (DPDK) environment and provides services like memory management.
The one library in the
lib directory that doesn't fit into the above classification is the
spdk_event library. This library defines a framework used by the applications contained in the
example directories. Much care has been taken to keep the SPDK libraries independent from this framework. The libraries in
lib are engineered to allow plugging directly into independent application frameworks such as Seastar or libuv with minimal effort.
Currently there are two exceptions in the
lib directory which still rely on
spdk_iscsi. There are efforts underway to remove all remaining dependencies these libraries have on the
Much like the
spdk_event library, the
spdk_env_dpdk library has been architected in such a way that it can be readily replaced by an alternate environment shim. More information on replacing the
spdk_env_dpdk module and the underlying
dpdk environment can be found in the environment section.
The component libraries in the
module directory represent specific implementations of the base libraries in the
lib directory. As with the
lib directory, much care has been taken to avoid dependencies on the
spdk_event framework except for those libraries which directly implement the
spdk_event module plugin system.
There are seven sub-directories in the
module directory which each hold a different class of libraries. These sub-directories can be divided into two types.
libdirectory and are registered with that library at runtime by way of a specific constructor function. The parent library in the
libdirectory then manages the module directly. These types of libraries each implement a function table defined by their parent library. The following table shows these directories and their corresponding parent libraries:
|module directory||parent library||dependent on event library|
libdirectory but are not explicitly registered to that library via a constructor. The libraries in the
env_dpdkdirectories fall into this category. None of the libraries in this category depend explicitly on the
The SPDK libraries follow strict conventions for naming functions, logging, versioning, and header files.
All public SPDK header files exist in the
include directory of the SPDK repository. These headers are divided into two sub-directories.
include/spdk contains headers intended to be used by consumers of the SPDK libraries. All of the functions, variables, and types in these functions are intended for public consumption. Multiple headers in this directory may depend upon the same underlying library and work together to expose different facets of the library. The
spdk_bdev library, for example, is exposed in three different headers.
bdev_module.h defines the interfaces a bdev module library would need to implement,
bdev.h contains general block device functions that would be used by an application consuming block devices exposed by SPDK, and
bdev_zone.h exposes zoned bdev specific functions. Many of the other libraries exhibit a similar behavior of splitting headers between consumers of the library and those wishing to register a module with that library.
include/spdk_internal, as its name suggests contains header files intended to be consumed only by other libraries inside of the SPDK repository. These headers are typically used for sharing lower level functions between two libraries that both require similar functions. For example
spdk_internal/nvme_tcp.h contains low level tcp functions used by both the
spdk_nvmf libraries. These headers are NOT intended for general consumption.
Other header files contained directly in the
module directories are intended to be consumed only by source files of their corresponding library. Any symbols intended to be used across libraries need to be included in a header in the
All public types and functions in SPDK libraries begin with the prefix
spdk_. They are also typically further namespaced using the spdk library name. The rest of the function or type name describes its purpose.
There are no internal library functions that begin with the
spdk_ prefix. This naming convention is enforced by the SPDK continuous Integration testing. Functions not intended for use outside of their home library should be namespaced with the name of the library only.
SPDK libraries can be built as both static and shared object files. To facilitate building libraries as shared objects, each one has a corresponding map file (e.g.
spdk_nvmf relies on
spdk_nvmf.map). SPDK libraries not exporting any symbols rely on a blank map file located at
SPDK shared objects follow a semantic versioning pattern with a major and minor version. Any changes which break backwards compatibility (symbol removal or change) will cause a shared object major increment and backwards compatible changes will cause a minor version increment; i.e. an application that relies on
libspdk_nvmf.so.3.0 will be compatible with
libspdk_nvmf.so.3.1 but not with
Shared object versions are incremented only once between each release cycle. This means that at most, the major version of each SPDK shared library will increment only once between each SPDK release.
There are currently no guarantees in SPDK of ABI compatibility between two major SPDK releases.
The point releases of an LTS release will be ABI compatible with the corresponding LTS major release.
Shared objects are versioned independently of one another. This means that
libspdk_bdev.so.3.0 do not necessarily belong to the same release. This also means that shared objects with the same suffix are not necessarily compatible with each other. It is important to source all of your SPDK libraries from the same repository and version to ensure inter-library compatibility.
Shared objects in SPDK are created on a per-library basis. There is a top level
libspdk.so object which is a linker script. It simply contains references to all of the other spdk shared objects.
There are essentially two ways of linking to SPDK libraries.
In the second instance, please note that applications need only link to the libraries upon which they directly depend. All SPDK libraries have their dependencies specified at object compile time. This means that when linking to
spdk_net, one does not also have to specify
spdk_rpc. However, this dependency inclusion does not extend to the application itself; i.e. if an application directly uses symbols from both
spdk_log, both libraries will need to be supplied to the linker when linking the application even though
spdk_log is a dependency of
Please also note that when linking to SPDK libraries, both the spdk_env shim library and the env library itself need to be supplied to the linker. In the examples above, these are
dpdk respectively. This was intentional and allows one to easily swap out both the environment and the environment shim.
SPDK depends on an environment abstraction that provides crucial pinned memory management and PCIe bus management operations. The interface for this environment abstraction is defined in the
include/env.h header file. The default implementation of this environment is located in
spdk_env_dpdk. This abstraction in turn relies upon the DPDK libraries. This two part implementation was deliberate and allows for easily swapping out the dpdk version upon which the spdk libraries rely without making modifications to the spdk source directly.
Any environment can replace the
spdk_env_dpdk environment by implementing the
include/env.h header file. The environment can either be implemented wholesale in a single library or as a two-part shim/implementation library system.
SPDK static objects are compiled by default even when no parameters are supplied to the build system. Unlike SPDK shared objects, the filename does not contain any versioning semantics. Linking against static objects is similar to shared objects but will always require the use of
-Wl,--whole-archive as argument. This is due to the use of constructor functions in SPDK such as those to register NVMe transports.
Due to the lack of versioning semantics, it is not recommended to install static libraries system wide. Instead the path to these static libraries should be added as argument at compile time using
-L/path/to/static/libs. The use of static objects instead of shared objects can also be forced through
-Wl,-Bstatic, otherwise some compilers might prefer to use the shared objects if both are available.