VMD driver

In this document

Introduction

Intel Volume Management Device is a hardware logic inside processor's Root Complex responsible for management of PCIe NVMe SSDs. It provides robust Hot Plug support and Status LED management.

The driver is responsible for enumeration and hooking NVMe devices behind VMD into SPDK PCIe subsystem. It also provides API for LED management and hot plug.

Public Interface

Key Functions

Function Description
spdk_vmd_init() Enumerate VMD devices and hook them into the spdk pci subsystem.
spdk_vmd_pci_device_list() Returns a list of nvme devices found on the given vmd pci BDF.
spdk_vmd_set_led_state() Sets the state of the LED on specified PCI device.
spdk_vmd_get_led_state() Retrieves the state of the LED on specified PCI device.
spdk_vmd_hotplug_monitor() Checks for hotplug/hotremove events of the devices behind the VMD.

Configuration

To enable VMD driver enumeration, the following steps are required:

Check for available VMD devices (VMD needs to be properly set up in BIOS first).

Example:

$ lspci | grep 201d
$ 5d:05.5 RAID bus controller: Intel Corporation Device 201d (rev 04)
$ d7:05.5 RAID bus controller: Intel Corporation Device 201d (rev 04)

Run setup.sh script with VMD devices set in PCI_WHITELIST.

Example:

$ PCI_WHITELIST="0000:5d:05.5 0000:d7:05.5" scripts/setup.sh

Check for available devices behind the VMD with spdk_lspci.

Example:

$ ./app/spdk_lspci/spdk_lspci
5d0505:01:00.0 (8086 a54) (NVMe disk behind VMD)
5d0505:03:00.0 (8086 a54) (NVMe disk behind VMD)
d70505:01:00.0 (8086 a54) (NVMe disk behind VMD)
d70505:03:00.0 (8086 a54) (NVMe disk behind VMD)
0000:5d:05.5 (8086 201d) (VMD)
0000:d7:05.5 (8086 201d) (VMD)

VMD NVMe BDF could be used as regular NVMe BDF.

Example:

$ ./scripts/rpc.py bdev_nvme_attach_controller -b NVMe1 -t PCIe -a 5d0505:01:00.0

Application framework

When application framework is used, VMD section needs to be added to the configuration file:

Text config:

[VMD]
Enable True

JSON config:

{
"subsystem": "vmd",
"config": [
{
"method": "enable_vmd",
"params": {}
}
]
}

or use RPC call before framework starts e.g.

$ ./app/spdk_tgt/spdk_tgt --wait_for_rpc
$ ./scripts/rpc.py enable_vmd
$ ./scripts/rpc.py framework_start_init

Applications w/o application framework

To enable VMD enumeration in SPDK application that are not using application framework e.g nvme/perf, nvme/identify -V flag is required - please refer to app help if it supports VMD.

Applications need to call spdk_vmd_init() to enumerate NVMe devices behind the VMD prior to calling spdk_nvme_(probe|connect). To support hot plugs spdk_vmd_hotplug_monitor() needs to be called periodically.

LED management

VMD LED utility in the examples/vmd/led could be used to set LED states.