ntb_amd.h File Reference

AMD EPYC 4th-Generation Non-Transparent Bridge (NTB) user-space driver public interface. More...

Macros
#define	SPDK_NTB_QUEUE_SZ_NAME   "queue_size"
#define	SPDK_NTB_QUEUE_NUM_NAME   "queue_num"
#define	SPDK_NTB_TOPO_NAME   "topo"
#define	SPDK_NTB_LINK_STATUS_NAME   "link_status"
#define	SPDK_NTB_SPEED_NAME   "speed"
#define	SPDK_NTB_WIDTH_NAME   "width"
#define	SPDK_NTB_MW_CNT_NAME   "mw_count"
#define	SPDK_NTB_DB_CNT_NAME   "db_count"
#define	SPDK_NTB_SPAD_CNT_NAME   "spad_count"

Typedefs
typedef bool(*	spdk_ntb_amd_probe_cb) (void *cb_ctx, struct spdk_pci_device *pci_dev)
	Callback for spdk_ntb_amd_probe() enumeration.
typedef void(*	spdk_ntb_amd_attach_cb) (void *cb_ctx, struct spdk_pci_device *pci_dev, struct spdk_ntb_amd_dev *ntb)
	Callback for spdk_ntb_amd_probe() to report a device that has been attached to the userspace NTB driver.

Functions
int	spdk_ntb_amd_get_link_status (struct spdk_ntb_amd_dev *ntb)
int	spdk_ntb_amd_spad_read (const struct spdk_ntb_amd_dev *ntb, int spad, bool peer, uint32_t *value)
uint64_t	spdk_ntb_amd_db_read (const struct spdk_ntb_amd_dev *ntb)
int	spdk_ntb_amd_db_clear (struct spdk_ntb_amd_dev *ntb, uint64_t db_bits)
int	spdk_ntb_amd_peer_db_set (struct spdk_ntb_amd_dev *ntb, uint8_t db_idx)
int	spdk_ntb_amd_attr_get (struct spdk_ntb_amd_dev *ntb, const char *attr_name, uint64_t *attr_value)
bool	spdk_ntb_amd_peer_ready_get (const struct spdk_ntb_amd_dev *ntb)
	Read the cached "peer ready" flag for this channel.
int	spdk_ntb_amd_peer_ready_set (struct spdk_ntb_amd_dev *ntb, bool ready)
	Update the cached "peer ready" flag for this channel.
int	spdk_ntb_amd_handshake_work (struct spdk_ntb_amd_dev *ntb, void *buffer, size_t buffer_size)
	Perform the peer-to-peer NTB handshake.
int	spdk_ntb_amd_dev_start (struct spdk_ntb_amd_dev *ntb)
	Finalize device setup once the link is up.
int	spdk_ntb_amd_set_active_mw (struct spdk_ntb_amd_dev *ntb, uint8_t mw_idx)
	Select which memory window (BAR) is used for the local mapping in spdk_ntb_amd_dev_start().
int	spdk_ntb_amd_get_peer_mw (struct spdk_ntb_amd_dev *ntb, uint8_t mw_idx, struct iovec *iov, struct spdk_memory_domain **domain, void **vaddr)
	Describe a peer memory window as a DMA target.
int	spdk_ntb_amd_probe (void *cb_ctx, spdk_ntb_amd_probe_cb probe_cb, spdk_ntb_amd_attach_cb attach_cb)
	Enumerate the NTB devices attached to the system and attach the userspace NTB driver to them if desired.
void	spdk_ntb_amd_detach (struct spdk_ntb_amd_dev *ntb)
	Detach specified device returned by spdk_ntb_amd_probe() from the NTB driver.

Detailed Description

AMD EPYC 4th-Generation Non-Transparent Bridge (NTB) user-space driver public interface.

Typedef Documentation

spdk_ntb_amd_attach_cb

typedef void(* spdk_ntb_amd_attach_cb) (void *cb_ctx, struct spdk_pci_device *pci_dev, struct spdk_ntb_amd_dev *ntb)

Callback for spdk_ntb_amd_probe() to report a device that has been attached to the userspace NTB driver.

Parameters

cb_ctx	User-specified opaque value corresponding to cb_ctx from spdk_ntb_amd_probe().
pci_dev	PCI device that was attached to the driver.
ntb	NTB channel that was attached to the driver.

spdk_ntb_amd_probe_cb

typedef bool(* spdk_ntb_amd_probe_cb) (void *cb_ctx, struct spdk_pci_device *pci_dev)

Callback for spdk_ntb_amd_probe() enumeration.

Parameters

cb_ctx	User-specified opaque value corresponding to cb_ctx from spdk_ntb_amd_probe().
pci_dev	PCI device that is being probed.

Returns

true to attach to this device.

Function Documentation

spdk_ntb_amd_detach()

void spdk_ntb_amd_detach

(

struct spdk_ntb_amd_dev *

ntb

)

Detach specified device returned by spdk_ntb_amd_probe() from the NTB driver.

Parameters

ntb	NTB channel to detach from the driver.

spdk_ntb_amd_dev_start()

int spdk_ntb_amd_dev_start

(

struct spdk_ntb_amd_dev *

ntb

)

Finalize device setup once the link is up.

Must be called after spdk_ntb_amd_handshake_work() and before any data movement.

Parameters

ntb	NTB channel handle returned by spdk_ntb_amd_probe().

Returns

0 on success, negative errno on failure.

spdk_ntb_amd_get_peer_mw()

int spdk_ntb_amd_get_peer_mw	(	struct spdk_ntb_amd_dev *	ntb,
		uint8_t	mw_idx,
		struct iovec *	iov,
		struct spdk_memory_domain **	domain,
		void **	vaddr )

Describe a peer memory window as a DMA target.

Fills iov with the window's device (bus) address (iov_base) and size (iov_len), and domain with the NTB device memory domain. The (iovec, memory-domain) pair can be passed directly to spdk_ae4dma_build_copy() as a copy endpoint: because the domain is non-NULL and describes device memory, the engine treats iov_base as an already-resolved bus address (no spdk_vtophys()) and as IO memory.

For CPU access to the same window (e.g. memcpy), vaddr optionally receives the locally-mapped virtual address.

Parameters

ntb	NTB channel handle returned by spdk_ntb_amd_probe().
mw_idx	Memory window index, less than the memory-window count.
iov	Output iovec describing the window (bus address + size).
domain	Output NTB memory domain for the window. May be NULL.
vaddr	Optional output for the locally-mapped virtual address. May be NULL if the caller only needs the DMA view.

Returns

0 on success, -EINVAL on invalid arguments, -ENODEV if the window is not mapped.

spdk_ntb_amd_handshake_work()

int spdk_ntb_amd_handshake_work	(	struct spdk_ntb_amd_dev *	ntb,
		void *	buffer,
		size_t	buffer_size )

Perform the peer-to-peer NTB handshake.

Publishes the memory-window count via scratchpad, programs each memory-window translation to a non-overlapping slice of buffer, and rings doorbell 0 to notify the peer.

Parameters

ntb	NTB channel handle returned by spdk_ntb_amd_probe().
buffer	Local DMA-capable staging buffer that backs all exposed memory windows. The buffer must be physically contiguous (e.g. allocated via spdk_dma_zmalloc()) and at least sum(mw_size[i]) bytes long, where mw_size[i] is the size of the i'th memory window. Memory window i is programmed to the slice starting at offset sum(mw_size[0..i-1]).
buffer_size	Size of buffer in bytes; rejected with -EINVAL if smaller than the sum of memory-window sizes, or if the buffer is not physically contiguous across that range.

Returns

0 on success, negative errno on failure.

spdk_ntb_amd_peer_ready_get()

bool spdk_ntb_amd_peer_ready_get

(

const struct spdk_ntb_amd_dev *

ntb

)

Read the cached "peer ready" flag for this channel.

The flag is true once the local side has observed the peer-ready doorbell at least once and is intended to gate the per-link-up setup (memory-window translations, mempool buffer publication, etc.) so that the application does not redo it for every duplicate doorbell.

The driver itself only reads this flag (it folds it into the value returned for the "link_status" attribute). Updating it is the application's responsibility — see spdk_ntb_amd_peer_ready_set().

Parameters

ntb	NTB channel handle returned by spdk_ntb_amd_probe().

Returns

true if the peer is currently considered ready, false otherwise (also returns false if ntb is NULL).

spdk_ntb_amd_peer_ready_set()

int spdk_ntb_amd_peer_ready_set	(	struct spdk_ntb_amd_dev *	ntb,
		bool	ready )

Update the cached "peer ready" flag for this channel.

Applications call this in their doorbell handler when they observe a peer-ready (set true) or peer-down / peer-ack-down (set false) notification. The flag is consumed by spdk_ntb_amd_attr_get(NTB_LINK_STATUS_NAME).

Parameters

ntb	NTB channel handle returned by spdk_ntb_amd_probe().
ready	New value for the flag.

Returns

0 on success, -EINVAL if ntb is NULL.

spdk_ntb_amd_probe()

int spdk_ntb_amd_probe	(	void *	cb_ctx,
		spdk_ntb_amd_probe_cb	probe_cb,
		spdk_ntb_amd_attach_cb	attach_cb )

Enumerate the NTB devices attached to the system and attach the userspace NTB driver to them if desired.

If called more than once, only devices that are not already attached to the SPDK NTB driver will be reported.

To stop using the controller and release its associated resources, call spdk_ntb_amd_detach() with the ntb_channel instance returned by this function.

Parameters

cb_ctx	Opaque value which will be passed back in cb_ctx parameter of the callbacks.
probe_cb	will be called once per NTB device found in the system.
attach_cb	will be called for devices for which probe_cb returned true once the NTB controller has been attached to the userspace driver.

Returns

0 on success, -1 on failure.

spdk_ntb_amd_set_active_mw()

int spdk_ntb_amd_set_active_mw	(	struct spdk_ntb_amd_dev *	ntb,
		uint8_t	mw_idx )

Select which memory window (BAR) is used for the local mapping in spdk_ntb_amd_dev_start().

Parameters

ntb	NTB device.
mw_idx	Memory window index, less than mw_cnt.

Returns

0 on success, -EINVAL if arguments are invalid.