muBLASLt API Reference

This document describes the muBLASLt APIs supported in MUSA SDK 5.2. It starts with the muBLASLt programming model, then documents the supported matrix multiplication and descriptor APIs.

1. Introduction

1.1 What Is muBLASLt

muBLASLt is the lightweight matrix multiplication extension library for muBLAS. It provides descriptor-based matrix multiplication APIs that let applications describe operation attributes, matrix layouts, algorithm choices, workspace, stream execution, and floating-point emulation settings.

1.2 Document Scope in MUSA SDK 5.2

This reference describes the muBLASLt APIs supported in MUSA SDK 5.2. Use it as the compatibility reference for the functions, types, descriptor attributes, and status values documented here.

Note: Some muBLASLt declarations may appear in mublasLt.h but are not described in this reference. These declarations are experimental in MUSA SDK 5.2 and are provided for reference only.

1.3 Supported API Groups

API group	Scoped APIs
Matrix multiplication	mublasLtMatmul
Matmul descriptor attributes	mublasLtMatmulDescSetAttribute
Floating-point emulation descriptors	mublasLtEmulationDescInit_internal, mublasLtEmulationDescCreate, mublasLtEmulationDescDestroy, mublasLtEmulationDescSetAttribute, mublasLtEmulationDescGetAttribute

1.4 How to Use This Document

Start with Chapter 2 for the descriptor-based usage model. Use Chapter 3 to check status values, datatypes, descriptor attributes, function signatures, parameter lists, return values, and supported API entries.

2. Using the muBLASLt API

2.1 Matmul Operation Model

mublasLtMatmul computes:

D = alpha * op(A) * op(B) + beta * C

The operation is configured by the function arguments and the supplied descriptors. The computeDesc argument describes operation-level attributes such as transpose mode, scaling behavior, epilogue behavior, and optional emulation configuration. The Adesc, Bdesc, Cdesc, and Ddesc arguments describe the memory layout of each matrix operand.

The scalar arguments alpha and beta are passed as pointers. Their interpretation depends on descriptor settings such as scale type and pointer mode. The matrix arguments A, B, C, and D point to the input and output matrix data used by the operation.

The algo, workspace, and workspaceSizeInBytes arguments describe the algorithm configuration and available temporary storage. If the configured algorithm requires more workspace than workspaceSizeInBytes provides, mublasLtMatmul returns MUBLAS_STATUS_INVALID_VALUE.

2.2 Descriptor Attribute Model

muBLASLt uses descriptors to separate operation configuration from execution. Attribute setter APIs use the same basic pattern:

1. Select a descriptor attribute enum value.
2. Store the new attribute value in a buffer.
3. Pass the buffer pointer and the exact value size in bytes.
4. Check the returned mublasStatus_t.

mublasLtMatmulDescSetAttribute sets attributes on a matmul operation descriptor. mublasLtEmulationDescSetAttribute and mublasLtEmulationDescGetAttribute set and query attributes on a floating-point emulation descriptor.

For setter APIs, buf must point to the new value and sizeInBytes must match the internal storage size for the selected attribute. For getter APIs, buf receives the attribute value. If sizeInBytes is 0, sizeWritten can be used to query the number of bytes required for the full value.

2.3 Floating-Point Emulation Descriptor

The emulation descriptor stores floating-point emulation settings that can be attached to a matmul descriptor through MUBLASLT_MATMUL_DESC_EMULATION_DESCRIPTOR.

Use mublasLtEmulationDescInit_internal to initialize an emulation descriptor in caller-provided storage when the storage lifetime is managed outside the library. The current header also defines the inline convenience wrapper mublasLtEmulationDescInit, which forwards sizeof(*emulationDesc) to mublasLtEmulationDescInit_internal.

Use mublasLtEmulationDescCreate to allocate a descriptor through the library, and release that descriptor with mublasLtEmulationDescDestroy when it is no longer needed. Use the set and get attribute APIs to configure or query the emulation strategy, special-values behavior, and fixed-point mantissa controls.

2.4 Stream Usage

mublasLtMatmul takes a musaStream_t argument. The stream identifies where the matrix multiplication work is issued. Keep the stream, workspace, descriptors, and data buffers valid for the duration of the queued operation.

2.5 Error Handling

The supported muBLASLt APIs return mublasStatus_t. Check the return value after each call, especially for descriptor creation, attribute updates, and matrix multiplication calls that depend on descriptor compatibility, workspace size, and device support.

For status values used by the supported APIs, see 3.1 Return Value mublasStatus_t.

3. API Reference

This chapter describes muBLASLt status values, datatypes, descriptor attributes, and library functions. Function entries include signatures, parameters, return values, and descriptions derived from public header comments. APIs that may appear in public headers but are not documented here should be treated as experimental and for reference only in this release.

3.1 Return Value mublasStatus_t

Most supported muBLASLt APIs return mublasStatus_t. A return value of MUBLAS_STATUS_SUCCESS indicates that the call completed successfully. Any other value indicates that the call did not complete as requested.

The following table lists the status values emitted by the supported muBLASLt API entries in this reference.

Status value	Meaning
MUBLAS_STATUS_SUCCESS	The operation completed successfully.
MUBLAS_STATUS_NOT_INITIALIZED	The muBLASLt handle has not been initialized.
MUBLAS_STATUS_INVALID_VALUE	One or more parameters are in conflict, invalid, null when required, or in an unsupported configuration.
MUBLAS_STATUS_NOT_SUPPORTED	The selected device or implementation does not support the configured operation.
MUBLAS_STATUS_ARCH_MISMATCH	The configured operation cannot run on the selected device architecture.
MUBLAS_STATUS_EXECUTION_FAILED	MUSA reported a device execution error.
MUBLAS_STATUS_ALLOC_FAILED	Memory allocation failed.

3.2 muBLASLt Datatypes Reference

This section summarizes the public types that appear in the supported MUSA SDK 5.2 muBLASLt API signatures or descriptor attributes.

Type	Purpose
mublasLtHandle_t	Opaque muBLASLt context handle passed to mublasLtMatmul.
mublasLtMatrixLayout_t	Opaque descriptor for matrix memory layout.
mublasLtMatmulAlgo_t	Semi-opaque algorithm descriptor used by mublasLtMatmul.
mublasLtMatmulDesc_t	Opaque descriptor for matrix multiplication operation details.
mublasLtMatmulDescAttributes_t	Attribute selector used by mublasLtMatmulDescSetAttribute.
mublasLtEmulationDesc_t	Opaque descriptor for floating-point emulation properties.
mublasLtEmulationDescAttributes_t	Attribute selector used by emulation descriptor set and get APIs.
mublasStatus_t	Return type used by the supported muBLASLt APIs.
mublasComputeType_t	Compute-precision selector used by matmul descriptor attributes.
mublasOperation_t	Operation selector used by matmul descriptor attributes to control transpose behavior.
mublasFillMode_t	Fill-mode selector used by matmul descriptor attributes.
mublasLtPointerMode_t	Pointer-location selector used by matmul descriptor attributes for alpha and beta.
mublasLtEpilogue_t	Epilogue selector used by matmul descriptor attributes.
musaDataType_t	Data-type selector used by matrix layouts and descriptor attributes.
musaStream_t	MUSA stream handle passed to mublasLtMatmul.

3.2.1 Matmul Descriptor Attributes

mublasLtMatmulDescAttributes_t selects the matmul descriptor attribute to set. Attribute support can depend on the configured data types, scale types, layouts, device, and epilogue. Unsupported combinations can return MUBLAS_STATUS_INVALID_VALUE or MUBLAS_STATUS_NOT_SUPPORTED.

Attribute	Value type and default	Purpose
MUBLASLT_MATMUL_DESC_COMPUTE_TYPE	int32_t	Selects the data type used for multiply and accumulate operations and the accumulator.
MUBLASLT_MATMUL_DESC_SCALE_TYPE	int32_t; default is the compute type	Selects the data type of alpha and beta.
MUBLASLT_MATMUL_DESC_POINTER_MODE	int32_t; default is MUBLASLT_POINTER_MODE_HOST	Selects the pointer mode for alpha and beta.
MUBLASLT_MATMUL_DESC_TRANSA	int32_t; default is MUBLAS_OP_N	Selects the transform for matrix A.
MUBLASLT_MATMUL_DESC_TRANSB	int32_t; default is MUBLAS_OP_N	Selects the transform for matrix B.
MUBLASLT_MATMUL_DESC_TRANSC	int32_t; default is MUBLAS_OP_N	Selects the transform for matrix C. Currently only MUBLAS_OP_N is supported.
MUBLASLT_MATMUL_DESC_FILL_MODE	int32_t; default is MUBLAS_FILL_MODE_FULL	Selects the matrix fill mode.
MUBLASLT_MATMUL_DESC_EPILOGUE	uint32_t; default is MUBLASLT_EPILOGUE_DEFAULT	Selects the epilogue function.
MUBLASLT_MATMUL_DESC_BIAS_POINTER	const void * or void *; default is NULL	Provides the device pointer for a bias or bias-gradient vector.
MUBLASLT_MATMUL_DESC_BIAS_BATCH_STRIDE	int64_t; default is 0	Provides the batch stride for the bias or bias-gradient vector.
MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_POINTER	void * or const void *; default is NULL	Provides the auxiliary buffer pointer used by supported epilogue modes.
MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_LD	int64_t; default is 0	Provides the leading dimension for the epilogue auxiliary buffer.
MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_BATCH_STRIDE	int64_t; default is 0	Provides the batch stride for the epilogue auxiliary buffer.
MUBLASLT_MATMUL_DESC_ALPHA_VECTOR_BATCH_STRIDE	int64_t; default is 0	Provides the batch stride for alpha-vector pointer modes.
MUBLASLT_MATMUL_DESC_SM_COUNT_TARGET	int32_t; default is 0	Targets a number of SMs for parallel execution. A value of 0 uses the device-reported count.
MUBLASLT_MATMUL_DESC_A_SCALE_POINTER	const void *; default is NULL	Provides the device pointer to the scale factor for matrix A.
MUBLASLT_MATMUL_DESC_B_SCALE_POINTER	const void *; default is NULL	Provides the device pointer to the scale factor for matrix B.
MUBLASLT_MATMUL_DESC_C_SCALE_POINTER	const void *; default is NULL	Provides the device pointer to the scale factor for matrix C.
MUBLASLT_MATMUL_DESC_D_SCALE_POINTER	const void *; default is NULL	Provides the device pointer to the scale factor for matrix D.
MUBLASLT_MATMUL_DESC_AMAX_D_POINTER	void *; default is NULL	Provides the device pointer that receives the maximum absolute value in the output matrix.
MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_DATA_TYPE	int32_t based on musaDataType_t; default is -1	Selects the data type stored in the epilogue auxiliary buffer.
MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_SCALE_POINTER	void *; default is NULL	Provides the device pointer to the scale factor for the auxiliary matrix.
MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_AMAX_POINTER	void *; default is NULL	Provides the device pointer that receives the maximum absolute value in the auxiliary buffer.
MUBLASLT_MATMUL_DESC_FAST_ACCUM	int8_t; default is 0	Enables or disables FP8 fast accumulation.
MUBLASLT_MATMUL_DESC_BIAS_DATA_TYPE	int32_t based on musaDataType_t; default is -1	Selects the data type of the bias or bias-gradient vector.
MUBLASLT_MATMUL_DESC_A_SCALE_MODE	int32_t; default is 0	Selects how the matrix A scale factor is interpreted.
MUBLASLT_MATMUL_DESC_B_SCALE_MODE	int32_t; default is 0	Selects how the matrix B scale factor is interpreted.
MUBLASLT_MATMUL_DESC_C_SCALE_MODE	int32_t; default is 0	Selects how the matrix C scale factor is interpreted.
MUBLASLT_MATMUL_DESC_D_SCALE_MODE	int32_t; default is 0	Selects how the matrix D scale factor is interpreted.
MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_SCALE_MODE	int32_t; default is 0	Selects how the auxiliary matrix scale factor is interpreted.
MUBLASLT_MATMUL_DESC_D_OUT_SCALE_POINTER	void *; default is NULL	Provides the device pointer to the scale factors used to convert matrix D output data.
MUBLASLT_MATMUL_DESC_D_OUT_SCALE_MODE	int32_t; default is 0	Selects how the output matrix D scale factor is interpreted.
MUBLASLT_MATMUL_DESC_EMULATION_DESCRIPTOR	mublasLtEmulationDesc_t; default is NULL	Attaches a floating-point emulation descriptor to the matmul operation.
MUBLASLT_MATMUL_DESC_ALPHA_BATCH_STRIDE	int64_t; default is 0	Provides the batch stride for alpha.
MUBLASLT_MATMUL_DESC_BETA_BATCH_STRIDE	int64_t; default is 0	Provides the batch stride for beta.

3.2.2 Emulation Descriptor Attributes

mublasLtEmulationDescAttributes_t selects the floating-point emulation descriptor attribute to set or query.

Attribute	Value type and default	Purpose
MUBLASLT_EMULATION_DESC_STRATEGY	int32_t; default is 0	Selects when to use floating-point emulation algorithms.
MUBLASLT_EMULATION_DESC_SPECIAL_VALUES_SUPPORT	int32_t; default is 0xFFFF	Selects special floating-point values that must be supported.
MUBLASLT_EMULATION_DESC_FIXEDPOINT_MANTISSA_CONTROL	int32_t; default is 0	Selects how fixed-point emulation computes retained mantissa bits.
MUBLASLT_EMULATION_DESC_FIXEDPOINT_MAX_MANTISSA_BIT_COUNT	int32_t; default is 0	Sets the maximum number of mantissa bits retained during fixed-point emulation.
MUBLASLT_EMULATION_DESC_FIXEDPOINT_MANTISSA_BIT_OFFSET	int32_t; default is 0	Biases the recommended mantissa bit count for dynamic mantissa control.
MUBLASLT_EMULATION_DESC_FIXEDPOINT_MANTISSA_BIT_COUNT_POINTER	int32_t *; default is NULL	Points to device memory that receives the number of mantissa bits retained. If emulation is not used, the value is -1.

3.3 Matrix Multiplication Reference

This section covers the supported matrix multiplication API.

mublasLtMatmul

mublasStatus_t MUBLASWINAPI mublasLtMatmul(
    mublasLtHandle_t lightHandle,
    mublasLtMatmulDesc_t computeDesc,
    const void *alpha,
    const void *A,
    mublasLtMatrixLayout_t Adesc,
    const void *B,
    mublasLtMatrixLayout_t Bdesc,
    const void *beta,
    const void *C,
    mublasLtMatrixLayout_t Cdesc,
    void *D,
    mublasLtMatrixLayout_t Ddesc,
    const mublasLtMatmulAlgo_t *algo,
    void *workspace,
    size_t workspaceSizeInBytes,
    musaStream_t stream)

Description

Executes matrix multiplication:

D = alpha * op(A) * op(B) + beta * C

Parameters

Parameter	Type	Description
lightHandle	mublasLtHandle_t	muBLASLt library context handle.
computeDesc	mublasLtMatmulDesc_t	Matmul operation descriptor.
alpha	const void *	Pointer to the scale factor applied to op(A) * op(B).
A	const void *	Pointer to matrix A data.
Adesc	mublasLtMatrixLayout_t	Matrix layout descriptor for A.
B	const void *	Pointer to matrix B data.
Bdesc	mublasLtMatrixLayout_t	Matrix layout descriptor for B.
beta	const void *	Pointer to the scale factor applied to matrix C.
C	const void *	Pointer to matrix C data.
Cdesc	mublasLtMatrixLayout_t	Matrix layout descriptor for C.
D	void *	Pointer to matrix D output data.
Ddesc	mublasLtMatrixLayout_t	Matrix layout descriptor for D.
algo	const mublasLtMatmulAlgo_t *	Algorithm descriptor for the operation.
workspace	void *	Temporary workspace buffer.
workspaceSizeInBytes	size_t	Size of the workspace buffer, in bytes.
stream	musaStream_t	MUSA stream used to issue the operation.

Returns

Status value	Meaning
MUBLAS_STATUS_NOT_INITIALIZED	The muBLASLt handle has not been initialized.
MUBLAS_STATUS_INVALID_VALUE	Parameters are in conflict or in an impossible configuration. This includes a workspace size smaller than the configured algorithm requires.
MUBLAS_STATUS_NOT_SUPPORTED	The current implementation on the selected device does not support the configured operation.
MUBLAS_STATUS_ARCH_MISMATCH	The configured operation cannot run on the selected device.
MUBLAS_STATUS_EXECUTION_FAILED	MUSA reported a device execution error.
MUBLAS_STATUS_SUCCESS	The operation completed successfully.

3.4 Matmul Descriptor Reference

This section covers the supported API for setting matmul operation descriptor attributes.

mublasLtMatmulDescSetAttribute

mublasStatus_t MUBLASWINAPI mublasLtMatmulDescSetAttribute(
    mublasLtMatmulDesc_t matmulDesc,
    mublasLtMatmulDescAttributes_t attr,
    const void *buf,
    size_t sizeInBytes)

Description

Sets an attribute on a matmul operation descriptor.

Parameters

Parameter	Type	Description
matmulDesc	mublasLtMatmulDesc_t	Matmul operation descriptor.
attr	mublasLtMatmulDescAttributes_t	Attribute to set.
buf	const void *	Pointer to the new attribute value.
sizeInBytes	size_t	Size of the value in buf, in bytes.

Returns

Status value	Meaning
MUBLAS_STATUS_INVALID_VALUE	buf is NULL, or sizeInBytes does not match the internal storage size for the selected attribute.
MUBLAS_STATUS_SUCCESS	The attribute was set successfully.

3.5 Emulation Descriptor Reference

This section covers the supported APIs for creating, destroying, setting, and querying floating-point emulation descriptors.

mublasLtEmulationDescInit_internal

mublasStatus_t MUBLASWINAPI mublasLtEmulationDescInit_internal(
    mublasLtEmulationDesc_t emulationDesc,
    size_t size)

Description

Initializes an emulation descriptor in pre-allocated space.

The current MUSA SDK 5.2 header also defines the inline convenience wrapper mublasLtEmulationDescInit, which forwards sizeof(*emulationDesc) to this exported symbol.

Parameters

Parameter	Type	Description
emulationDesc	mublasLtEmulationDesc_t	Emulation descriptor to initialize.
size	size_t	Size of the pre-allocated descriptor storage, in bytes.

Returns

Status value	Meaning
MUBLAS_STATUS_ALLOC_FAILED	The size of the pre-allocated space is insufficient.
MUBLAS_STATUS_SUCCESS	The descriptor was initialized successfully.

mublasLtEmulationDescCreate

mublasStatus_t MUBLASWINAPI mublasLtEmulationDescCreate(
    mublasLtEmulationDesc_t *emulationDesc)

Description

Creates an emulation descriptor.

Parameters

Parameter	Type	Description
emulationDesc	mublasLtEmulationDesc_t *	Output pointer that receives the created emulation descriptor.

Returns

Status value	Meaning
MUBLAS_STATUS_ALLOC_FAILED	Memory could not be allocated.
MUBLAS_STATUS_SUCCESS	The descriptor was created successfully.

mublasLtEmulationDescDestroy

mublasStatus_t MUBLASWINAPI mublasLtEmulationDescDestroy(
    mublasLtEmulationDesc_t emulationDesc)

Description

Destroys an emulation descriptor.

Parameters

Parameter	Type	Description
emulationDesc	mublasLtEmulationDesc_t	Emulation descriptor to destroy.

Returns

Status value	Meaning
MUBLAS_STATUS_SUCCESS	The operation completed successfully.

mublasLtEmulationDescSetAttribute

mublasStatus_t MUBLASWINAPI mublasLtEmulationDescSetAttribute(
    mublasLtEmulationDesc_t emulationDesc,
    mublasLtEmulationDescAttributes_t attr,
    const void *buf,
    size_t sizeInBytes)

Description

Sets an attribute on an emulation descriptor.

Parameters

Parameter	Type	Description
emulationDesc	mublasLtEmulationDesc_t	Emulation descriptor.
attr	mublasLtEmulationDescAttributes_t	Attribute to set.
buf	const void *	Pointer to the new attribute value.
sizeInBytes	size_t	Size of the value in buf, in bytes.

Returns

Status value	Meaning
MUBLAS_STATUS_INVALID_VALUE	buf is NULL, or sizeInBytes does not match the internal storage size for the selected attribute.
MUBLAS_STATUS_SUCCESS	The attribute was set successfully.

mublasLtEmulationDescGetAttribute

mublasStatus_t MUBLASWINAPI mublasLtEmulationDescGetAttribute(
    mublasLtEmulationDesc_t emulationDesc,
    mublasLtEmulationDescAttributes_t attr,
    void *buf,
    size_t sizeInBytes,
    size_t *sizeWritten)

Description

Gets an attribute from an emulation descriptor.

Parameters

Parameter	Type	Description
emulationDesc	mublasLtEmulationDesc_t	Emulation descriptor.
attr	mublasLtEmulationDescAttributes_t	Attribute to query.
buf	void *	Destination buffer that receives the attribute value.
sizeInBytes	size_t	Size of the destination buffer, in bytes.
sizeWritten	size_t *	Output pointer that receives the number of bytes written when sizeInBytes is non-zero, or the number of bytes required when sizeInBytes is 0.

Returns

Status value	Meaning
MUBLAS_STATUS_INVALID_VALUE	sizeInBytes is 0 and sizeWritten is NULL, or sizeInBytes is non-zero and buf is NULL, or sizeInBytes does not match the internal storage size for the selected attribute.
MUBLAS_STATUS_SUCCESS	The attribute value was written to user memory successfully.