muBLASLt API Reference

1 Introduction

The muBLASLt library is a new lightweight library dedicated to GEneral Matrix-to-matrix Multiply (GEMM) operations with a new flexible API. This new library adds flexibility in matrix data layouts, input types, compute types, and also in choosing the algorithmic implementations and heuristics through parameter programmability.

2 Module Index

2.1 Modules

Here is a list of all modules:

Date Types
muBLASLt Library Functions

3 Class Index

3.1 Class List

Here are the classes, structs, unions and interfaces with brief descriptions:

   - mublasLtMatmulAlgo_t
   - mublasLtMatmulDescOpaque_t
   - mublasLtMatmulHeuristicResult_t
   - mublasLtMatmulPreferenceOpaque_t
   - mublasLtMatrixLayoutOpaque_t
   - mublasLtMatrixTransformDescOpaque_t

4 Class Documentation

4.1 mublasLtMatmulAlgo_t Struct Reference

#include <mublasLt.h>

Public Attributes

   - uint64_t data [8]

4.1.1 Detailed Description

Semi-opaque algorithm descriptor (to avoid complicated alloc/free schemes)

This structure can be trivially serialized and later restored for use with the same version of mublas library to save on selecting the right configuration again.

4.1.2 Member Data Documentation

4.1.2.1 data

uint64_t mublasLtMatmulAlgo_t::data[8]

The documentation for this struct was generated from the following file:

   - mublasLt.h

4.2 mublasLtMatmulDescOpaque_t Struct Reference

#include <mublasLt.h>

Public Attributes

   - uint64_t data [23]

4.2.1 Detailed Description

Semi-opaque descriptor for mublasLtMatmul() operation details

4.2.2 Member Data Documentation

4.2.2.1 data

uint64_t mublasLtMatmulDescOpaque_t::data[23]

The documentation for this struct was generated from the following file:

• mublasLt.h

4.3 mublasLtMatmulHeuristicResult_t Struct Reference

#include <mublasLt.h>

Public Attributes

   - mublasLtMatmulAlgo_t algo
   - size_t workspaceSize
   - mublasStatus_t state
   - float wavesCount
   - int reserved [4]

4.3.1 Detailed Description

Results structure used by mublasLtMatmulGetAlgo. Holds returned configured algo descriptor and its runtime properties.

4.3.2 Member Data Documentation

4.3.2.1 algo

mublasLtMatmulAlgo_t mublasLtMatmulHeuristicResult_t::algo

Matmul algorithm descriptor. Must be initialized with mublasLtMatmulAlgoInit() if preferences' MUBLASLT_MATMUL_PERF_SEARCH_MODE is set to MUBLASLT_SEARCH_LIMITED_BY_ALGO_ID

4.3.2.2 reserved

int mublasLtMatmulHeuristicResult_t::reserved[4]

4.3.2.3 state

mublasStatus_t mublasLtMatmulHeuristicResult_t::state Result status, other fields are only valid if after call to mublasLtMatmulAlgoGetHeuristic() this member is set to MUBLAS_STATUS_SUCCESS.

4.3.2.4 wavesCount

float mublasLtMatmulHeuristicResult_t::wavesCount

Waves count - a device utilization metric. wavesCount value of 1.0f suggests that when kernel is launched it will fully occupy the GPU.

4.3.2.5 workspaceSize

size_t mublasLtMatmulHeuristicResult_t::workspaceSize

Actual size of workspace memory required. The documentation for this struct was generated from the following file:

   - mublasLt.h

4.4 mublasLtMatmulPreferenceOpaque_t Struct Reference

#include <mublasLt.h>

Public Attributes

   - uint64_t data [8]

4.4.1 Detailed Description

Semi-opaque descriptor for mublasLtMatmulPreference() operation details

4.4.2 Member Data Documentation

4.4.2.1 data

uint64_t mublasLtMatmulPreferenceOpaque_t::data[8] The documentation for this struct was generated from the following file:

   - mublasLt.h

4.5 mublasLtMatrixLayoutOpaque_t Struct Reference

#include <mublasLt.h>

Public Attributes

   - uint64_t data [8]

4.5.1 Detailed Description

Semi-opaque descriptor for matrix memory layout

4.5.2 Member Data Documentation

4.5.2.1 data

uint64_t mublasLtMatrixLayoutOpaque_t::data[8] The documentation for this struct was generated from the following file:

   - mublasLt.h

4.6 mublasLtMatrixTransformDescOpaque_t Struct Reference

#include <mublasLt.h>

Public Attributes

   - uint64_t data [8]

4.6.1 Detailed Description

Semi-opaque descriptor for mublasLtMatrixTransform() operation details

4.6.2 Member Data Documentation

4.6.2.1 data

uint64_t mublasLtMatrixTransformDescOpaque_t::data[8] The documentation for this struct was generated from the following file:

   - mublasLt.h

5 File Documentation

5.1 mublasLt.h File Reference

#include <mublas.h>
#include <stdint.h>
#include <stddef.h>
#include <stdio.h>

Classes

• struct mublasLtMatrixLayoutOpaque_t
• struct mublasLtMatmulAlgo_t
• struct mublasLtMatmulDescOpaque_t
• struct mublasLtMatrixTransformDescOpaque_t
• struct mublasLtMatmulPreferenceOpaque_t
• struct mublasLtMatmulHeuristicResult_t

Macros

• #define MUBLASAPI
• #define MUBLASWINAPI
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_FMA (0x01ull << 0)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_HMMA (0x02ull << 0)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_IMMA (0x04ull << 0)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_DMMA (0x08ull << 0)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_TENSOR_OP_MASK (0xfeull << 0)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_OP_TYPE_MASK (0xffull << 0)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_16F (0x01ull << 8)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_32F (0x02ull << 8)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_64F (0x04ull << 8)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_32I (0x08ull << 8)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_TYPE_MASK (0xffull << 8)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_16F (0x01ull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_16BF (0x02ull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_TF32 (0x04ull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_32F (0x08ull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_64F (0x10ull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8I (0x20ull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8F_E4M3 (0x40ull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8F_E5M2 (0x80ull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_OP_INPUT_TYPE_MASK (0xffull << 16)
• #define MUBLASLT_NUMERICAL_IMPL_FLAGS_GAUSSIAN (0x01ull << 32)

Typedefs

• typedef struct mublasLtContext ∗ mublasLtHandle_t
• typedef mublasLtMatrixLayoutOpaque_t ∗ mublasLtMatrixLayout_t
• typedef mublasLtMatmulDescOpaque_t ∗ mublasLtMatmulDesc_t
• typedef mublasLtMatrixTransformDescOpaque_t ∗ mublasLtMatrixTransformDesc_t
• typedef mublasLtMatmulPreferenceOpaque_t ∗ mublasLtMatmulPreference_t
• typedef uint64_t mublasLtNumericalImplFlags_t
• typedef void(∗ mublasLtLoggerCallback_t) (int logLevel, const char ∗functionName, const char ∗message)

Enumerations

   - enum mublasLtMatmulTile_t {
        MUBLASLT_MATMUL_TILE_UNDEFINED = 0 ,
        MUBLASLT_MATMUL_TILE_8x8 = 1 ,
        MUBLASLT_MATMUL_TILE_8x16 = 2 ,
        MUBLASLT_MATMUL_TILE_16x8 = 3 ,
        MUBLASLT_MATMUL_TILE_8x32 = 4 ,
        MUBLASLT_MATMUL_TILE_16x16 = 5 ,
        MUBLASLT_MATMUL_TILE_32x8 = 6 ,
        MUBLASLT_MATMUL_TILE_8x64 = 7 ,
        MUBLASLT_MATMUL_TILE_16x32 = 8 ,
        MUBLASLT_MATMUL_TILE_32x16 = 9 ,
        MUBLASLT_MATMUL_TILE_64x8 = 10 ,
        MUBLASLT_MATMUL_TILE_32x32 = 11 ,
        MUBLASLT_MATMUL_TILE_32x64 = 12 ,
        MUBLASLT_MATMUL_TILE_64x32 = 13 ,
        MUBLASLT_MATMUL_TILE_32x128 = 14 ,
        MUBLASLT_MATMUL_TILE_64x64 = 15 ,
        MUBLASLT_MATMUL_TILE_128x32 = 16 ,
        MUBLASLT_MATMUL_TILE_64x128 = 17 ,
        MUBLASLT_MATMUL_TILE_128x64 = 18 ,
        MUBLASLT_MATMUL_TILE_64x256 = 19 ,
        MUBLASLT_MATMUL_TILE_128x128 = 20 ,
        MUBLASLT_MATMUL_TILE_256x64 = 21 ,
        MUBLASLT_MATMUL_TILE_64x512 = 22 ,
        MUBLASLT_MATMUL_TILE_128x256 = 23 ,
        MUBLASLT_MATMUL_TILE_256x128 = 24 ,
        MUBLASLT_MATMUL_TILE_512x64 = 25 ,
        MUBLASLT_MATMUL_TILE_64x96 = 26 ,
        MUBLASLT_MATMUL_TILE_96x64 = 27 ,
        MUBLASLT_MATMUL_TILE_96x128 = 28 ,
        MUBLASLT_MATMUL_TILE_128x160 = 29 ,
        MUBLASLT_MATMUL_TILE_160x128 = 30 ,
        MUBLASLT_MATMUL_TILE_192x128 = 31 ,
        MUBLASLT_MATMUL_TILE_128x192 = 32 ,
        MUBLASLT_MATMUL_TILE_128x96 = 33 ,
        MUBLASLT_MATMUL_TILE_END }
   - enum mublasLtMatmulStages_t {
        MUBLASLT_MATMUL_STAGES_UNDEFINED = 0 ,
        MUBLASLT_MATMUL_STAGES_16x1 = 1 ,
        MUBLASLT_MATMUL_STAGES_16x2 = 2 ,
        MUBLASLT_MATMUL_STAGES_16x3 = 3 ,
        MUBLASLT_MATMUL_STAGES_16x4 = 4 ,
        MUBLASLT_MATMUL_STAGES_16x5 = 5 ,
        MUBLASLT_MATMUL_STAGES_16x6 = 6 ,
        MUBLASLT_MATMUL_STAGES_32x1 = 7 ,
        MUBLASLT_MATMUL_STAGES_32x2 = 8 ,
        MUBLASLT_MATMUL_STAGES_32x3 = 9 ,
        MUBLASLT_MATMUL_STAGES_32x4 = 10 ,
        MUBLASLT_MATMUL_STAGES_32x5 = 11 ,
        MUBLASLT_MATMUL_STAGES_32x6 = 12 ,
        MUBLASLT_MATMUL_STAGES_64x1 = 13 ,
        MUBLASLT_MATMUL_STAGES_64x2 = 14 ,
        MUBLASLT_MATMUL_STAGES_64x3 = 15 ,
        MUBLASLT_MATMUL_STAGES_64x4 = 16 ,
        MUBLASLT_MATMUL_STAGES_64x5 = 17 ,
        MUBLASLT_MATMUL_STAGES_64x6 = 18 ,
        MUBLASLT_MATMUL_STAGES_128x1 = 19 ,
        MUBLASLT_MATMUL_STAGES_128x2 = 20 ,
        MUBLASLT_MATMUL_STAGES_128x3 = 21 ,
        MUBLASLT_MATMUL_STAGES_128x4 = 22 ,
        MUBLASLT_MATMUL_STAGES_128x5 = 23 ,
        MUBLASLT_MATMUL_STAGES_128x6 = 24 ,
        MUBLASLT_MATMUL_STAGES_32x10 = 25 ,
        MUBLASLT_MATMUL_STAGES_8x4 = 26 ,
        MUBLASLT_MATMUL_STAGES_16x10 = 27 ,
        MUBLASLT_MATMUL_STAGES_8x5 = 28 ,
        MUBLASLT_MATMUL_STAGES_8x3 = 31 ,
        MUBLASLT_MATMUL_STAGES_8xAUTO = 32 ,
        MUBLASLT_MATMUL_STAGES_16xAUTO = 33 ,
        MUBLASLT_MATMUL_STAGES_32xAUTO = 34 ,
        MUBLASLT_MATMUL_STAGES_64xAUTO = 35 ,
        MUBLASLT_MATMUL_STAGES_128xAUTO = 36 ,
        MUBLASLT_MATMUL_STAGES_END }
   - enum mublasLtClusterShape_t {
        MUBLASLT_CLUSTER_SHAPE_AUTO = 0 ,
        MUBLASLT_CLUSTER_SHAPE_1x1x1 = 2 ,
        MUBLASLT_CLUSTER_SHAPE_2x1x1 = 3 ,
        MUBLASLT_CLUSTER_SHAPE_4x1x1 = 4 ,
        MUBLASLT_CLUSTER_SHAPE_1x2x1 = 5 ,
        MUBLASLT_CLUSTER_SHAPE_2x2x1 = 6 ,
        MUBLASLT_CLUSTER_SHAPE_4x2x1 = 7 ,
        MUBLASLT_CLUSTER_SHAPE_1x4x1 = 8 ,
        MUBLASLT_CLUSTER_SHAPE_2x4x1 = 9 ,
        MUBLASLT_CLUSTER_SHAPE_4x4x1 = 10 ,
        MUBLASLT_CLUSTER_SHAPE_8x1x1 = 11 ,
        MUBLASLT_CLUSTER_SHAPE_1x8x1 = 12 ,
        MUBLASLT_CLUSTER_SHAPE_8x2x1 = 13 ,
        MUBLASLT_CLUSTER_SHAPE_2x8x1 = 14 ,
        MUBLASLT_CLUSTER_SHAPE_16x1x1 = 15 ,
        MUBLASLT_CLUSTER_SHAPE_1x16x1 = 16 ,
        MUBLASLT_CLUSTER_SHAPE_3x1x1 = 17 ,
        MUBLASLT_CLUSTER_SHAPE_5x1x1 = 18 ,
        MUBLASLT_CLUSTER_SHAPE_6x1x1 = 19 ,
        MUBLASLT_CLUSTER_SHAPE_7x1x1 = 20 ,
        MUBLASLT_CLUSTER_SHAPE_9x1x1 = 21 ,
        MUBLASLT_CLUSTER_SHAPE_10x1x1 = 22 ,
        MUBLASLT_CLUSTER_SHAPE_11x1x1 = 23 ,
        MUBLASLT_CLUSTER_SHAPE_12x1x1 = 24 ,
        MUBLASLT_CLUSTER_SHAPE_13x1x1 = 25 ,
        MUBLASLT_CLUSTER_SHAPE_14x1x1 = 26 ,
        MUBLASLT_CLUSTER_SHAPE_15x1x1 = 27 ,
        MUBLASLT_CLUSTER_SHAPE_3x2x1 = 28 ,
        MUBLASLT_CLUSTER_SHAPE_5x2x1 = 29 ,
        MUBLASLT_CLUSTER_SHAPE_6x2x1 = 30 ,
        MUBLASLT_CLUSTER_SHAPE_7x2x1 = 31 ,
        MUBLASLT_CLUSTER_SHAPE_1x3x1 = 32 ,
        MUBLASLT_CLUSTER_SHAPE_2x3x1 = 33 ,
        MUBLASLT_CLUSTER_SHAPE_3x3x1 = 34 ,
        MUBLASLT_CLUSTER_SHAPE_4x3x1 = 35 ,
        MUBLASLT_CLUSTER_SHAPE_5x3x1 = 36 ,
        MUBLASLT_CLUSTER_SHAPE_3x4x1 = 37 ,
        MUBLASLT_CLUSTER_SHAPE_1x5x1 = 38 ,
        MUBLASLT_CLUSTER_SHAPE_2x5x1 = 39 ,
        MUBLASLT_CLUSTER_SHAPE_3x5x1 = 40 ,
        MUBLASLT_CLUSTER_SHAPE_1x6x1 = 41 ,
        MUBLASLT_CLUSTER_SHAPE_2x6x1 = 42 ,
        MUBLASLT_CLUSTER_SHAPE_1x7x1 = 43 ,
        MUBLASLT_CLUSTER_SHAPE_2x7x1 = 44 ,
        MUBLASLT_CLUSTER_SHAPE_1x9x1 = 45 ,
        MUBLASLT_CLUSTER_SHAPE_1x10x1 = 46 ,
        MUBLASLT_CLUSTER_SHAPE_1x11x1 = 47 ,
        MUBLASLT_CLUSTER_SHAPE_1x12x1 = 48 ,
        MUBLASLT_CLUSTER_SHAPE_1x13x1 = 49 ,
        MUBLASLT_CLUSTER_SHAPE_1x14x1 = 50 ,
        MUBLASLT_CLUSTER_SHAPE_1x15x1 = 51 ,
        MUBLASLT_CLUSTER_SHAPE_END }
   - enum mublasLtMatmulInnerShape_t {
        MUBLASLT_MATMUL_INNER_SHAPE_UNDEFINED = 0 ,
        MUBLASLT_MATMUL_INNER_SHAPE_MMA884 = 1 ,
        MUBLASLT_MATMUL_INNER_SHAPE_MMA1684 = 2 ,
        MUBLASLT_MATMUL_INNER_SHAPE_MMA1688 = 3 ,
        MUBLASLT_MATMUL_INNER_SHAPE_MMA16816 = 4 ,
        MUBLASLT_MATMUL_INNER_SHAPE_END }
   - enum mublasLtPointerMode_t {
        MUBLASLT_POINTER_MODE_HOST = MUBLAS_POINTER_MODE_HOST ,
        MUBLASLT_POINTER_MODE_DEVICE = MUBLAS_POINTER_MODE_DEVICE ,
        MUBLASLT_POINTER_MODE_DEVICE_VECTOR = 2 ,
        MUBLASLT_POINTER_MODE_ALPHA_DEVICE_VECTOR_BETA_ZERO = 3 ,
        MUBLASLT_POINTER_MODE_ALPHA_DEVICE_VECTOR_BETA_HOST = 4 }
   - enum mublasLtPointerModeMask_t {
        MUBLASLT_POINTER_MODE_MASK_HOST = 1 ,
        MUBLASLT_POINTER_MODE_MASK_DEVICE = 2 ,
        MUBLASLT_POINTER_MODE_MASK_DEVICE_VECTOR = 4 ,
        MUBLASLT_POINTER_MODE_MASK_ALPHA_DEVICE_VECTOR_BETA_ZERO = 8 ,
        MUBLASLT_POINTER_MODE_MASK_ALPHA_DEVICE_VECTOR_BETA_HOST = 16 }
   - enum mublasLtOrder_t {
        MUBLASLT_ORDER_COL = 0 ,
        MUBLASLT_ORDER_ROW = 1 ,
        MUBLASLT_ORDER_COL32 = 2 ,
        MUBLASLT_ORDER_COL4_4R2_8C = 3 ,
        MUBLASLT_ORDER_COL32_2R_4R4 = 4 }
   - enum mublasLtMatrixLayoutAttribute_t {
        MUBLASLT_MATRIX_LAYOUT_TYPE = 0 ,
        MUBLASLT_MATRIX_LAYOUT_ORDER = 1 ,
        MUBLASLT_MATRIX_LAYOUT_ROWS = 2 ,
        MUBLASLT_MATRIX_LAYOUT_COLS = 3 ,
        MUBLASLT_MATRIX_LAYOUT_LD = 4 ,
        MUBLASLT_MATRIX_LAYOUT_BATCH_COUNT = 5 ,
        MUBLASLT_MATRIX_LAYOUT_STRIDED_BATCH_OFFSET = 6 ,
        MUBLASLT_MATRIX_LAYOUT_PLANE_OFFSET = 7 }
   - enum mublasLtMatmulDescAttributes_t {
        MUBLASLT_MATMUL_DESC_COMPUTE_TYPE = 0 ,
        MUBLASLT_MATMUL_DESC_SCALE_TYPE = 1 ,
        MUBLASLT_MATMUL_DESC_POINTER_MODE = 2 ,
        MUBLASLT_MATMUL_DESC_TRANSA = 3 ,
        MUBLASLT_MATMUL_DESC_TRANSB = 4 ,
        MUBLASLT_MATMUL_DESC_TRANSC = 5 ,
        MUBLASLT_MATMUL_DESC_FILL_MODE = 6 ,
        MUBLASLT_MATMUL_DESC_EPILOGUE = 7 ,
        MUBLASLT_MATMUL_DESC_BIAS_POINTER = 8 ,
        MUBLASLT_MATMUL_DESC_BIAS_BATCH_STRIDE = 10 ,
        MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_POINTER = 11 ,
        MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_LD = 12 ,
        MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_BATCH_STRIDE = 13 ,
        MUBLASLT_MATMUL_DESC_ALPHA_VECTOR_BATCH_STRIDE = 14 ,
        MUBLASLT_MATMUL_DESC_SM_COUNT_TARGET = 15 ,
        MUBLASLT_MATMUL_DESC_A_SCALE_POINTER = 17 ,
        MUBLASLT_MATMUL_DESC_B_SCALE_POINTER = 18 ,
        MUBLASLT_MATMUL_DESC_C_SCALE_POINTER = 19 ,
        MUBLASLT_MATMUL_DESC_D_SCALE_POINTER = 20 ,
        MUBLASLT_MATMUL_DESC_AMAX_D_POINTER = 21 ,
        MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_DATA_TYPE = 22 ,
        MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_SCALE_POINTER = 23 ,
        MUBLASLT_MATMUL_DESC_EPILOGUE_AUX_AMAX_POINTER = 24 ,
        MUBLASLT_MATMUL_DESC_FAST_ACCUM = 25 ,
        MUBLASLT_MATMUL_DESC_BIAS_DATA_TYPE = 26 }
   - enum mublasLtMatrixTransformDescAttributes_t {
        MUBLASLT_MATRIX_TRANSFORM_DESC_SCALE_TYPE ,
        MUBLASLT_MATRIX_TRANSFORM_DESC_POINTER_MODE ,
        MUBLASLT_MATRIX_TRANSFORM_DESC_TRANSA ,
        MUBLASLT_MATRIX_TRANSFORM_DESC_TRANSB }
   - enum mublasLtReductionScheme_t {
        MUBLASLT_REDUCTION_SCHEME_NONE = 0 ,
        MUBLASLT_REDUCTION_SCHEME_INPLACE = 1 ,
        MUBLASLT_REDUCTION_SCHEME_COMPUTE_TYPE = 2 ,
        MUBLASLT_REDUCTION_SCHEME_OUTPUT_TYPE = 4 ,
        MUBLASLT_REDUCTION_SCHEME_MASK = 0x7 }
   - enum mublasLtEpilogue_t {
        MUBLASLT_EPILOGUE_DEFAULT = 1 ,
        MUBLASLT_EPILOGUE_RELU = 2 ,
        MUBLASLT_EPILOGUE_RELU_AUX = (MUBLASLT_EPILOGUE_RELU | 128) ,
        MUBLASLT_EPILOGUE_BIAS = 4 ,
        MUBLASLT_EPILOGUE_RELU_BIAS = (MUBLASLT_EPILOGUE_RELU | MUBLASLT_EPILOGUE_BIAS) ,
        MUBLASLT_EPILOGUE_RELU_AUX_BIAS = (MUBLASLT_EPILOGUE_RELU_AUX | MUBLASLT_EPILOGUE_BIAS) ,
        MUBLASLT_EPILOGUE_DRELU = 8 | 128 ,
        MUBLASLT_EPILOGUE_DRELU_BGRAD = MUBLASLT_EPILOGUE_DRELU | 16 ,
        MUBLASLT_EPILOGUE_GELU = 32 ,
        MUBLASLT_EPILOGUE_GELU_AUX = (MUBLASLT_EPILOGUE_GELU | 128) ,
        MUBLASLT_EPILOGUE_GELU_BIAS = (MUBLASLT_EPILOGUE_GELU | MUBLASLT_EPILOGUE_BIAS)
        ,
        MUBLASLT_EPILOGUE_GELU_AUX_BIAS = (MUBLASLT_EPILOGUE_GELU_AUX | MUBLASLT_EPILOGUE_BIAS) ,
        MUBLASLT_EPILOGUE_DGELU = 64 | 128 ,
        MUBLASLT_EPILOGUE_DGELU_BGRAD = MUBLASLT_EPILOGUE_DGELU | 16 ,
        MUBLASLT_EPILOGUE_BGRADA = 256 ,
        MUBLASLT_EPILOGUE_BGRADB = 512 }
   - enum mublasLtMatmulSearch_t {
        MUBLASLT_SEARCH_BEST_FIT = 0 ,
        MUBLASLT_SEARCH_LIMITED_BY_ALGO_ID = 1 ,
        MUBLASLT_SEARCH_RESERVED_02 = 2 ,
        MUBLASLT_SEARCH_RESERVED_03 = 3 ,
        MUBLASLT_SEARCH_RESERVED_04 = 4 ,
        MUBLASLT_SEARCH_RESERVED_05 = 5 }
   - enum mublasLtMatmulPreferenceAttributes_t {
        MUBLASLT_MATMUL_PREF_SEARCH_MODE = 0 ,
        MUBLASLT_MATMUL_PREF_MAX_WORKSPACE_BYTES = 1 ,
        MUBLASLT_MATMUL_PREF_REDUCTION_SCHEME_MASK = 3 ,
        MUBLASLT_MATMUL_PREF_MIN_ALIGNMENT_A_BYTES = 5 ,
        MUBLASLT_MATMUL_PREF_MIN_ALIGNMENT_B_BYTES = 6 ,
        MUBLASLT_MATMUL_PREF_MIN_ALIGNMENT_C_BYTES = 7 ,
        MUBLASLT_MATMUL_PREF_MIN_ALIGNMENT_D_BYTES = 8 ,
        MUBLASLT_MATMUL_PREF_MAX_WAVES_COUNT = 9 ,
        MUBLASLT_MATMUL_PREF_IMPL_MASK = 12 }
   - enum mublasLtMatmulAlgoCapAttributes_t {
        MUBLASLT_ALGO_CAP_SPLITK_SUPPORT = 0 ,
        MUBLASLT_ALGO_CAP_REDUCTION_SCHEME_MASK = 1 ,
        MUBLASLT_ALGO_CAP_CTA_SWIZZLING_SUPPORT = 2 ,
        MUBLASLT_ALGO_CAP_STRIDED_BATCH_SUPPORT = 3 ,
        MUBLASLT_ALGO_CAP_OUT_OF_PLACE_RESULT_SUPPORT = 4 ,
        MUBLASLT_ALGO_CAP_UPLO_SUPPORT = 5 ,
        MUBLASLT_ALGO_CAP_TILE_IDS = 6 ,
        MUBLASLT_ALGO_CAP_CUSTOM_OPTION_MAX = 7 ,
        MUBLASLT_ALGO_CAP_CUSTOM_MEMORY_ORDER = 10 ,
        MUBLASLT_ALGO_CAP_POINTER_MODE_MASK = 11 ,
        MUBLASLT_ALGO_CAP_EPILOGUE_MASK = 12 ,
        MUBLASLT_ALGO_CAP_STAGES_IDS = 13 ,
        MUBLASLT_ALGO_CAP_LD_NEGATIVE = 14 ,
        MUBLASLT_ALGO_CAP_NUMERICAL_IMPL_FLAGS = 15 ,
        MUBLASLT_ALGO_CAP_MIN_ALIGNMENT_A_BYTES = 16 ,
        MUBLASLT_ALGO_CAP_MIN_ALIGNMENT_B_BYTES = 17 ,
        MUBLASLT_ALGO_CAP_MIN_ALIGNMENT_C_BYTES = 18 ,
        MUBLASLT_ALGO_CAP_MIN_ALIGNMENT_D_BYTES = 19 }
   - enum mublasLtMatmulAlgoConfigAttributes_t {
        MUBLASLT_ALGO_CONFIG_ID = 0 ,
        MUBLASLT_ALGO_CONFIG_TILE_ID = 1 ,
        MUBLASLT_ALGO_CONFIG_SPLITK_NUM = 2 ,
        MUBLASLT_ALGO_CONFIG_REDUCTION_SCHEME = 3 ,
        MUBLASLT_ALGO_CONFIG_CTA_SWIZZLING = 4 ,
        MUBLASLT_ALGO_CONFIG_CUSTOM_OPTION = 5 ,
        MUBLASLT_ALGO_CONFIG_STAGES_ID = 6 ,
        MUBLASLT_ALGO_CONFIG_INNER_SHAPE_ID = 7 ,
        MUBLASLT_ALGO_CONFIG_CLUSTER_SHAPE_ID = 8 }

Functions

• mublasStatus_t MUBLASWINAPI mublasLtCreate (mublasLtHandle_t ∗lightHandle)
• mublasStatus_t MUBLASWINAPI mublasLtDestroy (mublasLtHandle_t lightHandle)
• const char ∗MUBLASWINAPI mublasLtGetStatusName (mublasStatus_t status)
• const char ∗MUBLASWINAPI mublasLtGetStatusString (mublasStatus_t status)
• size_t MUBLASWINAPI mublasLtGetVersion (void)
• size_t MUBLASWINAPI mublasLtGetCudartVersion (void)
• mublasStatus_t MUBLASWINAPI mublasLtGetProperty (libraryPropertyType type, int ∗value)
• mublasStatus_t MUBLASWINAPI mublasLtHeuristicsCacheGetCapacity (size_t ∗capacity)
• mublasStatus_t MUBLASWINAPI mublasLtHeuristicsCacheSetCapacity (size_t capacity)
• mublasStatus_t MUBLASWINAPI mublasLtMatmul (mublasLtHandle_t lightHandle, mublasLtMatmulDesc_t computeDesc, const void ∗alpha, const void ∗A, mublasLtMatrixLayout_t Adesc, const void ∗B, mublasLtMatrixLayout_t Bdesc, const void ∗beta, const void ∗C, mublasLtMatrixLayout_t Cdesc, void ∗D, mublasLtMatrixLayout_t Ddesc, const mublasLtMatmulAlgo_t ∗algo, void ∗workspace, size_t workspace←- SizeInBytes, musaStream_t stream)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixTransform (mublasLtHandle_t lightHandle, mublasLtMatrixTransformDesc_t transformDesc, const void ∗alpha, const void ∗A, mublasLtMatrixLayout_t Adesc, const void ∗beta, const void ∗B, mublasLtMatrixLayout_t Bdesc, void ∗C, mublasLtMatrixLayout_t Cdesc, musaStream_t stream)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutInit_internal (mublasLtMatrixLayout_t matLayout, size_t size, musaDataType type, uint64_t rows, uint64_t cols, int64_t ld)
• static mublasStatus_t mublasLtMatrixLayoutInit (mublasLtMatrixLayout_t matLayout, musaDataType type, uint64_t rows, uint64_t cols, int64_t ld)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutCreate (mublasLtMatrixLayout_t ∗matLayout, musaDataType type, uint64_t rows, uint64_t cols, int64_t ld)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutDestroy (mublasLtMatrixLayout_t matLayout)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutSetAttribute (mublasLtMatrixLayout_t matLayout, mublasLtMatrixLayoutAttribute_t attr, const void ∗buf, size_t sizeInBytes)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutGetAttribute (mublasLtMatrixLayout_t matLayout, mublasLtMatrixLayoutAttribute_t attr, void ∗buf, size_t sizeInBytes, size_t ∗sizeWritten)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulDescInit_internal (mublasLtMatmulDesc_t matmulDesc, size_t size, mublasComputeType_t computeType, musaDataType_t scaleType)
• static mublasStatus_t mublasLtMatmulDescInit (mublasLtMatmulDesc_t matmulDesc, mublasCompute←- Type_t computeType, musaDataType_t scaleType)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulDescCreate (mublasLtMatmulDesc_t ∗matmulDesc, mublasComputeType_t computeType, musaDataType_t scaleType)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulDescDestroy (mublasLtMatmulDesc_t matmulDesc)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulDescSetAttribute (mublasLtMatmulDesc_t matmulDesc, mublasLtMatmulDescAttributes_t attr, const void ∗buf, size_t sizeInBytes) • mublasStatus_t MUBLASWINAPI mublasLtMatmulDescGetAttribute (mublasLtMatmulDesc_t matmulDesc, mublasLtMatmulDescAttributes_t attr, void ∗buf, size_t sizeInBytes, size_t ∗sizeWritten)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescInit_internal (mublasLtMatrixTransformDesc_t transformDesc, size_t size, musaDataType scaleType)
• static mublasStatus_t mublasLtMatrixTransformDescInit (mublasLtMatrixTransformDesc_t transformDesc, musaDataType scaleType)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescCreate (mublasLtMatrixTransformDesc_t ∗transformDesc, musaDataType scaleType)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescDestroy (mublasLtMatrixTransformDesc_t transformDesc)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescSetAttribute (mublasLtMatrixTransformDesc_t transformDesc, mublasLtMatrixTransformDescAttributes_t attr, const void ∗buf, size_t sizeInBytes)
• mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescGetAttribute (mublasLtMatrixTransformDesc_t transformDesc, mublasLtMatrixTransformDescAttributes_t attr, void ∗buf, size_t sizeInBytes, size_t ∗size←- Written)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceInit_internal (mublasLtMatmulPreference_t pref, size_t size)
• static mublasStatus_t mublasLtMatmulPreferenceInit (mublasLtMatmulPreference_t pref)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceCreate (mublasLtMatmulPreference_t ∗pref)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceDestroy (mublasLtMatmulPreference_t pref)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceSetAttribute (mublasLtMatmulPreference_t pref, mublasLtMatmulPreferenceAttributes_t attr, const void ∗buf, size_t sizeInBytes)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceGetAttribute (mublasLtMatmulPreference_t pref, mublasLtMatmulPreferenceAttributes_t attr, void ∗buf, size_t sizeInBytes, size_t ∗sizeWritten)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoGetHeuristic (mublasLtHandle_t lightHandle, mublasLtMatmulDesc_t operationDesc, mublasLtMatrixLayout_t Adesc, mublasLtMatrixLayout_t Bdesc, mublasLtMatrixLayout_t Cdesc, mublasLtMatrixLayout_t Ddesc, mublasLtMatmulPreference_t preference, int requestedAlgoCount, mublasLtMatmulHeuristicResult_t heuristicResultsArray[ ], int ∗returnAlgoCount)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoGetIds (mublasLtHandle_t lightHandle, mublas←- ComputeType_t computeType, musaDataType_t scaleType, musaDataType_t Atype, musaDataType_t Btype, musaDataType_t Ctype, musaDataType_t Dtype, int requestedAlgoCount, int algoIdsArray[ ], int ∗return←- AlgoCount)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoInit (mublasLtHandle_t lightHandle, mublas←- ComputeType_t computeType, musaDataType_t scaleType, musaDataType_t Atype, musaDataType_t Btype, musaDataType_t Ctype, musaDataType_t Dtype, int algoId, mublasLtMatmulAlgo_t ∗algo)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoCheck (mublasLtHandle_t lightHandle, mublasLtMatmulDesc_t operationDesc, mublasLtMatrixLayout_t Adesc, mublasLtMatrixLayout_t Bdesc, mublasLtMatrixLayout_t Cdesc, mublasLtMatrixLayout_t Ddesc, const mublasLtMatmulAlgo_t ∗algo, mublasLtMatmulHeuristicResult_t ∗result)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoCapGetAttribute (const mublasLtMatmulAlgo_t ∗algo, mublasLtMatmulAlgoCapAttributes_t attr, void ∗buf, size_t sizeInBytes, size_t ∗sizeWritten)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoConfigSetAttribute (mublasLtMatmulAlgo_t ∗algo, mublasLtMatmulAlgoConfigAttributes_t attr, const void ∗buf, size_t sizeInBytes)
• mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoConfigGetAttribute (const mublasLtMatmulAlgo_t ∗algo, mublasLtMatmulAlgoConfigAttributes_t attr, void ∗buf, size_t sizeInBytes, size_t ∗sizeWritten)
• mublasStatus_t MUBLASWINAPI mublasLtLoggerSetCallback (mublasLtLoggerCallback_t callback)
• mublasStatus_t MUBLASWINAPI mublasLtLoggerSetFile (FILE ∗file)
• mublasStatus_t MUBLASWINAPI mublasLtLoggerOpenFile (const char ∗logFile)
• mublasStatus_t MUBLASWINAPI mublasLtLoggerSetLevel (int level)
• mublasStatus_t MUBLASWINAPI mublasLtLoggerSetMask (int mask)
• mublasStatus_t MUBLASWINAPI mublasLtLoggerForceDisable ()

5.1.1 Macro Definition Documentation

5.1.1.1 MUBLASAPI

#define MUBLASAPI

5.1.1.2 MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_16F

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_16F (0x01ull \<\< 8)

5.1.1.3 MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_32F

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_32F (0x02ull \<\< 8)

5.1.1.4 MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_32I

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_32I (0x08ull \<\< 8)

5.1.1.5 MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_64F

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_64F (0x04ull \<\< 8)

5.1.1.6 MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_TYPE_MASK

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_ACCUMULATOR_TYPE_MASK (0xffull \<\< 8)

5.1.1.7 MUBLASLT_NUMERICAL_IMPL_FLAGS_DMMA

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_DMMA (0x08ull \<\< 0)

5.1.1.8 MUBLASLT_NUMERICAL_IMPL_FLAGS_FMA

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_FMA (0x01ull \<\< 0)

Implementation details that may affect numerical behavior of algorithms.

5.1.1.9 MUBLASLT_NUMERICAL_IMPL_FLAGS_GAUSSIAN

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_GAUSSIAN (0x01ull \<\< 32)

5.1.1.10 MUBLASLT_NUMERICAL_IMPL_FLAGS_HMMA

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_HMMA (0x02ull \<\< 0)

5.1.1.11 MUBLASLT_NUMERICAL_IMPL_FLAGS_IMMA

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_IMMA (0x04ull \<\< 0)

5.1.1.12 MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_16BF

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_16BF (0x02ull \<\< 16)

5.1.1.13 MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_16F

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_16F (0x01ull \<\< 16)

5.1.1.14 MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_32F

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_32F (0x08ull \<\< 16)

5.1.1.15 MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_64F

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_64F (0x10ull \<\< 16)

5.1.1.16 MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8F_E4M3

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8F_E4M3 (0x40ull \<\< 16)

5.1.1.17 MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8F_E5M2

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8F_E5M2 (0x80ull \<\< 16)

5.1.1.18 MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8I

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_8I (0x20ull \<\< 16)

5.1.1.19 MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_TF32

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_INPUT_TF32 (0x04ull \<\< 16)

5.1.1.20 MUBLASLT_NUMERICAL_IMPL_FLAGS_OP_INPUT_TYPE_MASK

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_OP_INPUT_TYPE_MASK (0xffull \<\< 16)

5.1.1.21 MUBLASLT_NUMERICAL_IMPL_FLAGS_OP_TYPE_MASK

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_OP_TYPE_MASK (0xffull \<\< 0)

5.1.1.22 MUBLASLT_NUMERICAL_IMPL_FLAGS_TENSOR_OP_MASK

#define MUBLASLT_NUMERICAL_IMPL_FLAGS_TENSOR_OP_MASK (0xfeull \<\< 0)

5.1.1.23 MUBLASWINAPI

#define MUBLASWINAPI

5.1.2 Typedef Documentation

5.1.2.1 mublasLtHandle_t

typedef struct mublasLtContext∗ mublasLtHandle_t

Opaque structure holding MUBLASLT context

5.1.2.2 mublasLtLoggerCallback_t

typedef void(∗ mublasLtLoggerCallback_t) (int logLevel, const char ∗functionName, const char
∗message)

Experimental: Logger callback type.

5.1.2.3 mublasLtMatmulDesc_t

typedef mublasLtMatmulDescOpaque_t∗ mublasLtMatmulDesc_t

Opaque descriptor for mublasLtMatmul() operation details

5.1.2.4 mublasLtMatmulPreference_t

typedef mublasLtMatmulPreferenceOpaque_t∗ mublasLtMatmulPreference_t

Opaque descriptor for mublasLtMatmulAlgoGetHeuristic() configuration

5.1.2.5 mublasLtMatrixLayout_t

typedef mublasLtMatrixLayoutOpaque_t∗ mublasLtMatrixLayout_t

Opaque descriptor for matrix memory layout

5.1.2.6 mublasLtMatrixTransformDesc_t

typedef mublasLtMatrixTransformDescOpaque_t∗ mublasLtMatrixTransformDesc_t

Opaque descriptor for mublasLtMatrixTransform() operation details

5.1.2.7 mublasLtNumericalImplFlags_t

typedef uint64_t mublasLtNumericalImplFlags_t

5.1.3 Function Documentation

5.1.3.1 mublasLtCreate()

mublasStatus_t MUBLASWINAPI mublasLtCreate (mublasLtHandle_t ∗ lightHandle)

5.1.3.2 mublasLtDestroy()

mublasStatus_t MUBLASWINAPI mublasLtDestroy (mublasLtHandle_t lightHandle)

5.1.3.3 mublasLtGetCudartVersion()

size_t MUBLASWINAPI mublasLtGetCudartVersion (void *value)

5.1.3.4 mublasLtGetProperty()

mublasStatus_t MUBLASWINAPI mublasLtGetProperty (libraryPropertyType type, int ∗ value)

5.1.3.5 mublasLtGetStatusName()

const char ∗MUBLASWINAPI mublasLtGetStatusName (mublasStatus_t status)

5.1.3.6 mublasLtGetStatusString()

const char ∗MUBLASWINAPI mublasLtGetStatusString (mublasStatus_t status)

5.1.3.7 mublasLtGetVersion()

size_t MUBLASWINAPI mublasLtGetVersion (void *value)

5.1.3.8 mublasLtHeuristicsCacheGetCapacity()

mublasStatus_t MUBLASWINAPI mublasLtHeuristicsCacheGetCapacity (size_t ∗ capacity)

5.1.3.9 mublasLtHeuristicsCacheSetCapacity()

mublasStatus_t MUBLASWINAPI mublasLtHeuristicsCacheSetCapacity (size_t capacity)

5.1.3.10 mublasLtLoggerForceDisable()

mublasStatus_t MUBLASWINAPI mublasLtLoggerForceDisable ()

Experimental: Disable logging for the entire session.

5.1.3.11 mublasLtLoggerOpenFile()

mublasStatus_t MUBLASWINAPI mublasLtLoggerOpenFile (const char ∗ logFile)

Experimental: Open log file.

5.1.3.12 mublasLtLoggerSetCallback()

mublasStatus_t MUBLASWINAPI mublasLtLoggerSetCallback (mublasLtLoggerCallback_t callback)

Experimental: Logger callback setter.

5.1.3.13 mublasLtLoggerSetFile()

mublasStatus_t MUBLASWINAPI mublasLtLoggerSetFile (FILE ∗ file)

Experimental: Log file setter.

5.1.3.14 mublasLtLoggerSetLevel()

mublasStatus_t MUBLASWINAPI mublasLtLoggerSetLevel (int level)

Experimental: Log level setter.

5.1.3.15 mublasLtLoggerSetMask()

mublasStatus_t MUBLASWINAPI mublasLtLoggerSetMask (int mask)

Experimental: Log mask setter.

5.1.3.16 mublasLtMatmul()

mublasStatus_t MUBLASWINAPI mublasLtMatmul (
               mublasLtHandle_t lightHandle,
               mublasLtMatmulDesc_t computeDesc,
               const void ∗ alpha,
               const void ∗ A,
               mublasLtMatrixLayout_t Adesc,
               const void ∗ B,
               mublasLtMatrixLayout_t Bdesc,
               const void ∗ beta,
               const void ∗ C,
               mublasLtMatrixLayout_t Cdesc,
               void ∗ D,
               mublasLtMatrixLayout_t Ddesc,
               const mublasLtMatmulAlgo_t ∗ algo,
               void ∗ workspace,
               size_t workspaceSizeInBytes,
               musaStream_t stream)

Execute matrix multiplication (D = alpha ∗ op(A) ∗ op(B) + beta ∗ C).

5.1.3.17 mublasLtMatmulAlgoCapGetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoCapGetAttribute (
      const mublasLtMatmulAlgo_t ∗ algo,
      mublasLtMatmulAlgoCapAttributes_t attr,
      void ∗ buf,
      size_t sizeInBytes,
      size_t ∗ sizeWritten)

Get algo capability attribute.

E.g. to get list of supported Tile IDs:
mublasLtMatmulTile_t tiles[MUBLASLT_MATMUL_TILE_END];
size_t num_tiles, size_written;
if (mublasLtMatmulAlgoCapGetAttribute(algo, MUBLASLT_ALGO_CAP_TILE_IDS, tiles,
sizeof(tiles), size_written) == MUBLAS_STATUS_SUCCESS) { num_tiles = size_written / sizeof(tiles[0]) }

5.1.3.18 mublasLtMatmulAlgoCheck()

mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoCheck (
            mublasLtHandle_t lightHandle,
            mublasLtMatmulDesc_t operationDesc,
            mublasLtMatrixLayout_t Adesc,
            mublasLtMatrixLayout_t Bdesc,
            mublasLtMatrixLayout_t Cdesc,
            mublasLtMatrixLayout_t Ddesc,
            const mublasLtMatmulAlgo_t ∗ algo,
            mublasLtMatmulHeuristicResult_t ∗ result)

Check configured algo descriptor for correctness and support on current device. Result includes required workspace size and calculated wave count. MUBLAS_STATUS_SUCCESS doesn't fully guarantee algo will run (will fail if e.g. buffers are not correctly aligned); but if mublasLtMatmulAlgoCheck fails, the algo will not run.

5.1.3.19 mublasLtMatmulAlgoConfigGetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoConfigGetAttribute (
            const mublasLtMatmulAlgo_t ∗ algo,
            mublasLtMatmulAlgoConfigAttributes_t attr,
            void ∗ buf,
            size_t sizeInBytes,
            size_t ∗ sizeWritten)

Get algo configuration attribute.

5.1.3.20 mublasLtMatmulAlgoConfigSetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoConfigSetAttribute (
            mublasLtMatmulAlgo_t ∗ algo,
            mublasLtMatmulAlgoConfigAttributes_t attr,
            const void ∗ buf,
            size_t sizeInBytes)

Set algo configuration attribute.

5.1.3.21 mublasLtMatmulAlgoGetHeuristic()

mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoGetHeuristic (
            mublasLtHandle_t lightHandle,
            mublasLtMatmulDesc_t operationDesc,
            mublasLtMatrixLayout_t Adesc,
            mublasLtMatrixLayout_t Bdesc,
            mublasLtMatrixLayout_t Cdesc,
            mublasLtMatrixLayout_t Ddesc,
            mublasLtMatmulPreference_t preference,
            int requestedAlgoCount,
            mublasLtMatmulHeuristicResult_t heuristicResultsArray[ ],
            int ∗ returnAlgoCount)

Query mublasLt heuristic for algorithm appropriate for given use case.

5.1.3.22 mublasLtMatmulAlgoGetIds()

mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoGetIds (
            mublasLtHandle_t lightHandle,
            mublasComputeType_t computeType,
            musaDataType_t scaleType,
            musaDataType_t Atype,
            musaDataType_t Btype,
            musaDataType_t Ctype,
            musaDataType_t Dtype,
            int requestedAlgoCount,
            int algoIdsArray[ ],
            int ∗ returnAlgoCount)

Routine to get all algo IDs that can potentially run

5.1.3.23 mublasLtMatmulAlgoInit()

mublasStatus_t MUBLASWINAPI mublasLtMatmulAlgoInit (
            mublasLtHandle_t lightHandle,
            mublasComputeType_t computeType,
            musaDataType_t scaleType,
            musaDataType_t Atype,
            musaDataType_t Btype,
            musaDataType_t Ctype,
            musaDataType_t Dtype,
            int algoId,
            mublasLtMatmulAlgo_t ∗ algo)

Initialize algo structure

5.1.3.24 mublasLtMatmulDescCreate()

mublasStatus_t MUBLASWINAPI mublasLtMatmulDescCreate (
         mublasLtMatmulDesc_t ∗ matmulDesc,
         mublasComputeType_t computeType,
         musaDataType_t scaleType)

Create new matmul operation descriptor.

5.1.3.25 mublasLtMatmulDescDestroy()

mublasStatus_t MUBLASWINAPI mublasLtMatmulDescDestroy (
         mublasLtMatmulDesc_t matmulDesc)

Destroy matmul operation descriptor.

5.1.3.26 mublasLtMatmulDescGetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatmulDescGetAttribute (
            mublasLtMatmulDesc_t matmulDesc,
            mublasLtMatmulDescAttributes_t attr,
            void ∗ buf,
            size_t sizeInBytes,
            size_t ∗ sizeWritten)

Get matmul operation descriptor attribute.

5.1.3.27 mublasLtMatmulDescInit()

mublasStatus_t mublasLtMatmulDescInit (
      mublasLtMatmulDesc_t matmulDesc,
      mublasComputeType_t computeType,
      musaDataType_t scaleType)

Initialize matmul operation descriptor in pre-allocated space.

5.1.3.28 mublasLtMatmulDescInit_internal()

mublasStatus_t MUBLASWINAPI mublasLtMatmulDescInit_internal (
         mublasLtMatmulDesc_t matmulDesc,
         size_t size,
         mublasComputeType_t computeType,
         musaDataType_t scaleType)

Internal. Do not use directly.

5.1.3.29 mublasLtMatmulDescSetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatmulDescSetAttribute (
         mublasLtMatmulDesc_t matmulDesc,
         mublasLtMatmulDescAttributes_t attr,
         const void ∗ buf,
         size_t sizeInBytes)

Set matmul operation descriptor attribute.

5.1.3.30 mublasLtMatmulPreferenceCreate()

mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceCreate (
mublasLtMatmulPreference_t ∗ pref)

Create new matmul heuristic search preference descriptor.

5.1.3.31 mublasLtMatmulPreferenceDestroy()

mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceDestroy (
            mublasLtMatmulPreference_t pref)

Destroy matmul heuristic search preference descriptor.

5.1.3.32 mublasLtMatmulPreferenceGetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceGetAttribute (
               mublasLtMatmulPreference_t pref,
               mublasLtMatmulPreferenceAttributes_t attr,
               void ∗ buf,
               size_t sizeInBytes,
               size_t ∗ sizeWritten)

Get matmul heuristic search preference descriptor attribute.

5.1.3.33 mublasLtMatmulPreferenceInit()

mublasStatus_t mublasLtMatmulPreferenceInit (
      mublasLtMatmulPreference_t pref)

Initialize matmul heuristic search preference descriptor in pre-allocated space.

5.1.3.34 mublasLtMatmulPreferenceInit_internal()

mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceInit_internal (
            mublasLtMatmulPreference_t pref,
            size_t size)

Internal. Do not use directly.

5.1.3.35 mublasLtMatmulPreferenceSetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatmulPreferenceSetAttribute (
            mublasLtMatmulPreference_t pref,
            mublasLtMatmulPreferenceAttributes_t attr,
            const void ∗ buf,
            size_t sizeInBytes)

Set matmul heuristic search preference descriptor attribute.

5.1.3.36 mublasLtMatrixLayoutCreate()

mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutCreate (
               mublasLtMatrixLayout_t ∗ matLayout,
               musaDataType type,
               uint64_t rows,
               uint64_t cols,
               int64_t ld)

Create new matrix layout descriptor.

5.1.3.37 mublasLtMatrixLayoutDestroy()

mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutDestroy (
            mublasLtMatrixLayout_t matLayout)

Destroy matrix layout descriptor.

5.1.3.38 mublasLtMatrixLayoutGetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutGetAttribute (
               mublasLtMatrixLayout_t matLayout,
               mublasLtMatrixLayoutAttribute_t attr,
               void ∗ buf,
               size_t sizeInBytes,
               size_t ∗ sizeWritten)

Get matrix layout descriptor attribute.

5.1.3.39 mublasLtMatrixLayoutInit()

mublasStatus_t mublasLtMatrixLayoutInit (
            mublasLtMatrixLayout_t matLayout,
            musaDataType type,
            uint64_t rows,
            uint64_t cols,
            int64_t ld)

Initialize matrix layout descriptor in pre-allocated space.

5.1.3.40 mublasLtMatrixLayoutInit_internal()

mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutInit_internal (
               mublasLtMatrixLayout_t matLayout,
               size_t size,
               musaDataType type,
               uint64_t rows,
               uint64_t cols,
               int64_t ld)

Internal. Do not use directly.

5.1.3.41 mublasLtMatrixLayoutSetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatrixLayoutSetAttribute (
            mublasLtMatrixLayout_t matLayout,
            mublasLtMatrixLayoutAttribute_t attr,
            const void ∗ buf,
            size_t sizeInBytes)

Set matrix layout descriptor attribute.

5.1.3.42 mublasLtMatrixTransform()

mublasStatus_t MUBLASWINAPI mublasLtMatrixTransform (
               mublasLtHandle_t lightHandle,
               mublasLtMatrixTransformDesc_t transformDesc,
               const void ∗ alpha,
               const void ∗ A,
               mublasLtMatrixLayout_t Adesc,
               const void ∗ beta,
               const void ∗ B,
               mublasLtMatrixLayout_t Bdesc,
               void ∗ C,
               mublasLtMatrixLayout_t Cdesc,
               musaStream_t stream)

Matrix layout conversion helper (C = alpha ∗ op(A) + beta ∗ op(B)) Can be used to change memory order of data or to scale and shift the values.

5.1.3.43 mublasLtMatrixTransformDescCreate()

mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescCreate (
                  mublasLtMatrixTransformDesc_t ∗ transformDesc,
                  musaDataType scaleType)

Create new matrix transform operation descriptor.

5.1.3.44 mublasLtMatrixTransformDescDestroy()

mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescDestroy (
         mublasLtMatrixTransformDesc_t transformDesc)

Destroy matrix transform operation descriptor.

5.1.3.45 mublasLtMatrixTransformDescGetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescGetAttribute (
            mublasLtMatrixTransformDesc_t transformDesc,
            mublasLtMatrixTransformDescAttributes_t attr,
            void ∗ buf,
            size_t sizeInBytes,
            size_t ∗ sizeWritten)

Get matrix transform operation descriptor attribute.

5.1.3.46 mublasLtMatrixTransformDescInit()

mublasStatus_t mublasLtMatrixTransformDescInit (
            mublasLtMatrixTransformDesc_t transformDesc,
            musaDataType scaleType)

Initialize matrix transform operation descriptor in pre-allocated space.

5.1.3.47 mublasLtMatrixTransformDescInit_internal()

mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescInit_internal (
               mublasLtMatrixTransformDesc_t transformDesc,
               size_t size,
               musaDataType scaleType)

Internal. Do not use directly.

5.1.3.48 mublasLtMatrixTransformDescSetAttribute()

mublasStatus_t MUBLASWINAPI mublasLtMatrixTransformDescSetAttribute (
            mublasLtMatrixTransformDesc_t transformDesc,
            mublasLtMatrixTransformDescAttributes_t attr,
            const void ∗ buf,
            size_t sizeInBytes)

Set matrix transform operation descriptor attribute.