'Using ML.net with an ONNX model and GPU

How do you get Microsof.tML to run on an NVDIA GPU using C#? I followed the instructions

https://github.com/dotnet/machinelearning/blob/master/docs/api-reference/tensorflow-usage.md

And my pipeline looks like the below. All versions dependencies (onnxruntime.gpu,Microsoft.ML etc) are 1.5.2 so this should be supported but I get the exception DllNotFoundException: Unable to load DLL 'onnxruntime' or one of its dependencies: The specified module could not be found - and yes onnxruntime does appear in the list of installed nuget packages.

Has anyone gotten this to work? The documentation is still pretty sparse and there don't seem to be any working examples online.

var pipeline = context.Transforms.ResizeImages(
    resizing: ImageResizingEstimator.ResizingKind.Fill,
    outputColumnName: "image", imageWidth: ImageSettings.imageWidth,
    imageHeight: ImageSettings.imageHeight,
    inputColumnName: nameof(ModelInput.Image)
)
.Append(context.Transforms.ExtractPixels(outputColumnName: "image"))
.Append(context.Transforms.ApplyOnnxModel(
    modelFile: "./ObjectDetection/tinyyolov2-8.onnx",
    outputColumnName: "grid", 
    inputColumnName: "image", 
    gpuDeviceId:0)
);
c++sharpmicrosoft.ml


Solution 1:[1]

OK - bad form to answer my own question but the link above saying you must use Cuda 10 seems inaccurate. If you are using the ML.net 1.5 nuget releases you need Cuda 10.2.

11 won't work unless you build from source (and who has the time for that).

Hopefully this solves it but if you were going to comment please do as well!

Solution 2:[2]

Here is the compatibility chart. Newer versions have different compatibilities.

The CUDA Execution Provider enables hardware accelerated computation on Nvidia CUDA-enabled GPUs:

https://onnxruntime.ai/docs/execution-providers/CUDA-ExecutionProvider.html

Sources

This article follows the attribution requirements of Stack Overflow and is licensed under CC BY-SA 3.0.

Source: Stack Overflow

Solution Source
Solution 1 hamishkeith
Solution 2 sckoralesky

Related Questions