Skip to content

Commit

Permalink
complete the README
Browse files Browse the repository at this point in the history
  • Loading branch information
gtbluesky committed Sep 14, 2023
1 parent 1a18f34 commit 95831dd
Show file tree
Hide file tree
Showing 4 changed files with 78 additions and 76 deletions.
4 changes: 2 additions & 2 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,3 @@
## 0.0.1
## 0.0.2

* TODO: Describe initial release.
* initial release.
22 changes: 21 additions & 1 deletion LICENSE
Original file line number Diff line number Diff line change
@@ -1 +1,21 @@
TODO: Add your license here.
MIT License

Copyright (c) 2023 gtbluesky

Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:

The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.

THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
118 changes: 50 additions & 68 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,92 +1,74 @@
# onnxruntime
<p align="center"><img width="50%" src="https://github.com/microsoft/onnxruntime/raw/main/docs/images/ONNX_Runtime_logo_dark.png" /></p>

A new Flutter FFI plugin project.
# OnnxRuntime Plugin
[![pub package](https://img.shields.io/pub/v/onnxruntime.svg)](https://pub.dev/packages/onnxruntime)

## Getting Started

This project is a starting point for a Flutter
[FFI plugin](https://docs.flutter.dev/development/platform-integration/c-interop),
a specialized package that includes native code directly invoked with Dart FFI.

## Project structure

This template uses the following structure:
## Overview

* `src`: Contains the native source code, and a CmakeFile.txt file for building
that source code into a dynamic library.
Flutter plugin for OnnxRuntime with `FFI` provides an easy, flexible, and fast Dart API to integrate Onnx models in flutter apps across mobile and desktop platforms.

* `lib`: Contains the Dart code that defines the API of the plugin, and which
calls into the native code using `dart:ffi`.
## Key Features

* platform folders (`android`, `ios`, `windows`, etc.): Contains the build files
for building and bundling the native code library with the platform application.
* Multi-platform Support for Android and iOS
* Flexibility to use any Onnx Model.
* Acceleration using multi-threading.
* Similar structure as OnnxRuntime Java and C# API.
* Inference speeds close to native Android/iOS Apps built using the Java/Objective-C API.
* Run inference in different isolates to prevent jank in UI thread.

## Building and bundling native code
## Getting Started

The `pubspec.yaml` specifies FFI plugins as follows:
In your flutter project add the dependency:

```yaml
plugin:
platforms:
some_platform:
ffiPlugin: true
```yml
dependencies:
...
onnxruntime:
```
This configuration invokes the native build for the various target platforms
and bundles the binaries in Flutter applications using these FFI plugins.
This can be combined with dartPluginClass, such as when FFI is used for the
implementation of one platform in a federated plugin:
For help getting started with Flutter, view the online
[documentation](https://flutter.io/).
```yaml
plugin:
implements: some_other_plugin
platforms:
some_platform:
dartPluginClass: SomeClass
ffiPlugin: true
```
## Usage example
A plugin can have both FFI and method channels:
### Import
```yaml
plugin:
platforms:
some_platform:
pluginClass: SomeName
ffiPlugin: true
```dart
import 'package:onnxruntime/onnxruntime.dart';
```

The native build systems that are invoked by FFI (and method channel) plugins are:
### Initializing environment

* For Android: Gradle, which invokes the Android NDK for native builds.
* See the documentation in android/build.gradle.
* For iOS and MacOS: Xcode, via CocoaPods.
* See the documentation in ios/onnxruntime.podspec.
* See the documentation in macos/onnxruntime.podspec.
* For Linux and Windows: CMake.
* See the documentation in linux/CMakeLists.txt.
* See the documentation in windows/CMakeLists.txt.
## Binding to native code
```dart
OrtEnv.instance.init();
```

To use the native code, bindings in Dart are needed.
To avoid writing these by hand, they are generated from the header file
(`src/onnxruntime.h`) by `package:ffigen`.
Regenerate the bindings by running `flutter pub run ffigen --config ffigen.yaml`.
### Creating the Session

## Invoking native code
```dart
final sessionOptions = OrtSessionOptions();
const assetFileName = 'assets/models/test.onnx';
final rawAssetFile = await rootBundle.load(assetFileName);
final bytes = rawAssetFile.buffer.asUint8List();
final session = OrtSession.fromBuffer(bytes, sessionOptions!);
```

Very short-running native functions can be directly invoked from any isolate.
For example, see `sum` in `lib/onnxruntime.dart`.
### Performing inference

Longer-running functions should be invoked on a helper isolate to avoid
dropping frames in Flutter applications.
For example, see `sumAsync` in `lib/onnxruntime.dart`.
```dart
final shape = [1, 2, 3];
final inputOrt = OrtValueTensor.createTensorWithDataList(data, shape);
final inputs = {'input': inputOrt};
final runOptions = OrtRunOptions();
final outputs = await _session?.runAsync(runOptions, inputs);
inputOrt.release();
runOptions.release();
```

## Flutter help
### Releasing environment

For help getting started with Flutter, view our
[online documentation](https://flutter.dev/docs), which offers tutorials,
samples, guidance on mobile development, and a full API reference.
```dart
OrtEnv.instance.release();
```

10 changes: 5 additions & 5 deletions pubspec.yaml
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
name: onnxruntime
description: A new Flutter FFI plugin project.
version: 0.0.1
homepage:
description: Flutter plugin for OnnxRuntime provides an easy, flexible, and fast Dart API to integrate Onnx models in flutter apps across mobile and desktop platforms.
version: 0.0.2
homepage: https://github.com/gtbluesky/onnxruntime_flutter

environment:
sdk: '>=2.18.0 <4.0.0'
flutter: '>=1.10.0'
sdk: '>=2.17.0 <4.0.0'
flutter: '>=3.0.0'

dependencies:
flutter:
Expand Down

0 comments on commit 95831dd

Please sign in to comment.