mirror of
https://github.com/blakeblackshear/frigate.git
synced 2026-02-20 13:54:36 +01:00
rockchip: update dependencies and add script for model conversion (#15699)
* rockchip: update dependencies and add script for model conversion * rockchip: update docs --------- Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
This commit is contained in:
committed by
Blake Blackshear
parent
458ca4a983
commit
4c7d4e6c0a
@@ -35,7 +35,7 @@ Frigate supports multiple different detectors that work on different types of ha
|
||||
|
||||
:::note
|
||||
|
||||
Multiple detectors can not be mixed for object detection (ex: OpenVINO and Coral EdgeTPU can not be used for object detection at the same time).
|
||||
Multiple detectors can not be mixed for object detection (ex: OpenVINO and Coral EdgeTPU can not be used for object detection at the same time).
|
||||
|
||||
This does not affect using hardware for accelerating other tasks such as [semantic search](./semantic_search.md)
|
||||
|
||||
@@ -582,7 +582,7 @@ Hardware accelerated object detection is supported on the following SoCs:
|
||||
- RK3576
|
||||
- RK3588
|
||||
|
||||
This implementation uses the [Rockchip's RKNN-Toolkit2](https://github.com/airockchip/rknn-toolkit2/), version v2.0.0.beta0. Currently, only [Yolo-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) is supported as object detection model.
|
||||
This implementation uses the [Rockchip's RKNN-Toolkit2](https://github.com/airockchip/rknn-toolkit2/), version v2.3.0. Currently, only [Yolo-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) is supported as object detection model.
|
||||
|
||||
### Prerequisites
|
||||
|
||||
@@ -656,3 +656,37 @@ $ cat /sys/kernel/debug/rknpu/load
|
||||
|
||||
- All models are automatically downloaded and stored in the folder `config/model_cache/rknn_cache`. After upgrading Frigate, you should remove older models to free up space.
|
||||
- You can also provide your own `.rknn` model. You should not save your own models in the `rknn_cache` folder, store them directly in the `model_cache` folder or another subfolder. To convert a model to `.rknn` format see the `rknn-toolkit2` (requires a x86 machine). Note, that there is only post-processing for the supported models.
|
||||
|
||||
### Converting your own onnx model to rknn format
|
||||
|
||||
To convert a onnx model to the rknn format using the [rknn-toolkit2](https://github.com/airockchip/rknn-toolkit2/) you have to:
|
||||
|
||||
- Place one ore more models in onnx format in the directory `config/model_cache/rknn_cache/onnx` on your docker host (this might require `sudo` privileges).
|
||||
- Save the configuration file under `config/conv2rknn.yaml` (see below for details).
|
||||
- Run `docker exec <frigate_container_id> python3 /opt/conv2rknn.py`. If the conversion was successful, the rknn models will be placed in `config/model_cache/rknn_cache`.
|
||||
|
||||
This is an example configuration file that you need to adjust to your specific onnx model:
|
||||
|
||||
```yaml
|
||||
soc: ["rk3562","rk3566", "rk3568", "rk3576", "rk3588"]
|
||||
quantization: false
|
||||
|
||||
output_name: "{input_basename}"
|
||||
|
||||
config:
|
||||
mean_values: [[0, 0, 0]]
|
||||
std_values: [[255, 255, 255]]
|
||||
quant_img_rgb2bgr: true
|
||||
```
|
||||
|
||||
Explanation of the paramters:
|
||||
|
||||
- `soc`: A list of all SoCs you want to build the rknn model for. If you don't specify this parameter, the script tries to find out your SoC and builds the rknn model for this one.
|
||||
- `quantization`: true: 8 bit integer (i8) quantization, false: 16 bit float (fp16). Default: false.
|
||||
- `output_name`: The output name of the model. The following variables are available:
|
||||
- `quant`: "i8" or "fp16" depending on the config
|
||||
- `input_basename`: the basename of the input model (e.g. "my_model" if the input model is calles "my_model.onnx")
|
||||
- `soc`: the SoC this model was build for (e.g. "rk3588")
|
||||
- `tk_version`: Version of `rknn-toolkit2` (e.g. "2.3.0")
|
||||
- **example**: Specifying `output_name = "frigate-{quant}-{input_basename}-{soc}-v{tk_version}"` could result in a model called `frigate-i8-my_model-rk3588-v2.3.0.rknn`.
|
||||
- `config`: Configuration passed to `rknn-toolkit2` for model conversion. For an explanation of all available parameters have a look at section "2.2. Model configuration" of [this manual](https://github.com/MarcA711/rknn-toolkit2/releases/download/v2.3.0/03_Rockchip_RKNPU_API_Reference_RKNN_Toolkit2_V2.3.0_EN.pdf).
|
||||
Reference in New Issue
Block a user