Mirrors/blakeblackshear.frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2025-07-26 13:47:03 +02:00
Code Issues Projects Releases Wiki Activity

Add YOLOv9 support to RKNN (#17791)

Browse Source 
* Add yolov9

* Undo

* Update docs for rknn yolov9

* Update docs notes

* Add infernece times table
This commit is contained in:
Nicolas Mowen 2025-04-18 16:51:04 -06:00 committed by GitHub
parent e8883a2a2e
commit 89b54f19c8
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
3 changed files with 32 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

23
docs/docs/configuration/object_detectors.md
View File

@ -849,6 +849,7 @@ The inference time was determined on a rk3588 with 3 NPU cores.
| deci-fp16-yolonas_s | 24 | 25 |
| deci-fp16-yolonas_m | 62 | 35 |
| deci-fp16-yolonas_l | 81 | 45 |
| yolov9_tiny | 8 | 35 |
| yolox_nano | 3 | 16 |
| yolox_tiny | 6 | 20 |
@ -864,7 +865,9 @@ model: # required
# - deci-fp16-yolonas_s
# - deci-fp16-yolonas_m
# - deci-fp16-yolonas_l
# your yolonas_model.rknn
path: deci-fp16-yolonas_s
model_type: yolonas
width: 320
height: 320
input_pixel_format: bgr
@ -878,6 +881,24 @@ The pre-trained YOLO-NAS weights from DeciAI are subject to their license and ca
:::
#### YOLO (v9)
```yaml
model: # required
# name of model (will be automatically downloaded) or path to your own .rknn model file
# possible values are:
# - yolov9-t
# - yolov9-s
# your yolo_model.rknn
path: /config/model_cache/rknn_cache/yolov9-t.rknn
model_type: yolo-generic
width: 320
height: 320
input_tensor: nhwc
input_dtype: float
labelmap_path: /labelmap/coco-80.txt
```
#### YOLOx
```yaml
@ -886,7 +907,9 @@ model: # required
# possible values are:
# - yolox_nano
# - yolox_tiny
# your yolox_model.rknn
path: yolox_tiny
model_type: yolox
width: 416
height: 416
input_tensor: nhwc

6
docs/docs/frigate/hardware.md
View File

@ -165,6 +165,12 @@ Frigate supports hardware video processing on all Rockchip boards. However, hard
- RK3576
- RK3588
| Name | YOLOv9 Inference Time | YOLO-NAS Inference Time | YOLOx Inference Time |
| --------------- | --------------------- | --------------------------- | ------------------------- |
| rk3588 3 cores | ~ 35 ms | small: ~ 20 ms med: ~ 30 ms | nano: 18 ms tiny: 20 ms |
| rk3566 1 core | | small: ~ 96 ms | |
The inference time of a rk3588 with all 3 cores enabled is typically 25-30 ms for yolo-nas s.
## What does Frigate use the CPU for and what does it use a detector for? (ELI5 Version)

3
frigate/detectors/plugins/rknn.py
View File

@ -11,6 +11,7 @@ from pydantic import Field
from frigate.const import MODEL_CACHE_DIR
from frigate.detectors.detection_api import DetectionApi
from frigate.detectors.detector_config import BaseDetectorConfig, ModelTypeEnum
from frigate.util.model import post_process_yolo
logger = logging.getLogger(__name__)
@ -284,6 +285,8 @@ class Rknn(DetectionApi):
def post_process(self, output):
if self.detector_config.model.model_type == ModelTypeEnum.yolonas:
return self.post_process_yolonas(output)
elif self.detector_config.model.model_type == ModelTypeEnum.yologeneric:
return post_process_yolo(output, self.width, self.height)
elif self.detector_config.model.model_type == ModelTypeEnum.yolox:
return self.post_process_yolox(output, self.grids, self.expanded_strides)
else: