Mirrors/blakeblackshear.frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2025-05-07 01:16:43 +02:00
Code Issues Projects Releases Wiki Activity

Update openvino hardware inference times (#16368)

Browse Source
This commit is contained in:
Nicolas Mowen 2025-02-07 09:52:21 -07:00 committed by GitHub
parent 4b429e440b
commit d6a2965cb2
No known key found for this signature in database
GPG Key ID: B5690EEEBB952194
1 changed files with 27 additions and 26 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

53
docs/docs/frigate/hardware.md
View File

@ -52,24 +52,25 @@ The OpenVINO detector type is able to run on:
More information is available [in the detector docs](/configuration/object_detectors#openvino-detector) More information is available [in the detector docs](/configuration/object_detectors#openvino-detector)
Inference speeds vary greatly depending on the CPU, GPU, or VPU used, some known examples are below: Inference speeds vary greatly depending on the CPU or GPU used, some known examples of GPU inference times are below:
| Name | MobileNetV2 Inference Speed | YOLO-NAS Inference Speed | Notes | | Name | MobileNetV2 Inference Time | YOLO-NAS Inference Time | Notes |
| -------------------- | --------------------------- | ------------------------- | -------------------------------------- | | -------------------- | -------------------------- | ------------------------- | -------------------------------------- |
| Intel Celeron J4105 | ~ 25 ms | | Can only run one detector instance | | Intel Celeron J4105 | ~ 25 ms | | Can only run one detector instance |
| Intel Celeron N3060 | 130 - 150 ms | | Can only run one detector instance | | Intel Celeron N3060 | 130 - 150 ms | | Can only run one detector instance |
| Intel Celeron N3205U | ~ 120 ms | | Can only run one detector instance | | Intel Celeron N3205U | ~ 120 ms | | Can only run one detector instance |
| Intel Celeron N4020 | 50 - 200 ms | | Inference speed depends on other loads | | Intel Celeron N4020 | 50 - 200 ms | | Inference speed depends on other loads |
| Intel i3 6100T | 15 - 35 ms | | Can only run one detector instance | | Intel i3 6100T | 15 - 35 ms | | Can only run one detector instance |
| Intel i3 8100 | ~ 15 ms | | | | Intel i3 8100 | ~ 15 ms | | |
| Intel i5 4590 | ~ 20 ms | | | | Intel i5 4590 | ~ 20 ms | | |
| Intel i5 6500 | ~ 15 ms | | | | Intel i5 6500 | ~ 15 ms | | |
| Intel i5 7200u | 15 - 25 ms | | | | Intel i5 7200u | 15 - 25 ms | | |
| Intel i5 7500 | ~ 15 ms | | | | Intel i5 7500 | ~ 15 ms | | |
| Intel i5 1135G7 | 10 - 15 ms | | | | Intel i5 1135G7 | 10 - 15 ms | | |
| Intel i5 12600K | ~ 15 ms | 320: ~ 20 ms 640: ~ 46 ms | | | Intel i3 12000 | | 320: ~ 19 ms 640: ~ 54 ms | |
| Intel Arc A380 | ~ 6 ms | 320: ~ 10 ms | | | Intel i5 12600K | ~ 15 ms | 320: ~ 20 ms 640: ~ 46 ms | |
| Intel Arc A750 | ~ 4 ms | 320: ~ 8 ms | | | Intel Arc A380 | ~ 6 ms | 320: ~ 10 ms | |
| Intel Arc A750 | ~ 4 ms | 320: ~ 8 ms | |
### TensorRT - Nvidia GPU ### TensorRT - Nvidia GPU
@ -78,15 +79,15 @@ The TensortRT detector is able to run on x86 hosts that have an Nvidia GPU which
Inference speeds will vary greatly depending on the GPU and the model used. Inference speeds will vary greatly depending on the GPU and the model used.
`tiny` variants are faster than the equivalent non-tiny model, some known examples are below: `tiny` variants are faster than the equivalent non-tiny model, some known examples are below:
| Name | YoloV7 Inference Speed | YOLO-NAS Inference Speed | | Name | YoloV7 Inference Time | YOLO-NAS Inference Time |
| --------------- | ---------------------- | ------------------------- | | --------------- | --------------------- | ------------------------- |
| GTX 1060 6GB | ~ 7 ms | | | GTX 1060 6GB | ~ 7 ms | |
| GTX 1070 | ~ 6 ms | | | GTX 1070 | ~ 6 ms | |
| GTX 1660 SUPER | ~ 4 ms | | | GTX 1660 SUPER | ~ 4 ms | |
| RTX 3050 | 5 - 7 ms | 320: ~ 10 ms 640: ~ 16 ms | | RTX 3050 | 5 - 7 ms | 320: ~ 10 ms 640: ~ 16 ms |
| RTX 3070 Mobile | ~ 5 ms | | | RTX 3070 Mobile | ~ 5 ms | |
| Quadro P400 2GB | 20 - 25 ms | | | Quadro P400 2GB | 20 - 25 ms | |
| Quadro P2000 | ~ 12 ms | | | Quadro P2000 | ~ 12 ms | |
### AMD GPUs ### AMD GPUs