From adb043e7aea26ef037f27a366539bb036b29cb56 Mon Sep 17 00:00:00 2001 From: Marc Altmann <40744649+MarcA711@users.noreply.github.com> Date: Sat, 25 May 2024 19:27:44 +0200 Subject: [PATCH] Update docs for rockchip platform (#11503) * improve docs for rockchip * update version info * fix typo * fix typo Co-authored-by: Nicolas Mowen * fix typo Co-authored-by: Nicolas Mowen --------- Co-authored-by: Nicolas Mowen --- .../configuration/hardware_acceleration.md | 32 +----------- docs/docs/configuration/object_detectors.md | 37 ++------------ docs/docs/frigate/hardware.md | 12 +++++ docs/docs/frigate/installation.md | 50 +++++++++++++++++++ 4 files changed, 68 insertions(+), 63 deletions(-) diff --git a/docs/docs/configuration/hardware_acceleration.md b/docs/docs/configuration/hardware_acceleration.md index 7bc80e0a7..bab2822d4 100644 --- a/docs/docs/configuration/hardware_acceleration.md +++ b/docs/docs/configuration/hardware_acceleration.md @@ -362,39 +362,11 @@ that NVDEC/NVDEC1 are in use. ## Rockchip platform -Hardware accelerated video de-/encoding is supported on all Rockchip SoCs using [Nyanmisaka's FFmpeg Fork](https://github.com/nyanmisaka/ffmpeg-rockchip) based on [Rockchip's mpp library](https://github.com/rockchip-linux/mpp). +Hardware accelerated video de-/encoding is supported on all Rockchip SoCs using [Nyanmisaka's FFmpeg 6.1 Fork](https://github.com/nyanmisaka/ffmpeg-rockchip) based on [Rockchip's mpp library](https://github.com/rockchip-linux/mpp). ### Prerequisites -Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and rkvdec2 driver. To check, enter the following commands: - -``` -$ uname -r -5.10.xxx-rockchip # or 6.1.xxx; the -rockchip suffix is important -$ ls /dev/dri -by-path card0 card1 renderD128 renderD129 # should list renderD128 -``` - -I recommend [Joshua Riek's Ubuntu for Rockchip](https://github.com/Joshua-Riek/ubuntu-rockchip), if your board is supported. - -### Setup - -Follow Frigate's default installation instructions, but use a docker image with `-rk` suffix for example `ghcr.io/blakeblackshear/frigate:stable-rk`. - -Next, you need to grant docker permissions to access your hardware: -- During the configuration process, you should run docker in privileged mode to avoid any errors due to insufficient permissions. To do so, add `privileged: true` to your `docker-compose.yml` file or the `--privileged` flag to your docker run command. -- After everything works, you should only grant necessary permissions to increase security. Add the lines below to your `docker-compose.yml` file or the following options to your docker run command: `--security-opt systempaths=unconfined --security-opt apparmor=unconfined --device /dev/dri:/dev/dri --device /dev/dma_heap:/dev/dma_heap --device /dev/rga:/dev/rga --device /dev/mpp_service:/dev/mpp_service`: - -```yaml - security_opt: - - apparmor=unconfined - - systempaths=unconfined - devices: - - /dev/dri:/dev/dri - - /dev/dma_heap:/dev/dma_heap - - /dev/rga:/dev/rga - - /dev/mpp_service:/dev/mpp_service -``` +Make sure to follow the [Rockchip specific installation instructions](/frigate/installation#rockchip-platform). ### Configuration diff --git a/docs/docs/configuration/object_detectors.md b/docs/docs/configuration/object_detectors.md index ed38fb214..9c4dc9893 100644 --- a/docs/docs/configuration/object_detectors.md +++ b/docs/docs/configuration/object_detectors.md @@ -313,39 +313,11 @@ Hardware accelerated object detection is supported on the following SoCs: - RK3576 - RK3588 -This implementation uses the [Rockchip's RKNN-Toolkit2](https://github.com/airockchip/rknn-toolkit2/) Currently, only [Yolo-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) is supported as object detection model. +This implementation uses the [Rockchip's RKNN-Toolkit2](https://github.com/airockchip/rknn-toolkit2/), version v2.0.0.beta0. Currently, only [Yolo-NAS](https://github.com/Deci-AI/super-gradients/blob/master/YOLONAS.md) is supported as object detection model. ### Prerequisites -Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and rknpu driver. To check, enter the following commands: - -``` -$ uname -r -5.10.xxx-rockchip # or 6.1.xxx; the -rockchip suffix is important -$ ls /dev/dri -by-path card0 card1 renderD128 renderD129 # should list renderD129 -$ sudo cat /sys/kernel/debug/rknpu/version -RKNPU driver: v0.9.2 # or later version -``` - -I recommend [Joshua Riek's Ubuntu for Rockchip](https://github.com/Joshua-Riek/ubuntu-rockchip), if your board is supported. - -### Setup - -Follow Frigate's default installation instructions, but use a docker image with `-rk` suffix for example `ghcr.io/blakeblackshear/frigate:stable-rk`. - -Next, you need to grant docker permissions to access your hardware: - -- During the configuration process, you should run docker in privileged mode to avoid any errors due to insufficient permissions. To do so, add `privileged: true` to your `docker-compose.yml` file or the `--privileged` flag to your docker run command. -- After everything works, you should only grant necessary permissions to increase security. Add the lines below to your `docker-compose.yml` file or the following options to your docker run command: `--security-opt systempaths=unconfined --security-opt apparmor=unconfined --device /dev/dri:/dev/dri`: - -```yaml -security_opt: - - apparmor=unconfined - - systempaths=unconfined -devices: - - /dev/dri:/dev/dri -``` +Make sure to follow the [Rockchip specific installation instrucitions](/frigate/installation#rockchip-platform). ### Configuration @@ -405,6 +377,5 @@ $ cat /sys/kernel/debug/rknpu/load ::: -- By default the rknn detector uses the yolonas_s model (`model: path: default-fp16-yolonas_s`). This model comes with the image, so no further steps than those mentioned above are necessary and no download happens. -- The other choices are automatically downloaded and stored in the folder `config/model_cache/rknn_cache`. After upgrading Frigate, you should remove older models to free up space. -- Finally, you can also provide your own `.rknn` model. You should not save your own models in the `rknn_cache` folder, store them directly in the `model_cache` folder or another subfolder. To convert a model to `.rknn` format see the `rknn-toolkit2` (requires a x86 machine). Note, that there is only post-processing for the supported models. +- All models are automatically downloaded and stored in the folder `config/model_cache/rknn_cache`. After upgrading Frigate, you should remove older models to free up space. +- You can also provide your own `.rknn` model. You should not save your own models in the `rknn_cache` folder, store them directly in the `model_cache` folder or another subfolder. To convert a model to `.rknn` format see the `rknn-toolkit2` (requires a x86 machine). Note, that there is only post-processing for the supported models. diff --git a/docs/docs/frigate/hardware.md b/docs/docs/frigate/hardware.md index 47e0e4ce1..602cc906f 100644 --- a/docs/docs/frigate/hardware.md +++ b/docs/docs/frigate/hardware.md @@ -95,6 +95,18 @@ Frigate supports all Jetson boards, from the inexpensive Jetson Nano to the powe Inference speed will vary depending on the YOLO model, jetson platform and jetson nvpmodel (GPU/DLA/EMC clock speed). It is typically 20-40 ms for most models. The DLA is more efficient than the GPU, but not faster, so using the DLA will reduce power consumption but will slightly increase inference time. +#### Rockchip platform + +Frigate supports hardware video processing on all Rockchip boards. However, hardware object detection is only supported on these boards: + +- RK3562 +- RK3566 +- RK3568 +- RK3576 +- RK3588 + +The inference time of a rk3588 with all 3 cores enabled is typically 25-30 ms for yolo-nas s. + ## What does Frigate use the CPU for and what does it use a detector for? (ELI5 Version) This is taken from a [user question on reddit](https://www.reddit.com/r/homeassistant/comments/q8mgau/comment/hgqbxh5/?utm_source=share&utm_medium=web2x&context=3). Modified slightly for clarity. diff --git a/docs/docs/frigate/installation.md b/docs/docs/frigate/installation.md index 91891194d..d3ba550f6 100644 --- a/docs/docs/frigate/installation.md +++ b/docs/docs/frigate/installation.md @@ -95,6 +95,56 @@ By default, the Raspberry Pi limits the amount of memory available to the GPU. I Additionally, the USB Coral draws a considerable amount of power. If using any other USB devices such as an SSD, you will experience instability due to the Pi not providing enough power to USB devices. You will need to purchase an external USB hub with it's own power supply. Some have reported success with this (affiliate link). +### Rockchip platform + +Make sure that you use a linux distribution that comes with the rockchip BSP kernel 5.10 or 6.1 and necessary drivers (especially rkvdec2 and rknpu). To check, enter the following commands: + +``` +$ uname -r +5.10.xxx-rockchip # or 6.1.xxx; the -rockchip suffix is important +$ ls /dev/dri +by-path card0 card1 renderD128 renderD129 # should list renderD128 (VPU) and renderD129 (NPU) +$ sudo cat /sys/kernel/debug/rknpu/version +RKNPU driver: v0.9.2 # or later version +``` + +I recommend [Joshua Riek's Ubuntu for Rockchip](https://github.com/Joshua-Riek/ubuntu-rockchip), if your board is supported. + +#### Setup + +Follow Frigate's default installation instructions, but use a docker image with `-rk` suffix for example `ghcr.io/blakeblackshear/frigate:stable-rk`. + +Next, you need to grant docker permissions to access your hardware: + +- During the configuration process, you should run docker in privileged mode to avoid any errors due to insufficient permissions. To do so, add `privileged: true` to your `docker-compose.yml` file or the `--privileged` flag to your docker run command. +- After everything works, you should only grant necessary permissions to increase security. Disable the privileged mode and add the lines below to your `docker-compose.yml` file: + +```yaml +security_opt: + - apparmor=unconfined + - systempaths=unconfined +devices: + - /dev/dri + - /dev/dma_heap + - /dev/rga + - /dev/mpp_service +``` + +or add these options to your `docker run` command: + +``` +--security-opt systempaths=unconfined \ +--security-opt apparmor=unconfined \ +--device /dev/dri \ +--device /dev/dma_heap \ +--device /dev/rga \ +--device /dev/mpp_service +``` + +#### Configuration + +Next, you should configure [hardware object detection](/configuration/object_detectors#rockchip-platform) and [hardware video processing](/configuration/hardware_acceleration#rockchip-platform). + ## Docker Running in Docker with compose is the recommended install method.