e5fe323aca
* Initial work for adding OpenVino detector. Not functional * Load model and submit for inference. Sucessfully load model and initialize OpenVino engine with either CPU or GPU as device. Does not parse results for objects. * Detection working with ssdlite_mobilenetv2 FP16 model * Add OpenVIno support and model to docker image * Add documentation for OpenVino detector configuration * Adds support for ARM32/ARM64 and the Myriad X hardware - Use custom-built openvino wheel for all platforms - Add libusb build without udev for NCS2 support * Add documentation around Intel CPU requirements and NCS2 setup * Print all available output tensors * Update documentation for config parameters
4.3 KiB
| id | title |
|---|---|
| detectors | Detectors |
By default, Frigate will use a single CPU detector. If you have a Coral, you will need to configure your detector devices in the config file. When using multiple detectors, they run in dedicated processes, but pull from a common queue of requested detections across all cameras.
Frigate supports edgetpu and cpu as detector types. The device value should be specified according to the Documentation for the TensorFlow Lite Python API.
Note: There is no support for Nvidia GPUs to perform object detection with tensorflow. It can be used for ffmpeg decoding, but not object detection.
Single USB Coral
detectors:
coral:
type: edgetpu
device: usb
Multiple USB Corals
detectors:
coral1:
type: edgetpu
device: usb:0
coral2:
type: edgetpu
device: usb:1
Native Coral (Dev Board)
warning: may have compatibility issues after v0.9.x
detectors:
coral:
type: edgetpu
device: ""
Multiple PCIE/M.2 Corals
detectors:
coral1:
type: edgetpu
device: pci:0
coral2:
type: edgetpu
device: pci:1
Mixing Corals
detectors:
coral_usb:
type: edgetpu
device: usb
coral_pci:
type: edgetpu
device: pci
CPU Detectors (not recommended)
detectors:
cpu1:
type: cpu
num_threads: 3
cpu2:
type: cpu
num_threads: 3
When using CPU detectors, you can add a CPU detector per camera. Adding more detectors than the number of cameras should not improve performance.
OpenVINO
The OpenVINO detector allows Frigate to run an OpenVINO IR model on Intel CPU, GPU and VPU hardware.
OpenVINO Devices
The OpenVINO detector supports the Intel-supplied device plugins and can specify one or more devices in the configuration. See OpenVINO's device naming conventions in the Device Documentation for more detail. Other supported devices could be AUTO, CPU, GPU, MYRIAD, etc.
detectors:
ov_detector:
type: openvino
device: GPU
OpenVINO is supported on 6th Gen Intel platforms (Skylake) and newer. A supported Intel platform is required to use the GPU device with OpenVINO. The MYRIAD device may be run on any platform, including Arm devices. For detailed system requirements, see OpenVINO System Requirements
Intel NCS2 VPU and Myriad X Setup
Intel produces a neural net inference accelleration chip called Myriad X. This chip was sold in their Neural Compute Stick 2 (NCS2) which has been discontinued. If intending to use the MYRIAD device for accelleration, additional setup is required to pass through the USB device. The host needs a udev rule installed to handle the NCS2 device.
sudo usermod -a -G users "$(whoami)"
cat <<EOF > 97-myriad-usbboot.rules
SUBSYSTEM=="usb", ATTRS{idProduct}=="2485", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
SUBSYSTEM=="usb", ATTRS{idProduct}=="f63b", ATTRS{idVendor}=="03e7", GROUP="users", MODE="0666", ENV{ID_MM_DEVICE_IGNORE}="1"
EOF
sudo cp 97-myriad-usbboot.rules /etc/udev/rules.d/
sudo udevadm control --reload-rules
sudo udevadm trigger
Additionally, the Frigate docker container needs to run with the following configuration:
--device-cgroup-rule='c 189:\* rmw' -v /dev/bus/usb:/dev/bus/usb
or in your compose file:
device_cgroup_rules:
- 'c 189:* rmw'
volumes:
- /dev/bus/usb:/dev/bus/usb
OpenVINO Models
The included model for an OpenVINO detector comes from Intel's Open Model Zoo SSDLite MobileNet V2 and is converted to an FP16 precision IR model. Use the model configuration shown below when using the OpenVINO detector.
model:
path: /openvino-model/ssdlite_mobilenet_v2.xml
width: 300
height: 300
input_tensor: nhwc
input_pixel_format: bgr
labelmap_path: /openvino-model/coco_91cl_bkgr.txt