Mirrors/blakeblackshear.frigate
1
0
Fork 0
mirror of https://github.com/blakeblackshear/frigate.git synced 2025-01-21 00:06:44 +01:00
Code Issues Projects Releases Wiki Activity

FEAT: Support for ffmpeg presets (#3840)

Browse Source 
* Add hwaccel presets

* Use hwaccel presets

* Add input arg presets

* Use input arg presets

* Make util to clean up redundant code

* Add support for output arg presets

* Add tests

* Update camera specific to use presets

* Update hwaccel to use presets

* Format files and fix tests

* Rewrite tests to test record correctly

* Move presets from string to list to avoid manually separating into a list

* Add mjpeg cuvid decoder preset

* Fix tests

* Fix comment
This commit is contained in:
Nicolas Mowen 2022-11-28 20:48:11 -07:00 committed by GitHub
parent 69560c8bde
commit 87144cd572
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
6 changed files with 480 additions and 75 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

56
docs/docs/configuration/camera_specific.md
View File

@ -8,15 +8,15 @@ title: Camera Specific Configurations
The input and output parameters need to be adjusted for MJPEG cameras
```yaml
input_args: -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -use_wallclock_as_timestamps 1 -c:v mjpeg
input_args: preset-http-mjpeg-generic
```
Note that mjpeg cameras require encoding the video into h264 for recording, and rtmp roles. This will use significantly more CPU than if the cameras supported h264 feeds directly.
```yaml
output_args:
record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v libx264 -an
rtmp: -c:v libx264 -an -f flv
record: preset-record-mjpeg
rtmp: preset-rtmp-mjpeg
```
## JPEG Stream Cameras
@ -24,25 +24,7 @@ output_args:
Cameras using a live changing jpeg image will need input parameters as below
```yaml
input_args:
- -r
- 5 # << enter FPS here
- -stream_loop
- -1
- -f
- image2
- -avoid_negative_ts
- make_zero
- -fflags
- nobuffer
- -flags
- low_delay
- -strict
- experimental
- -fflags
- +genpts+discardcorrupt
- -use_wallclock_as_timestamps
- 1
input_args: preset-http-jpeg-generic
```
Outputting the stream will have the same args and caveats as per [MJPEG Cameras](#mjpeg-cameras)
@ -53,7 +35,7 @@ The input parameters need to be adjusted for RTMP cameras
```yaml
ffmpeg:
input_args: -avoid_negative_ts make_zero -fflags nobuffer -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rw_timeout 5000000 -use_wallclock_as_timestamps 1 -f live_flv
input_args: preset-rtmp-generic
```
## UDP Only Cameras
@ -62,7 +44,7 @@ If your cameras do not support TCP connections for RTSP, you can use UDP.
```yaml
ffmpeg:
input_args: -avoid_negative_ts make_zero -fflags +genpts+discardcorrupt -rtsp_transport udp -timeout 5000000 -use_wallclock_as_timestamps 1
input_args: preset-rtsp-udp
```
## Model/vendor specific setup
@ -77,7 +59,7 @@ cameras:
output_args:
record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -tag:v hvc1 -bsf:v hevc_mp4toannexb -c:a aac
rtmp: -c:v copy -c:a aac -f flv
inputs:
- path: rtsp://user:password@camera-ip:554/H264/ch1/main/av_stream # <----- Update for your camera
roles:
@ -99,7 +81,7 @@ You will need to remove `nobuffer` flag for Blue Iris RTSP cameras
```yaml
ffmpeg:
input_args: -avoid_negative_ts make_zero -flags low_delay -strict experimental -fflags +genpts+discardcorrupt -rtsp_transport tcp -timeout 5000000 -use_wallclock_as_timestamps 1
input_args: preset-rtsp-blue-iris
```
### Reolink 410/520 (possibly others)
@ -112,21 +94,7 @@ According to [this discussion](https://github.com/blakeblackshear/frigate/issues
cameras:
reolink:
ffmpeg:
input_args:
- -avoid_negative_ts
- make_zero
- -fflags
- +genpts+discardcorrupt
- -flags
- low_delay
- -strict
- experimental
- -analyzeduration
- 1000M
- -probesize
- 1000M
- -rw_timeout
- "5000000"
input_args: preset-http-reolink
inputs:
- path: http://reolink_ip/flv?port=1935&app=bcs&stream=channel0_main.bcs&user=username&password=password
roles:
@ -148,6 +116,6 @@ In the Unifi 2.0 update Unifi Protect Cameras had a change in audio sample rate
```yaml
ffmpeg:
output_args:
record: -f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c:v copy -ar 44100 -c:a aac
rtmp: -c:v copy -f flv -ar 44100 -c:a aac
```
record: preset-record-ubiquiti
rtmp: preset-rtmp-ubiquiti
```

14
docs/docs/configuration/hardware_acceleration.md
View File

@ -12,22 +12,22 @@ Ensure you increase the allocated RAM for your GPU to at least 128 (raspi-config
```yaml
ffmpeg:
hwaccel_args: -c:v h264_v4l2m2m
hwaccel_args: preset-rpi-64-h264
```
### Intel-based CPUs (<10th Generation) via Quicksync
```yaml
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
hwaccel_args: preset-intel-vaapi
```
**NOTICE**: With some of the processors, like the J4125, the default driver `iHD` doesn't seem to work correctly for hardware acceleration. You may need to change the driver to `i965` by adding the following environment variable `LIBVA_DRIVER_NAME=i965` to your docker-compose file or [in the frigate.yml for HA OS users](advanced.md#environment_vars).
**NOTICE**: With some of the processors, like the J4125, the default driver `iHD` doesn't seem to work correctly for hardware acceleration. You may need to change the driver to `i965` by adding the following environment variable `LIBVA_DRIVER_NAME=i965` to your docker-compose file or [in the frigate.yml for HA OS users](advanced.md#environment_vars).
### Intel-based CPUs (>=10th Generation) via Quicksync
```yaml
ffmpeg:
hwaccel_args: -c:v h264_qsv
hwaccel_args: preset-intel-qsv-h264
```
### AMD/ATI GPUs (Radeon HD 2000 and newer GPUs) via libva-mesa-driver
@ -36,7 +36,7 @@ ffmpeg:
```yaml
ffmpeg:
hwaccel_args: -hwaccel vaapi -hwaccel_device /dev/dri/renderD128 -hwaccel_output_format yuv420p
hwaccel_args: preset-amd-vaapi
```
### NVIDIA GPU
@ -79,11 +79,11 @@ A list of supported codecs (you can use `ffmpeg -decoders | grep cuvid` in the c
V..... vp9_cuvid Nvidia CUVID VP9 decoder (codec vp9)
```
For example, for H264 video, you'll select `h264_cuvid`.
For example, for H264 video, you'll select `preset-nvidia-h264`.
```yaml
ffmpeg:
hwaccel_args: -c:v h264_cuvid
hwaccel_args: preset-nvidia-h264
```
If everything is working correctly, you should see a significant improvement in performance.

49
frigate/config.py
View File

@ -21,10 +21,17 @@ from frigate.const import (
from frigate.util import (
create_mask,
deep_merge,
get_ffmpeg_arg_list,
escape_special_characters,
load_config_with_no_duplicates,
load_labels,
)
from frigate.ffmpeg_presets import (
parse_preset_hardware_acceleration,
parse_preset_input,
parse_preset_output_record,
parse_preset_output_rtmp,
)
logger = logging.getLogger(__name__)
@ -646,11 +653,8 @@ class CameraConfig(FrigateBaseModel):
def _get_ffmpeg_cmd(self, ffmpeg_input: CameraInput):
ffmpeg_output_args = []
if "detect" in ffmpeg_input.roles:
detect_args = (
self.ffmpeg.output_args.detect
if isinstance(self.ffmpeg.output_args.detect, list)
else self.ffmpeg.output_args.detect.split(" ")
)
detect_args = get_ffmpeg_arg_list(self.ffmpeg.output_args.detect)
ffmpeg_output_args = (
[
"-r",
@ -663,19 +667,18 @@ class CameraConfig(FrigateBaseModel):
+ ["pipe:"]
)
if "rtmp" in ffmpeg_input.roles and self.rtmp.enabled:
rtmp_args = (
self.ffmpeg.output_args.rtmp
if isinstance(self.ffmpeg.output_args.rtmp, list)
else self.ffmpeg.output_args.rtmp.split(" ")
rtmp_args = get_ffmpeg_arg_list(
parse_preset_output_rtmp(self.ffmpeg.output_args.rtmp)
or self.ffmpeg.output_args.rtmp
)
ffmpeg_output_args = (
rtmp_args + [f"rtmp://127.0.0.1/live/{self.name}"] + ffmpeg_output_args
)
if "record" in ffmpeg_input.roles and self.record.enabled:
record_args = (
self.ffmpeg.output_args.record
if isinstance(self.ffmpeg.output_args.record, list)
else self.ffmpeg.output_args.record.split(" ")
record_args = get_ffmpeg_arg_list(
parse_preset_output_record(self.ffmpeg.output_args.record)
or self.ffmpeg.output_args.record
)
ffmpeg_output_args = (
@ -688,18 +691,18 @@ class CameraConfig(FrigateBaseModel):
if len(ffmpeg_output_args) == 0:
return None
global_args = ffmpeg_input.global_args or self.ffmpeg.global_args
hwaccel_args = ffmpeg_input.hwaccel_args or self.ffmpeg.hwaccel_args
input_args = ffmpeg_input.input_args or self.ffmpeg.input_args
global_args = (
global_args if isinstance(global_args, list) else global_args.split(" ")
global_args = get_ffmpeg_arg_list(
ffmpeg_input.global_args or self.ffmpeg.global_args
)
hwaccel_args = (
hwaccel_args if isinstance(hwaccel_args, list) else hwaccel_args.split(" ")
hwaccel_args = get_ffmpeg_arg_list(
ffmpeg_input.hwaccel_args
or parse_preset_hardware_acceleration(self.ffmpeg.hwaccel_args)
or self.ffmpeg.hwaccel_args
)
input_args = (
input_args if isinstance(input_args, list) else input_args.split(" ")
input_args = get_ffmpeg_arg_list(
ffmpeg_input.input_args
or parse_preset_input(self.ffmpeg.input_args, self.detect.fps)
or self.ffmpeg.input_args
)
cmd = (

277
frigate/ffmpeg_presets.py Normal file
View File

@ -0,0 +1,277 @@
"""Handles inserting and maintaining ffmpeg presets."""
from typing import Any
PRESETS_HW_ACCEL = {
"preset-rpi-32-h264": ["-c:v", "h264_v4l2m2m"],
"preset-rpi-64-h264": ["-c:v", "h264_v4l2m2m"],
"preset-intel-vaapi": [
"-hwaccel",
"vaapi",
"-hwaccel_device",
"/dev/dri/renderD128",
"-hwaccel_output_format",
"yuv420p",
],
"preset-intel-qsv-h264": ["-c:v", "h264_qsv"],
"preset-intel-qsv-h265": ["-c:v", "hevc_qsv"],
"preset-amd-vaapi": [
"-hwaccel",
"vaapi",
"-hwaccel_device",
"/dev/dri/renderD128",
"-hwaccel_output_format",
"yuv420p",
],
"preset-nvidia-h264": ["-c:v", "h264_cuvid"],
"preset-nvidia-h265": ["-c:v", "hevc_cuvid"],
"preset-nvidia-mjpeg": ["-c:v", "mjpeg_cuvid"],
}
def parse_preset_hardware_acceleration(arg: Any) -> list[str]:
"""Return the correct preset if in preset format otherwise return None."""
if not isinstance(arg, str):
return None
return PRESETS_HW_ACCEL.get(arg, None)
PRESETS_INPUT = {
"preset-http-jpeg-generic": [
"-r",
"{}",
"-stream_loop",
"-1",
"-f",
"image2",
"-avoid_negative_ts",
"make_zero",
"-fflags",
"nobuffer",
"-flags",
"low_delay",
"-strict",
"experimental",
"-fflags",
"+genpts+discardcorrupt",
"-use_wallclock_as_timestamps",
"1",
],
"preset-http-mjpeg-generic": [
"-avoid_negative_ts",
"make_zero",
"-fflags",
"nobuffer",
"-flags",
"low_delay",
"-strict",
"experimental",
"-fflags",
"+genpts+discardcorrupt",
"-use_wallclock_as_timestamps",
"1",
],
"preset-http-reolink": [
"-avoid_negative_ts",
"make_zero",
"-fflags",
"+genpts+discardcorrupt",
"-flags",
"low_delay",
"-strict",
"experimental",
"-analyzeduration",
"1000M",
"-probesize",
"1000M",
"-rw_timeout",
"5000000",
],
"preset-rtmp-generic": [
"-avoid_negative_ts",
"make_zero",
"-fflags",
"nobuffer",
"-flags",
"low_delay",
"-strict",
"experimental",
"-fflags",
"+genpts+discardcorrupt",
"-rw_timeout",
"5000000",
"-use_wallclock_as_timestamps",
"1",
"-f",
"live_flv",
],
"preset-rtsp-generic": [
"-avoid_negative_ts",
"make_zero",
"-fflags",
"+genpts+discardcorrupt",
"-rtsp_transport",
"tcp",
"-timeout",
"5000000",
"-use_wallclock_as_timestamps",
"1",
],
"preset-rtsp-udp": [
"-avoid_negative_ts",
"make_zero",
"-fflags",
"+genpts+discardcorrupt",
"-rtsp_transport",
"udp",
"-timeout",
"5000000",
"-use_wallclock_as_timestamps",
"1",
],
"preset-rtsp-blue-iris": [
"-avoid_negative_ts",
"make_zero",
"-flags",
"low_delay",
"-strict",
"experimental",
"-fflags",
"+genpts+discardcorrupt",
"-rtsp_transport",
"tcp",
"-timeout",
"5000000",
"-use_wallclock_as_timestamps",
"1",
],
}
def parse_preset_input(arg: Any, detect_fps: int) -> list[str]:
"""Return the correct preset if in preset format otherwise return None."""
if not isinstance(arg, str):
return None
if arg == "preset-jpeg-generic":
return PRESETS_INPUT[arg].format(f"{detect_fps}")
return PRESETS_INPUT.get(arg, None)
PRESETS_RECORD_OUTPUT = {
"preset-record-generic": [
"-f",
"segment",
"-segment_time",
"10",
"-segment_format",
"mp4",
"-reset_timestamps",
"1",
"-strftime",
"1",
"-c",
"copy",
"-an",
],
"preset-record-generic-audio": [
"-f",
"segment",
"-segment_time",
"10",
"-segment_format",
"mp4",
"-reset_timestamps",
"1",
"-strftime",
"1",
"-c:v",
"copy",
"-c:a",
"aac",
],
"preset-record-mjpeg": [
"-f",
"segment",
"-segment_time",
"10",
"-segment_format",
"mp4",
"-reset_timestamps",
"1",
"-strftime",
"1",
"-c:v",
"libx264",
"-an",
],
"preset-record-jpeg": [
"-f",
"segment",
"-segment_time",
"10",
"-segment_format",
"mp4",
"-reset_timestamps",
"1",
"-strftime",
"1",
"-c:v",
"libx264",
"-an",
],
"preset-record-ubiquiti": [
"-f",
"segment",
"-segment_time",
"10",
"-segment_format",
"mp4",
"-reset_timestamps",
"1",
"-strftime",
"1",
"-c:v",
"copy",
"-ar",
"44100",
"-c:a",
"aac",
],
}
def parse_preset_output_record(arg: Any) -> list[str]:
"""Return the correct preset if in preset format otherwise return None."""
if not isinstance(arg, str):
return None
return PRESETS_RECORD_OUTPUT.get(arg, None)
PRESETS_RTMP_OUTPUT = {
"preset-rtmp-generic": ["-c", "copy", "-f", "flv"],
"preset-rtmp-mjpeg": ["-c:v", "libx264", "-an", "-f", "flv"],
"preset-rtmp-jpeg": ["-c:v", "libx264", "-an", "-f", "flv"],
"preset-rtmp-ubiquiti": [
"-c:v",
"copy",
"-f",
"flv",
"-ar",
"44100",
"-c:a",
"aac",
],
}
def parse_preset_output_rtmp(arg: Any) -> list[str]:
"""Return the correct preset if in preset format otherwise return None."""
if not isinstance(arg, str):
return None
return PRESETS_RTMP_OUTPUT.get(arg, None)

152
frigate/test/test_ffmpeg_presets.py Normal file
View File

@ -0,0 +1,152 @@
import unittest
from frigate.config import FrigateConfig
from frigate.ffmpeg_presets import parse_preset_input
class TestFfmpegPresets(unittest.TestCase):
def setUp(self):
self.default_ffmpeg = {
"mqtt": {"host": "mqtt"},
"cameras": {
"back": {
"ffmpeg": {
"inputs": [
{
"path": "rtsp://10.0.0.1:554/video",
"roles": ["detect", "rtmp"],
}
],
"output_args": {
"detect": "-f rawvideo -pix_fmt yuv420p",
"record": "-f segment -segment_time 10 -segment_format mp4 -reset_timestamps 1 -strftime 1 -c copy -an",
"rtmp": "-c copy -f flv",
},
},
"detect": {
"height": 1080,
"width": 1920,
"fps": 5,
},
"record": {
"enabled": True,
},
"rtmp": {
"enabled": True,
},
"name": "back",
}
},
}
def test_default_ffmpeg(self):
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert self.default_ffmpeg == frigate_config.dict(exclude_unset=True)
def test_ffmpeg_hwaccel_preset(self):
self.default_ffmpeg["cameras"]["back"]["ffmpeg"][
"hwaccel_args"
] = "preset-rpi-64-h264"
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert "preset-rpi-64-h264" not in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
assert "-c:v h264_v4l2m2m" in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
def test_ffmpeg_hwaccel_not_preset(self):
self.default_ffmpeg["cameras"]["back"]["ffmpeg"][
"hwaccel_args"
] = "-other-hwaccel args"
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert "-other-hwaccel args" in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
def test_default_ffmpeg_input_arg_preset(self):
frigate_config = FrigateConfig(**self.default_ffmpeg)
self.default_ffmpeg["cameras"]["back"]["ffmpeg"][
"input_args"
] = "preset-rtsp-generic"
frigate_preset_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
frigate_preset_config.cameras["back"].create_ffmpeg_cmds()
assert (
frigate_preset_config.cameras["back"].ffmpeg_cmds[0]["cmd"]
== frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"]
)
def test_ffmpeg_input_preset(self):
self.default_ffmpeg["cameras"]["back"]["ffmpeg"][
"input_args"
] = "preset-rtmp-generic"
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert "preset-rtmp-generic" not in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
assert (" ".join(parse_preset_input("preset-rtmp-generic", 5))) in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
def test_ffmpeg_input_not_preset(self):
self.default_ffmpeg["cameras"]["back"]["ffmpeg"]["input_args"] = "-some inputs"
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert "-some inputs" in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
def test_ffmpeg_output_record_preset(self):
self.default_ffmpeg["cameras"]["back"]["ffmpeg"]["output_args"][
"record"
] = "preset-record-generic-audio"
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert "preset-record-generic-audio" not in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
assert "-c:v copy -c:a aac" in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
def test_ffmpeg_output_record_not_preset(self):
self.default_ffmpeg["cameras"]["back"]["ffmpeg"]["output_args"][
"record"
] = "-some output"
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert "-some output" in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
def test_ffmpeg_output_rtmp_preset(self):
self.default_ffmpeg["cameras"]["back"]["ffmpeg"]["output_args"][
"rtmp"
] = "preset-rtmp-jpeg"
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert "preset-rtmp-jpeg" not in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
assert "-c:v libx264" in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
def test_ffmpeg_output_rtmp_not_preset(self):
self.default_ffmpeg["cameras"]["back"]["ffmpeg"]["output_args"][
"rtmp"
] = "-some output"
frigate_config = FrigateConfig(**self.default_ffmpeg)
frigate_config.cameras["back"].create_ffmpeg_cmds()
assert "-some output" in (
" ".join(frigate_config.cameras["back"].ffmpeg_cmds[0]["cmd"])
)
if __name__ == "__main__":
unittest.main(verbosity=2)

7
frigate/util.py
View File

@ -13,7 +13,7 @@ from abc import ABC, abstractmethod
from collections import Counter
from collections.abc import Mapping
from multiprocessing import shared_memory
from typing import AnyStr
from typing import Any, AnyStr
import cv2
import numpy as np
@ -886,6 +886,11 @@ def vainfo_hwaccel() -> sp.CompletedProcess:
return sp.run(ffprobe_cmd, capture_output=True)
def get_ffmpeg_arg_list(arg: Any) -> list:
"""Use arg if list or convert to list format."""
return arg if isinstance(arg, list) else arg.split(" ")
class FrameManager(ABC):
@abstractmethod
def create(self, name, size) -> AnyStr: