Commit Graph

84 Commits

Author SHA1 Message Date
Nicolas Mowen
09cc4251f0
Fix dict getting overwritten (#6830) 2023-06-17 10:31:35 -05:00
Nicolas Mowen
2be2050d57
Don't fail to save segment when cv2 fails (#6823)
* Don't fail when cv2 fails

* Clean up exception handling
2023-06-17 09:56:00 -05:00
Nicolas Mowen
b66810247e
Optimize stacked recordings (#6809)
* Make camera recordings mover asynchronous

* Formatting

* Move to using cv2 instead of external ffmpeg process

* Use ffprobe if cv2 failed

* Formatting

* Fix bad access

* Formatting

* Update frigate/record/maintainer.py

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>

* Update name of caller

---------

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2023-06-16 07:09:13 -06:00
Nicolas Mowen
ec4d79eafc
Fix bug in intersection logic (#6780)
* Fix bug in intersection logic

* Fix isort

* Remove unrelated test

* Formatting

* Fix type in test
2023-06-12 17:38:22 -05:00
Sergey Krashevich
8d941e5e26
Add function to get physical interfaces for bandwidth calculation (#6618)
* Add function to get physical interfaces for bandwidth calculation in get_bandwidth_stats() function

* Add telemetry configuration option for enabled network interfaces, with default values for monitoring bandwidth stats for camera ffmpeg processes, go2rtc, and object detectors. Also add support for FrigateConfig in set_bandwidth_stats function to get bandwidth stats for specified network interfaces
2023-06-11 07:34:03 -05:00
Martin Weinelt
ab50d0b006
Add isort and ruff linter (#6575)
* Add isort and ruff linter

Both linters are pretty common among modern python code bases.

The isort tool provides stable sorting and grouping, as well as pruning
of unused imports.

Ruff is a modern linter, that is very fast due to being written in rust.
It can detect many common issues in a python codebase.

Removes the pylint dev requirement, since ruff replaces it.

* treewide: fix issues detected by ruff

* treewide: fix bare except clauses

* .devcontainer: Set up isort

* treewide: optimize imports

* treewide: apply black

* treewide: make regex patterns raw strings

This is necessary for escape sequences to be properly recognized.
2023-05-29 05:31:17 -05:00
Sergey Krashevich
6d0c2ec5c8
Add go2rtc & remote detectors network bandwidth usage to System table (#6526)
* Add network bandwidth usage to System table display in System.jsx and update get_bandwidth_stats function in util.py to include go2rtc processes

* black...

* Add network bandwidth usage to system table in web UI and improve regex in get_bandwidth_stats function to include frigate detector processes

* black...

* Update bandwidth calculation to include both incoming and outgoing traffic

* black:(
2023-05-18 06:40:24 -05:00
Korneliusz Jarzębski
17e8a46c7d
add ffmpeg bandwidth stats (#6492)
* add ffmpeg bandwidth stats

* add ffmpeg bandwidth stats

* Change column name

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* fix lint formatting

---------

Co-authored-by: Korneliusz Jarzębski <k.jarzebski@netverse.pl>
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-05-17 20:01:56 -05:00
Sergey Krashevich
5fb96c777a
Add cmdline information to CPU usage stats in get_cpu_stats() function and display it in the System page's ffmpeg table with a copy-to-clipboard button (#6430) 2023-05-15 07:39:03 -05:00
Sergey Krashevich
0fcfcb85ab
Implement NVML for NVIDIA GPU Stats (#6359)
* nvml

* black...black...black...

* small fix for avoid errors on strange GPUs and old drivers

* fix type errors

* fix type errors

* fix unittest process crash

where the tests for tests?..

* it's impossible to mock low-level library

* fix double % for other GPU types

* remove space before gpu statistic values
2023-05-04 18:02:01 -05:00
Sergey Krashevich
2add675d42
Refactor Process Stats and Bugfixes (#6344)
* test refactor process stats

* Update util.py

* bugfix

* black formatting

* add missing processes field to StatsTrackingTypes class

* fix python checks and tests...

* use psutil for calcilate cpu utilization on get_cpu_stats

* black...black...black...

* add cpu average

* calculate statiscts for logger process

* add P-ID for other processes in System.jsx

* Apply suggestions from code review

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* make page beautiful again :)

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-05-04 17:58:59 -05:00
Sergey Krashevich
b38c9e82e2
Replace subprocess usage with os module for better performance and maintainability (#6298)
* avoid executing external tools by using Python's built-in os module to interact with the filesystem directly

* Refactor recording cleanup script to use os module instead of subprocess

* black format util.py

* Ooooops

* Refactor get_cpu_stats() to properly identify recording process
2023-05-01 21:22:35 -05:00
Nicolas Mowen
37360edbc1
Show other system processes in system page (#6276) 2023-04-28 07:47:49 -05:00
Blake Blackshear
0d16bd0144
False positives (#6217)
* add false positive submission

* switch timeline events to x,y,w,h

* update docs

* fix type checks

* convert to upsert

* fix config test
2023-04-24 07:24:28 -05:00
Blake Blackshear
15415bab10
update deps (#6093)
* update dev deps

* update deps

* docs deps

* python dev deps

* update python deps
2023-04-15 14:17:59 -05:00
Nicolas Mowen
3f17f871fa
Fix cleaning logs with rtsp in middle (#5800) 2023-03-21 17:21:37 -05:00
Nicolas Mowen
562e2627c2
Auto select gpu for hwaccel presets (#5406)
* Add ability to GPU device to be automatically detected when multiple exist

* Add logging info

* Fix access

* Fix

* Formatting

* Fix path of device

* Use log error instead of raise

* Remove log which could apply to other caess

* Set default value

* rework logic and support auto gpu selection for encoding gpu as well
2023-02-11 08:00:58 -06:00
jvrobert
7083a5c9b6
Try to limit nvidia GPU queries to included GPUs (#5356)
* Try to limit nvidia GPU queries to included GPUs

* ignore non digit GPU indexes

* formatting

* Formatting

* Remove trailing spaces

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-02-03 19:34:07 -06:00
Felipe Santos
feeb7107d5
Fix Frigate stuck at finishing when restarting from UI (#5216) 2023-01-24 07:27:57 -06:00
Felipe Santos
02df2a8bbd
Refactor s6 scripts to the new format (#5135)
* Refator s6 scripts to the new format

* Remove unneeded workaround

* Migrate logging to new s6 format

* Remove more unnecessary s6 variables

* Fix prepare-log and when go2rtc is not present in config

* Restart the whole container if either Frigate or go2rtc fails

* D

* Fix service name in finish

* Fix nginx finish comment

* Restart improvements

* Fix devcontainer

* Fix format

* Update Dockerfile

Co-authored-by: Felipe Santos <felipecassiors@gmail.com>

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-01-18 07:53:53 -06:00
Blake Blackshear
3bec28ffef
handle timezones with partial hour offsets (#5115)
* handle timezones with partial hour offsets

* cleanup
2023-01-16 17:17:03 -06:00
James L
ddcae2d4aa
Fix mem usage reporting when using docker limits (#5011)
* Fix mem usage reporting when using docker limits

* format code

* wip
2023-01-11 17:39:26 -06:00
Nicolas Mowen
61243ad34b
Make error messages for gpu stats more clear (#4962)
* Make error messages for gpu stats more clear

* Make error messages for gpu stats more clear
2023-01-08 14:00:46 -06:00
Nate Meyer
8ac3114f9a
Cleanup Detector labelmap (#4932)
* Add missing labels to default labelmap.  Fill any holes with "unknown".  Remove unique labelmap for tensorrt.

* Replace "truck" with "car" on Openvino labelmap
2023-01-06 07:03:16 -06:00
Nicolas Mowen
43c2761308
System page improvements (#4707)
* Catch case where camera is disabled

* Add ffprobe timeout and catch case where camera is disabled
2022-12-14 17:20:47 -06:00
Nicolas Mowen
4f79ca1bf0
Fix ffprobe with special characters (#4646)
* Clean path from scpecial characters before running ffprobe

* Fix camera paths
2022-12-08 19:02:11 -06:00
Nicolas Mowen
007fa75294
Fix intel gpu stats (#4581)
* Fix return code for timeout call

* Rewrit intel gpu logic

* Fix test and output
2022-12-03 10:28:01 -06:00
Felipe Santos
4523c9b06d
Set User Agent for FFmpeg calls (#4555)
* Set User Agent for FFmpeg calls

* Allow to use shell-like string args

* Add test for arg as string with space
2022-11-30 16:53:45 -06:00
Nicolas Mowen
87144cd572
FEAT: Support for ffmpeg presets (#3840)
* Add hwaccel presets

* Use hwaccel presets

* Add input arg presets

* Use input arg presets

* Make util to clean up redundant code

* Add support for output arg presets

* Add tests

* Update camera specific to use presets

* Update hwaccel to use presets

* Format files and fix tests

* Rewrite tests to test record correctly

* Move presets from string to list to avoid manually separating into a list

* Add mjpeg cuvid decoder preset

* Fix tests

* Fix comment
2022-11-28 21:48:11 -06:00
Nicolas Mowen
aaedd24f37
Add GPU stats to the /stats API and debug screen (#3931)
* Add ffprobe endpoint

* Get ffprobe for multiple inputs

* Copy ffprobe in output

* Fix bad if statement

* Return full output of ffprobe process

* Return full output of ffprobe process

* Make ffprobe button show dialog with output and option to copy

* Add driver names to consts

* Add driver env var name

* Setup general tracking for GPU stats

* Catch RPi args as well

* Add util to get radeontop results

* Add real amd GPU stats

* Fix missed arg

* pass config

* Use only the values

* Fix vram

* Add nvidia gpu stats

* Use nvidia stats

* Add chart for gpu stats

* Format AMD with space between percent

* Get correct nvidia %

* Start to add support for intel GPU stats

* Block out RPi as util is not currently available

* Formatting

* Fix mypy

* Strip for float conversion

* Strip for float conversion

* Fix percent formatting

* Remove name from gpu map

* Add tests and fix AMD formatting

* Add nvidia gpu stats test

* Formatting

* Add intel_gpu_top for testing

* Formatting

* Handle case where hwaccel is not setup

* Formatting

* Check to remove none

* Don't use set

* Cleanup and fix types

* Handle case where args is list

* Fix mypy

* Cast to str

* Fix type checking

* Return none instead of empty

* Fix organization

* Make keys consistent

* Make gpu match style

* Get support for vainfo

* Add vainfo endpoint

* Set vainfo output in error correctly

* Remove duplicate function

* Fix errors

* Do cpu & gpu work asynchonously

* Fix async

* Fix event loop

* Fix crash

* Fix naming

* Send empty data for gpu if error occurs

* Show error if gpu stats could not be retrieved

* Fix mypy

* Fix test

* Don't use json for vainfo

* Fix cross references

* Strip unicode still

* await vainfo response

* Add gpu deps

* Formatting

* remove comments

* Use empty string

* Add vainfo back in
2022-11-28 19:24:20 -06:00
Nicolas Mowen
68248cc274
Clean http password, clean path in debug config as well (#4519)
* Clean ffmpeg paths

* Clean http passwords too

* Make check optional
2022-11-26 19:18:33 -06:00
Blake Blackshear
047c2408d2
fix support for yuv models (#4523) 2022-11-26 19:15:47 -06:00
Nicolas Mowen
c97aac6c94
Revamped debug UI and add camera / process info, ffprobe copying (#4349)
* Move each camera to a separate card and show per process info

* Install top

* Add support for cpu usage stats

* Use cpu usage stats in debug

* Increase number of runs to ensure good results

* Add ffprobe endpoint

* Get ffprobe for multiple inputs

* Copy ffprobe in output

* Add fps to camera metrics

* Fix lint errors

* Update stats config

* Add ffmpeg pid

* Use grid display so more cameras can take less vertical space

* Fix hanging characters

* Only show the current detector

* Fix bad if statement

* Return full output of ffprobe process

* Return full output of ffprobe process

* Don't specify rtsp_transport

* Make ffprobe button show dialog with output and option to copy

* Adjust ffprobe api to take paths directly

* Add docs for ffprobe api
2022-11-13 12:48:14 -06:00
Nicolas Mowen
9e31873520
More config checks (#4310)
* Move existing checks to own functions

* Add config check for zone objects that are not tracked

* Add tests for config error

* Formatting

* Catch case where field is defined multiple times and add test

* Add warning for rtmp
2022-11-08 19:47:45 -06:00
Nate Meyer
40cb5102b0
Support more special chars in camera passwords (#4259)
* Support more special chars in camera passwords

* End password test with double @ chars

* Escape passwords in paths for go2rtc

* Fixes for formatting
2022-11-05 21:42:32 -05:00
Nate Meyer
4383b883c0
Refactor to simplify support for additional detector types (#3656)
* Refactor EdgeTPU and CPU model handling to detector submodules.

* Fix selecting the correct detection device type from the config

* Remove detector type check when creating ObjectDetectProcess

* Fixes after rebasing to 0.11

* Add init file to detector folder

* Rename to detect_api

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* Add unit test for LocalObjectDetector class

* Add configuration for model inputs
Support transforming detection regions to RGB or BGR.
Support specifying the input tensor shape.  The tensor shape has a standard format ["BHWC"] when handed to the detector, but can be transformed in the detector to match the model shape using the model  input_tensor config.

* Add documentation for new model config parameters

* Add input tensor transpose to LocalObjectDetector

* Change the model input tensor config to use an enumeration

* Updates for model config documentation

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2022-11-03 21:23:09 -05:00
Nicolas Mowen
1bc9efd529
Support special characters in passwords, redacted logs & debug config (#4057)
* Consts for regex

* Add regex for camera username and password

* Redact user:pass from ffmpeg logs

* Redact ffmpeg commands

* Move common function to util

* Add tests

* Formatting

* Remove unused imports

* Fix test

* Add port to test

* Support special characters in passwords

* Add tests for special character handling

* Remove docs about not supporting special characters
2022-11-02 07:00:54 -05:00
Sebastian Englbrecht
c6234bf548 fix depreciated import from collections 2022-04-15 07:01:43 -05:00
Nick
045aac8933
Add object filter ratio (#2952)
* Add object ratio config parameters

Issue: #2948

* Add config test for object filter ratios

Issue: #2948

* Address review comments

- Accept `ratio` default
- Rename `bounds` to `box` for consistency
- Add migration for new field

Issue: #2948

* Fix logical errors

- field migrations require default values
- `clipped` referenced the wrong index for region, since it shifted
- missed an inclusion of `ratio` for detections in `process_frames`
- revert naming `o[2]` as `box` since it is out of scope!

This has now been test-run against a video, so I believe the kinks are
worked out.

Issue: #2948

* Update contributing notes for `make`

Issue: #2948

* Fix migration

- Ensure that defaults match between Event and migration script
- Deconflict migration script number (from rebase)

Issue: #2948

* Filter objects out of ratio bounds

Issue: #2948

* Update migration file to 009

Issue: #2948
2022-04-10 08:25:18 -05:00
Blake Blackshear
b82d75b79e avoid rare divide by zero 2022-02-18 21:18:26 -06:00
Yuriy Sannikov
80627e4989 safe refactoring (#2552)
Co-authored-by: YS <ys@gm.com>
2022-02-18 21:18:26 -06:00
Blake Blackshear
e36099a342 improve box merging and keep tracking 2022-02-18 21:18:26 -06:00
Blake Blackshear
a943ac1308 use s6 to shutdown frigate 2021-09-18 07:40:27 -05:00
Blake Blackshear
f63a7cb6c0 remove font_scale in timestamp_style and calculate dynamically again 2021-09-04 16:34:48 -05:00
ElMoribond
fe2e32c84f Add ability to restart 2021-07-06 07:36:37 -05:00
ElMoribond
c6e53d0ea8 Add ability to restart 2021-07-06 07:36:37 -05:00
ElMoribond
11d8095cdd Add ability to restart 2021-07-06 07:36:37 -05:00
Jason Hunter
8d01cc4807 fix ffmpeg config and remove side effects 2021-06-25 06:13:08 -05:00
Jason Hunter
c664bd63f6 initial conversion to pydantic 2021-06-24 06:53:05 -05:00
Blake Blackshear
f009897eca clear position before copy 2021-06-23 08:15:15 -05:00