Commit Graph

61 Commits

Author SHA1 Message Date
Nicolas Mowen
5ad391977e
FEAT: Storage API & Frontend (#3409)
* Get storage output stats for each camera

* Add storage route

* Add storage route

* Add storage page

* Cleanup

* Add stats and show more storage

* Add tests for mb abbrev util fun

* Rewrite storage logic to use storage maintainer and segment sizes

* Include storage maintainer for http

* Use correct format

* Remove utils

* Fix tests

* Remove total from equation

* Multiply by 100 to get percent

* Add basic storage info

* Fix storage stats

* Fix endpoint and ui

* Fix formatting
2022-11-29 19:59:56 -06:00
Nicolas Mowen
87144cd572
FEAT: Support for ffmpeg presets (#3840)
* Add hwaccel presets

* Use hwaccel presets

* Add input arg presets

* Use input arg presets

* Make util to clean up redundant code

* Add support for output arg presets

* Add tests

* Update camera specific to use presets

* Update hwaccel to use presets

* Format files and fix tests

* Rewrite tests to test record correctly

* Move presets from string to list to avoid manually separating into a list

* Add mjpeg cuvid decoder preset

* Fix tests

* Fix comment
2022-11-28 21:48:11 -06:00
Nicolas Mowen
aaedd24f37
Add GPU stats to the /stats API and debug screen (#3931)
* Add ffprobe endpoint

* Get ffprobe for multiple inputs

* Copy ffprobe in output

* Fix bad if statement

* Return full output of ffprobe process

* Return full output of ffprobe process

* Make ffprobe button show dialog with output and option to copy

* Add driver names to consts

* Add driver env var name

* Setup general tracking for GPU stats

* Catch RPi args as well

* Add util to get radeontop results

* Add real amd GPU stats

* Fix missed arg

* pass config

* Use only the values

* Fix vram

* Add nvidia gpu stats

* Use nvidia stats

* Add chart for gpu stats

* Format AMD with space between percent

* Get correct nvidia %

* Start to add support for intel GPU stats

* Block out RPi as util is not currently available

* Formatting

* Fix mypy

* Strip for float conversion

* Strip for float conversion

* Fix percent formatting

* Remove name from gpu map

* Add tests and fix AMD formatting

* Add nvidia gpu stats test

* Formatting

* Add intel_gpu_top for testing

* Formatting

* Handle case where hwaccel is not setup

* Formatting

* Check to remove none

* Don't use set

* Cleanup and fix types

* Handle case where args is list

* Fix mypy

* Cast to str

* Fix type checking

* Return none instead of empty

* Fix organization

* Make keys consistent

* Make gpu match style

* Get support for vainfo

* Add vainfo endpoint

* Set vainfo output in error correctly

* Remove duplicate function

* Fix errors

* Do cpu & gpu work asynchonously

* Fix async

* Fix event loop

* Fix crash

* Fix naming

* Send empty data for gpu if error occurs

* Show error if gpu stats could not be retrieved

* Fix mypy

* Fix test

* Don't use json for vainfo

* Fix cross references

* Strip unicode still

* await vainfo response

* Add gpu deps

* Formatting

* remove comments

* Use empty string

* Add vainfo back in
2022-11-28 19:24:20 -06:00
Nicolas Mowen
4eb3e98076
Update go2rtc, simplify manual stream, and support audio for MSE & WebRTC (#4391)
* Update go2rtc to rc3

* Simplify ffmpeg / audio conversions

* Set ffmpeg bin location

* Manually set video as copied

* Run go2rtc with env vars

* Remove manual ffmpeg declaration

* Enable force_audio by default

* Fix test
2022-11-16 06:29:47 -06:00
Nicolas Mowen
9e31873520
More config checks (#4310)
* Move existing checks to own functions

* Add config check for zone objects that are not tracked

* Add tests for config error

* Formatting

* Catch case where field is defined multiple times and add test

* Add warning for rtmp
2022-11-08 19:47:45 -06:00
Nate Meyer
40cb5102b0
Support more special chars in camera passwords (#4259)
* Support more special chars in camera passwords

* End password test with double @ chars

* Escape passwords in paths for go2rtc

* Fixes for formatting
2022-11-05 21:42:32 -05:00
Nate Meyer
4383b883c0
Refactor to simplify support for additional detector types (#3656)
* Refactor EdgeTPU and CPU model handling to detector submodules.

* Fix selecting the correct detection device type from the config

* Remove detector type check when creating ObjectDetectProcess

* Fixes after rebasing to 0.11

* Add init file to detector folder

* Rename to detect_api

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* Add unit test for LocalObjectDetector class

* Add configuration for model inputs
Support transforming detection regions to RGB or BGR.
Support specifying the input tensor shape.  The tensor shape has a standard format ["BHWC"] when handed to the detector, but can be transformed in the detector to match the model shape using the model  input_tensor config.

* Add documentation for new model config parameters

* Add input tensor transpose to LocalObjectDetector

* Change the model input tensor config to use an enumeration

* Updates for model config documentation

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2022-11-03 21:23:09 -05:00
Nicolas Mowen
1bc9efd529
Support special characters in passwords, redacted logs & debug config (#4057)
* Consts for regex

* Add regex for camera username and password

* Redact user:pass from ffmpeg logs

* Redact ffmpeg commands

* Move common function to util

* Add tests

* Formatting

* Remove unused imports

* Fix test

* Add port to test

* Support special characters in passwords

* Add tests for special character handling

* Remove docs about not supporting special characters
2022-11-02 07:00:54 -05:00
Nicolas Mowen
d8123d2497
Add go2rtc and add restream role / live source (#4082)
* Pull go2rtc dependency

* Add go2rtc to local services and add to s6

* Add relay controller for go2rtc

* Add restream role

* Add restream role

* Add restream to nginx

* Add camera live source config

* Disable RTMP by default and use restream

* Use go2rtc for camera config

* Fix go2rtc move

* Start restream on frigate start

* Send restream to camera level

* Fix restream

* Make sure jsmpeg works as expected

* Make view rspect live size config

* Tweak player options to fit live view

* Adjust VideoPlayer to accept live option which disables irrelevant controls

* Add multiple options from restream live view

* Add base for webrtc option

* Setup specific restream modules

* Make mp4 the default streaming for now

* Expose 8554 for rtsp relay from go2rtc

* Formatting

* Update docs to suggest new restream method.

* Update docs to reflect restream role

* Update docs to reflect restream role

* Add webrtc player

* Improvements to webRTC

* Support webrtc

* Cleanup

* Adjust rtmp test and add restream test

* Fix tests

* Add restream tests

* Add live view docs and show different options

* Small docs tweak

* Support all stream types

* Update to beta 9 of go2rtc

* Formatting

* Make jsmpeg the default

* Support wss if made from https

* Support wss if made from https

* Use onEffect

* Set url outside onEffect

* Fix passed deps

* Update docs about required host mode

* Try memo instead

* Close websocket on changing camera

* Formatting

* Close pc connection

* Set video source to null on cleanup

* Use full path since go2rtc can't see PATH var

* Adjust audio codec to enable browser audio by default

* Cleanup stream creation

* Add restream tests

* Format tests

* Mock requests

* Adjust paths

* Move stream configs to restream

* Remove live source

* Remove live config

* Use live persistence for which view to use on each camera

* Fix live sizes

* Only use jsmpeg sizes for jsmpeg live

* Set max live size

* Remove access of live config

* Add selector for live view source in web view

* Remove RTMP from default list of roles

* Update docs

* Fix tests

* Fix docs for live view modes

* make default undefined to avoid race condition

* Wait until camera source is loaded to avoid race condition

* Fix tests

* Add config to go2rtc

* Work with config

* Set full path for config

* Set to use stun

* Check for mounted file

* Look for frigate-go2rtc

* Update docs to reflect webRTC configuration.

* Add link to go2rtc config

* Update docs to be more clear

* Update docs to be more clear

* Update format

Co-authored-by: Felipe Santos <felipecassiors@gmail.com>

* Update live docs

* Improve bash startup script

* Add option to force audio compatibility

* Formatting

* Fix mapping

* Fix broken link

* Update go2rtc version

* Get go2rtc webui working

* Add support for mse

* Remove mp4 option

* Undo changes to video player

* Update docs for new live view options

* Make separate path for mse

* Remove unused

* Remove mp4 path

* Try to get go2rtc proxy working

* Try to get go2rtc proxy working

* Remove unused callback

* Allow websocket on restrea dashboard

* Make mse default stream option

* Fix mse sizing

* don't assume roles is defined

* Remove nginx mapping to go2rtc ui

Co-authored-by: Felipe Santos <felipecassiors@gmail.com>
Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2022-11-02 06:36:09 -05:00
Nicolas Mowen
b4d4adb75b
Limit recording retention to available storage (#3942)
* Add field and migration for segment size

* Store the segment size in db

* Add comment

* Add default

* Fix size parsing

* Include segment size in recordings endpoint

* Start adding storage maintainer

* Add storage maintainer and calculate average sizes

* Update comment

* Store segment and hour avg sizes per camera

* Formatting

* Keep track of total segment and hour averages

* Remove unused files

* Cleanup 2 hours of recordings at a time

* Formatting

* Fix bug

* Round segment size

* Cleanup some comments

* Handle case where segments are not deleted on initial run or is only retained segments

* Improve cleanup log

* Formatting

* Fix typo and improve logging

* Catch case where no recordings exist for camera

* Specifically define sort

* Handle edge case for cameras that only record part time

* Increase definition of part time recorder

* Remove warning about not supported storage based retention

* Add note about storage based retention to recording docs

* Add tests for storage maintenance calculation and cleanup

* Format tests

* Don't run for a camera with no recording segments

* Get size of file from cache

* Rework camera stats to be more efficient

* Remove total and other inefficencies

* Rewrite storage cleanup logic to be much more efficient

* Fix existing tests

* Fix bugs from tests

* Add another test

* Improve logging

* Formatting

* Set back correct loop time

* Update name

* Update comment

* Only include segments that have a nonzero size

* Catch case where camera has 0 nonzero segment durations

* Add test to cover zero bandwidth migration case

* Fix test

* Incorrect boolean logic

* Formatting

* Explicity re-define iterator
2022-10-09 06:28:26 -05:00
Blake Blackshear
37325c70ba fix plus enabled for addons 2022-07-22 08:12:58 -05:00
Nicolas Mowen
c2465a46a8
Http tests (#3350)
* Set up for http tests

* Setup basics for testing and first test

* Add testing consts

* Cleanup db creation

* Add one more check to test

* Get event that does not exist

* Get events working with cleaner db

* Test retain / un-retain

* Test setting and deleting sub label

* Test getting list of sub labels

* Fix bug caught in tests

* Test deleting event

* Test geting list of events

* Expand test

* Test more event filters

* Write version module so tests don't fail on version import

* Test config

* Test recordings endpoint

* Formatting

* Remove unused imports

* Test stats

* Add cleanup files in const

* Add name to match other checks
2022-06-30 07:53:46 -05:00
Blake Blackshear
a1afade9ba fix birdseye config 2022-04-16 09:08:10 -05:00
Blake Blackshear
12abbc59d6 add lint check to workflow 2022-04-16 09:08:10 -05:00
Nicolas Mowen
d749cf2e6b
Allow birdseye to be overridden at the camera level (#3083)
* Add camera level processing for birdseye

* Add camera level birdseye configruation

* Propogate birdseye from global

* Update docs to show that birdseye is overridable

* Fix incorrect default factory

* Update note to indicate values that can be overridden

* Cleanup config accessing

* Add tests for birdseye config behavior

* Fix mistake on test format

* Update tests
2022-04-15 06:59:30 -05:00
Nick
045aac8933
Add object filter ratio (#2952)
* Add object ratio config parameters

Issue: #2948

* Add config test for object filter ratios

Issue: #2948

* Address review comments

- Accept `ratio` default
- Rename `bounds` to `box` for consistency
- Add migration for new field

Issue: #2948

* Fix logical errors

- field migrations require default values
- `clipped` referenced the wrong index for region, since it shifted
- missed an inclusion of `ratio` for detections in `process_frames`
- revert naming `o[2]` as `box` since it is out of scope!

This has now been test-run against a video, so I believe the kinks are
worked out.

Issue: #2948

* Update contributing notes for `make`

Issue: #2948

* Fix migration

- Ensure that defaults match between Event and migration script
- Deconflict migration script number (from rebase)

Issue: #2948

* Filter objects out of ratio bounds

Issue: #2948

* Update migration file to 009

Issue: #2948
2022-04-10 08:25:18 -05:00
Blake Blackshear
69aaf1f8e6 require url safe camera names 2022-02-18 21:18:26 -06:00
Yuriy Sannikov
6eecb6780e Run python unit tests in a github actions (#2589)
* tox tests initial commit

* run tests in the Dockerfile during the build phase

* remove local tests

Co-authored-by: YS <ys@gm.com>
2022-02-18 21:18:26 -06:00
Blake Blackshear
e36099a342 improve box merging and keep tracking 2022-02-18 21:18:26 -06:00
Blake Blackshear
26ae6084ea fix rtmp again 2021-10-24 13:53:43 -05:00
Blake Blackshear
a67b8ab84d validate with runtime config (fixes #2055) 2021-10-23 08:21:15 -05:00
Blake Blackshear
c3109f808c allow partial days in retention settings 2021-09-20 18:59:16 -05:00
Blake Blackshear
661f7baa21 fix global live config 2021-09-13 20:33:00 -05:00
Blake Blackshear
288b1a0562 remove nested enabled config setting on events 2021-09-08 08:02:26 -05:00
Blake Blackshear
e8eb3125a5 disallow extra keys in config 2021-09-04 16:56:01 -05:00
drinfernoo
65855e23d9
Add RTMP and timestamp style to global config (#1674)
* 📝🔧 - Make RTMP config global

Fixes #1671

* 📝🔧 - Make timestamp style config global

Fixes #1656

* fix test function names

* formatter

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2021-09-03 07:03:36 -05:00
Blake Blackshear
fbea51372f sync global snapshot options (fixes #1621) 2021-08-28 09:14:00 -05:00
Blake Blackshear
fa5ec8d019 cleanup global and camera detect config (fixes #1615) 2021-08-28 08:51:51 -05:00
Blake Blackshear
11c425a7eb error on invalid role 2021-08-28 08:16:25 -05:00
Blake Blackshear
45798d6d14 clean house on clips 2021-08-15 08:30:27 -05:00
Blake Blackshear
f3a1c1de0a move width/height/fps under detect and make required
also resizes the output from ffmpeg to specified size
2021-08-14 14:18:35 -05:00
Jason Hunter
a476bc9885 initial commit 2021-08-07 15:51:16 -05:00
Blake Blackshear
ebee204ca0 ensure default args propogate 2021-07-10 06:40:50 -05:00
Blake Blackshear
92aa16c556 add labelmap customization to the config (fixes #507) 2021-07-07 22:58:18 -05:00
Blake Blackshear
a3853af47a fix test 2021-07-07 22:58:18 -05:00
Blake Blackshear
b4e7e51e63 make contour_area dynamic 2021-07-01 07:54:09 -05:00
Blake Blackshear
2a41a9d3ff set min motion frame to 180 2021-07-01 07:54:09 -05:00
Blake Blackshear
f9457be109 add test for max_disappeared default 2021-06-27 08:13:48 -05:00
Jason Hunter
8d01cc4807 fix ffmpeg config and remove side effects 2021-06-25 06:13:08 -05:00
Jason Hunter
c664bd63f6 initial conversion to pydantic 2021-06-24 06:53:05 -05:00
Blake Blackshear
fa61e9da29 maintain aspect ratio for birdseye 2021-06-23 08:15:15 -05:00
Blake Blackshear
c70419bd0b update birdseye layout calculations 2021-06-12 07:23:14 -05:00
Sean Vig
84a0827aee Use dataclasses for config handling
Use config data classes to eliminate some of the boilerplate associated
with setting up the configuration.  In particular, using dataclasses
removes a lot of the boilerplate around assigning properties to the
object and allows these to be easily immutable by freezing them.  In the
case of simple, non-nested dataclasses, this also provides more
convenient `asdict` helpers.

To set this up, where previously the objects would be parsed from the
config via the `__init__` method, create a `build` classmethod that does
this and calls the dataclass initializer.

Some of the objects are mutated at runtime, in particular some of the
zones are mutated to set the color (this might be able to be refactored
out) and some of the camera functionality can be enabled/disabled.  Some
of the configs with `enabled` properties don't seem to have mqtt hooks
to be able to toggle this, in particular, the clips, snapshots, and
detect can be toggled but rtmp and record configs do not, but all of
these configs are still not frozen in case there is some other
functionality I am missing.

There are a couple other minor fixes here, one that was introduced
by me recently where `max_seconds` was not defined, the other to
properly `get()` the message payload when handling publishing mqtt
messages sent via websocket.
2021-05-23 20:38:57 -05:00
Blake Blackshear
39ff49e054 formatting cleanup 2021-02-25 07:01:57 -06:00
Blake Blackshear
ce90ae343c add global object mask 2021-02-20 08:20:17 -06:00
Blake Blackshear
d51e9446ff add camera level ffmpeg params 2021-02-20 08:20:17 -06:00
Blake Blackshear
3fab321045 update object filters to inherit like motion settings 2021-01-26 21:40:33 -06:00
Blake Blackshear
63e14a98f9 add retention settings for snapshots 2021-01-26 21:40:33 -06:00
Blake Blackshear
80a5a7b129 cleanup save_Clips/clips inconsistency 2021-01-26 21:40:33 -06:00
Blake Blackshear
8f68df60c7 fix test 2021-01-26 21:40:33 -06:00