Commit Graph

844 Commits

Author SHA1 Message Date
Sergey Krashevich
5c27cb7e9b
Optimization of Sync Records: Implementing Pagination and Temporary Table (#6585)
* Update pull_request.yml

* Add temporary table for deletion and use pagination to process recordings in chunks for deletion of recordings with missing files

* move RecordingsToDelete class to models.py

* recording cleanup: bugfixes

* Update cleanup.py

* improve log message in cleanup.py

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-06-11 08:01:50 -05:00
Nicolas Mowen
b160abac0d
Ability to configure min frames for zone presence (#6680)
* Objects need to be in zones multiple times to be considered present in the zone

* Add a field to configure inertia per zone

* Formatting

* Use correct default method

* Clarify zone presence behavior

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>

---------

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2023-06-11 08:00:53 -05:00
Nicolas Mowen
fd6eb78f41
Birdseye Autolayout (#6734)
* Calculate possible layout

* Working in ideal conditions

* Fix issues with different heights

* Remove logs

* Optimally handle cameras that don't match the canvas aspect ratio

* Make sure to copy so list is not overwritten

* Remove unused import

* Remove try catch

* Optimize layout for low amount of cameras

* Try to scale frames up if not enough space is used
2023-06-11 07:54:18 -05:00
Sergey Krashevich
8d941e5e26
Add function to get physical interfaces for bandwidth calculation (#6618)
* Add function to get physical interfaces for bandwidth calculation in get_bandwidth_stats() function

* Add telemetry configuration option for enabled network interfaces, with default values for monitoring bandwidth stats for camera ffmpeg processes, go2rtc, and object detectors. Also add support for FrigateConfig in set_bandwidth_stats function to get bandwidth stats for specified network interfaces
2023-06-11 07:34:03 -05:00
Nicolas Mowen
8bc76d19db
DB Optimizations (#6712)
* Enable auto vacuums

* Enable auto vacuum

* Fix separator

* Fix separator and remove incorrect log

* Limit to 1 row since that is all that is used

* Add index on camera + segment_size

* Formatting

* Increase timeout and cache_size

* Set DB mode to NORMAL synchronous level

* Formatting

* Vacuum every 2 weeks

* Remove fstring

* Use string

* Use consts
2023-06-11 07:23:18 -05:00
Nicolas Mowen
435d152423
Fix bug introduced in new linter (#6754)
* Fix bug introduced in new linter

* Ignore this error

* Fix more

* Ignore boolean error too
2023-06-11 07:18:47 -05:00
Nicolas Mowen
d3949eebfa
Add API and WebUI to export recordings (#6550)
* Add ability to export frigate clips

* Add http endpoint

* Add dir to nginx

* Add webUI

* Formatting

* Cleanup unused

* Optimize timelapse

* Fix pts

* Use JSON body for params

* Use hwaccel to encode when available

* Print ffmpeg command when fail

* Print ffmpeg command when fail

* Add separate ffmpeg preset for timelapse

* Add docs outlining the export directory

* Add export docs

* Use ''

* Fix playlist max time

* Lower max playlist time

* Add api docs for export

* isort fixes
2023-06-08 07:32:35 -04:00
Nicolas Mowen
62f36b221e
Remove ffmpeg banner (#6600)
* Hide banner in maintainer

* Hide banner in event download
2023-06-01 05:46:34 -05:00
Blake Blackshear
ae0aba44dc
Improve tracking (#6516) 2023-05-31 08:12:43 -06:00
Martin Weinelt
ab50d0b006
Add isort and ruff linter (#6575)
* Add isort and ruff linter

Both linters are pretty common among modern python code bases.

The isort tool provides stable sorting and grouping, as well as pruning
of unused imports.

Ruff is a modern linter, that is very fast due to being written in rust.
It can detect many common issues in a python codebase.

Removes the pylint dev requirement, since ruff replaces it.

* treewide: fix issues detected by ruff

* treewide: fix bare except clauses

* .devcontainer: Set up isort

* treewide: optimize imports

* treewide: apply black

* treewide: make regex patterns raw strings

This is necessary for escape sequences to be properly recognized.
2023-05-29 05:31:17 -05:00
Martin Weinelt
1e17dbaa91
deepstack: pass api key in request (#6579)
The POST data was prepared, but not passed into the request call.
2023-05-23 17:32:08 -05:00
Nicolas Mowen
f66ccb04f6
Fix pydantic (#6584) 2023-05-23 08:00:21 -06:00
Sergey Krashevich
11738110dc
Add handling for non-existent or empty model file paths (#6525)
* Add handling for non-existent or empty model file paths in `compute_model_hash()` method

* Remove print() in detector_config.py

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>

---------

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2023-05-21 08:29:57 -05:00
Nicolas Mowen
4940826011
Fix false positives (#6544) 2023-05-19 13:26:19 -06:00
Nicolas Mowen
fec1dd3f46
Fix breakages by manual_event_api branch (#6542)
* Fix breakages by manual_event_api branch

* More fixes
2023-05-19 10:59:24 -06:00
Blake Blackshear
42f3ee3de5 sync master 2023-05-19 05:47:49 -05:00
Nicolas Mowen
e357715a8c
Ability to manually create events through the API (#3184)
* Move to events package

* Improve handling of external events

* Handle external events in the event queue

* Pass in event processor

* Check event json

* Fix json parsing and change defaults

* Fix snapshot saving

* Hide % score when not available

* Correct docs and add json example

* Save event png db

* Adjust image

* Formatting

* Add catch for failure ending event

* Add init to modules

* Fix naming

* Formatting

* Fix http creation

* fix test

* Change to PUT and include response in docs

* Add ability to set bounding box locations in snapshot

* Support multiple box annotations

* Cleanup docs example response

Co-authored-by: Blake Blackshear <blake@frigate.video>

* Cleanup docs wording

Co-authored-by: Blake Blackshear <blake@frigate.video>

* Store full frame for thumbnail

* Formatting

* Set thumbnail height to 175

* Formatting

---------

Co-authored-by: Blake Blackshear <blake@frigate.video>
2023-05-19 05:16:11 -05:00
Sergey Krashevich
6d0c2ec5c8
Add go2rtc & remote detectors network bandwidth usage to System table (#6526)
* Add network bandwidth usage to System table display in System.jsx and update get_bandwidth_stats function in util.py to include go2rtc processes

* black...

* Add network bandwidth usage to system table in web UI and improve regex in get_bandwidth_stats function to include frigate detector processes

* black...

* Update bandwidth calculation to include both incoming and outgoing traffic

* black:(
2023-05-18 06:40:24 -05:00
Korneliusz Jarzębski
17e8a46c7d
add ffmpeg bandwidth stats (#6492)
* add ffmpeg bandwidth stats

* add ffmpeg bandwidth stats

* Change column name

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* fix lint formatting

---------

Co-authored-by: Korneliusz Jarzębski <k.jarzebski@netverse.pl>
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-05-17 20:01:56 -05:00
Nicolas Mowen
b568a29fa8
Don't fail if camera does not support presets (#6497) 2023-05-17 07:42:56 -05:00
Sergey Krashevich
b4038821cc
Update DeepStack detector to set width and height from input image size (#6429) 2023-05-17 07:40:41 -05:00
Sergey Krashevich
5fb96c777a
Add cmdline information to CPU usage stats in get_cpu_stats() function and display it in the System page's ffmpeg table with a copy-to-clipboard button (#6430) 2023-05-15 07:39:03 -05:00
Sergey Krashevich
181b53a55d
Fix error in parsing DeepStack response JSON and handle cases where predictions field is missing (#6463) 2023-05-15 07:37:34 -05:00
Nicolas Mowen
5951a740d2
Prevent recordings from being turned on if disabled in config (#6444)
* Prevent enabling recordings if not enabled in config

* Fix conflict

* Fix spacing

* Update wording

* Update wording

---------

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2023-05-15 07:36:26 -05:00
Sergey Krashevich
ede1dedbbd
Add Deepstack/CodeProject-AI.Server detector plugin (#6143)
* Add Deepstack detector plugin with configurable API URL, timeout, and API key

* Update DeepStack plugin to recognize 'truck' as 'car' for label indexing

* Add debug logging to DeepStack plugin for better monitoring and troubleshooting

* Refactor DeepStack label loading from file to use merged labelmap

* Black format

* add documentation draft

* fix link to codeproject website

* Apply suggestions from code review

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-05-04 18:06:07 -05:00
Sergey Krashevich
0fcfcb85ab
Implement NVML for NVIDIA GPU Stats (#6359)
* nvml

* black...black...black...

* small fix for avoid errors on strange GPUs and old drivers

* fix type errors

* fix type errors

* fix unittest process crash

where the tests for tests?..

* it's impossible to mock low-level library

* fix double % for other GPU types

* remove space before gpu statistic values
2023-05-04 18:02:01 -05:00
Sergey Krashevich
03b45c153b
Increase maximum event sub_label length to 100 characters (#6350)
* Increase maximum sub label length to 100 characters and update corresponding fields in models and API

* black format...
2023-05-04 17:59:44 -05:00
Sergey Krashevich
2add675d42
Refactor Process Stats and Bugfixes (#6344)
* test refactor process stats

* Update util.py

* bugfix

* black formatting

* add missing processes field to StatsTrackingTypes class

* fix python checks and tests...

* use psutil for calcilate cpu utilization on get_cpu_stats

* black...black...black...

* add cpu average

* calculate statiscts for logger process

* add P-ID for other processes in System.jsx

* Apply suggestions from code review

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>

* make page beautiful again :)

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2023-05-04 17:58:59 -05:00
Nicolas Mowen
305323c9e9
Update Rpi preset (#6385)
* Update Rpi preset

* Fix rpi test
2023-05-04 17:55:44 -05:00
Sergey Krashevich
b38c9e82e2
Replace subprocess usage with os module for better performance and maintainability (#6298)
* avoid executing external tools by using Python's built-in os module to interact with the filesystem directly

* Refactor recording cleanup script to use os module instead of subprocess

* black format util.py

* Ooooops

* Refactor get_cpu_stats() to properly identify recording process
2023-05-01 21:22:35 -05:00
Blake Blackshear
19890310fe
Bug fixes (#6332)
* fix timeline delete

* fix labelmap in config response

* create cache dirs if needed
2023-04-30 14:58:39 -05:00
Blake Blackshear
9bf98f908d
add plus integration for models (#6328) 2023-04-30 13:32:36 -05:00
Nicolas Mowen
ad52e238ce
Refactor events to be more generic (#6320)
* Organize event table to be more generalized

* Add appropriate fields to data

* Move tracked object logic to own function

* Add source type to event queue

* rename enum

* Fix types that are used in webUI

* remove redundant

* Formatting

* fix typing

* Rename enum
2023-04-30 12:07:14 -05:00
Nicolas Mowen
ca7790ff65
Cleanup timeline entries when relevant recording segments are removed (#6319)
* Cleanup timeline entries when relevant recording segments are removed

* Make timeline cleanup simpler

* Formatting
2023-04-30 09:28:21 -05:00
Nicolas Mowen
37360edbc1
Show other system processes in system page (#6276) 2023-04-28 07:47:49 -05:00
Nicolas Mowen
df016ddd0d
Fix http-jpeg template insertion (#6291) 2023-04-28 07:11:01 -05:00
Nicolas Mowen
b48f6d750f
Add annotation support (#6288) 2023-04-28 06:02:06 -05:00
Alin Balutoiu
83006eeb65
Add option to have cameras sorted inside the Birdseye stream (#5450)
* Add option to sort cameras inside Birdseye

* Make default order to be sorted alphabetically

* Add docs for sorting cameras

* Update index.md for cameras

* Remove irelevant comments
2023-04-26 19:29:01 -05:00
Nicolas Mowen
e451f44ced
Move recording management to separate process (#6248)
* Move recordings management to own process and ensure db multiprocess access

* remove reference to old threads

* Cleanup directory remover

* Mypy fixes

* Fix mypy

* Add support back for setting record via MQTT and WS

* Formatting

* Fix rebase issue
2023-04-26 07:25:26 -06:00
Nicolas Mowen
43ade86796
Support Controlling PTZ Cameras Via WebUI (#4715)
* Add support for ptz commands via websocket

* Fix startup issues

* Fix bugs

* Set config manually

* Add more commands

* Add presets

* Add zooming

* Fixes

* Set name

* Cleanup

* Add ability to set presets from UI

* Add ability to set preset from UI

* Cleanup for errors

* Ui tweaks

* Add visual design for pan / tilt

* Add pan/tilt support

* Support zooming

* Try to set wsdl

* Fix duplicate logs

* Catch auth errors

* Don't init onvif for disabled cameras

* Fix layout sizing

* Don't comment out

* Fix formatting

* Add ability to control camera with keyboard shortcuts

* Disallow user selection

* Fix mobile pressing

* Remove logs

* Substitute onvif password

* Add ptz controls ot birdseye

* Put wsdl back

* Add padding

* Formatting

* Catch onvif error

* Optimize layout for mobile and web

* Place ptz controls next to birdseye view in large layout

* Fix pt support

* Center text titles

* Update tests

* Update docs

* Write camera docs for PTZ

* Add MQTT docs for PTZ

* Add ptz info docs for http

* Fix test

* Make half width when full screen

* Fix preset panel logic

* Fix parsing

* Update mqtt.md

* Catch preset error

* Add onvif example to docs

* Remove template example from main camera docs
2023-04-26 06:08:53 -05:00
Blake Blackshear
0d16bd0144
False positives (#6217)
* add false positive submission

* switch timeline events to x,y,w,h

* update docs

* fix type checks

* convert to upsert

* fix config test
2023-04-24 07:24:28 -05:00
Felipe Santos
cdd8e6fd4e
Migrate default database path from /media/frigate to /config (#5219)
* Migrate db path to /config

* Ensure oneshot runs

* Put logic inside of Frigate's run

* Use new db default path in code

* Fix missing config dir

* Upgrade yq to 4.33.3
2023-04-23 11:35:40 -05:00
Nicolas Mowen
fbaab71d78
Metadata Timeline (#6194)
* Create timeline table

* Fix indexes

* Add other fields

* Adjust schema to be less descriptive

* Handle timeline queue from tracked object data

* Setup timeline queue in events

* Add source id for index

* Add other fields

* Fixes

* Formatting

* Store better data

* Add api with filtering

* Setup basic UI for timeline in events

* Cleanups

* Add recordings snapshot url

* Start working on timeline ui

* Add tooltip with info

* Improve icons

* Fix start time with clip

* Move player logic back to clips

* Make box in timeline relative coordinates

* Make region relative

* Get box overlay working

* Remove overlay when playing again

* Add disclaimer when selecting overlay points

* Add docs for new apis

* Fix mobile

* Fix docs

* Change color of bottom center box

* Fix vscode formatting
2023-04-23 10:45:19 -05:00
KtAlumi
366c31259f
Edit dsize order (#6172)
dsize could be (width, height).

Co-authored-by: kt.lu <kt.lu@latticeworkinc.com>
2023-04-22 21:38:58 -05:00
Martin Weinelt
cb73d0cd39
Migrate away from deprecated np.float (#5898)
It is an alias for the python float type, and got deprecated in 1.20 and
was removed in 1.24. The rest of the project already uses float32
(single), so I believe this is also correct here, as opposed to float64
(double).
2023-04-16 07:13:33 -05:00
Blake Blackshear
15415bab10
update deps (#6093)
* update dev deps

* update deps

* docs deps

* python dev deps

* update python deps
2023-04-15 14:17:59 -05:00
Nicolas Mowen
3b62ff093a
Ensure the frigate config validators are also checked when saving config (#6069) 2023-04-15 07:24:13 -05:00
Nicolas Mowen
6986949d84
Remove deprecated retain_days config (#5952) 2023-04-09 11:23:34 -05:00
Nicolas Mowen
3f17f871fa
Fix cleaning logs with rtsp in middle (#5800) 2023-03-21 17:21:37 -05:00
Blake Blackshear
2d9556f5f3
set threads to 2 (#5747) 2023-03-17 17:14:57 -05:00