Commit Graph

27 Commits

Author SHA1 Message Date
Nicolas Mowen
93260f6cfd
Add region count to database and use for motion activity (#10480)
* Add region count to database and use for motion activity

* Fix test
2024-03-15 09:29:22 -06:00
Nicolas Mowen
cb30450060
Migrate pydantic to V2 (#10142)
* Run pydantic migration tool

* Finish removing deprecated functions

* Formatting

* Fix movement weights type

* Fix movement weight test

* Fix config checks

* formatting

* fix typing

* formatting

* Fix

* Fix serialization issues

* Formatting

* fix model namespace warnings

* Update formatting

* Format go2rtc file

* Cleanup migrations

* Fix warnings

* Don't include null values in config json

* Formatting

* Fix test

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-02-29 16:10:13 -07:00
Nicolas Mowen
940be5dc6f
Create ReviewSegment table in DB for organizing detections to be reviewed (#9918)
* Add review to database

* Create main manager for review segments

* Upsert and maintain review segments

* Update logic for adding new segments

* Add api

* Support deleting review segments on recording cleanup

* Add field for alert labels

* Formatting

* Logic fixes

* Save 16:9 thumbnail for review segment

* Ensure that crop is 16:9

* Fix non detected objects being added

* Only include true positives

* Add sub labels to data
2024-02-20 16:26:09 -07:00
Nicolas Mowen
cfda531f5a Write a low resolution low fps stream from decoded frames (#8673)
* Generate low res low fps previews for recordings viewer

* Make sure previews end on the hour

* Fix durations and decrase keyframe interval to ensure smooth scrubbing

* Ensure minimized resolution is compatible with yuv

* Add ability to configure preview quality

* Fix

* Clean up previews more efficiently

* Use iterator
2024-01-31 12:56:11 +00:00
tpjanssen
3359123364
Performance increase with lots of recordings (#8525)
* Performance increase with lots of recordings

* Formatting + review

* Update 020_update_index_recordings.py

Formatting

* Feedback + check fix

* Update 020_update_index_recordings.py
2023-11-07 23:18:26 +00:00
Nicolas Mowen
91f7d67c5e
Smarter Regions (#8194)
* Smarter Regions

* Formatting

* Cleanup

* Fix motion region checking logic

* Add database table and migration for regions

* Update region grid on startup

* Revert init delay change

* Fix mypy

* Move object related functions to util

* Remove unused

* Fix tests

* Remove log

* Update the region daily at 2

* Fix logic

* Formatting

* Initialize grid before starting processing frames

* Move back to creating grid in main process

* Formatting

* Fixes

* Formating

* Fix region check

* Accept all but true

* Use regions grid for startup scan

* Add clarifying comment

* Fix new grid requests

* Add tests

* Delete stale region grids from DB
2023-10-18 18:21:52 -05:00
Nicolas Mowen
00016b7499
Save average dBFS and retain segment with dBFS in motion mode (#7158)
* Hold audio info queue for recordings

* Add dBFS to db

* Cleanup

* Formatting

* Fix check
2023-07-14 19:05:14 -05:00
Nicolas Mowen
8bc76d19db
DB Optimizations (#6712)
* Enable auto vacuums

* Enable auto vacuum

* Fix separator

* Fix separator and remove incorrect log

* Limit to 1 row since that is all that is used

* Add index on camera + segment_size

* Formatting

* Increase timeout and cache_size

* Set DB mode to NORMAL synchronous level

* Formatting

* Vacuum every 2 weeks

* Remove fstring

* Use string

* Use consts
2023-06-11 07:23:18 -05:00
Martin Weinelt
ab50d0b006
Add isort and ruff linter (#6575)
* Add isort and ruff linter

Both linters are pretty common among modern python code bases.

The isort tool provides stable sorting and grouping, as well as pruning
of unused imports.

Ruff is a modern linter, that is very fast due to being written in rust.
It can detect many common issues in a python codebase.

Removes the pylint dev requirement, since ruff replaces it.

* treewide: fix issues detected by ruff

* treewide: fix bare except clauses

* .devcontainer: Set up isort

* treewide: optimize imports

* treewide: apply black

* treewide: make regex patterns raw strings

This is necessary for escape sequences to be properly recognized.
2023-05-29 05:31:17 -05:00
Sergey Krashevich
03b45c153b
Increase maximum event sub_label length to 100 characters (#6350)
* Increase maximum sub label length to 100 characters and update corresponding fields in models and API

* black format...
2023-05-04 17:59:44 -05:00
Nicolas Mowen
ad52e238ce
Refactor events to be more generic (#6320)
* Organize event table to be more generalized

* Add appropriate fields to data

* Move tracked object logic to own function

* Add source type to event queue

* rename enum

* Fix types that are used in webUI

* remove redundant

* Formatting

* fix typing

* Rename enum
2023-04-30 12:07:14 -05:00
Blake Blackshear
0d16bd0144
False positives (#6217)
* add false positive submission

* switch timeline events to x,y,w,h

* update docs

* fix type checks

* convert to upsert

* fix config test
2023-04-24 07:24:28 -05:00
Nicolas Mowen
fbaab71d78
Metadata Timeline (#6194)
* Create timeline table

* Fix indexes

* Add other fields

* Adjust schema to be less descriptive

* Handle timeline queue from tracked object data

* Setup timeline queue in events

* Add source id for index

* Add other fields

* Fixes

* Formatting

* Store better data

* Add api with filtering

* Setup basic UI for timeline in events

* Cleanups

* Add recordings snapshot url

* Start working on timeline ui

* Add tooltip with info

* Improve icons

* Fix start time with clip

* Move player logic back to clips

* Make box in timeline relative coordinates

* Make region relative

* Get box overlay working

* Remove overlay when playing again

* Add disclaimer when selecting overlay points

* Add docs for new apis

* Fix mobile

* Fix docs

* Change color of bottom center box

* Fix vscode formatting
2023-04-23 10:45:19 -05:00
Nicolas Mowen
b4d4adb75b
Limit recording retention to available storage (#3942)
* Add field and migration for segment size

* Store the segment size in db

* Add comment

* Add default

* Fix size parsing

* Include segment size in recordings endpoint

* Start adding storage maintainer

* Add storage maintainer and calculate average sizes

* Update comment

* Store segment and hour avg sizes per camera

* Formatting

* Keep track of total segment and hour averages

* Remove unused files

* Cleanup 2 hours of recordings at a time

* Formatting

* Fix bug

* Round segment size

* Cleanup some comments

* Handle case where segments are not deleted on initial run or is only retained segments

* Improve cleanup log

* Formatting

* Fix typo and improve logging

* Catch case where no recordings exist for camera

* Specifically define sort

* Handle edge case for cameras that only record part time

* Increase definition of part time recorder

* Remove warning about not supported storage based retention

* Add note about storage based retention to recording docs

* Add tests for storage maintenance calculation and cleanup

* Format tests

* Don't run for a camera with no recording segments

* Get size of file from cache

* Rework camera stats to be more efficient

* Remove total and other inefficencies

* Rewrite storage cleanup logic to be much more efficient

* Fix existing tests

* Fix bugs from tests

* Add another test

* Improve logging

* Formatting

* Set back correct loop time

* Update name

* Update comment

* Only include segments that have a nonzero size

* Catch case where camera has 0 nonzero segment durations

* Add test to cover zero bandwidth migration case

* Fix test

* Incorrect boolean logic

* Formatting

* Explicity re-define iterator
2022-10-09 06:28:26 -05:00
Blake Blackshear
78e1782084 optimize query performance 2022-05-12 06:29:43 -05:00
Blake Blackshear
e724fe3da6 add endpoint to submit to plus 2022-04-10 09:11:16 -05:00
Nick
045aac8933
Add object filter ratio (#2952)
* Add object ratio config parameters

Issue: #2948

* Add config test for object filter ratios

Issue: #2948

* Address review comments

- Accept `ratio` default
- Rename `bounds` to `box` for consistency
- Add migration for new field

Issue: #2948

* Fix logical errors

- field migrations require default values
- `clipped` referenced the wrong index for region, since it shifted
- missed an inclusion of `ratio` for detections in `process_frames`
- revert naming `o[2]` as `box` since it is out of scope!

This has now been test-run against a video, so I believe the kinks are
worked out.

Issue: #2948

* Update contributing notes for `make`

Issue: #2948

* Fix migration

- Ensure that defaults match between Event and migration script
- Deconflict migration script number (from rebase)

Issue: #2948

* Filter objects out of ratio bounds

Issue: #2948

* Update migration file to 009

Issue: #2948
2022-04-10 08:25:18 -05:00
Nicolas Mowen
b1cc64d4fa
FEAT: Ability to set sub labels for specific events (#2949)
* Add sub label to model and set / delete funs

* Add migrations for sub label

* Tweaks to API and model

* Show sublabel if available

* Cleanups

* Update docs

* Show person in UI title

* Fix typo and don't fail on no json

* Transfer sub labels for in progress events

* Remove sublabel reset

* Remove person only check

* Make default null

* Update docs and formatting

* Make default null

* Make nullable in migration

* Undo null

* Update model to accept null

* Update migration to accept null

* Don't set to default values

* Remove redundant defaults and update http logic

* Only need a single route

* Enforce 20 character limit in http

* Update docs to mention 20 character limit

* Cleanup

* Separate insert and update to make sure updated values are retained when event ends

* Use insert instead of replace

* Remove redundant if and have should_update_db include clip or snapshot requirement.
2022-03-17 07:18:43 -05:00
Nicolas Mowen
4004048add
Ability to retain specific clips / events indefinitely (#2831) 2022-02-21 22:03:01 -06:00
Blake Blackshear
7c74bf2566 fix migrations 2022-02-18 21:18:26 -06:00
Blake Blackshear
188b202836 store objects and motion counts in the db 2022-02-18 21:18:26 -06:00
Blake Blackshear
2f2329ba44 only save recordings when an event is in progress 2022-02-18 21:18:26 -06:00
Blake Blackshear
1d25936f31 add region/bbox/area to event table 2021-09-23 07:31:48 -05:00
Jason Hunter
f7975cbbc4 remove backfill - only store rows moving forward 2021-06-10 07:04:28 -05:00
Jason Hunter
055bd22138 initial implementation of recordings table 2021-06-10 07:04:28 -05:00
Blake Blackshear
39ff49e054 formatting cleanup 2021-02-25 07:01:57 -06:00
Blake Blackshear
c770470b58 add database migrations 2021-01-26 21:40:33 -06:00