Commit Graph

70 Commits

Author SHA1 Message Date
Josh Hawkins
24ac9f3e5a
Use sqlite-vec extension instead of chromadb for embeddings (#14163)
* swap sqlite_vec for chroma in requirements

* load sqlite_vec in embeddings manager

* remove chroma and revamp Embeddings class for sqlite_vec

* manual minilm onnx inference

* remove chroma in clip model

* migrate api from chroma to sqlite_vec

* migrate event cleanup from chroma to sqlite_vec

* migrate embedding maintainer from chroma to sqlite_vec

* genai description for sqlite_vec

* load sqlite_vec in main thread db

* extend the SqliteQueueDatabase class and use peewee db.execute_sql

* search with Event type for similarity

* fix similarity search

* install and add comment about transformers

* fix normalization

* add id filter

* clean up

* clean up

* fully remove chroma and add transformers env var

* readd uvicorn for fastapi

* readd tokenizer parallelism env var

* remove chroma from docs

* remove chroma from UI

* try removing custom pysqlite3 build

* hard code limit

* optimize queries

* revert explore query

* fix query

* keep building pysqlite3

* single pass fetch and process

* remove unnecessary re-embed

* update deps

* move SqliteVecQueueDatabase to db directory

* make search thumbnail take up full size of results box

* improve typing

* improve model downloading and add status screen

* daemon downloading thread

* catch case when semantic search is disabled

* fix typing

* build sqlite_vec from source

* resolve conflict

* file permissions

* try build deps

* remove sources

* sources

* fix thread start

* include git in build

* reorder embeddings after detectors are started

* build with sqlite amalgamation

* non-platform specific

* use wget instead of curl

* remove unzip -d

* remove sqlite_vec from requirements and load the compiled version

* fix build

* avoid race in db connection

* add scale_factor and bias to description zscore normalization
2024-10-07 14:30:45 -06:00
gtsiam
a468ed316d
Added stop_event to util.Process (#14142)
* Added stop_event to util.Process

util.Process will take care of receiving signals when the stop_event is
accessed in the subclass. If it never is, SystemExit is raised instead.

This has the effect of still behaving like multiprocessing.Process when
stop_event is not accessed, while still allowing subclasses to not deal
with the hassle of setting it up.

* Give each util.Process their own logger

This will help to reduce boilerplate in subclasses.

* Give explicit types to util.Process.__init__

This gives better type hinting in the editor.

* Use util.Process facilities in AudioProcessor

Boilerplate begone!

* Removed pointless check in util.Process

The log_listener.queue should never be None, unless something has gone
extremely wrong in the log setup code. If we're that far gone, crashing
is better.

* Make sure faulthandler is enabled in all processes

This has no effect currently since we're using the fork start_method.
However, when we inevidably switch to forkserver (either by choice, or
by upgrading to python 3.14+) not having this makes for some really fun
failure modes :D
2024-10-03 11:03:43 -06:00
gtsiam
54900ae318
Properly call super() in subclasses (#14124) 2024-10-02 19:35:46 -06:00
Nicolas Mowen
5b0c1e5b9e
Fix event entries (#14093) 2024-10-01 11:07:13 -05:00
Josh Hawkins
95d6da3111
Add ability to configure genai to use snapshot instead of thumbnails (#14077)
* Allow embedding of snapshot for description via config option

* docs

* frontend button

* Backend

* crop snapshot to region

* only show dropdown when event has snapshot

* fix cursor on dropdown

* crop on initial generation as well

* use enum for type

* fix type
2024-09-30 15:54:53 -06:00
gtsiam
c0bd3b362c
Custom classes for Process and Metrics (#13950)
* Subclass Process for audio_process

* Introduce custom mp.Process subclass

In preparation to switch the multiprocessing startup method away from
"fork", we cannot rely on os.fork cloning the log state at fork time.
Instead, we have to set up logging before we run the business logic of
each process.

* Make camera_metrics into a class

* Make ptz_metrics into a class

* Fixed PtzMotionEstimator.ptz_metrics type annotation

* Removed pointless variables

* Do not start audio processor when no audio cameras are configured
2024-09-27 07:53:23 -05:00
Nicolas Mowen
ddfdb71783
Fix snapshot access (#13996) 2024-09-26 20:05:48 -05:00
Nicolas Mowen
a65aaab849
Fixes (#13990)
* Fix ROCm input name

* Fix incorrect parsing of None
2024-09-26 12:42:21 -05:00
gtsiam
dc54981784
Move more things out of FrigateApp (#13897)
* Moved FrigateApp.init_config() into FrigateConfig.load()

* Move frigate config loading into main

* Store PlusApi in FrigateConfig

* Register SIGTERM handler in main

* Ensure logging is setup during config parsing

* Removed pointless try

* Moved config initialization out of FrigateApp

* Made FrigateApp.shm_frame_count into a function

* Removed log calls from signal handlers

python's logging calls are not re-entrant, which caused at least one of
these to deadlock randomly.

* Reopen stdout/err on process fork

This helps avoid deadlocks (https://github.com/python/cpython/issues/91776).

* Make mypy happy

* Whoops. I might have forgotten to save.

Truly an amateur mistake.

* Always call FrigateApp.stop()
2024-09-24 07:07:47 -05:00
Josh Hawkins
0f813962be
Chroma fixes (#13902)
* Ensure descriptions saved in chroma are non-empty

* delete only existing ids in event cleanup

* add debug logging
2024-09-23 06:53:19 -06:00
Nicolas Mowen
5ff476c6f9
Configurable ffmpeg (#13722)
* Install multiple ffmpeg versions and add config to make it configurable

* Update docs

* Run ffprobe too

* Cleanup

* Apply config to go2rtc as well

* Fix ffmpeg bin

* Docs

* Restore path

* Cleanup env var

* Fix ffmpeg path for encoding

* Fix export

* Formatting
2024-09-13 15:14:51 -05:00
Nicolas Mowen
7626dd239a
Fix incorrect recording config access (#13524)
* Fix incorrect access

* Formatting
2024-09-03 16:27:19 -05:00
Nicolas Mowen
e66f47bdf6
Cleanup after recordings refactor (#13518)
* Cleanup after recordings refactor

* Fix default retention mode

* Formatting

* Fix review item query

* Default review retention
2024-09-03 08:49:50 -06:00
Nicolas Mowen
0acbd3d5e8
Refactor recordings retention to be based on review items (#13355)
* Refactor recordings config to be based off of review items

* Update object processing logic for when an event is created

* Migrate to deciding recording retention based on review items

* Refactor recording expiration to be based off of review items

* Remove remainder of recording events access

* Handle migration automatically

* Update version and cleanup

* Update docs

* Clarify docs

* Cleanup

* Target camera config

* Safely access all fields
2024-09-02 07:22:53 -06:00
Nicolas Mowen
347d54f388 Chunk timeline deletes (#12900) 2024-08-29 20:19:50 -06:00
Nicolas Mowen
3428baa3fa Fix embeddings failing to start 2024-08-29 20:19:50 -06:00
Jason Hunter
36cbffcc5e Semantic Search for Detections (#11899)
* Initial re-implementation of semantic search

* put docker-compose back and make reindex match docs

* remove debug code and fix import

* fix docs

* manually build pysqlite3 as binaries are only available for x86-64

* update comment in build_pysqlite3.sh

* only embed objects

* better error handling when genai fails

* ask ollama to pull requested model at startup

* update ollama docs

* address some PR review comments

* fix lint

* use IPC to write description, update docs for reindex

* remove gemini-pro-vision from docs as it will be unavailable soon

* fix OpenAI doc available models

* fix api error in gemini and metadata for embeddings
2024-08-29 20:19:50 -06:00
Nicolas Mowen
e563692fa2 Add camera name to audio debug line (#12799)
* Add camera name to audio debug line

* Formatting
2024-08-08 07:54:13 -06:00
Blake Blackshear
b4384a1be3
Shutdown hang (#11793)
* intentionally handle queues during shutdown and carefully manage shutdown order

* more carefully manage shutdown to avoid threadlocks

* use debug for signal logging

* ensure disabled cameras dont break shutdown

* typo
2024-06-06 18:54:38 -05:00
Nicolas Mowen
142641b387
Adjust nginx proc count based on available CPUs (#11653)
* Restrict nginx to 4 processes if more are available

* Fix bash

* Different sed structure

* Limit ffmpeg thread counts for secondary ffmpeg processes

* Add up / down keyboard shortcut
2024-05-30 12:34:01 -05:00
Nicolas Mowen
9680f2a574
Bug fixes (#11332)
* Fix external events saving

* Only show relevant labels

* Set on recordings view too

* Fix video control width on motion page

* use overlay so player state is maintained between camera switches

* use overlay so player state is maintained between camera switches

* mobile only

* Formatting

* Use higher amount

* Only wrap when needed
2024-05-10 11:42:56 -06:00
Nicolas Mowen
216e44bc34
Event cleanup (#11225)
* remove duplicate event purging

* Slightly increase stationary object sensitivity

* Add toast for export starting

* Increase stationary threshold as well

* Update message

* Make text consistent

* Update message
2024-05-03 08:03:41 -06:00
Nicolas Mowen
03e25b3f94
Improve config validation for zones and object masks (#11022)
* Add verification for required zone names

* Make global object masks use relative coordinates as well

* Ensure event image cleanup doesn't fail

* Return passed value
2024-04-18 11:35:16 -05:00
Nicolas Mowen
049f27d710
Ongoing review segments (#10924)
* Update review maintainer to save events when ongoing

* Handle previews for in progress review items

* Reset DB items in app

* Handle in progress review items

* Scroll back down to selected event item

* Handle undefined end time

* Formatting

* remove unused

* Make export handles have full resolution

* reduce preview thumbnail props

* fix missing return

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-04-11 07:42:16 -05:00
Nicolas Mowen
8163c036ef
Use zmq for event end queue and update python deps (#10886)
* Use zmq for events ended

* Cleanup

* Update deps

* formatting
2024-04-08 18:19:45 -05:00
Nicolas Mowen
d1082ec305
Support manual detections in review items (#10784)
* Only update frame time if it is older

* Support manual detections as review items

* Don't handle api detections in recordings

* Store recordings for manual events
2024-04-02 07:07:50 -06:00
Nicolas Mowen
76a114a3cd
Rewrite events communication to use zmq instead of mp.Queue (#10627)
* Move to using zmq for events updating

* Use event updater in manual events handler

* Formatting
2024-03-23 11:11:32 -05:00
Nicolas Mowen
cb30450060
Migrate pydantic to V2 (#10142)
* Run pydantic migration tool

* Finish removing deprecated functions

* Formatting

* Fix movement weights type

* Fix movement weight test

* Fix config checks

* formatting

* fix typing

* formatting

* Fix

* Fix serialization issues

* Formatting

* fix model namespace warnings

* Update formatting

* Format go2rtc file

* Cleanup migrations

* Fix warnings

* Don't include null values in config json

* Formatting

* Fix test

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-02-29 16:10:13 -07:00
Nicolas Mowen
dba21b606d
Convert config updating and video/audio queues to use zmq (#9893)
* Add config pub / sub pattern

* remove recording from feature metrics

* remove audio and feature metrics

* Check for updates from all cameras

* remove birdseye from camera metrics

* remove motion and detection camera metrics

* Ensure that all processes are stopped

* Stop communicators

* Detections

* Cleanup video output queue

* Use select for time sensitive polls

* Use ipc instead of tcp
2024-02-19 07:26:59 -06:00
Nicolas Mowen
dd3dc7949a
Use zmq for inter process communication (#9309)
* Use zmq for inter process communication

* Use localhost for reply and request

* Use pyobj instead of json and Need to use separate requestors for each audio listener

* Cleanup port defining
2024-02-14 18:24:36 -06:00
Nicolas Mowen
a1e5c658d5 Add support for filtering history page and add support for creating timeline entries for audio / custom events (#9034)
* Add filter popover

* Add api filter hook and use UI with filtering

* Get history filtering working for cameras and labels

* Allow filtering on detail level

* Save timeline entries for api events

* reset

* fix width
2024-01-31 12:56:11 +00:00
Nicolas Mowen
4524d9440c Add initial implementation of history view in new webui framework (#8895)
* Add support for review grid

* Cleanup reloading on focus

* Adjust timeline api to include metadata and before

* Be more efficient about getting info

* Adjust to new data format

* Cleanup types

* Cleanup text

* Transition to history

* Cleanup

* remove old web implementations

* Cleanup
2024-01-31 12:56:11 +00:00
Nicolas Mowen
ca4e0dbc75
Fix ffmpeg input arg parsing (#8924) 2023-12-12 10:48:09 +00:00
Nicolas Mowen
7270eef6bf
Don't fail on 0 rms (#8447) 2023-11-04 02:18:23 +00:00
Nicolas Mowen
f1efd8dbe2
Use int for drawing box (#8388) 2023-10-30 19:53:29 -04:00
Nicolas Mowen
cf2b56613f
Don't overwrite event while cleaning up expired cameras (#8320) 2023-10-26 06:20:06 -05:00
Nicolas Mowen
1a9e00ee49
Add count of audio labels to active count (#8310)
* Add count of audio labels to active count

* Formatting
2023-10-24 19:26:46 -04:00
Blake Blackshear
9fc93c72a0
more consistent use of iterators in select queries (#8258) 2023-10-21 10:53:33 -05:00
Nicolas Mowen
34b315cc8c
Fix bug forcing rtsp for audio detection ffmpeg process (#8156)
* Fix forcing rtsp for audio process

* Send as list
2023-10-13 17:03:04 -05:00
tpjanssen
e32bd4ab15
Added audio sensors to camera metrics and API stats (#8109)
* Added audio sensor to camera metrics and API stats

* Update types.py

* Update app.py
2023-10-13 06:17:41 -05:00
Nicolas Mowen
49e7723405
Add debug logs for audio labels (#8080) 2023-10-07 09:23:11 -05:00
Nicolas Mowen
8bde914939
Fix a couple bugs (#8045) 2023-10-07 09:18:34 -05:00
Nicolas Mowen
08ef69bac4
Add recordings timeline entry for frigate+ attributes (#8063)
* Add attribute item to timeline

* Add face icon

* Add support for other icons

* Cleanup

* Ensure attributes are only updated once

* don't show _ in attributes
2023-10-07 09:17:18 -05:00
Nicolas Mowen
a2206bb177
Reduce database queries to necessary fields (#7751)
* Reduce database queries to necessary labels

* Set columns for other queries

* skip creating model instances

---------

Co-authored-by: Blake Blackshear <blakeb@blakeshome.com>
2023-09-11 17:07:04 -05:00
Nicolas Mowen
bbdf5e1bee
Don't count in progress events as duplicates (#7568)
* Don't count in progress events as duplicates

* Fix sql
2023-09-01 07:02:35 -05:00
Sergey Krashevich
731743c7e5
Refactor AudioTfl class to accept the number of detection threads as a parameter in the constructor, and update the usage of the num_threads attribute accordingly (#7588) 2023-09-01 07:00:11 -05:00
Nicolas Mowen
fbed3fbb8b
Fix audio filter config (#7401) 2023-08-06 12:19:28 -05:00
Nicolas Mowen
36ccaa4cff
Improve debug logs for audio & ptz (#7387)
* Update audio.py

* Add debug log for ptz info

* Formatting

* Fix check
2023-08-05 05:47:56 -05:00
Nicolas Mowen
b7ff6735f6
Add support for stationary and active timeline entires (#7344)
* Add support for stationary and active timeline entires

* Be sure to update stored event data copy with stationary
2023-07-31 21:43:48 -05:00
Nicolas Mowen
9016a48dc7
Refactor Audio Events To Use stdout Pipe (#7291)
* Cleanup audio detection

* Read audio frames from ffmpeg process directly

* Handle case where process has stopped
2023-07-26 05:51:45 -05:00