Commit Graph

106 Commits

Author SHA1 Message Date
Nicolas Mowen
faadea8e1f Dynamic Management of Cameras (#18671)
* Add base class for global config updates

* Add or remove camera states

* Move camera process management to separate thread

* Move camera management fully to separate class

* Cleanup

* Stop camera processes when stop command is sent

* Start processes dynamically when needed

* Adjust

* Leave extra room in tracked object queue for two cameras

* Dynamically set extra config pieces

* Add some TODOs

* Fix type check

* Simplify config updates

* Improve typing

* Correctly handle indexed entries

* Cleanup

* Create out SHM

* Use ZMQ for signaling object detectoin is completed

* Get camera correctly created

* Cleanup for updating the cameras config

* Cleanup

* Don't enable audio if no cameras have audio transcription

* Use exact string so similar camera names don't interfere

* Add ability to update config via json body to config/set endpoint

Additionally, update the config in a single rather than multiple calls for each updated key

* fix autotracking calibration to support new config updater function

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-08-16 10:20:33 -05:00
Nicolas Mowen
4b57e5e265 Refactor TensorRT (#18643)
* Combine base and arm trt detectors

* Remove unused deps for amd64 build

* Add missing packages and cleanup ldconfig

* Expand packages for tensorflow model training

* Cleanup

* Refactor training to not reserve memory
2025-08-16 10:20:33 -05:00
Josh Hawkins
40ab7d6c38 Make Birdseye clickable (#18628)
* keep track of layout changes and publish on change

* websocket hook

* clickable overlay div to navigate to full camera view
2025-08-16 10:20:33 -05:00
Nicolas Mowen
765a28d812 Live classification model training (#18583)
* Implement model training via ZMQ and add model states to represent training

* Get model updates working

* Improve toasts and model state

* Clean up logging

* Add back in
2025-08-16 10:20:33 -05:00
Josh Hawkins
6dc36fcbb4 Audio transcription support (#18398)
* install new packages for transcription support

* add config options

* audio maintainer modifications to support transcription

* pass main config to audio process

* embeddings support

* api and transcription post processor

* embeddings maintainer support for post processor

* live audio transcription with sherpa and faster-whisper

* update dispatcher with live transcription topic

* frontend websocket

* frontend live transcription

* frontend changes for speech events

* i18n changes

* docs

* mqtt docs

* fix linter

* use float16 and small model on gpu for real-time

* fix return value and use requestor to embed description instead of passing embeddings

* run real-time transcription in its own thread

* tweaks

* publish live transcriptions on their own topic instead of tracked_object_update

* config validator and docs

* clarify docs
2025-08-16 10:20:33 -05:00
Nicolas Mowen
53ff33135b Update ROCm to 6.4.1 (#18364)
* Update rocm to 6.4.1

* Quick fix
2025-08-16 10:20:33 -05:00
Nicolas Mowen
dc187eee1c Dynamic Config Updates (#18353)
* Create classes to handle publishing and subscribing config updates

* Cleanup

* Use config updater

* Update handling for enabled config

* Cleanup

* Recording config updates

* Birdseye config updates

* Handle notifications

* handle review

* Update motion
2025-08-16 10:20:33 -05:00
Josh Hawkins
0c92c3ccfa Add more notification debug logs (#19252) 2025-07-22 08:20:42 -06:00
Josh Hawkins
5d13925d2b Fixes (#18275) 2025-05-17 16:11:19 -06:00
Josh Hawkins
ebae6cb1ed Fixes (#18262)
* Don't use timezone in export dialog timestamps

Revert an unnecessary change made in https://github.com/blakeblackshear/frigate/pull/18257

* Ensure notifications register button is only disabled when both all cameras and every individual camera is disabled

* Send test notification if any cameras are enabled

* clarify docs about disabling cameras

* fix crash in autotracking zoom

* clean up

* masks and zones i18n fixes

* Check if camera is enabled in config

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-05-16 16:37:25 -06:00
Martin Weinelt
4d4d54d030 Fix various typing issues (#18187)
* Fix the `Any` typing hint treewide

There has been confusion between the Any type[1] and the any function[2]
in typing hints.

[1] https://docs.python.org/3/library/typing.html#typing.Any
[2] https://docs.python.org/3/library/functions.html#any

* Fix typing for various frame_shape members

Frame shapes are most likely defined by height and width, so a single int
cannot express that.

* Wrap gpu stats functions in Optional[]

These can return `None`, so they need to be `Type | None`, which is what
`Optional` expresses very nicely.

* Fix return type in get_latest_segment_datetime

Returns a datetime object, not an integer.

* Make the return type of FrameManager.write optional

This is necessary since the SharedMemoryFrameManager.write function can
return None.

* Fix total_seconds() return type in get_tz_modifiers

The function returns a float, not an int.

https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds

* Account for floating point results in to_relative_box

Because the function uses division the return types may either be int or
float.

* Resolve ruff deprecation warning

The config has been split into formatter and linter, and the global
options are deprecated.
2025-05-13 08:27:20 -06:00
Josh Hawkins
f39ddbc00d Fixes (#18139)
* Catch error and show toast when failing to delete review items

* i18n keys

* add link to speed estimation docs in zone edit pane

* Implement reset of tracked object update for each camera

* Cleanup

* register mqtt callbacks for toggling alerts and detections

* clarify snapshots docs

* clarify semantic search reindexing

* add ukrainian

* adjust date granularity for last recording time

The api endpoint only returns granularity down to the day

* Add amd hardware

* fix crash in face library on initial start after enabling

* Fix recordings view for mobile landscape

The events view incorrectly was displaying two columns on landscape view and it only took up 20% of the screen width. Additionally, in landscape view the timeline was too wide (especially on iPads of various screen sizes) and would overlap the main video

* face rec overfitting instructions

* Clarify

* face docs

* clarify

* clarify

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-05-11 12:03:53 -06:00
Josh Hawkins
8094dd4075 Fixes (#18117)
* face library i18n fixes

* face library i18n fixes

* add ability to use ctrl/cmd S to save in the config editor

* Use datetime as ID

* Update metrics inference speed to start with 0 ms

* fix android formatted thumbnail

* ensure role is comma separated and stripped correctly

* improve face library deletion

- add a confirmation dialog
- add ability to select all / delete faces in collections

* Implement lazy loading for video previews

* Force GPU for large embedding model

* GPU is required

* settings i18n fixes

* Don't delete train tab

* webpush debugging logs

* Fix incorrectly copying zones

* copy path data

* Ensure that cache dir exists for Frigate+

* face docs update

* Add description to upload image step to clarify the image

* Clean up

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-05-09 07:36:44 -06:00
Josh Hawkins
377b78af11 Change default zmq timeout (#17971)
* update default zmq timeout

* use lower default timeout

* keep longer timeout for event maintainer

* remove default

* remove
2025-04-30 08:29:16 -06:00
Nicolas Mowen
4b4053d174 Add face and lpr to tracked object update topic (#17940)
* Send tracked object updates for face and license_plate objects

* Update docs

* Add to type enum

* Add camera to object description update

* Formatting

* Consolidate yue-Hant

* Add missing
2025-04-28 16:43:03 -06:00
Josh Hawkins
23c3323871 Dynamic embeddings reindexing (#17418)
* reindex with api endpoint and zmq

* threading

* frontend

* require admin role
2025-03-27 11:29:34 -06:00
Josh Hawkins
fa4643fddf LPR improvements (#17289)
* config options

* processing in maintainer

* detect and process dedicated lpr plates

* create camera type, add manual event and save snapshot

* use const

* ensure lpr events are always detections, typing fixes

* docs

* docs tweaks

* add preprocessing and penalization for low confidence chars
2025-03-23 13:30:48 -06:00
Nicolas Mowen
d4d5c4aac8 Disabled cameras fixing (#17273)
* Fix case where objects are returned as null

* Fix enabled status not being persisted

* Use config as source of truth when refreshed

* Ensure camera always have config object updated

* Cleanup typing
2025-03-20 11:20:44 -05:00
Nicolas Mowen
e33fa96599 Face recognize api (#17233)
* Add api to run face recognition on image

* Rework save attempts option

* Cleanup mobile object pane buttons

* Adjust api signature

* Remove param

* Cleanup
2025-03-19 09:02:25 -06:00
Josh Hawkins
124cc4c9cc Rename identifier field (#17128)
* backend rename

* frontend

* docs

* fix api path
2025-03-12 17:45:16 -06:00
Josh Hawkins
6360802612 Use identifier field for unknown license plates (#17123)
* backend

* backend fixes

* api for search queries

* frontend

* docs

* add filterable scroll list to more filters pane for identifiers

* always publish identifier
2025-03-12 14:38:28 -06:00
Nicolas Mowen
b3d5cd9e4b Refactor manual event api to use ZMQ (#17105)
* Don't wait for topic

* Refactor object processing and camera state

* Move manual event handling to camera state / tracked object

* Cleanup

* Refactor audio to use internal zmq

* Cleanup

* Clenaup

* Cleanup

* Quick label fix

* Fix tests

* Cleanup
2025-03-11 22:31:05 -05:00
Nicolas Mowen
0cc5d66e5b Refactor sub label api (#17079)
* Use event metadata updater to handle sub label operations

* Use event metadata publisher for sub label setting

* Formatting

* fix tests

* Cleanup
2025-03-10 16:29:29 -06:00
Nicolas Mowen
2946c935ee Disabled camera output (#16920)
* Fix live cameras not showing on refresh

* Fix live dashboard when birdseye is added

* Handle cameras that are offline / disabled

* Use black instead of green frame

* Fix missing mqtt topics
2025-03-03 15:05:49 -06:00
Josh Hawkins
531042467a Dynamically enable/disable cameras (#16894)
* config options

* metrics

* stop and restart ffmpeg processes

* dispatcher

* frontend websocket

* buttons for testing

* don't recreate log pipe

* add/remove cam from birdseye when enabling/disabling

* end all objects and send empty camera activity

* enable/disable switch in ui

* disable buttons when camera is disabled

* use enabled_in_config for some frontend checks

* tweaks

* handle settings pane with disabled cameras

* frontend tweaks

* change to debug log

* mqtt docs

* tweak

* ensure all ffmpeg processes are initially started

* clean up

* use zmq

* remove camera metrics

* remove camera metrics

* tweaks

* frontend tweaks
2025-03-03 08:30:52 -07:00
Josh Hawkins
60b34bcfca Refactor processors and add LPR postprocessing (#16722)
* recordings data pub/sub

* function to process recording stream frames

* model runner

* lpr model runner

* refactor to mixin class and use model runner

* separate out realtime and post processors

* move model and mixin folders

* basic postprocessor

* clean up

* docs

* postprocessing logic

* clean up

* return none if recordings are disabled

* run postprocessor handle_requests too

* tweak expansion

* add put endpoint

* postprocessor tweaks with endpoint
2025-02-21 06:51:37 -07:00
Nicolas Mowen
c7db0f479d Only match on exact config name in some cases (#16704)
* Only match on exact config name in some cases

* Cleanup
2025-02-20 10:04:06 -06:00
Josh Hawkins
3f07d2d37c Improve notifications (#16632)
* add notification cooldown

* cooldown docs

* show alert box when notifications are used in an insecure context

* add ability to suspend notifications from dashboard context menu
2025-02-17 07:19:03 -07:00
Josh Hawkins
82f8694464 Toggle review alerts and detections (#16482)
* backend

* frontend

* docs

* fix topic name and initial websocket state

* update reference config

* fix mqtt docs

* fix initial topics

* don't apply max severity when alerts/detections are disabled

* fix ws merge

* tweaks
2025-02-11 07:46:25 -07:00
Nicolas Mowen
ac3dfbc30d Set stop event first (#16466) 2025-02-10 21:22:33 -06:00
Josh Hawkins
9a0211a71c Improve Notifications (#16453)
* backend

* frontend

* add notification config at camera level

* camera level notifications in dispatcher

* initial onconnect

* frontend

* backend for suspended notifications

* frontend

* use base communicator

* initialize all cameras in suspended array and use 0 for unsuspended

* remove switch and use select for suspending in frontend

* use timestamp instead of datetime

* frontend tweaks

* mqtt docs

* fix button width

* use grid for layout

* use thread and queue for processing notifications with 10s timeout

* clean up

* move async code to main class

* tweaks

* docs

* remove warning message
2025-02-10 19:47:15 -07:00
Josh Hawkins
bc96db8612 Add ability to set mqtt qos in config (#16435) 2025-02-09 16:45:04 -06:00
Nicolas Mowen
1c3527f5c4 Face recognition reprocess (#16212)
* Implement update topic

* Add API for reprocessing face

* Get reprocess working

* Fix crash when no faces exist

* Simplify
2025-02-08 12:47:01 -06:00
Nicolas Mowen
3f99ff65ed Face recognition improvements (#16034) 2025-02-08 12:47:01 -06:00
Nicolas Mowen
1edbd2d498 Refactor camera activity processing (#15803)
* Replace object label sensors with new manager

* Implement zone topics

* remove unused
2025-02-08 12:47:01 -06:00
Nicolas Mowen
aa19ec3ddb Face recognition backend (#14495)
* Add basic config and face recognition table

* Reconfigure updates processing to handle face

* Crop frame to face box

* Implement face embedding calculation

* Get matching face embeddings

* Add support face recognition based on existing faces

* Use arcface face embeddings instead of generic embeddings model

* Add apis for managing faces

* Implement face uploading API

* Build out more APIs

* Add min area config

* Handle larger images

* Add more debug logs

* fix calculation

* Reduce timeout

* Small tweaks

* Use webp images

* Use facenet model
2025-02-08 12:47:01 -06:00
Blake Blackshear
c4727f19e1 Simplify plus submit (#15941)
* remove unused annotate file

* improve plus error messages

* formatting
2025-01-11 07:04:11 -07:00
Nicolas Mowen
66277fbb6c Fix embeddings (#15072)
* Fix embeddings reading frames

* Fix event update reading

* Formatting

* Pin AIO http to fix build failure

* Pin starlette
2024-11-19 12:20:04 -06:00
Josh Hawkins
9ae839ad72 Tracked object metadata changes (#15055)
* add enum and change topic name

* frontend renaming

* docs

* only display sublabel score if it it exists

* remove debug print
2024-11-18 11:26:44 -07:00
Evan Jarrett
e4a6b29279 fix string comparison on mqtt error message for Server unavailable (#14675) 2024-10-30 05:05:58 -06:00
Nicolas Mowen
eca504cb07 More bug fixes (#14593)
* Adjust mqtt logging behavior

* Set disconnect

* Only consider intel gpu stats error if None is returned
2024-10-25 09:45:11 -05:00
Nicolas Mowen
94f7528885 Bug fixes (#14588)
* Get intel stats manually if parsing fails

* Fix assignment

* Clean up mqtt

* Formatting

* Fix logic
2024-10-25 06:47:56 -06:00
Josh Hawkins
4ca267ea17 Search UI tweaks and bugfixes (#14328)
* Publish model state and embeddings reindex in dispatcher onConnect

* remove unneeded from explore

* add embeddings reindex progress to statusbar

* don't allow right click or show similar button if semantic search is disabled

* fix status bar
2024-10-13 19:36:49 -06:00
Nicolas Mowen
6df541e1fd Openvino models (#14283)
* Enable model conversion cache for openvino

* Use openvino directly for onnx embeddings if available

* Don't fail if zmq is busy
2024-10-11 10:47:23 -06:00
Josh Hawkins
ee8091ba91 Correctly handle camera command in dispatcher (#14273) 2024-10-10 18:48:56 -06:00
Nicolas Mowen
dd6276e706 Embeddings fixes (#14269)
* Add debugging logs for more info

* Improve timeout handling

* Fix event cleanup

* Handle zmq error and empty data

* Don't run download

* Remove unneeded embeddings creations

* Update timouts

* Init models immediately

* Fix order of init

* Cleanup
2024-10-10 16:37:43 -05:00
Josh Hawkins
f67ec241d4 Add embeddings reindex progress to the UI (#14268)
* refactor dispatcher

* add reindex to dictionary

* add circular progress bar component

* Add progress to UI when embeddings are reindexing

* readd comments to dispatcher for clarity

* Only report progress every 10 events so we don't spam the logs and websocket

* clean up
2024-10-10 13:28:43 -06:00
Nicolas Mowen
8ade85edec Restructure embeddings (#14266)
* Restructure embeddings

* Use ZMQ to proxy embeddings requests

* Handle serialization

* Formatting

* Remove unused
2024-10-10 09:42:24 -06:00
Josh Hawkins
24ac9f3e5a Use sqlite-vec extension instead of chromadb for embeddings (#14163)
* swap sqlite_vec for chroma in requirements

* load sqlite_vec in embeddings manager

* remove chroma and revamp Embeddings class for sqlite_vec

* manual minilm onnx inference

* remove chroma in clip model

* migrate api from chroma to sqlite_vec

* migrate event cleanup from chroma to sqlite_vec

* migrate embedding maintainer from chroma to sqlite_vec

* genai description for sqlite_vec

* load sqlite_vec in main thread db

* extend the SqliteQueueDatabase class and use peewee db.execute_sql

* search with Event type for similarity

* fix similarity search

* install and add comment about transformers

* fix normalization

* add id filter

* clean up

* clean up

* fully remove chroma and add transformers env var

* readd uvicorn for fastapi

* readd tokenizer parallelism env var

* remove chroma from docs

* remove chroma from UI

* try removing custom pysqlite3 build

* hard code limit

* optimize queries

* revert explore query

* fix query

* keep building pysqlite3

* single pass fetch and process

* remove unnecessary re-embed

* update deps

* move SqliteVecQueueDatabase to db directory

* make search thumbnail take up full size of results box

* improve typing

* improve model downloading and add status screen

* daemon downloading thread

* catch case when semantic search is disabled

* fix typing

* build sqlite_vec from source

* resolve conflict

* file permissions

* try build deps

* remove sources

* sources

* fix thread start

* include git in build

* reorder embeddings after detectors are started

* build with sqlite amalgamation

* non-platform specific

* use wget instead of curl

* remove unzip -d

* remove sqlite_vec from requirements and load the compiled version

* fix build

* avoid race in db connection

* add scale_factor and bias to description zscore normalization
2024-10-07 14:30:45 -06:00
gtsiam
54900ae318 Properly call super() in subclasses (#14124) 2024-10-02 19:35:46 -06:00