* recordings data pub/sub
* function to process recording stream frames
* model runner
* lpr model runner
* refactor to mixin class and use model runner
* separate out realtime and post processors
* move model and mixin folders
* basic postprocessor
* clean up
* docs
* postprocessing logic
* clean up
* return none if recordings are disabled
* run postprocessor handle_requests too
* tweak expansion
* add put endpoint
* postprocessor tweaks with endpoint
* Improve ffmpeg versions handling
* Remove fallback from LIBAVFORMAT_VERSION_MAJOR, it should always be set
* Mention ffprobe in custom ffmpeg docs
* Fix ffmpeg extraction
* Fix go2rtc example formatting
* Add fallback back to LIBAVFORMAT_VERSION_MAJOR
* Fix linter
* Move onnx runner
* Build out base embedding
* Convert text embedding to separate class
* Move image embedding to separate
* Move LPR to separate class
* Remove mono embedding
* Simplify model downloading
* Reorganize jina v1 embeddings
* Cleanup
* Cleanup for review
* Remove thumbnail from dict
* Create thumbnail diectory
* Cleanup handling of tracked object images
* Make thumbnail optional
* Handle cases where thumbnail is used
* Expand options for thumbnail api
* Fix up the does not exist condition
* Remove absolute usages of thumbnails
* Write thumbnails for external events
* Reduce webp quality
* Use webp everywhere in frontend
* Formatting
* Always consider all events when re-indexing
* Add thumbnail deletion and cleanup path management
* Cleanup imports
* Rename def
* Don't save thumbnail for every object
* Correct event count
* Use correct function
* Include thumbnail in query
* Remove unused
* Fix requiring exception
* add path point tracking to backend
* types
* draw paths on lifecycle pane
* make points clickable
* don't display a path if we don't have any saved path points
* only object lifecycle points should have a click handler
* change to debug log
* better debug log message
* add notification cooldown
* cooldown docs
* show alert box when notifications are used in an insecure context
* add ability to suspend notifications from dashboard context menu
* use a small yolov9 model for detection
* use yolov9 for users without frigate+ and update retention algorithm
* new lpr config fields
* levenshtein distance package
* tweaks
* docs
* norfair tracker config per object type
* change default R back to 3.4
* separate trackers for static and autotracking cameras
* tweak params and fix debug draw
* ensure all trackers are correctly updated even when there are no detections
* basic reid with histograms
* check mp value
* check mp value again
* stationary objects won't have embeddings
* don't switch trackers when autotracking is toggled after startup
* improve motion detection during autotracking
* use helper function
* get histogram in tracker instead of detect
* backend
* frontend
* add notification config at camera level
* camera level notifications in dispatcher
* initial onconnect
* frontend
* backend for suspended notifications
* frontend
* use base communicator
* initialize all cameras in suspended array and use 0 for unsuspended
* remove switch and use select for suspending in frontend
* use timestamp instead of datetime
* frontend tweaks
* mqtt docs
* fix button width
* use grid for layout
* use thread and queue for processing notifications with 10s timeout
* clean up
* move async code to main class
* tweaks
* docs
* remove warning message
* WIP yolov9
* Implement post processing for yolov9
* Cleanup detection
* Update docs to make note of supported yolov9
* Move post processing to separate utility
* Add note about other models
* utility functions
* backend config
* backend object speed tracking
* draw speed on debug view
* basic frontend zone editor
* remove line sorting
* fix types
* highlight line on canvas when entering value in zone edit pane
* rename vars and add validation
* ensure speed estimation is disabled when user adds more than 4 points
* pixel velocity in debug
* unit_system in config
* ability to define unit system in config
* save max speed to db
* frontend
* docs
* clarify docs
* utility functions
* backend config
* backend object speed tracking
* draw speed on debug view
* basic frontend zone editor
* remove line sorting
* fix types
* highlight line on canvas when entering value in zone edit pane
* rename vars and add validation
* ensure speed estimation is disabled when user adds more than 4 points
* pixel velocity in debug
* unit_system in config
* ability to define unit system in config
* save max speed to db
* frontend
* docs
* clarify docs
* fix duplicates from merge
* include max_estimated_speed in api responses
* add units to zone edit pane
* catch undefined
* add average speed
* clarify docs
* only track average speed when object is active
* rename vars
* ensure points and distances are ordered clockwise
* only store the last 10 speeds like score history
* remove max estimated speed
* update docs
* update docs
* fix point ordering
* improve readability
* docs inertia recommendation
* fix point ordering
* check object frame time
* add velocity angle to frontend
* docs clarity
* add frontend speed filter
* fix mqtt docs
* fix mqtt docs
* don't try to remove distances if they weren't already defined
* don't display estimates on debug view/snapshots if object is not in a speed tracking zone
* docs
* implement speed_threshold for zone presence
* docs for threshold
* better ground plane image
* improve image zone size
* add inertia to speed threshold example
* config file changes
* config migrator
* stream selection on single camera live view
* camera streaming settings dialog
* manage persistent group streaming settings
* apply streaming settings in camera groups
* add ability to clear all streaming settings from settings
* docs
* update reference config
* fixes
* clarify docs
* use first stream as default in dialog
* ensure still image is visible after switching stream type to none
* docs
* clarify docs
* add ability to continue playing stream in background
* fix props
* put stream selection inside dropdown on desktop
* add capabilities to live mode hook
* live context menu component
* resize observer: only return new dimensions if they've actually changed
* pass volume prop to players
* fix slider bug, https://github.com/shadcn-ui/ui/issues/1448
* update react-grid-layout
* prevent animated transitions on draggable grid layout
* add context menu to dashboards
* use provider
* streaming dialog from context menu
* docs
* add jsmpeg warning to context menu
* audio and two way talk indicators in single camera view
* add link to debug view
* don't use hook
* create manual events from live camera view
* maintain grow classes on grid items
* fix initial volume state on default dashboard
* fix pointer events causing context menu to end up underneath image on iOS
* mobile drawer tweaks
* stream stats
* show settings menu for non-restreamed cameras
* consistent settings icon
* tweaks
* optional stats to fix birdseye player
* add toaster to live camera view
* fix crash on initial save in streaming dialog
* don't require restreaming for context menu streaming settings
* add debug view to context menu
* stats fixes
* update docs
* always show stream info when restreamed
* update camera streaming dialog
* make note of no h265 support for webrtc
* docs clarity
* ensure docs show streams as a dict
* docs clarity
* fix css file
* tweaks
* Start working on bird processor
* Initial setup for bird processing
* Improvements to handling
* Get classification working
* Cleanup classification
* Add classification config
* Update sort
* Actually send result to face registration
* Define postprocessing api and move face processing to fit
* Standardize request handling
* Standardize handling of processors
* Rename processing metrics
* Cleanup
* Standardize object end
* Update to newer formatting
* One more
* One more
* Get stats for embeddings inferences
* cleanup embeddings inferences
* Enable UI for feature metrics
* Change threshold
* Fix check
* Update python for actions
* Set python version
* Ignore type for now
* Support downloading face models
* Handle download and loading correctly
* Add face dir creation
* Fix error
* Fix
* Formatting
* Move upload to button
* Show number of faces in library for each name
* Add text color for score
* Cleanup
* rockchip: update dependencies and add script for model conversion
* rockchip: update docs
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* use ruamel to parse and preserve line numbers for config validation
* maintain exception for non validation errors
* fix types
* include input in log messages
* Validate faces using cosine distance and SVC
* Formatting
* Use opencv instead of face embedding
* Update docs for training data
* Adjust to score system
* Set bounds
* remove face embeddings
* Update writing images
* Add face library page
* Add ability to select file
* Install opencv deps
* Cleanup
* Use different deps
* Move deps
* Cleanup
* Only show face library for desktop
* Implement deleting
* Add ability to upload image
* Add support for uploading images
* Add margin to detected faces for embeddings
* Standardize pixel values for face input
* Use SVC to classify faces
* Clear classifier when new face is added
* Formatting
* Add dependency
* Update version
* Face recognition backend (#14495)
* Add basic config and face recognition table
* Reconfigure updates processing to handle face
* Crop frame to face box
* Implement face embedding calculation
* Get matching face embeddings
* Add support face recognition based on existing faces
* Use arcface face embeddings instead of generic embeddings model
* Add apis for managing faces
* Implement face uploading API
* Build out more APIs
* Add min area config
* Handle larger images
* Add more debug logs
* fix calculation
* Reduce timeout
* Small tweaks
* Use webp images
* Use facenet model
* Improve face recognition (#14537)
* Increase requirements for face to be set
* Manage faces properly
* Add basic docs
* Simplify
* Separate out face recognition frome semantic search
* Update docs
* Formatting
* Fix access (#14540)
* Face detection (#14544)
* Add support for face detection
* Add support for detecting faces during registration
* Set body size to be larger
* Undo
* Update version
* Face recognition backend (#14495)
* Add basic config and face recognition table
* Reconfigure updates processing to handle face
* Crop frame to face box
* Implement face embedding calculation
* Get matching face embeddings
* Add support face recognition based on existing faces
* Use arcface face embeddings instead of generic embeddings model
* Add apis for managing faces
* Implement face uploading API
* Build out more APIs
* Add min area config
* Handle larger images
* Add more debug logs
* fix calculation
* Reduce timeout
* Small tweaks
* Use webp images
* Use facenet model
* Improve face recognition (#14537)
* Increase requirements for face to be set
* Manage faces properly
* Add basic docs
* Simplify
* Separate out face recognition frome semantic search
* Update docs
* Formatting
* Fix access (#14540)
* Face detection (#14544)
* Add support for face detection
* Add support for detecting faces during registration
* Set body size to be larger
* Undo
* initial foundation for alpr with paddleocr
* initial foundation for alpr with paddleocr
* initial foundation for alpr with paddleocr
* config
* config
* lpr maintainer
* clean up
* clean up
* fix processing
* don't process for stationary cars
* fix order
* fixes
* check for known plates
* improved length and character by character confidence
* model fixes and small tweaks
* docs
* placeholder for non frigate+ model lp detection
---------
Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
* Increase requirements for face to be set
* Manage faces properly
* Add basic docs
* Simplify
* Separate out face recognition frome semantic search
* Update docs
* Formatting
* Add basic config and face recognition table
* Reconfigure updates processing to handle face
* Crop frame to face box
* Implement face embedding calculation
* Get matching face embeddings
* Add support face recognition based on existing faces
* Use arcface face embeddings instead of generic embeddings model
* Add apis for managing faces
* Implement face uploading API
* Build out more APIs
* Add min area config
* Handle larger images
* Add more debug logs
* fix calculation
* Reduce timeout
* Small tweaks
* Use webp images
* Use facenet model
* Started unit tests for the review controller
* Revert "Started unit tests for the review controller"
This reverts commit 7746eb146f.
* Started unit tests for GET /review/activity/motion Endpoint
* Started unit tests for GET /review/event/{event_id} Endpoint
* Continued unit tests for GET /review/event/{event_id} Endpoint
* Continued unit tests for GET /review/{event_id} Endpoint
* Continued unit tests for GET /review/{review_id} Endpoint
* Added unit tests for GET /review/{review_id}/viewed Endpoint
* Added unit tests for GET /stats Endpoint
* Added unit tests for GET /events Endpoint
* Updated unit tests for GET /events Endpoint
* Deleted unit tests for /events from test_http (updated tests are now in test_http_event.py)
* Removed duplicated test for GET /review/activity/motion Endpoint
* GenAI: add ability to save JPGs sent to provider
* Remove mention from GenAI docs
* Change config name to debug_save_thumbnails
* Change folder structure to clips/genai-requests/{event_id}/{1.jpg}
* Organize api files
* Add more API definitions for events
* Add export select by ID
* Typing fixes
* Update openapi spec
* Change type
* Fix test
* Fix message
* Fix tests
* use id instead of index for object details and scrolling
* long press package and hook
* fix long press in review
* search action group
* multi select in explore
* add bulk deletion to backend api
* clean up
* mimic behavior of review
* don't open dialog on left click when mutli selecting
* context menu on container ref
* revert long press code
* clean up
* Fix audio events in explore section
Make sure that audio events are listed in the explore section
* Update audio.py
* Hide other submit options
Only allow submits for objects only
* Started unit tests for the review controller
* Revert "Started unit tests for the review controller"
This reverts commit 7746eb146f.
* Started unit tests for the review controller
* FIrst test
* Added test for review endpoint (time filter - after + before)
* Assert expected event
* Added more tests for review endpoint
* Added test for review endpoint with all filters
* Added test for review endpoint with limit
* Comment
* Renamed tests to increase readability
* fix regex for cookie_name to be general snake case
* Update frigate/config/auth.py
Co-authored-by: Blake Blackshear <blake.blackshear@gmail.com>
---------
Co-authored-by: Blake Blackshear <blake.blackshear@gmail.com>
* Don't track shared memory in frame tracker
* Don't track any instance
* Don't assign sub label to objects when multiple cars are overlapping
* Formatting
* Fix assignment
* Use custom body for the export recordings endpoint
* Fixed usage of ExportRecordingsBody
* Updated docs to reflect changes to export endpoint
* Fix friendly name and source
* Updated openAPI spec
* Updated documentation for the review endpoint
* Updated documentation for the review/summary endpoint
* Updated documentation for the review/summary endpoint
* Documentation for the review activity audio and motion endpoints
* Added responses for more review.py endpoints
* Added responses for more review.py endpoints
* Fixed review.py responses and proper path parameter names
* Added body model for /reviews/viewed and /reviews/delete
* Updated OpenAPI specification for the review controller endpoints
* Run ruff format frigate
* Drop significant_motion
* Updated frigate-api.yaml
* Deleted total_motion
* Combine 2 models into generic
* Add reindex progress to mobile bottom bar status alert
* move menu to new component
* actions component in search footer thumbnail
* context menu for explore summary thumbnail images
* readd top_score to search query for old events
* Add service manager infrastructure
The changes are (This will be a bit long):
- A ServiceManager class that spawns a background thread and deals with
service lifecycle management. The idea is that service lifecycle code
will run in async functions, so a single thread is enough to manage
any (reasonable) amount of services.
- A Service class, that offers start(), stop() and restart() methods
that simply notify the service manager to... well. Start, stop or
restart a service.
(!) Warning: Note that this differs from mp.Process.start/stop in that
the service commands are sent asynchronously and will complete
"eventually". This is good because it means that business logic is
fast when booting up and shutting down, but we need to make sure
that code does not rely on start() and stop() being instant
(Mainly pid assignments).
Subclasses of the Service class should use the on_start and on_stop
methods to monitor for service events. These will be run by the
service manager thread, so we need to be careful not to block
execution here. Standard async stuff.
(!) Note on service names: Service names should be unique within a
ServiceManager. Make sure that you pass the name you want to
super().__init__(name="...") if you plan to spawn multiple instances
of a service.
- A ServiceProcess class: A Service that wraps a multiprocessing.Process
into a Service. It offers a run() method subclasses can override and
can support in-place restarting using the service manager.
And finally, I lied a bit about this whole thing using a single thread.
I can't find any way to run python multiprocessing in async, so there is
a MultiprocessingWaiter thread that waits for multiprocessing events and
notifies any pending futures. This was uhhh... fun? No, not really.
But it works. Using this part of the code just involves calling the
provided wait method. See the implementation of ServiceProcess for more
details.
Mirror util.Process hooks onto service process
Remove Service.__name attribute
Do not serialize process object on ServiceProcess start.
asd
* Update frigate dictionary
* Convert AudioProcessor to service process
* only save a fixed number of thumbnails if genai is enabled
* disable cpu_mem_arena to save on memory until its actually needed
* fix search settings pane so it actually saves to the config
* Fix access
* Reorganize tracked object for imports
* Separate out rockchip build
* Formatting
* Use original ffmpeg build
* Fix build
* Update default search type value
* backend score filtering and sorting
* score filter frontend
* use input for score filtering
* use correct score on search thumbnail
* add popover to explain top_score
* revert sublabel score calc
* update filters logic
* fix rounding on score
* wait until default view is loaded
* don't turn button to selected style for similarity searches
* clarify language
* fix alert dialog buttons to use correct destructive variant
* use root level top_score for very old events
* better arrangement of thumbnail footer items on smaller screens
* Handle Frigate+ submitted case
* Add search settings and rename general to ui settings
* Add platform aware sheet component
* use two columns on mobile view
* Add cameras page to more filters
* clean up search settings view
* Add time range to side filter
* better match with ui settings
* fix icon size
* use two columns on mobile view
* clean up search settings view
* Add zones and saving logic
* Add all filters to side panel
* better match with ui settings
* fix icon size
* Fix mobile fitler page
* Fix embeddings access
* Cleanup
* Fix scroll
* fix double scrollbars and add separators on mobile too
* two columns on mobile
* italics for emphasis
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Publish model state and embeddings reindex in dispatcher onConnect
* remove unneeded from explore
* add embeddings reindex progress to statusbar
* don't allow right click or show similar button if semantic search is disabled
* fix status bar
* Convert peewee model to dict before formatting for genai description
* add embeddings reindex progress to statusbar
* fix status bar
* Convert peewee model to dict before formatting for genai description
* Publish model state and embeddings reindex in dispatcher onConnect
* remove unneeded from explore
* add embeddings reindex progress to statusbar
* don't allow right click or show similar button if semantic search is disabled
* fix status bar
* Remove device config and use model size to configure device used
* Don't show Frigate+ submission when in progress
* Add docs link for bounding box colors
* Use cosine distance metric for vec tables
* Only apply normalization to multi modal searches
* Catch possible edge case in stddev calc
* Use sigmoid function for normalization for multi modal searches only
* Ensure we get model state on initial page load
* Only save stats for multi modal searches and only use cosine similarity for image -> image search
* Add config option to select fp16 or quantized jina vision model
* requires_fp16 for text and large models only
* fix model type check
* fix cpu
* pass model size
* refactor dispatcher
* add reindex to dictionary
* add circular progress bar component
* Add progress to UI when embeddings are reindexing
* readd comments to dispatcher for clarity
* Only report progress every 10 events so we don't spam the logs and websocket
* clean up
* add generic onnx model class and use jina ai clip models for all embeddings
* fix merge confligt
* add generic onnx model class and use jina ai clip models for all embeddings
* fix merge confligt
* preferred providers
* fix paths
* disable download progress bar
* remove logging of path
* drop and recreate tables on reindex
* use cache paths
* fix model name
* use trust remote code per transformers docs
* ensure tokenizer and feature extractor are correctly loaded
* revert
* manually download and cache feature extractor config
* remove unneeded
* remove old clip and minilm code
* docs update
* :Add support for nvidia driver info
* Don't show temperature if detector isn't called coral
* Add encoder and decoder info for Nvidia GPUs
* Fix device info
* Implement GPU info for nvidia GPU
* Update web/src/views/system/GeneralMetrics.tsx
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* Update web/src/views/system/GeneralMetrics.tsx
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
---------
Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
* swap sqlite_vec for chroma in requirements
* load sqlite_vec in embeddings manager
* remove chroma and revamp Embeddings class for sqlite_vec
* manual minilm onnx inference
* remove chroma in clip model
* migrate api from chroma to sqlite_vec
* migrate event cleanup from chroma to sqlite_vec
* migrate embedding maintainer from chroma to sqlite_vec
* genai description for sqlite_vec
* load sqlite_vec in main thread db
* extend the SqliteQueueDatabase class and use peewee db.execute_sql
* search with Event type for similarity
* fix similarity search
* install and add comment about transformers
* fix normalization
* add id filter
* clean up
* clean up
* fully remove chroma and add transformers env var
* readd uvicorn for fastapi
* readd tokenizer parallelism env var
* remove chroma from docs
* remove chroma from UI
* try removing custom pysqlite3 build
* hard code limit
* optimize queries
* revert explore query
* fix query
* keep building pysqlite3
* single pass fetch and process
* remove unnecessary re-embed
* update deps
* move SqliteVecQueueDatabase to db directory
* make search thumbnail take up full size of results box
* improve typing
* improve model downloading and add status screen
* daemon downloading thread
* catch case when semantic search is disabled
* fix typing
* build sqlite_vec from source
* resolve conflict
* file permissions
* try build deps
* remove sources
* sources
* fix thread start
* include git in build
* reorder embeddings after detectors are started
* build with sqlite amalgamation
* non-platform specific
* use wget instead of curl
* remove unzip -d
* remove sqlite_vec from requirements and load the compiled version
* fix build
* avoid race in db connection
* add scale_factor and bias to description zscore normalization
* Added stop_event to util.Process
util.Process will take care of receiving signals when the stop_event is
accessed in the subclass. If it never is, SystemExit is raised instead.
This has the effect of still behaving like multiprocessing.Process when
stop_event is not accessed, while still allowing subclasses to not deal
with the hassle of setting it up.
* Give each util.Process their own logger
This will help to reduce boilerplate in subclasses.
* Give explicit types to util.Process.__init__
This gives better type hinting in the editor.
* Use util.Process facilities in AudioProcessor
Boilerplate begone!
* Removed pointless check in util.Process
The log_listener.queue should never be None, unless something has gone
extremely wrong in the log setup code. If we're that far gone, crashing
is better.
* Make sure faulthandler is enabled in all processes
This has no effect currently since we're using the fork start_method.
However, when we inevidably switch to forkserver (either by choice, or
by upgrading to python 3.14+) not having this makes for some really fun
failure modes :D
I just saw this, and I would be very surprised by that behaviour as a
user. Changing the db path would randomly move the database, and
changing it back (or to anything, really) would not. These kinds of
advanced settings are generally expected to do one thing: Change the
path frigate opens the database from. The end.
