Commit Graph

1498 Commits

Author SHA1 Message Date
Josh Hawkins
6c70e56059
Misc bugfixes and improvements (#14460)
* only save a fixed number of thumbnails if genai is enabled

* disable cpu_mem_arena to save on memory until its actually needed

* fix search settings pane so it actually saves to the config
2024-10-20 14:14:51 -06:00
Josh Hawkins
b24d292ade
Improve Explore SQL query memory usage (#14451)
* Remove sql window function in explore endpoint

* don't revalidate first page on every fetch
2024-10-19 22:12:54 -06:00
Nicolas Mowen
2137de37b9
Fix snapshot call (#14448) 2024-10-19 14:11:49 -05:00
Josh Hawkins
3c591ad8a9
Explore snapshot and clip filter (#14439)
* backend

* add ToggleButton component

* boolean type

* frontend

* allow setting filter in input

* better padding on dual slider

* use shadcn toggle group instead of custom component
2024-10-18 16:16:43 -05:00
Josh Hawkins
5d8bcb42c6
Fix autotrack to work with new tracked object package (#14414) 2024-10-17 10:21:27 -06:00
Josh Hawkins
b299652e86
Generative AI changes (#14413)
* Update default genai prompt

* Update docs

* improve wording

* clarify wording
2024-10-17 10:15:44 -06:00
Nicolas Mowen
8ac4b001a2
Various fixes (#14410)
* Fix access

* Reorganize tracked object for imports

* Separate out rockchip build

* Formatting

* Use original ffmpeg build

* Fix build

* Update default search type value
2024-10-17 11:02:27 -05:00
Josh Hawkins
8173cd7776
Add score filter to Explore view (#14397)
* backend score filtering and sorting

* score filter frontend

* use input for score filtering

* use correct score on search thumbnail

* add popover to explain top_score

* revert sublabel score calc

* update filters logic

* fix rounding on score

* wait until default view is loaded

* don't turn button to selected style for similarity searches

* clarify language

* fix alert dialog buttons to use correct destructive variant

* use root level top_score for very old events

* better arrangement of thumbnail footer items on smaller screens
2024-10-17 05:30:52 -06:00
Nicolas Mowen
5f77408956
Update logos handling (#14396)
* Add attribute for logos

* Clean up tracked object to pass model data

* Update default attributes map
2024-10-16 16:22:34 -05:00
Nicolas Mowen
3f1ab66899
Embeddings UI updates (#14378)
* Handle Frigate+ submitted case

* Add search settings and rename general to ui settings

* Add platform aware sheet component

* use two columns on mobile view

* Add cameras page to more filters

* clean up search settings view

* Add time range to side filter

* better match with ui settings

* fix icon size

* use two columns on mobile view

* clean up search settings view

* Add zones and saving logic

* Add all filters to side panel

* better match with ui settings

* fix icon size

* Fix mobile fitler page

* Fix embeddings access

* Cleanup

* Fix scroll

* fix double scrollbars and add separators on mobile too

* two columns on mobile

* italics for emphasis

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-10-15 19:25:59 -05:00
Nicolas Mowen
b75efcbca2
UI tweaks (#14369)
* Adjust text size

* Make cursor consistent

* Fix lint
2024-10-15 09:37:04 -06:00
Nicolas Mowen
25043278ab
Always run embedding descs one by one (#14365) 2024-10-15 07:40:45 -06:00
Nicolas Mowen
0eccb6a610
Db fixes (#14364)
* Handle case where embeddings overflow token limit

* Set notification tokens

* Fix sort
2024-10-15 07:17:54 -06:00
Nicolas Mowen
3879fde06d
Don't allow unlimited unprocessed segments to stay in cache (#14341)
* Don't allow unlimited unprocessed frames to stay in cache

* Formatting
2024-10-14 16:11:43 -06:00
Nicolas Mowen
887433fc6a
Streaming download (#14346)
* Send downloaded mp4 as a streaming response instead of a file

* Add download button to UI

* Formatting

* Fix CSS and text

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

* download video button component

* use download button component in review detail dialog

* better filename

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-10-14 15:23:02 -06:00
Josh Hawkins
0ee32cf110
Fix yaml bug and ensure embeddings progress doesn't show until all models are loaded (#14338) 2024-10-14 08:23:08 -06:00
Josh Hawkins
72aa68cedc
Fix genai labels (#14330)
* Publish model state and embeddings reindex in dispatcher onConnect

* remove unneeded from explore

* add embeddings reindex progress to statusbar

* don't allow right click or show similar button if semantic search is disabled

* fix status bar

* Convert peewee model to dict before formatting for genai description

* add embeddings reindex progress to statusbar

* fix status bar

* Convert peewee model to dict before formatting for genai description
2024-10-14 06:23:10 -06:00
Nicolas Mowen
9adffa1ef5
Detection adjustments (#14329) 2024-10-13 21:34:51 -05:00
Josh Hawkins
4ca267ea17
Search UI tweaks and bugfixes (#14328)
* Publish model state and embeddings reindex in dispatcher onConnect

* remove unneeded from explore

* add embeddings reindex progress to statusbar

* don't allow right click or show similar button if semantic search is disabled

* fix status bar
2024-10-13 19:36:49 -06:00
Josh Hawkins
833768172d
UI tweaks (#14326)
* small tweaks for frigate+ submission and debug object list

* exclude attributes from labels colormap
2024-10-13 15:48:54 -06:00
Josh Hawkins
1ec459ea3a
Batch embeddings fixes (#14325)
* fixes

* more readable loops

* more robust key check and warning message

* ensure we get reindex progress on mount

* use correct var for length
2024-10-13 15:25:13 -06:00
Nicolas Mowen
e8b2fde753
Support batch embeddings when reindexing (#14320)
* Refactor onnx embeddings to handle multiple inputs by default

* Process items in batches when reindexing
2024-10-13 12:33:27 -05:00
Josh Hawkins
0fc7999780
Improve reindex completion flag (#14308) 2024-10-12 14:44:01 -05:00
Nicolas Mowen
3a403392e7
Fixes for model downloading (#14305)
* Use different requestor for downloaders

* Handle case where lock is left over from failed partial download

* close requestor

* Formatting
2024-10-12 13:36:10 -05:00
Josh Hawkins
48c60621b6
Fix substitution on genai prompts (#14298) 2024-10-12 06:19:24 -06:00
Nicolas Mowen
6e332bbdf8
Remove device config and use model size to configure device used (#14290)
* Remove device config and use model size to configure device used

* Don't show Frigate+ submission when in progress

* Add docs link for bounding box colors
2024-10-11 17:08:14 -05:00
Josh Hawkins
8a8a0c7dec
Embeddings normalization fixes (#14284)
* Use cosine distance metric for vec tables

* Only apply normalization to multi modal searches

* Catch possible edge case in stddev calc

* Use sigmoid function for normalization for multi modal searches only

* Ensure we get model state on initial page load

* Only save stats for multi modal searches and only use cosine similarity for image -> image search
2024-10-11 13:11:11 -05:00
Nicolas Mowen
d4b9b5a7dd
Reduce onnx memory usage (#14285) 2024-10-11 13:03:47 -05:00
Nicolas Mowen
6df541e1fd
Openvino models (#14283)
* Enable model conversion cache for openvino

* Use openvino directly for onnx embeddings if available

* Don't fail if zmq is busy
2024-10-11 10:47:23 -06:00
Josh Hawkins
ae91fa6a39
Add time remaining to embedding reindex pane (#14279)
* Add function to convert seconds to human readable duration

* Add estimated time remaining to reindexing pane
2024-10-11 07:04:25 -06:00
Josh Hawkins
2897afce41
Reset saved search stats on reindex (#14280) 2024-10-11 06:59:29 -06:00
Josh Hawkins
ee8091ba91
Correctly handle camera command in dispatcher (#14273) 2024-10-10 18:48:56 -06:00
Josh Hawkins
30b5faebae
chunk is already a list (#14272) 2024-10-10 17:53:11 -06:00
Josh Hawkins
8d753f821d
Allow empty description for tracked objects (#14271)
* Allow tracked object description to be saved as an empty string

* ensure event_ids is passed as list
2024-10-10 18:12:05 -05:00
Josh Hawkins
54eb03d2a1
Add config option to select fp16 or quantized jina vision model (#14270)
* Add config option to select fp16 or quantized jina vision model

* requires_fp16 for text and large models only

* fix model type check

* fix cpu

* pass model size
2024-10-10 16:46:21 -06:00
Nicolas Mowen
dd6276e706
Embeddings fixes (#14269)
* Add debugging logs for more info

* Improve timeout handling

* Fix event cleanup

* Handle zmq error and empty data

* Don't run download

* Remove unneeded embeddings creations

* Update timouts

* Init models immediately

* Fix order of init

* Cleanup
2024-10-10 16:37:43 -05:00
Josh Hawkins
f67ec241d4
Add embeddings reindex progress to the UI (#14268)
* refactor dispatcher

* add reindex to dictionary

* add circular progress bar component

* Add progress to UI when embeddings are reindexing

* readd comments to dispatcher for clarity

* Only report progress every 10 events so we don't spam the logs and websocket

* clean up
2024-10-10 13:28:43 -06:00
Nicolas Mowen
8ade85edec
Restructure embeddings (#14266)
* Restructure embeddings

* Use ZMQ to proxy embeddings requests

* Handle serialization

* Formatting

* Remove unused
2024-10-10 09:42:24 -06:00
Nicolas Mowen
a2ca18a714
Bug fixes (#14263)
* Simplify loitering logic

* Fix divide by zero

* Add device config for semantic search

* Add docs
2024-10-10 07:09:12 -06:00
Nicolas Mowen
bc3a06178b
Embedding gpu (#14253) 2024-10-09 19:46:31 -06:00
Josh Hawkins
9fda259c0c
Ensure genai prompt is properly formatted (#14256) 2024-10-09 19:19:40 -06:00
Josh Hawkins
d4925622f9
Use JinaAI models for embeddings (#14252)
* add generic onnx model class and use jina ai clip models for all embeddings

* fix merge confligt

* add generic onnx model class and use jina ai clip models for all embeddings

* fix merge confligt

* preferred providers

* fix paths

* disable download progress bar

* remove logging of path

* drop and recreate tables on reindex

* use cache paths

* fix model name

* use trust remote code per transformers docs

* ensure tokenizer and feature extractor are correctly loaded

* revert

* manually download and cache feature extractor config

* remove unneeded

* remove old clip and minilm code

* docs update
2024-10-09 15:31:54 -06:00
Nicolas Mowen
0b71cfaf06
Handle loitering objects (#14221) 2024-10-08 09:41:54 -05:00
Nicolas Mowen
2541a345d0
Improve Nvidia GPU stats (#14206)
* :Add support for nvidia driver info

* Don't show temperature if detector isn't called coral

* Add encoder and decoder info for Nvidia GPUs

* Fix device info

* Implement GPU info for nvidia GPU

* Update web/src/views/system/GeneralMetrics.tsx

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

* Update web/src/views/system/GeneralMetrics.tsx

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-10-07 20:15:31 -06:00
Josh Hawkins
23ce1e930d
Adjust zscore bias for descriptions (#14205) 2024-10-07 18:51:37 -06:00
Josh Hawkins
6ebad84160
initialize path before calling super() (#14203) 2024-10-07 16:17:57 -05:00
Josh Hawkins
24ac9f3e5a
Use sqlite-vec extension instead of chromadb for embeddings (#14163)
* swap sqlite_vec for chroma in requirements

* load sqlite_vec in embeddings manager

* remove chroma and revamp Embeddings class for sqlite_vec

* manual minilm onnx inference

* remove chroma in clip model

* migrate api from chroma to sqlite_vec

* migrate event cleanup from chroma to sqlite_vec

* migrate embedding maintainer from chroma to sqlite_vec

* genai description for sqlite_vec

* load sqlite_vec in main thread db

* extend the SqliteQueueDatabase class and use peewee db.execute_sql

* search with Event type for similarity

* fix similarity search

* install and add comment about transformers

* fix normalization

* add id filter

* clean up

* clean up

* fully remove chroma and add transformers env var

* readd uvicorn for fastapi

* readd tokenizer parallelism env var

* remove chroma from docs

* remove chroma from UI

* try removing custom pysqlite3 build

* hard code limit

* optimize queries

* revert explore query

* fix query

* keep building pysqlite3

* single pass fetch and process

* remove unnecessary re-embed

* update deps

* move SqliteVecQueueDatabase to db directory

* make search thumbnail take up full size of results box

* improve typing

* improve model downloading and add status screen

* daemon downloading thread

* catch case when semantic search is disabled

* fix typing

* build sqlite_vec from source

* resolve conflict

* file permissions

* try build deps

* remove sources

* sources

* fix thread start

* include git in build

* reorder embeddings after detectors are started

* build with sqlite amalgamation

* non-platform specific

* use wget instead of curl

* remove unzip -d

* remove sqlite_vec from requirements and load the compiled version

* fix build

* avoid race in db connection

* add scale_factor and bias to description zscore normalization
2024-10-07 14:30:45 -06:00
Nicolas Mowen
74047453ef
Bug Fixes (#14193)
* fix getting device from empty list

* Overwrite all fields

* Fix header too low
2024-10-07 07:18:09 -06:00
Rémi Bédard-Couture
dcaed0e90f
Log correct tensorRT version when debugging (#14182) 2024-10-06 15:33:03 -06:00
gtsiam
a468ed316d
Added stop_event to util.Process (#14142)
* Added stop_event to util.Process

util.Process will take care of receiving signals when the stop_event is
accessed in the subclass. If it never is, SystemExit is raised instead.

This has the effect of still behaving like multiprocessing.Process when
stop_event is not accessed, while still allowing subclasses to not deal
with the hassle of setting it up.

* Give each util.Process their own logger

This will help to reduce boilerplate in subclasses.

* Give explicit types to util.Process.__init__

This gives better type hinting in the editor.

* Use util.Process facilities in AudioProcessor

Boilerplate begone!

* Removed pointless check in util.Process

The log_listener.queue should never be None, unless something has gone
extremely wrong in the log setup code. If we're that far gone, crashing
is better.

* Make sure faulthandler is enabled in all processes

This has no effect currently since we're using the fork start_method.
However, when we inevidably switch to forkserver (either by choice, or
by upgrading to python 3.14+) not having this makes for some really fun
failure modes :D
2024-10-03 11:03:43 -06:00