Commit Graph

3496 Commits

Author SHA1 Message Date
Josh Hawkins
0ee32cf110
Fix yaml bug and ensure embeddings progress doesn't show until all models are loaded (#14338) 2024-10-14 08:23:08 -06:00
Josh Hawkins
72aa68cedc
Fix genai labels (#14330)
* Publish model state and embeddings reindex in dispatcher onConnect

* remove unneeded from explore

* add embeddings reindex progress to statusbar

* don't allow right click or show similar button if semantic search is disabled

* fix status bar

* Convert peewee model to dict before formatting for genai description

* add embeddings reindex progress to statusbar

* fix status bar

* Convert peewee model to dict before formatting for genai description
2024-10-14 06:23:10 -06:00
Nicolas Mowen
9adffa1ef5
Detection adjustments (#14329) 2024-10-13 21:34:51 -05:00
Josh Hawkins
4ca267ea17
Search UI tweaks and bugfixes (#14328)
* Publish model state and embeddings reindex in dispatcher onConnect

* remove unneeded from explore

* add embeddings reindex progress to statusbar

* don't allow right click or show similar button if semantic search is disabled

* fix status bar
2024-10-13 19:36:49 -06:00
Josh Hawkins
833768172d
UI tweaks (#14326)
* small tweaks for frigate+ submission and debug object list

* exclude attributes from labels colormap
2024-10-13 15:48:54 -06:00
Josh Hawkins
1ec459ea3a
Batch embeddings fixes (#14325)
* fixes

* more readable loops

* more robust key check and warning message

* ensure we get reindex progress on mount

* use correct var for length
2024-10-13 15:25:13 -06:00
Josh Hawkins
66d0ad5803
See a preview when using the timeline to export footage (#14321)
* custom hook and generic video player component

* add export preview dialog

* export preview dialog when using timeline export

* refactor search detail dialog to use new generic video player component

* clean up
2024-10-13 12:46:40 -05:00
Josh Hawkins
92ac025e43
Don't show submit to frigate plus card if plus is disabled (#14319) 2024-10-13 11:34:39 -06:00
Nicolas Mowen
e8b2fde753
Support batch embeddings when reindexing (#14320)
* Refactor onnx embeddings to handle multiple inputs by default

* Process items in batches when reindexing
2024-10-13 12:33:27 -05:00
Josh Hawkins
0fc7999780
Improve reindex completion flag (#14308) 2024-10-12 14:44:01 -05:00
Nicolas Mowen
3a403392e7
Fixes for model downloading (#14305)
* Use different requestor for downloaders

* Handle case where lock is left over from failed partial download

* close requestor

* Formatting
2024-10-12 13:36:10 -05:00
Josh Hawkins
acccc6fd93
Only revalidate if event update is valid (#14302) 2024-10-12 08:32:11 -06:00
Nicolas Mowen
40bb4765d4
Add support for more icons (#14299) 2024-10-12 08:37:22 -05:00
Josh Hawkins
48c60621b6
Fix substitution on genai prompts (#14298) 2024-10-12 06:19:24 -06:00
Nicolas Mowen
51509760e3
Update object docs (#14295) 2024-10-12 07:13:00 -05:00
Josh Hawkins
1e1610671e
Add info icons for popovers in debug view (#14296) 2024-10-12 06:12:02 -06:00
Josh Hawkins
de86c37687
Prevent single letter words from matching filter suggestions (#14297) 2024-10-12 06:11:22 -06:00
Nicolas Mowen
6e332bbdf8
Remove device config and use model size to configure device used (#14290)
* Remove device config and use model size to configure device used

* Don't show Frigate+ submission when in progress

* Add docs link for bounding box colors
2024-10-11 17:08:14 -05:00
Josh Hawkins
8a8a0c7dec
Embeddings normalization fixes (#14284)
* Use cosine distance metric for vec tables

* Only apply normalization to multi modal searches

* Catch possible edge case in stddev calc

* Use sigmoid function for normalization for multi modal searches only

* Ensure we get model state on initial page load

* Only save stats for multi modal searches and only use cosine similarity for image -> image search
2024-10-11 13:11:11 -05:00
Nicolas Mowen
d4b9b5a7dd
Reduce onnx memory usage (#14285) 2024-10-11 13:03:47 -05:00
Nicolas Mowen
6df541e1fd
Openvino models (#14283)
* Enable model conversion cache for openvino

* Use openvino directly for onnx embeddings if available

* Don't fail if zmq is busy
2024-10-11 10:47:23 -06:00
Josh Hawkins
748087483c
Use number keys on keyboard to move ptz camera to presets (#14278)
* Use number keys on keyboard to move ptz camera to presets

* clean up
2024-10-11 07:05:28 -06:00
Josh Hawkins
ae91fa6a39
Add time remaining to embedding reindex pane (#14279)
* Add function to convert seconds to human readable duration

* Add estimated time remaining to reindexing pane
2024-10-11 07:04:25 -06:00
Josh Hawkins
2897afce41
Reset saved search stats on reindex (#14280) 2024-10-11 06:59:29 -06:00
Josh Hawkins
ee8091ba91
Correctly handle camera command in dispatcher (#14273) 2024-10-10 18:48:56 -06:00
Josh Hawkins
30b5faebae
chunk is already a list (#14272) 2024-10-10 17:53:11 -06:00
Josh Hawkins
8d753f821d
Allow empty description for tracked objects (#14271)
* Allow tracked object description to be saved as an empty string

* ensure event_ids is passed as list
2024-10-10 18:12:05 -05:00
Josh Hawkins
54eb03d2a1
Add config option to select fp16 or quantized jina vision model (#14270)
* Add config option to select fp16 or quantized jina vision model

* requires_fp16 for text and large models only

* fix model type check

* fix cpu

* pass model size
2024-10-10 16:46:21 -06:00
Nicolas Mowen
dd6276e706
Embeddings fixes (#14269)
* Add debugging logs for more info

* Improve timeout handling

* Fix event cleanup

* Handle zmq error and empty data

* Don't run download

* Remove unneeded embeddings creations

* Update timouts

* Init models immediately

* Fix order of init

* Cleanup
2024-10-10 16:37:43 -05:00
Josh Hawkins
f67ec241d4
Add embeddings reindex progress to the UI (#14268)
* refactor dispatcher

* add reindex to dictionary

* add circular progress bar component

* Add progress to UI when embeddings are reindexing

* readd comments to dispatcher for clarity

* Only report progress every 10 events so we don't spam the logs and websocket

* clean up
2024-10-10 13:28:43 -06:00
Nicolas Mowen
8ade85edec
Restructure embeddings (#14266)
* Restructure embeddings

* Use ZMQ to proxy embeddings requests

* Handle serialization

* Formatting

* Remove unused
2024-10-10 09:42:24 -06:00
Nicolas Mowen
a2ca18a714
Bug fixes (#14263)
* Simplify loitering logic

* Fix divide by zero

* Add device config for semantic search

* Add docs
2024-10-10 07:09:12 -06:00
Josh Hawkins
6a83ff2511
Fix config editor error pane (#14264) 2024-10-10 07:09:03 -06:00
Nicolas Mowen
bc3a06178b
Embedding gpu (#14253) 2024-10-09 19:46:31 -06:00
Josh Hawkins
9fda259c0c
Ensure genai prompt is properly formatted (#14256) 2024-10-09 19:19:40 -06:00
Josh Hawkins
d4925622f9
Use JinaAI models for embeddings (#14252)
* add generic onnx model class and use jina ai clip models for all embeddings

* fix merge confligt

* add generic onnx model class and use jina ai clip models for all embeddings

* fix merge confligt

* preferred providers

* fix paths

* disable download progress bar

* remove logging of path

* drop and recreate tables on reindex

* use cache paths

* fix model name

* use trust remote code per transformers docs

* ensure tokenizer and feature extractor are correctly loaded

* revert

* manually download and cache feature extractor config

* remove unneeded

* remove old clip and minilm code

* docs update
2024-10-09 15:31:54 -06:00
Nicolas Mowen
dbeaf43b8f
Fix detector config help template (#14249)
* Fix detector config

* Fix general support
2024-10-09 16:04:31 -05:00
JC
f86957e5e1
Improve docs on exports API endpoints (#14224)
* Add (optional) export name to the create-export API endpoint docs

* Add the exports list endpoint to the docs
2024-10-08 19:15:10 -05:00
Nicolas Mowen
a2f42d51fd
Fix install docs (#14226) 2024-10-08 15:48:54 -05:00
Nicolas Mowen
0b71cfaf06
Handle loitering objects (#14221) 2024-10-08 09:41:54 -05:00
Josh Hawkins
d558ac83b6
Search fixes (#14217)
* Ensure semantic search is enabled before checking model download state

* Only clear similarity search when removing similarity pill
2024-10-08 07:01:31 -06:00
Josh Hawkins
74efc94649
Always display confidence chip on similarity searches (#14207) 2024-10-07 20:15:49 -06:00
Nicolas Mowen
2541a345d0
Improve Nvidia GPU stats (#14206)
* :Add support for nvidia driver info

* Don't show temperature if detector isn't called coral

* Add encoder and decoder info for Nvidia GPUs

* Fix device info

* Implement GPU info for nvidia GPU

* Update web/src/views/system/GeneralMetrics.tsx

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

* Update web/src/views/system/GeneralMetrics.tsx

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-10-07 20:15:31 -06:00
Josh Hawkins
23ce1e930d
Adjust zscore bias for descriptions (#14205) 2024-10-07 18:51:37 -06:00
Josh Hawkins
6ebad84160
initialize path before calling super() (#14203) 2024-10-07 16:17:57 -05:00
Josh Hawkins
24ac9f3e5a
Use sqlite-vec extension instead of chromadb for embeddings (#14163)
* swap sqlite_vec for chroma in requirements

* load sqlite_vec in embeddings manager

* remove chroma and revamp Embeddings class for sqlite_vec

* manual minilm onnx inference

* remove chroma in clip model

* migrate api from chroma to sqlite_vec

* migrate event cleanup from chroma to sqlite_vec

* migrate embedding maintainer from chroma to sqlite_vec

* genai description for sqlite_vec

* load sqlite_vec in main thread db

* extend the SqliteQueueDatabase class and use peewee db.execute_sql

* search with Event type for similarity

* fix similarity search

* install and add comment about transformers

* fix normalization

* add id filter

* clean up

* clean up

* fully remove chroma and add transformers env var

* readd uvicorn for fastapi

* readd tokenizer parallelism env var

* remove chroma from docs

* remove chroma from UI

* try removing custom pysqlite3 build

* hard code limit

* optimize queries

* revert explore query

* fix query

* keep building pysqlite3

* single pass fetch and process

* remove unnecessary re-embed

* update deps

* move SqliteVecQueueDatabase to db directory

* make search thumbnail take up full size of results box

* improve typing

* improve model downloading and add status screen

* daemon downloading thread

* catch case when semantic search is disabled

* fix typing

* build sqlite_vec from source

* resolve conflict

* file permissions

* try build deps

* remove sources

* sources

* fix thread start

* include git in build

* reorder embeddings after detectors are started

* build with sqlite amalgamation

* non-platform specific

* use wget instead of curl

* remove unzip -d

* remove sqlite_vec from requirements and load the compiled version

* fix build

* avoid race in db connection

* add scale_factor and bias to description zscore normalization
2024-10-07 14:30:45 -06:00
Nicolas Mowen
2a15b95f18
Docs updates (#14202)
* Clarify live docs

* Link out to common config examples in getting started guide

* Add tip for go2rtc name configuration

* direct link
2024-10-07 15:28:24 -05:00
Rui Alves
757150dec1
Use Swagger documentation for Frigate HTTP API (#14178)
* Updated documentation

* docusaurus.config and sidebars converted to Typescript to allow for typings

* Added type for sidebars.ts

* Replaced integrations/api.md with automatically generated openAPI specification. Make sidebar collapsible to increase readability

* Fix HTTP API links in the documentation

* Added rust as language in the openapi sidebar

* Make sure configuration/pwa is present

* Fix API slug

* Fix links

* Revert sidebarCollapsible configuration

* Make HTTP API sidebar collapsed by default. Added CSS for OpenAPI methods

* Proper localhost server path

* Proper localhost server path

* No introduction page

* Lint
2024-10-07 14:27:35 -06:00
Josh Hawkins
ddcec82b61
Explicitly set video tag dimensions to fit inside dialog (#14184) 2024-10-07 07:19:22 -06:00
Nicolas Mowen
74047453ef
Bug Fixes (#14193)
* fix getting device from empty list

* Overwrite all fields

* Fix header too low
2024-10-07 07:18:09 -06:00