Commit Graph

3384 Commits

Author SHA1 Message Date
Nicolas Mowen
e8b2fde753
Support batch embeddings when reindexing (#14320)
* Refactor onnx embeddings to handle multiple inputs by default

* Process items in batches when reindexing
2024-10-13 12:33:27 -05:00
Josh Hawkins
0fc7999780
Improve reindex completion flag (#14308) 2024-10-12 14:44:01 -05:00
Nicolas Mowen
3a403392e7
Fixes for model downloading (#14305)
* Use different requestor for downloaders

* Handle case where lock is left over from failed partial download

* close requestor

* Formatting
2024-10-12 13:36:10 -05:00
Josh Hawkins
acccc6fd93
Only revalidate if event update is valid (#14302) 2024-10-12 08:32:11 -06:00
Nicolas Mowen
40bb4765d4
Add support for more icons (#14299) 2024-10-12 08:37:22 -05:00
Josh Hawkins
48c60621b6
Fix substitution on genai prompts (#14298) 2024-10-12 06:19:24 -06:00
Josh Hawkins
1e1610671e
Add info icons for popovers in debug view (#14296) 2024-10-12 06:12:02 -06:00
Josh Hawkins
de86c37687
Prevent single letter words from matching filter suggestions (#14297) 2024-10-12 06:11:22 -06:00
Nicolas Mowen
6e332bbdf8
Remove device config and use model size to configure device used (#14290)
* Remove device config and use model size to configure device used

* Don't show Frigate+ submission when in progress

* Add docs link for bounding box colors
2024-10-11 17:08:14 -05:00
Josh Hawkins
8a8a0c7dec
Embeddings normalization fixes (#14284)
* Use cosine distance metric for vec tables

* Only apply normalization to multi modal searches

* Catch possible edge case in stddev calc

* Use sigmoid function for normalization for multi modal searches only

* Ensure we get model state on initial page load

* Only save stats for multi modal searches and only use cosine similarity for image -> image search
2024-10-11 13:11:11 -05:00
Nicolas Mowen
d4b9b5a7dd
Reduce onnx memory usage (#14285) 2024-10-11 13:03:47 -05:00
Nicolas Mowen
6df541e1fd
Openvino models (#14283)
* Enable model conversion cache for openvino

* Use openvino directly for onnx embeddings if available

* Don't fail if zmq is busy
2024-10-11 10:47:23 -06:00
Josh Hawkins
748087483c
Use number keys on keyboard to move ptz camera to presets (#14278)
* Use number keys on keyboard to move ptz camera to presets

* clean up
2024-10-11 07:05:28 -06:00
Josh Hawkins
ae91fa6a39
Add time remaining to embedding reindex pane (#14279)
* Add function to convert seconds to human readable duration

* Add estimated time remaining to reindexing pane
2024-10-11 07:04:25 -06:00
Josh Hawkins
2897afce41
Reset saved search stats on reindex (#14280) 2024-10-11 06:59:29 -06:00
Josh Hawkins
ee8091ba91
Correctly handle camera command in dispatcher (#14273) 2024-10-10 18:48:56 -06:00
Josh Hawkins
30b5faebae
chunk is already a list (#14272) 2024-10-10 17:53:11 -06:00
Josh Hawkins
8d753f821d
Allow empty description for tracked objects (#14271)
* Allow tracked object description to be saved as an empty string

* ensure event_ids is passed as list
2024-10-10 18:12:05 -05:00
Josh Hawkins
54eb03d2a1
Add config option to select fp16 or quantized jina vision model (#14270)
* Add config option to select fp16 or quantized jina vision model

* requires_fp16 for text and large models only

* fix model type check

* fix cpu

* pass model size
2024-10-10 16:46:21 -06:00
Nicolas Mowen
dd6276e706
Embeddings fixes (#14269)
* Add debugging logs for more info

* Improve timeout handling

* Fix event cleanup

* Handle zmq error and empty data

* Don't run download

* Remove unneeded embeddings creations

* Update timouts

* Init models immediately

* Fix order of init

* Cleanup
2024-10-10 16:37:43 -05:00
Josh Hawkins
f67ec241d4
Add embeddings reindex progress to the UI (#14268)
* refactor dispatcher

* add reindex to dictionary

* add circular progress bar component

* Add progress to UI when embeddings are reindexing

* readd comments to dispatcher for clarity

* Only report progress every 10 events so we don't spam the logs and websocket

* clean up
2024-10-10 13:28:43 -06:00
Nicolas Mowen
8ade85edec
Restructure embeddings (#14266)
* Restructure embeddings

* Use ZMQ to proxy embeddings requests

* Handle serialization

* Formatting

* Remove unused
2024-10-10 09:42:24 -06:00
Nicolas Mowen
a2ca18a714
Bug fixes (#14263)
* Simplify loitering logic

* Fix divide by zero

* Add device config for semantic search

* Add docs
2024-10-10 07:09:12 -06:00
Josh Hawkins
6a83ff2511
Fix config editor error pane (#14264) 2024-10-10 07:09:03 -06:00
Nicolas Mowen
bc3a06178b
Embedding gpu (#14253) 2024-10-09 19:46:31 -06:00
Josh Hawkins
9fda259c0c
Ensure genai prompt is properly formatted (#14256) 2024-10-09 19:19:40 -06:00
Josh Hawkins
d4925622f9
Use JinaAI models for embeddings (#14252)
* add generic onnx model class and use jina ai clip models for all embeddings

* fix merge confligt

* add generic onnx model class and use jina ai clip models for all embeddings

* fix merge confligt

* preferred providers

* fix paths

* disable download progress bar

* remove logging of path

* drop and recreate tables on reindex

* use cache paths

* fix model name

* use trust remote code per transformers docs

* ensure tokenizer and feature extractor are correctly loaded

* revert

* manually download and cache feature extractor config

* remove unneeded

* remove old clip and minilm code

* docs update
2024-10-09 15:31:54 -06:00
Nicolas Mowen
dbeaf43b8f
Fix detector config help template (#14249)
* Fix detector config

* Fix general support
2024-10-09 16:04:31 -05:00
Nicolas Mowen
a2f42d51fd
Fix install docs (#14226) 2024-10-08 15:48:54 -05:00
Nicolas Mowen
0b71cfaf06
Handle loitering objects (#14221) 2024-10-08 09:41:54 -05:00
Josh Hawkins
d558ac83b6
Search fixes (#14217)
* Ensure semantic search is enabled before checking model download state

* Only clear similarity search when removing similarity pill
2024-10-08 07:01:31 -06:00
Josh Hawkins
74efc94649
Always display confidence chip on similarity searches (#14207) 2024-10-07 20:15:49 -06:00
Nicolas Mowen
2541a345d0
Improve Nvidia GPU stats (#14206)
* :Add support for nvidia driver info

* Don't show temperature if detector isn't called coral

* Add encoder and decoder info for Nvidia GPUs

* Fix device info

* Implement GPU info for nvidia GPU

* Update web/src/views/system/GeneralMetrics.tsx

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

* Update web/src/views/system/GeneralMetrics.tsx

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-10-07 20:15:31 -06:00
Josh Hawkins
23ce1e930d
Adjust zscore bias for descriptions (#14205) 2024-10-07 18:51:37 -06:00
Josh Hawkins
6ebad84160
initialize path before calling super() (#14203) 2024-10-07 16:17:57 -05:00
Josh Hawkins
24ac9f3e5a
Use sqlite-vec extension instead of chromadb for embeddings (#14163)
* swap sqlite_vec for chroma in requirements

* load sqlite_vec in embeddings manager

* remove chroma and revamp Embeddings class for sqlite_vec

* manual minilm onnx inference

* remove chroma in clip model

* migrate api from chroma to sqlite_vec

* migrate event cleanup from chroma to sqlite_vec

* migrate embedding maintainer from chroma to sqlite_vec

* genai description for sqlite_vec

* load sqlite_vec in main thread db

* extend the SqliteQueueDatabase class and use peewee db.execute_sql

* search with Event type for similarity

* fix similarity search

* install and add comment about transformers

* fix normalization

* add id filter

* clean up

* clean up

* fully remove chroma and add transformers env var

* readd uvicorn for fastapi

* readd tokenizer parallelism env var

* remove chroma from docs

* remove chroma from UI

* try removing custom pysqlite3 build

* hard code limit

* optimize queries

* revert explore query

* fix query

* keep building pysqlite3

* single pass fetch and process

* remove unnecessary re-embed

* update deps

* move SqliteVecQueueDatabase to db directory

* make search thumbnail take up full size of results box

* improve typing

* improve model downloading and add status screen

* daemon downloading thread

* catch case when semantic search is disabled

* fix typing

* build sqlite_vec from source

* resolve conflict

* file permissions

* try build deps

* remove sources

* sources

* fix thread start

* include git in build

* reorder embeddings after detectors are started

* build with sqlite amalgamation

* non-platform specific

* use wget instead of curl

* remove unzip -d

* remove sqlite_vec from requirements and load the compiled version

* fix build

* avoid race in db connection

* add scale_factor and bias to description zscore normalization
2024-10-07 14:30:45 -06:00
Rui Alves
757150dec1
Use Swagger documentation for Frigate HTTP API (#14178)
* Updated documentation

* docusaurus.config and sidebars converted to Typescript to allow for typings

* Added type for sidebars.ts

* Replaced integrations/api.md with automatically generated openAPI specification. Make sidebar collapsible to increase readability

* Fix HTTP API links in the documentation

* Added rust as language in the openapi sidebar

* Make sure configuration/pwa is present

* Fix API slug

* Fix links

* Revert sidebarCollapsible configuration

* Make HTTP API sidebar collapsed by default. Added CSS for OpenAPI methods

* Proper localhost server path

* Proper localhost server path

* No introduction page

* Lint
2024-10-07 14:27:35 -06:00
Josh Hawkins
ddcec82b61
Explicitly set video tag dimensions to fit inside dialog (#14184) 2024-10-07 07:19:22 -06:00
Nicolas Mowen
74047453ef
Bug Fixes (#14193)
* fix getting device from empty list

* Overwrite all fields

* Fix header too low
2024-10-07 07:18:09 -06:00
Rémi Bédard-Couture
dcaed0e90f
Log correct tensorRT version when debugging (#14182) 2024-10-06 15:33:03 -06:00
Josh Hawkins
cae304e07f
Revamp object snapshot tab (#14180)
* Revamp object snapshot tab

* Make snapshots and thumbnails left justified in the review pane
2024-10-06 11:43:36 -06:00
Blake Blackshear
47ad0ca993 Merge remote-tracking branch 'origin/master' into dev 2024-10-05 10:48:14 -05:00
Nicolas Mowen
9c751230a1
Include lsof in container (#14145) 2024-10-03 17:23:37 -05:00
gtsiam
a468ed316d
Added stop_event to util.Process (#14142)
* Added stop_event to util.Process

util.Process will take care of receiving signals when the stop_event is
accessed in the subclass. If it never is, SystemExit is raised instead.

This has the effect of still behaving like multiprocessing.Process when
stop_event is not accessed, while still allowing subclasses to not deal
with the hassle of setting it up.

* Give each util.Process their own logger

This will help to reduce boilerplate in subclasses.

* Give explicit types to util.Process.__init__

This gives better type hinting in the editor.

* Use util.Process facilities in AudioProcessor

Boilerplate begone!

* Removed pointless check in util.Process

The log_listener.queue should never be None, unless something has gone
extremely wrong in the log setup code. If we're that far gone, crashing
is better.

* Make sure faulthandler is enabled in all processes

This has no effect currently since we're using the fork start_method.
However, when we inevidably switch to forkserver (either by choice, or
by upgrading to python 3.14+) not having this makes for some really fun
failure modes :D
2024-10-03 11:03:43 -06:00
Nicolas Mowen
e725730982
Fix shared memory frames being stuck when a camera capture crashed (#14140)
* Fix shared memory frames being stuck when a camera capture crashed

* Update ffmpeg build
2024-10-03 09:31:07 -06:00
Nicolas Mowen
21c12d118b
Correct preview docs (#14136) 2024-10-03 08:40:49 -05:00
Josh Hawkins
b9e74ee9ab
Fix autotracking absolute zoom interpolation logic (#14139) 2024-10-03 08:40:27 -05:00
Nicolas Mowen
0f2cff5078
Fix mjpeg api (#14138) 2024-10-03 07:33:06 -06:00
gtsiam
e5e196bd7f
Fix logging (#14122)
Fixes logging without introducing more junk into FrigateApp.
2024-10-03 06:33:53 -06:00
Nicolas Mowen
077402406b
Make env vars warning more clear (#14128) 2024-10-02 21:52:58 -05:00