Commit Graph

77 Commits

Author SHA1 Message Date
Nicolas Mowen
1c3527f5c4 Face recognition reprocess (#16212)
* Implement update topic

* Add API for reprocessing face

* Get reprocess working

* Fix crash when no faces exist

* Simplify
2025-02-08 12:47:01 -06:00
Nicolas Mowen
3f99ff65ed Face recognition improvements (#16034) 2025-02-08 12:47:01 -06:00
Nicolas Mowen
c7c8575c9b Bird classification (#15966)
* Start working on bird processor

* Initial setup for bird processing

* Improvements to handling

* Get classification working

* Cleanup classification

* Add classification config

* Update sort
2025-02-08 12:47:01 -06:00
Nicolas Mowen
9dc85d4a76 Processing refactor (#15935)
* Refactor post processor to be real time processor

* Build out generic API for post processing

* Cleanup

* Fix
2025-02-08 12:47:01 -06:00
Nicolas Mowen
88686c44fe Generalize postprocessing (#15931)
* Actually send result to face registration

* Define postprocessing api and move face processing to fit

* Standardize request handling

* Standardize handling of processors

* Rename processing metrics

* Cleanup

* Standardize object end

* Update to newer formatting

* One more

* One more
2025-02-08 12:47:01 -06:00
Nicolas Mowen
4f85b18b08 Improvements to face recognition (#15854)
* Do not add margin to face images

* remove margin

* Correctly clear
2025-02-08 12:47:01 -06:00
Nicolas Mowen
a6ae208fe7 Add metrics page for embeddings and face / license plate processing times (#15818)
* Get stats for embeddings inferences

* cleanup embeddings inferences

* Enable UI for feature metrics

* Change threshold

* Fix check

* Update python for actions

* Set python version

* Ignore type for now
2025-02-08 12:47:01 -06:00
Nicolas Mowen
0c13227f7d Fix facedet download (#15811)
* Support downloading face models

* Handle download and loading correctly

* Add face dir creation

* Fix error

* Fix

* Formatting

* Move upload to button

* Show number of faces in library for each name

* Add text color for score

* Cleanup
2025-02-08 12:47:01 -06:00
Nicolas Mowen
281407247b Implement face recognition training in UI (#15786)
* Rename debug to train

* Add api to train image as person

* Cleanup model running

* Formatting

* Fix

* Set face recognition page title
2025-02-08 12:47:01 -06:00
Nicolas Mowen
172e7d494f Add UI for managing face recognitions (#15757)
* Add ability to view attempts

* Improve UI

* Cleanup

* Correctly refresh ui when item is deleted

* Select correct library by default

* Add min score

* Cleanup
2025-02-08 12:47:01 -06:00
Nicolas Mowen
8763390dfe Face recognition logic improvements (#15679)
* Always initialize face model on startup

* Add ability to save face images for debugging

* Implement better face recognition reasonability
2025-02-08 12:47:01 -06:00
Nicolas Mowen
c26144da75 Change folder 2025-02-08 12:47:01 -06:00
Nicolas Mowen
d025495374 Set model size 2025-02-08 12:47:01 -06:00
Nicolas Mowen
f58fc4c367 Improve face recognition (#15670)
* Face recognition tuning

* Support face alignment

* Cleanup

* Correctly download model
2025-02-08 12:47:01 -06:00
Nicolas Mowen
0e4ff91d6b Improve face recognition (#15205)
* Validate faces using cosine distance and SVC

* Formatting

* Use opencv instead of face embedding

* Update docs for training data

* Adjust to score system

* Set bounds

* remove face embeddings

* Update writing images

* Add face library page

* Add ability to select file

* Install opencv deps

* Cleanup

* Use different deps

* Move deps

* Cleanup

* Only show face library for desktop

* Implement deleting

* Add ability to upload image

* Add support for uploading images
2025-02-08 12:47:01 -06:00
Nicolas Mowen
dd7b1be7f4 Remove standardization 2025-02-08 12:47:01 -06:00
Nicolas Mowen
755c9eea1c Remove hardcoded face name 2025-02-08 12:47:01 -06:00
Nicolas Mowen
e5fcc50ae2 Use SVC to normalize and classify faces for recognition (#14835)
* Add margin to detected faces for embeddings

* Standardize pixel values for face input

* Use SVC to classify faces

* Clear classifier when new face is added

* Formatting

* Add dependency
2025-02-08 12:47:01 -06:00
Josh Hawkins
8bb037f82e Use regular expressions for plate matching (#14727) 2025-02-08 12:47:01 -06:00
Nicolas Mowen
a0c35101fb Update facenet model (#14647) 2025-02-08 12:47:01 -06:00
Josh Hawkins
af1eaac5ff LPR improvements (#14641) 2025-02-08 12:47:01 -06:00
Josh Hawkins
dbbfc735f0 Prevent division by zero in lpr confidence checks (#14615) 2025-02-08 12:47:01 -06:00
Nicolas Mowen
711575736d Fix label check (#14610)
* Create config for parsing object

* Use in maintainer
2025-02-08 12:47:01 -06:00
Josh Hawkins
c4ce7f9800 License plate recognition (ALPR) backend (#14564)
* Update version

* Face recognition backend (#14495)

* Add basic config and face recognition table

* Reconfigure updates processing to handle face

* Crop frame to face box

* Implement face embedding calculation

* Get matching face embeddings

* Add support face recognition based on existing faces

* Use arcface face embeddings instead of generic embeddings model

* Add apis for managing faces

* Implement face uploading API

* Build out more APIs

* Add min area config

* Handle larger images

* Add more debug logs

* fix calculation

* Reduce timeout

* Small tweaks

* Use webp images

* Use facenet model

* Improve face recognition (#14537)

* Increase requirements for face to be set

* Manage faces properly

* Add basic docs

* Simplify

* Separate out face recognition frome semantic search

* Update docs

* Formatting

* Fix access (#14540)

* Face detection (#14544)

* Add support for face detection

* Add support for detecting faces during registration

* Set body size to be larger

* Undo

* Update version

* Face recognition backend (#14495)

* Add basic config and face recognition table

* Reconfigure updates processing to handle face

* Crop frame to face box

* Implement face embedding calculation

* Get matching face embeddings

* Add support face recognition based on existing faces

* Use arcface face embeddings instead of generic embeddings model

* Add apis for managing faces

* Implement face uploading API

* Build out more APIs

* Add min area config

* Handle larger images

* Add more debug logs

* fix calculation

* Reduce timeout

* Small tweaks

* Use webp images

* Use facenet model

* Improve face recognition (#14537)

* Increase requirements for face to be set

* Manage faces properly

* Add basic docs

* Simplify

* Separate out face recognition frome semantic search

* Update docs

* Formatting

* Fix access (#14540)

* Face detection (#14544)

* Add support for face detection

* Add support for detecting faces during registration

* Set body size to be larger

* Undo

* initial foundation for alpr with paddleocr

* initial foundation for alpr with paddleocr

* initial foundation for alpr with paddleocr

* config

* config

* lpr maintainer

* clean up

* clean up

* fix processing

* don't process for stationary cars

* fix order

* fixes

* check for known plates

* improved length and character by character confidence

* model fixes and small tweaks

* docs

* placeholder for non frigate+ model lp detection

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-02-08 12:47:01 -06:00
Nicolas Mowen
594a4e0ba3 Face detection (#14544)
* Add support for face detection

* Add support for detecting faces during registration

* Set body size to be larger

* Undo
2025-02-08 12:47:01 -06:00
Nicolas Mowen
c1d5510428 Fix access (#14540) 2025-02-08 12:47:01 -06:00
Nicolas Mowen
a3d6266d96 Improve face recognition (#14537)
* Increase requirements for face to be set

* Manage faces properly

* Add basic docs

* Simplify

* Separate out face recognition frome semantic search

* Update docs

* Formatting
2025-02-08 12:47:01 -06:00
Nicolas Mowen
aa19ec3ddb Face recognition backend (#14495)
* Add basic config and face recognition table

* Reconfigure updates processing to handle face

* Crop frame to face box

* Implement face embedding calculation

* Get matching face embeddings

* Add support face recognition based on existing faces

* Use arcface face embeddings instead of generic embeddings model

* Add apis for managing faces

* Implement face uploading API

* Build out more APIs

* Add min area config

* Handle larger images

* Add more debug logs

* fix calculation

* Reduce timeout

* Small tweaks

* Use webp images

* Use facenet model
2025-02-08 12:47:01 -06:00
leccelecce
00371546a3
GenAI: add ability to save JPGs sent to provider (#15643)
* GenAI: add ability to save JPGs sent to provider

* Remove mention from GenAI docs

* Change config name to debug_save_thumbnails

* Change  folder structure to clips/genai-requests/{event_id}/{1.jpg}
2024-12-23 07:05:34 -07:00
Giorgio Ughini
d9ef8fa206
Fix always the same image is sent to GenAI (#15550)
* Fix always the same image is sent to GenAI

* Fix typo for bug where identical images are sent to GenAI

* Correct formatting
2024-12-17 07:44:00 -06:00
Josh Hawkins
717493e668
Improve handling of error conditions with ollama and snapshot regeneration (#15527) 2024-12-15 20:51:23 -06:00
Josh Hawkins
d49f958d4d
Don't crop by region for genai snapshot for manual events (#15525) 2024-12-15 17:03:19 -06:00
Nicolas Mowen
66277fbb6c
Fix embeddings (#15072)
* Fix embeddings reading frames

* Fix event update reading

* Formatting

* Pin AIO http to fix build failure

* Pin starlette
2024-11-19 12:20:04 -06:00
Josh Hawkins
9ae839ad72
Tracked object metadata changes (#15055)
* add enum and change topic name

* frontend renaming

* docs

* only display sublabel score if it it exists

* remove debug print
2024-11-18 11:26:44 -07:00
Josh Hawkins
143bab87f1
Genai bugfix (#14880)
* Fix genai init when disabled at global level

* use genai config for class init
2024-11-09 06:48:53 -07:00
Nicolas Mowen
40c6fda19d
Various fixes and improvements (#14492)
* Refactor preprocessing of images

* Cleanup preprocessing

* Improve naming and handling of embeddings

* Handle invalid intel json

* remove unused

* Use enum for model types

* Formatting
2024-10-21 16:19:34 -06:00
Josh Hawkins
6c70e56059
Misc bugfixes and improvements (#14460)
* only save a fixed number of thumbnails if genai is enabled

* disable cpu_mem_arena to save on memory until its actually needed

* fix search settings pane so it actually saves to the config
2024-10-20 14:14:51 -06:00
Nicolas Mowen
3f1ab66899
Embeddings UI updates (#14378)
* Handle Frigate+ submitted case

* Add search settings and rename general to ui settings

* Add platform aware sheet component

* use two columns on mobile view

* Add cameras page to more filters

* clean up search settings view

* Add time range to side filter

* better match with ui settings

* fix icon size

* use two columns on mobile view

* clean up search settings view

* Add zones and saving logic

* Add all filters to side panel

* better match with ui settings

* fix icon size

* Fix mobile fitler page

* Fix embeddings access

* Cleanup

* Fix scroll

* fix double scrollbars and add separators on mobile too

* two columns on mobile

* italics for emphasis

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2024-10-15 19:25:59 -05:00
Nicolas Mowen
b75efcbca2
UI tweaks (#14369)
* Adjust text size

* Make cursor consistent

* Fix lint
2024-10-15 09:37:04 -06:00
Nicolas Mowen
25043278ab
Always run embedding descs one by one (#14365) 2024-10-15 07:40:45 -06:00
Nicolas Mowen
0eccb6a610
Db fixes (#14364)
* Handle case where embeddings overflow token limit

* Set notification tokens

* Fix sort
2024-10-15 07:17:54 -06:00
Josh Hawkins
1ec459ea3a
Batch embeddings fixes (#14325)
* fixes

* more readable loops

* more robust key check and warning message

* ensure we get reindex progress on mount

* use correct var for length
2024-10-13 15:25:13 -06:00
Nicolas Mowen
e8b2fde753
Support batch embeddings when reindexing (#14320)
* Refactor onnx embeddings to handle multiple inputs by default

* Process items in batches when reindexing
2024-10-13 12:33:27 -05:00
Josh Hawkins
0fc7999780
Improve reindex completion flag (#14308) 2024-10-12 14:44:01 -05:00
Nicolas Mowen
3a403392e7
Fixes for model downloading (#14305)
* Use different requestor for downloaders

* Handle case where lock is left over from failed partial download

* close requestor

* Formatting
2024-10-12 13:36:10 -05:00
Josh Hawkins
48c60621b6
Fix substitution on genai prompts (#14298) 2024-10-12 06:19:24 -06:00
Nicolas Mowen
6e332bbdf8
Remove device config and use model size to configure device used (#14290)
* Remove device config and use model size to configure device used

* Don't show Frigate+ submission when in progress

* Add docs link for bounding box colors
2024-10-11 17:08:14 -05:00
Josh Hawkins
8a8a0c7dec
Embeddings normalization fixes (#14284)
* Use cosine distance metric for vec tables

* Only apply normalization to multi modal searches

* Catch possible edge case in stddev calc

* Use sigmoid function for normalization for multi modal searches only

* Ensure we get model state on initial page load

* Only save stats for multi modal searches and only use cosine similarity for image -> image search
2024-10-11 13:11:11 -05:00
Nicolas Mowen
6df541e1fd
Openvino models (#14283)
* Enable model conversion cache for openvino

* Use openvino directly for onnx embeddings if available

* Don't fail if zmq is busy
2024-10-11 10:47:23 -06:00
Josh Hawkins
ae91fa6a39
Add time remaining to embedding reindex pane (#14279)
* Add function to convert seconds to human readable duration

* Add estimated time remaining to reindexing pane
2024-10-11 07:04:25 -06:00