Commit Graph

59 Commits

Author SHA1 Message Date
Nicolas Mowen
87efebde19 Cleanup event metadata updater 2025-07-18 12:28:29 -06:00
Nicolas Mowen
b88fa9ece6
Object attribute classification (#19205)
* Add enum for type of classification for objects

* Update recognized license plate topic to be used as attribute updater

* Update attribute for attribute type object classification

* Cleanup
2025-07-18 09:28:02 -05:00
Nicolas Mowen
7ea288fe32
Classification train updates (#19173)
* Improve model train button

* Add filters for classification

* Cleanup

* Don't run classification on false positives

* Cleanup filter

* Fix icon color
2025-07-16 21:46:59 -05:00
Nicolas Mowen
0f4cac736a
Improve classification UI (#18910)
* Move threhsold to base model config

* Improve score handling

* Add back button
2025-06-27 09:35:02 -05:00
Nicolas Mowen
bd6dee5b38 Remove TFLite init logs 2025-06-27 06:54:02 -06:00
Nicolas Mowen
53315342c0
Improve object classification (#18908)
* Ui improvements

* Improve image cropping and model saving

* Improve naming

* Add logs for training

* Improve model labeling

* Don't set sub label for none object classification

* Cleanup
2025-06-27 06:28:40 -06:00
Nicolas Mowen
4b18d54d3d
0.17 tweaks (#18892)
* Set version

* Cleanup more logs

* Don't log matplotlib
2025-06-26 07:32:48 -06:00
Nicolas Mowen
28fba7122d
Refactor TensorRT (#18643)
* Combine base and arm trt detectors

* Remove unused deps for amd64 build

* Add missing packages and cleanup ldconfig

* Expand packages for tensorflow model training

* Cleanup

* Refactor training to not reserve memory
2025-06-09 08:25:33 -06:00
Nicolas Mowen
8409100623
Classification Model Metrics (#18595)
* Add speed and rate metrics for custom classification models

* Use metrics for classification models

* Use keys

* Cast to list
2025-06-06 11:29:44 -05:00
Nicolas Mowen
be8ee068e2
Live classification model training (#18583)
* Implement model training via ZMQ and add model states to represent training

* Get model updates working

* Improve toasts and model state

* Clean up logging

* Add back in
2025-06-05 09:13:12 -06:00
Nicolas Mowen
85d721eb6b
Classification Model UI (#18571)
* Setup basic training structure

* Build out route

* Handle model configs

* Add image fetch APIs

* Implement model training screen with dataset selection

* Implement viewing of training images

* Adjust directories

* Implement viewing of images

* Add support for deleting images

* Implement full deletion

* Implement classification model training

* Improve naming

* More renaming

* Improve layout

* Reduce logging

* Cleanup
2025-06-04 18:09:55 -05:00
Josh Hawkins
eb1fe9fe20
Audio transcription tweaks (#18540)
* use model runner

* unload whisper model when live transcription is complete
2025-06-03 06:53:48 -05:00
Nicolas Mowen
20e0addae1
Implement API to train classification models (#18475) 2025-05-29 17:51:32 -06:00
Josh Hawkins
2bd6fa53fe
Audio transcription support (#18398)
* install new packages for transcription support

* add config options

* audio maintainer modifications to support transcription

* pass main config to audio process

* embeddings support

* api and transcription post processor

* embeddings maintainer support for post processor

* live audio transcription with sherpa and faster-whisper

* update dispatcher with live transcription topic

* frontend websocket

* frontend live transcription

* frontend changes for speech events

* i18n changes

* docs

* mqtt docs

* fix linter

* use float16 and small model on gpu for real-time

* fix return value and use requestor to embed description instead of passing embeddings

* run real-time transcription in its own thread

* tweaks

* publish live transcriptions on their own topic instead of tracked_object_update

* config validator and docs

* clarify docs
2025-05-27 10:26:00 -05:00
Nicolas Mowen
87d0102624
Add ability to configure when custom classification models run (#18380)
* Add config to control when classification models are run

* Cleanup
2025-05-24 11:18:46 -05:00
Nicolas Mowen
8a1da3a89f
Initial custom classification model config support (#18362)
* Add basic config for defining a teachable machine model

* Add model type

* Add basic config for teachable machine models

* Adjust config for state and object

* Use config to process

* Correctly check for objects

* Remove debug

* Rename to not be teachable machine specific

* Cleanup
2025-05-23 09:46:53 -05:00
Nicolas Mowen
b18572a3b6
Min face configuration option (#18284)
* Add option to specify minimum face attempts for an object

* Set upper bound
2025-05-18 07:20:59 -05:00
Nicolas Mowen
f48356cbee
Fixes (#18245)
* Only check if an object is stationary to avoid mqtt snapshot

* docs heading tweak

* Add more API descriptions

* Add missing lib for new rocm onnxruntime whl

* Update inference times to reflect better rocm performance

* Cleanup resetting tracked object activity

* remove print

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-05-15 17:13:18 -05:00
Martin Weinelt
4d4d54d030
Fix various typing issues (#18187)
* Fix the `Any` typing hint treewide

There has been confusion between the Any type[1] and the any function[2]
in typing hints.

[1] https://docs.python.org/3/library/typing.html#typing.Any
[2] https://docs.python.org/3/library/functions.html#any

* Fix typing for various frame_shape members

Frame shapes are most likely defined by height and width, so a single int
cannot express that.

* Wrap gpu stats functions in Optional[]

These can return `None`, so they need to be `Type | None`, which is what
`Optional` expresses very nicely.

* Fix return type in get_latest_segment_datetime

Returns a datetime object, not an integer.

* Make the return type of FrameManager.write optional

This is necessary since the SharedMemoryFrameManager.write function can
return None.

* Fix total_seconds() return type in get_tz_modifiers

The function returns a float, not an int.

https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds

* Account for floating point results in to_relative_box

Because the function uses division the return types may either be int or
float.

* Resolve ruff deprecation warning

The config has been split into formatter and linter, and the global
options are deprecated.
2025-05-13 08:27:20 -06:00
Josh Hawkins
2c9bfaa49c
Fixes (#18176)
* Add camera name tooltip to previews in recording view

* Apply face area check to cv2 face detection

* Delete review thumbnails

* Don't import hailo until it is used

* Add comment

* Clean up camera name

* Filter out empty keys when updating yaml config

HA ingress seems to randomly add an equal sign to the PUT urls for updating the config from the UI. This fix prevents empty keys from being processed, but still allows empty values.

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-05-13 08:27:07 -06:00
Josh Hawkins
f39ddbc00d
Fixes (#18139)
* Catch error and show toast when failing to delete review items

* i18n keys

* add link to speed estimation docs in zone edit pane

* Implement reset of tracked object update for each camera

* Cleanup

* register mqtt callbacks for toggling alerts and detections

* clarify snapshots docs

* clarify semantic search reindexing

* add ukrainian

* adjust date granularity for last recording time

The api endpoint only returns granularity down to the day

* Add amd hardware

* fix crash in face library on initial start after enabling

* Fix recordings view for mobile landscape

The events view incorrectly was displaying two columns on landscape view and it only took up 20% of the screen width. Additionally, in landscape view the timeline was too wide (especially on iPads of various screen sizes) and would overlap the main video

* face rec overfitting instructions

* Clarify

* face docs

* clarify

* clarify

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-05-11 12:03:53 -06:00
Josh Hawkins
8094dd4075
Fixes (#18117)
* face library i18n fixes

* face library i18n fixes

* add ability to use ctrl/cmd S to save in the config editor

* Use datetime as ID

* Update metrics inference speed to start with 0 ms

* fix android formatted thumbnail

* ensure role is comma separated and stripped correctly

* improve face library deletion

- add a confirmation dialog
- add ability to select all / delete faces in collections

* Implement lazy loading for video previews

* Force GPU for large embedding model

* GPU is required

* settings i18n fixes

* Don't delete train tab

* webpush debugging logs

* Fix incorrectly copying zones

* copy path data

* Ensure that cache dir exists for Frigate+

* face docs update

* Add description to upload image step to clarify the image

* Clean up

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-05-09 07:36:44 -06:00
Nicolas Mowen
4b4053d174
Add face and lpr to tracked object update topic (#17940)
* Send tracked object updates for face and license_plate objects

* Update docs

* Add to type enum

* Add camera to object description update

* Formatting

* Consolidate yue-Hant

* Add missing
2025-04-28 16:43:03 -06:00
Nicolas Mowen
e94b24ec39
Face fixes (#17530)
* Catch - in name

* Add note about docs
2025-04-04 07:03:08 -06:00
Nicolas Mowen
207d1d2806
Face UI cleanup (#17472)
* Add note

* Sort by event id

* Fix reprocess causing shift

* Move event group to separate comp

* Handle selecting events

* implement event selection

* Implement selected handler

* handle right click

* Toggle ctrl + a

* Stop propogation

* Fix
2025-03-31 16:49:56 -05:00
Nicolas Mowen
9e8b85a957
Implement enchrichments events per second graph (#17436)
* Cleanup existing naming

* Add face recognitions per second

* Add lpr fps

* Add all eps

* Clean up line graph

* Translations

* Change wording

* Fix incorrect access

* Don't require plates

* Add comment

* Fix
2025-03-28 18:13:35 -05:00
Nicolas Mowen
b14abffea3
Refactor face library page (#17424)
* Section faces by event id

* Make score keeping more robust

* layout improvements

* Cleanup dialog

* Fix clicking behavior

* Add view in explore option

* math.round

* Don't require events

* Cleanup

* Remove selection

* Don't require

* Change dialog size with snapshot

* Use filename as key

* fix key

* Rework layout for mobile

* Handle mobile landscape

* Fix train issue

* Match logic

* Move deletion logic

* Fix reprocessing

* Support creating a new face

* Translations

* Do sorting in frontend

* Adjust unknown

* Cleanup

* Set max limit to faces to recognize

* Fix sorting

* Fix
2025-03-28 13:52:12 -05:00
Nicolas Mowen
a35146ab61
Various fixes (#17411)
* Remove initial requirement for history

* Clenaup conf

* Handle symlinks
2025-03-27 08:28:09 -06:00
Nicolas Mowen
36446ceded
Implement facenet tflite for small face recognition model (#17402) 2025-03-27 06:31:29 -05:00
Nicolas Mowen
e3d4b84803
Face recognition improvements (#17387)
* Increase frequency of updates when internal face detection is used

* Adjust number of required faces based on detection type

* Adjust min_score config to unknown_score

* Only for person

* Improve typing

* Update face rec docs

* Cleanup ui colors

* Cleanup
2025-03-26 07:23:01 -06:00
Nicolas Mowen
b18d1fb970
Refactor face recognition (#17368)
* Refactor face recognition to allow for running lbph or embedding

* Cleanup

* Use weighted average for faces

* Set correct url

* Cleanup

* Update docs

* Update docs

* Use scipy trimmed mean

* Normalize

* Handle color and gray landmark detection

* Upgrade to new arcface model

* Implement sigmoid function

* Rename

* Rename to arcface

* Fix

* Add face recognition model size to ui config

* Update toast
2025-03-25 19:59:03 -05:00
Josh Hawkins
fa4643fddf
LPR improvements (#17289)
* config options

* processing in maintainer

* detect and process dedicated lpr plates

* create camera type, add manual event and save snapshot

* use const

* ensure lpr events are always detections, typing fixes

* docs

* docs tweaks

* add preprocessing and penalization for low confidence chars
2025-03-23 13:30:48 -06:00
Nicolas Mowen
17e14cefd9
Various fixes & tweaks (#17308)
* Catch case where returned face box is invalid

* Update detector docs

* Add note for customizing rfdetr resolution
2025-03-22 13:58:27 -05:00
Nicolas Mowen
08cf0def6e
Face tweaks (#17290)
* Ensure doesn't fail due to missing dir

* Remove redundant settings from tabs

* Adjust selection method for mobile

* Fix button descendent error

* Ensure train is option on mobile

* Cleanup face images

* Cleanup
2025-03-21 12:47:32 -05:00
Nicolas Mowen
ce43b7b3d9
Add per-camera face and lpr configs (#17235)
* Add per-camera face and lpr configs

* Formatting

* Cleanup
2025-03-19 09:02:40 -06:00
Nicolas Mowen
e33fa96599
Face recognize api (#17233)
* Add api to run face recognition on image

* Rework save attempts option

* Cleanup mobile object pane buttons

* Adjust api signature

* Remove param

* Cleanup
2025-03-19 09:02:25 -06:00
leccelecce
dda7be99eb
Face rec: only consider webp files in /faces and handle_request (#17244)
* Face rec: only consider webp files

* Allow png/jpg/jpeg as well as webp
2025-03-19 06:44:07 -06:00
Nicolas Mowen
bf22d89f67
Improve Face Library Management (#17213)
* Set maximum number of face images to be kept

* Fix vertical camera scaling

* adjust wording

* Add attributes to search data

* Add button to train face from event

* Handle event id saving in API
2025-03-17 16:57:46 -05:00
Nicolas Mowen
ff8e145b90
Face setup wizard (#17203)
* Fix login page

* Increase face image size and add time ago

* Add component for indicating steps in a wizard

* Split out form inputs from dialog

* Add wizard for adding new face to library

* Simplify dialog

* Translations

* Fix scaling bug

* Fix key missing

* Improve multi select

* Adjust wording and spacing

* Add tip for face training

* Fix padding

* Remove text for buttons on mobile
2025-03-17 13:50:13 -06:00
Nicolas Mowen
95e141ed15
Improve face detection (#17202) 2025-03-17 08:05:53 -06:00
Nicolas Mowen
db541abed4
Fix model errors (#17171) 2025-03-16 06:01:15 -05:00
Nicolas Mowen
fbd22c8124
Cleanup bird classification (#17146)
* Cleanup bird classification

* Cleanup
2025-03-14 07:21:50 -06:00
Nicolas Mowen
8d05e7c5b9
Make detection threhsold configurable (#17136) 2025-03-13 12:22:14 -05:00
Nicolas Mowen
300f85720c
Face blur factor (#17099)
* Add option to apply factor to face blurring

* Adjust blur factors

* Add debug log
2025-03-11 14:18:43 -05:00
Nicolas Mowen
0cc5d66e5b
Refactor sub label api (#17079)
* Use event metadata updater to handle sub label operations

* Use event metadata publisher for sub label setting

* Formatting

* fix tests

* Cleanup
2025-03-10 16:29:29 -06:00
Martin Weinelt
458134de5d
Reuse constants (#16874) 2025-02-28 21:35:09 -07:00
Josh Hawkins
447f26e1b9
Fix lpr metrics and add yolov9 plate detection metric (#16827) 2025-02-26 07:29:34 -07:00
Josh Hawkins
b3c1b21f80
Don't require model_runner for realtime processors (#16728) 2025-02-21 13:26:03 -07:00
Josh Hawkins
60b34bcfca
Refactor processors and add LPR postprocessing (#16722)
* recordings data pub/sub

* function to process recording stream frames

* model runner

* lpr model runner

* refactor to mixin class and use model runner

* separate out realtime and post processors

* move model and mixin folders

* basic postprocessor

* clean up

* docs

* postprocessing logic

* clean up

* return none if recordings are disabled

* run postprocessor handle_requests too

* tweak expansion

* add put endpoint

* postprocessor tweaks with endpoint
2025-02-21 06:51:37 -07:00
Nicolas Mowen
c736b1dae5
Refactor ONNX embedding class to use a base class and type-specific classes (#16703)
* Move onnx runner

* Build out base embedding

* Convert text embedding to separate class

* Move image embedding to separate

* Move LPR to separate class

* Remove mono embedding

* Simplify model downloading

* Reorganize jina v1 embeddings

* Cleanup

* Cleanup for review
2025-02-20 10:17:07 -06:00