Commit Graph

119 Commits

Author SHA1 Message Date
Josh Hawkins
92a0dad2c2
Catch json decode exception (#19323)
* Catch json decode exception when semantic search stats file is corrupted

* clear stats file if corrupted
2025-07-29 13:38:13 -05:00
Josh Hawkins
ca1c98eab8
Fixes (#19125)
* fix embeddings reindex

- always increment processed objects to prevent division by zero
- ensure description still gets processed even if there is no thumbnail

* clean up

* Add newer labels to default attribute map

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-07-17 07:29:50 -06:00
Josh Hawkins
fd96cd5dae
Verify images before adding to batch for embedding (#18885) 2025-06-26 07:22:31 -06:00
Josh Hawkins
5d13925d2b
Fixes (#18275) 2025-05-17 16:11:19 -06:00
Martin Weinelt
4d4d54d030
Fix various typing issues (#18187)
* Fix the `Any` typing hint treewide

There has been confusion between the Any type[1] and the any function[2]
in typing hints.

[1] https://docs.python.org/3/library/typing.html#typing.Any
[2] https://docs.python.org/3/library/functions.html#any

* Fix typing for various frame_shape members

Frame shapes are most likely defined by height and width, so a single int
cannot express that.

* Wrap gpu stats functions in Optional[]

These can return `None`, so they need to be `Type | None`, which is what
`Optional` expresses very nicely.

* Fix return type in get_latest_segment_datetime

Returns a datetime object, not an integer.

* Make the return type of FrameManager.write optional

This is necessary since the SharedMemoryFrameManager.write function can
return None.

* Fix total_seconds() return type in get_tz_modifiers

The function returns a float, not an int.

https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds

* Account for floating point results in to_relative_box

Because the function uses division the return types may either be int or
float.

* Resolve ruff deprecation warning

The config has been split into formatter and linter, and the global
options are deprecated.
2025-05-13 08:27:20 -06:00
Josh Hawkins
f39ddbc00d
Fixes (#18139)
* Catch error and show toast when failing to delete review items

* i18n keys

* add link to speed estimation docs in zone edit pane

* Implement reset of tracked object update for each camera

* Cleanup

* register mqtt callbacks for toggling alerts and detections

* clarify snapshots docs

* clarify semantic search reindexing

* add ukrainian

* adjust date granularity for last recording time

The api endpoint only returns granularity down to the day

* Add amd hardware

* fix crash in face library on initial start after enabling

* Fix recordings view for mobile landscape

The events view incorrectly was displaying two columns on landscape view and it only took up 20% of the screen width. Additionally, in landscape view the timeline was too wide (especially on iPads of various screen sizes) and would overlap the main video

* face rec overfitting instructions

* Clarify

* face docs

* clarify

* clarify

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-05-11 12:03:53 -06:00
Josh Hawkins
8094dd4075
Fixes (#18117)
* face library i18n fixes

* face library i18n fixes

* add ability to use ctrl/cmd S to save in the config editor

* Use datetime as ID

* Update metrics inference speed to start with 0 ms

* fix android formatted thumbnail

* ensure role is comma separated and stripped correctly

* improve face library deletion

- add a confirmation dialog
- add ability to select all / delete faces in collections

* Implement lazy loading for video previews

* Force GPU for large embedding model

* GPU is required

* settings i18n fixes

* Don't delete train tab

* webpush debugging logs

* Fix incorrectly copying zones

* copy path data

* Ensure that cache dir exists for Frigate+

* face docs update

* Add description to upload image step to clarify the image

* Clean up

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-05-09 07:36:44 -06:00
idxlics
976863518b
Use HF_ENDPOINT env instead of hardcoding https://huggingface.co (#18036)
* Update jina_v1_embedding.py

* Update jina_v2_embedding.py
2025-05-04 19:38:17 -05:00
Josh Hawkins
377b78af11
Change default zmq timeout (#17971)
* update default zmq timeout

* use lower default timeout

* keep longer timeout for event maintainer

* remove default

* remove
2025-04-30 08:29:16 -06:00
Josh Hawkins
c91c6970de
Autotracking improvements (#17955)
* add zoom time to movement predictions

* config migrator

* add space to face rename regex

* more debug

* only calculate zoom time of relative move

* fix test

* make migrated movement weight a zero

* check for str and bool for movestatus support
2025-04-29 10:17:56 -06:00
Nicolas Mowen
e57dde7bb0
Support automatic language selection based on system language (#17953)
* Support automatic language selection

* Handle non-matching keys

* Cleanup

* Handle region specific language codes from browser

* Fix passing in requestor
2025-04-29 10:02:50 -05:00
Nicolas Mowen
4b4053d174
Add face and lpr to tracked object update topic (#17940)
* Send tracked object updates for face and license_plate objects

* Update docs

* Add to type enum

* Add camera to object description update

* Formatting

* Consolidate yue-Hant

* Add missing
2025-04-28 16:43:03 -06:00
Josh Hawkins
d11f46bbce
Add ability to rename faces in the Face Library (#17879)
* api endpoint

* embeddings rename function

* frontend and i18n keys

* lazy load train tab images

* only log exception to make codeql happy
2025-04-23 17:27:46 -06:00
Josh Hawkins
760ed25f0c
LPR improvements (#17716)
* add support for multi-line plates

* config for model size

* default to small model

* add license plate as attribute to motorcycle

* use model size

* docs

* attribute map

* i18n key fix
2025-04-15 09:40:12 -06:00
Josh Hawkins
f618f25d7e
catch not implemented exception (#17594) 2025-04-07 20:41:53 -06:00
Josh Hawkins
921908fe33
LPR device selection (#17592)
* config changes

* use device

* docs

* docs

* reset state

* remove auto

* use device

* docs
2025-04-07 19:30:08 -06:00
Josh Hawkins
348e728220
LPR improvements (#17549)
* auto select LPR model backend for inference

* docs update
2025-04-05 11:03:17 -05:00
Josh Hawkins
2920127ada
Dedicated LPR improvements (#17453)
* remove license plate from attributes for dedicated lpr cameras

* ensure we always have a color

* use frigate+ models with dedicated lpr cameras

* docs

* docs clarity

* docs enrichments

* use license_plate as object type
2025-03-30 07:43:24 -06:00
Josh Hawkins
37c3ac5413
Misc fixes (#17443)
* ensure semantic search is enabled before updating embeddings stats

* add camera and label to snapshot download filename
2025-03-29 05:58:50 -06:00
Nicolas Mowen
c1b06d63b6
Fix embedding eps (#17437)
* Fix embedding eps

* Fix thumbnail cleanup
2025-03-28 18:35:50 -06:00
Nicolas Mowen
9e8b85a957
Implement enchrichments events per second graph (#17436)
* Cleanup existing naming

* Add face recognitions per second

* Add lpr fps

* Add all eps

* Clean up line graph

* Translations

* Change wording

* Fix incorrect access

* Don't require plates

* Add comment

* Fix
2025-03-28 18:13:35 -05:00
Josh Hawkins
23c3323871
Dynamic embeddings reindexing (#17418)
* reindex with api endpoint and zmq

* threading

* frontend

* require admin role
2025-03-27 11:29:34 -06:00
Nicolas Mowen
36446ceded
Implement facenet tflite for small face recognition model (#17402) 2025-03-27 06:31:29 -05:00
Nicolas Mowen
ff34739f40
Face recognition bug fixes (#17401)
* Simplify normalization

* Fix confidence check
2025-03-26 19:43:10 -05:00
Josh Hawkins
4edf0d8cd3
LPR bugfix (#17384)
* ensure image is numpy array

* clean up debugging

* clean up postprocessor

* process raw input as img
2025-03-26 07:41:00 -05:00
Nicolas Mowen
b18d1fb970
Refactor face recognition (#17368)
* Refactor face recognition to allow for running lbph or embedding

* Cleanup

* Use weighted average for faces

* Set correct url

* Cleanup

* Update docs

* Update docs

* Use scipy trimmed mean

* Normalize

* Handle color and gray landmark detection

* Upgrade to new arcface model

* Implement sigmoid function

* Rename

* Rename to arcface

* Fix

* Add face recognition model size to ui config

* Update toast
2025-03-25 19:59:03 -05:00
Nicolas Mowen
a8fa77c6b9
Classification fixes (#17337)
* Cleanup classification settings behavior

* Cleanup response

* restore
2025-03-24 09:12:42 -05:00
Josh Hawkins
fa4643fddf
LPR improvements (#17289)
* config options

* processing in maintainer

* detect and process dedicated lpr plates

* create camera type, add manual event and save snapshot

* use const

* ensure lpr events are always detections, typing fixes

* docs

* docs tweaks

* add preprocessing and penalization for low confidence chars
2025-03-23 13:30:48 -06:00
Nicolas Mowen
e33fa96599
Face recognize api (#17233)
* Add api to run face recognition on image

* Rework save attempts option

* Cleanup mobile object pane buttons

* Adjust api signature

* Remove param

* Cleanup
2025-03-19 09:02:25 -06:00
Nicolas Mowen
db541abed4
Fix model errors (#17171) 2025-03-16 06:01:15 -05:00
Nicolas Mowen
0cc5d66e5b
Refactor sub label api (#17079)
* Use event metadata updater to handle sub label operations

* Use event metadata publisher for sub label setting

* Formatting

* fix tests

* Cleanup
2025-03-10 16:29:29 -06:00
leccelecce
c23653338f
GenAI: allow configuring additional send trigger after_significant_updates as well as event_end (#16919) 2025-03-04 09:23:51 -07:00
Josh Hawkins
76c35307b2
Ensure genai thumbnails are always jpegs (#16939) 2025-03-04 07:34:19 -07:00
leccelecce
f3765bc391
GenAI minor refactor (#16916) 2025-03-03 11:01:02 -06:00
Martin Weinelt
458134de5d
Reuse constants (#16874) 2025-02-28 21:35:09 -07:00
Josh Hawkins
8d2f461350
Embeddings tweaks (#16864)
* make semantic search optional

* config

* frontend metrics

* docs

* tweak

* fixes

* also check genai cameras for embeddings context
2025-02-28 11:43:08 -07:00
Josh Hawkins
d0e9bcbfdc
Add ability to use Jina CLIP V2 for semantic search (#16826)
* add wheels

* move extra index url to bottom

* config model option

* add postprocess

* fix config

* jina v2 embedding class

* use jina v2 in embeddings

* fix ov inference

* frontend

* update reference config

* revert device

* fix truncation

* return np tensors

* use correct embeddings from inference

* manual preprocess

* clean up

* docs

* lower batch size for v2 only

* docs clarity

* wording
2025-02-26 07:58:25 -07:00
Josh Hawkins
60b34bcfca
Refactor processors and add LPR postprocessing (#16722)
* recordings data pub/sub

* function to process recording stream frames

* model runner

* lpr model runner

* refactor to mixin class and use model runner

* separate out realtime and post processors

* move model and mixin folders

* basic postprocessor

* clean up

* docs

* postprocessing logic

* clean up

* return none if recordings are disabled

* run postprocessor handle_requests too

* tweak expansion

* add put endpoint

* postprocessor tweaks with endpoint
2025-02-21 06:51:37 -07:00
Nicolas Mowen
c736b1dae5
Refactor ONNX embedding class to use a base class and type-specific classes (#16703)
* Move onnx runner

* Build out base embedding

* Convert text embedding to separate class

* Move image embedding to separate

* Move LPR to separate class

* Remove mono embedding

* Simplify model downloading

* Reorganize jina v1 embeddings

* Cleanup

* Cleanup for review
2025-02-20 10:17:07 -06:00
Nicolas Mowen
f49a8009ec
Remove thumb from database field (#16647)
* Remove thumbnail from dict

* Create thumbnail diectory

* Cleanup handling of tracked object images

* Make thumbnail optional

* Handle cases where thumbnail is used

* Expand options for thumbnail api

* Fix up the does not exist condition

* Remove absolute usages of thumbnails

* Write thumbnails for external events

* Reduce webp quality

* Use webp everywhere in frontend

* Formatting

* Always consider all events when re-indexing

* Add thumbnail deletion and cleanup path management

* Cleanup imports

* Rename def

* Don't save thumbnail for every object

* Correct event count

* Use correct function

* Include thumbnail in query

* Remove unused

* Fix requiring exception
2025-02-18 07:46:29 -07:00
Josh Hawkins
729f5c0833
LPR improvements (#16559)
* use a small yolov9 model for detection

* use yolov9 for users without frigate+ and update retention algorithm

* new lpr config fields

* levenshtein distance package

* tweaks

* docs
2025-02-13 16:08:56 -07:00
Josh Hawkins
2458f667c4
Refactor lpr into real time data processor (#16497) 2025-02-11 13:45:13 -07:00
Nicolas Mowen
1c3527f5c4 Face recognition reprocess (#16212)
* Implement update topic

* Add API for reprocessing face

* Get reprocess working

* Fix crash when no faces exist

* Simplify
2025-02-08 12:47:01 -06:00
Nicolas Mowen
3f99ff65ed Face recognition improvements (#16034) 2025-02-08 12:47:01 -06:00
Nicolas Mowen
c7c8575c9b Bird classification (#15966)
* Start working on bird processor

* Initial setup for bird processing

* Improvements to handling

* Get classification working

* Cleanup classification

* Add classification config

* Update sort
2025-02-08 12:47:01 -06:00
Nicolas Mowen
9dc85d4a76 Processing refactor (#15935)
* Refactor post processor to be real time processor

* Build out generic API for post processing

* Cleanup

* Fix
2025-02-08 12:47:01 -06:00
Nicolas Mowen
88686c44fe Generalize postprocessing (#15931)
* Actually send result to face registration

* Define postprocessing api and move face processing to fit

* Standardize request handling

* Standardize handling of processors

* Rename processing metrics

* Cleanup

* Standardize object end

* Update to newer formatting

* One more

* One more
2025-02-08 12:47:01 -06:00
Nicolas Mowen
4f85b18b08 Improvements to face recognition (#15854)
* Do not add margin to face images

* remove margin

* Correctly clear
2025-02-08 12:47:01 -06:00
Nicolas Mowen
a6ae208fe7 Add metrics page for embeddings and face / license plate processing times (#15818)
* Get stats for embeddings inferences

* cleanup embeddings inferences

* Enable UI for feature metrics

* Change threshold

* Fix check

* Update python for actions

* Set python version

* Ignore type for now
2025-02-08 12:47:01 -06:00
Nicolas Mowen
0c13227f7d Fix facedet download (#15811)
* Support downloading face models

* Handle download and loading correctly

* Add face dir creation

* Fix error

* Fix

* Formatting

* Move upload to button

* Show number of faces in library for each name

* Add text color for score

* Cleanup
2025-02-08 12:47:01 -06:00