Commit Graph

44 Commits

Author SHA1 Message Date
Josh Hawkins
fa6dda6735
Miscellaneous Fixes (#21208)
* conditionally display actions for admin role only

* only allow admins to save annotation offset

* Fix classification reset filter

* fix explore context menu from blocking pointer events on the body element after dialog close

applying modal=false to the menu (not to the dialog) to fix this in the same way as elsewhere in the codebase

* add select all link to face library, classification, and explore

* Disable iOS image dragging for classification card

* add proxmox ballooning comment

* lpr docs tweaks

* yaml list

* clarify tls_insecure

* Improve security summary format and usefulness

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-12-11 07:23:34 -07:00
Nicolas Mowen
c136e5e8bd
Miscellaneous fixes (#21141)
* Remove source_type from API

* Don't require state classification models to select all classes

* Specifically validate provided end_time for manual events

* Remove yolov9 specification for warning

* Remove warning for coral

* clarify zone name tip

* clarify replace rules in lpr docs

* remove periods

* Add explanation for review report

* adjust HLS gap controller params

defaults to false, should help to recover from hangs and stalling in tracking details videos on chrome

* only redirect to login page once on 401

attempt to fix ios pwa safari redirect storm

* Use contextual information from other cameras to inform report summary

* Formatting and prompt improvements for review summary report

* More improvements to prompt

* Remove examples

* Don't show admin action buttons on export card

* fix redirect race condition

Coordinate 401 redirect logic between ApiProvider and ProtectedRoute using a shared flag to prevent multiple simultaneous redirects that caused UI flashing. Ensure both auth error paths check and set the redirect flag before navigating to login, eliminating race conditions where both mechanisms could trigger at once

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-12-04 12:19:07 -06:00
Nicolas Mowen
de2144f158
Miscellaneous Fixes (#21050)
* Don't add to history when opening search dialog

* Update caniuse

* Revamp the history handling for dialog components

* clarify audio transcription docs

* Use titlecase helper

* Allow running object clasasification on stationary objects

* small spacing tweaks for tablets

* require admin role to delete users

* explicitly prevent deletion of admin user

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-11-26 07:23:51 -06:00
Josh Hawkins
aa8b423b68
Miscellaneous Fixes (#21024)
* fix wording in reference config

* spacing tweaks

* make live view settings drawer scrollable

* clarify audio transcription docs

* change audio transcription icon to activity indicator when transcription is in progress

the backend doesn't implement any kind of queueing for speech event transcription

* tracking details tweaks

- Add attribute box overlay and area
- Add score
- Throttle swr revalidation during video component rerendering

* add mse codecs to console debug on errors

* add camera name
2025-11-24 06:34:56 -07:00
Josh Hawkins
fbf4388b37
Miscellaneous Fixes (#20897)
* don't flatten the search result cache when updating

this would cause an infinite swr fetch if something was mutated and then fetch was called again

* Properly sort keys for recording summary in StorageMetrics

* tracked object description box tweaks

* Remove ability to right click on elements inside of face popup

* Update reprocess message

* don't show object track until video metadata is loaded

* fix blue line height calc for in progress events

* Use timeline tab by default for notifications but add a query arg for customization

* Try and improve notification opening behavior

* Reduce review item buffering behavior

* ensure logging config is passed to camera capture and tracker processes

* ensure on demand recording stops when browser closes

* improve active line progress height with resize observer

* remove icons and duplicate find similar link in explore context menu

* fix for initial broken image when creating trigger from explore

* display friendly names for triggers in toasts

* lpr and triggers docs updates

* remove icons from dropdowns in face and classification

* fix comma dangle linter issue

* re-add incorrectly removed face library button icons

* fix sidebar nav links on < 768px desktop layout

* allow text to wrap on mark as reviewed button

* match exact pixels

* clarify LPR docs

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-11-17 08:12:05 -06:00
Josh Hawkins
c371fc0c87
Miscellaneous Fixes (#20866)
* Don't warn when event ids have expired for trigger sync

* Import faster_whisper conditinally to avoid illegal instruction

* Catch OpenVINO runtime error

* fix race condition in detail stream context

navigating between tracked objects in Explore would sometimes prevent the object track from appearing

* Handle case where classification images are deleted

* Adjust default rounded corners on larger screens

* Improve flow handling for classification state

* Remove images when wizard is cancelled

* Improve deletion handling for classes

* Set constraints on review buffers

* Update to support correct data format

* Set minimum duration for recording based review items

* Use friendly name in review genai prompt

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-11-10 10:03:56 -07:00
Nicolas Mowen
d41ee4ff88
Miscellaneous Fixes (#20848)
* Fix filtering for classification

* Adjust prompt to account for response tokens

* Correctly return response for reprocess

* Use API response to update data instead of trying to re-parse all of the values

* Implement rename class api

* Fix model deletion / rename dialog

* Remove camera spatial context

* Catch error
2025-11-08 13:13:40 -07:00
Josh Hawkins
530b69b877
Miscellaneous fixes (#20833)
* remove frigate+ icon from explore grid footer

* add margin

* pointer cursor on event menu items in detail stream

* don't show submit to plus for non-objects and if plus is disabled

* tweak spacing in annotation settings popover

* Fix deletion of classification images and library

* Ensure after creating a class that things are correct

* Fix dialog getting stuck

* Only show the genai summary popup on mobile when timeline is open

* fix audio transcription embedding

* spacing

* hide x icon on restart sheet to prevent closure issues

* prevent x overflow in detail stream on mobile safari

* ensure name is valid for search effect trigger

* add trigger to detail actions menu

* move find similar to actions menu

* Use a column layout for MobilePageContent in PlatformAwareSheet

 This is so the header is outside the scrolling area and the content can grow/scroll independently. This now matches the way it's done in classification

* Skip azure execution provider

* add optional ref to always scroll to top

the more filters in explore was not scrolled to the top on open due to the use of framer motion

* fix title classes on desktop

---------

Co-authored-by: Nicolas Mowen <nickmowen213@gmail.com>
2025-11-07 06:53:27 -07:00
Nicolas Mowen
35ce275071
Add ability to define Review Summary camera context (#20828)
* Add ability to define GenAI camera context

* Cleanup

* Only show example with list
2025-11-06 07:39:44 -07:00
Nicolas Mowen
b751228476
Various Tweaks (#20800)
* Fix incorrectly picking start time when date was selected

* Implement shared file locking utility

* Cleanup
2025-11-04 17:06:14 -06:00
Nicolas Mowen
685f2c5030
Mark delivery objects with marker so LLM knows what is happening (#20736) 2025-10-31 07:14:34 -05:00
Nicolas Mowen
971521cd8e
Review description updates (#20723)
* Update docs for review descriptions

* Add logging for context tokens used

* Incrase number of images due to lower than expected context usage

* Re-balance the suspicious activity checks

* Adjustments to context sizing

* optimize context usage

* Adjust context usage

* Make title more direct

* Update docs
2025-10-30 08:52:55 -06:00
Nicolas Mowen
29bc213c04
Various Tweaks (#20713)
* Adjust for commutes

* Tweaks

* Don't show no models view in grid

* Add text-md to inputs

* Adjust train title for mobile

* Cleanup prompt more

* Use i18n functions for tooltip

* Fix model complexity causing crash

* Cleanup
2025-10-29 09:40:50 -05:00
Josh Hawkins
576f692dae
Trigger actions (#20709)
* add backend trigger actions

* config

* frontend types

* add actions to form and wizard

* i18n

* docs

* use camera level notification enabled check
2025-10-28 15:13:04 -06:00
Nicolas Mowen
16e17e027d
Review prompt adjustments (#20704)
* Make prompt more fair and reduce time extension

* Adjust naming of unrecognized objects

* Improve object naming behavior

* Add more context image levels
2025-10-28 08:28:36 -05:00
Nicolas Mowen
710a77679b
Improve default genai review prompt structure (#20690) 2025-10-27 11:34:39 -05:00
Nicolas Mowen
5ff7a47ba9
Unify list of objects under dedicated section (#20684)
* Unify list of objects under dedicated section

* Use helper fuction
2025-10-26 16:37:57 -05:00
Nicolas Mowen
094a0a6e05
Add ability to change source of images for review descriptions (#20676)
* Add ability to change source of images for review descriptions

* Undo
2025-10-26 08:40:38 -05:00
Nicolas Mowen
b1a5896b53
Map verified objects to their sub label directly (#20413)
* Map verified objects to their sub label directly

* Simplify access

* Cleanup

* Add protection for mismatched object and index

* Keep track of verified objects separately
2025-10-10 08:07:00 -05:00
Nicolas Mowen
7a8f93e9f5
Review summary popup (#20383)
* Add title to prompt

* Add popup for genai review summary

* Add animation
2025-10-07 19:11:04 -05:00
Josh Hawkins
d818dbb6ba
Triggers tweaks (#20339)
* backend

* frontend

* use correct camera name param

* i18n

* change log message to debug level

* docs tweaks
2025-10-03 06:36:14 -06:00
Nicolas Mowen
2d45ea271e
Refactor object genai to be a post-processor (#20331)
* Refactor object genai to be a post-processor

* Include function correctly
2025-10-02 12:48:11 -06:00
Nicolas Mowen
37999abbe6
Improve review summary performance (#20328)
* Undo vite

* Balance the prompt

* Round duration

* Calculate context size to determine number of images

* Increase number of images
2025-10-02 10:17:25 -05:00
Nicolas Mowen
28e3aa39f0
Customizable GenAI Review prompt (#20296)
* Add customizable prompt

* Update docs
2025-09-30 18:07:16 -05:00
Nicolas Mowen
8b293449f9
Improve review summary (#20216)
* Add debug logging for review summaries report

* Improve debug logging

* Improve review report prompt

* Cleanup

* Add date to report
2025-09-25 21:05:22 -05:00
Nicolas Mowen
ccbaa74a8b Improve natural language of prompt (#19515)
* Make sequence details human-readable so they are used in natural language response

* Cleanup

* Improve prompt and image selection

* Adjust

* Adjust sligtly

* Format time

* Adjust frame selection logic

* Debug save response

* Ignore extra fields

* Adjust docs
2025-08-16 10:20:33 -05:00
Nicolas Mowen
7740b08bd9 Add ability to pass additional args to Ollama (#19484)
* Call out recognized objects more specifically

* Cleanup

* Make keep_alive and options configurable

* Generalize

* Use for other providers
2025-08-16 10:20:33 -05:00
Nicolas Mowen
dace88bfce Genai review summaries (#19473)
* Generate review item summaries with requests

* Adjust logic to only send important items

* Don't mention ladder

* Adjust prompt to be more specific

* Add more relaxed nature for normal activity

* Cleanup summary

* Update ollama client

* Add more directions to analyze the frames in order

* Remove environment from prompt
2025-08-16 10:20:33 -05:00
Nicolas Mowen
cc18d7f786 Add config for users to define additional concerns that GenAI should make note of in review summary (#19463)
* Don't default to openai

* Improve UI

* Allow configuring additional concerns that users may want the AI to note

* Formatting

* Add preferred language config

* Remove unused
2025-08-16 10:20:33 -05:00
Nicolas Mowen
3cf86767f1 Use preview frames for Review Descriptions (#19450)
* Use preview frames for genai

* Cleanup

* Adjust
2025-08-16 10:20:33 -05:00
Nicolas Mowen
92417a1b9c Review genai updates (#19448)
* Include extra level for normal activity

* Add dynamic toggling

* Update docs

* Add different threshold for genai

* Adjust webUI for object and review description feature

* Adjust config

* Send on startup

* Cleanup config setting

* Set config

* Fix config name
2025-08-16 10:20:33 -05:00
Nicolas Mowen
2cf8dd693c Review Item GenAI metadata (#19442)
* Rename existing function

* Keep track of thumbnial updates

* Tinkering with genai prompt

* Adjust input format

* Create model for review description output

* testing prompt changes

* Prompt improvements and image saving

* Add config for review items genai

* Use genai review config

* Actual config usage

* Adjust debug image saving

* Fix

* Fix review creation

* Adjust prompt

* Prompt adjustment

* Run genai in thread

* Fix detections block

* Adjust prompt

* Prompt changes

* Save genai response to metadata model

* Handle metadata

* Send review update to dispatcher

* Save review metadata to DB

* Send review notification updates

* Quick fix

* Fix name

* Fix update type

* Correctly dump model

* Add card

* Add card

* Remove message

* Cleanup typing and UI

* Adjust prompt

* Formatting

* Add log

* Formatting

* Add inference speed and keep alive
2025-08-16 10:20:33 -05:00
Nicolas Mowen
e9e3c481b2 Implement start for review item description processor (#19352)
* Add review item data transmission

* Publish review updates

* Add review item subscriber

* Basic implementation for testing review processor

* Formatting
2025-08-16 10:20:33 -05:00
Josh Hawkins
3609b41217 Semantic Search Triggers (#18969)
* semantic trigger test

* database and model

* config

* embeddings maintainer and trigger post-processor

* api to create, edit, delete triggers

* frontend and i18n keys

* use thumbnail and description for trigger types

* image picker tweaks

* initial sync

* thumbnail file management

* clean up logs and use saved thumbnail on frontend

* publish mqtt messages

* webpush changes to enable trigger notifications

* add enabled switch

* add triggers from explore

* renaming and deletion fixes

* fix typing

* UI updates and add last triggering event time and link

* log exception instead of return in endpoint

* highlight entry in UI when triggered

* save and delete thumbnails directly

* remove alert action for now and add descriptions

* tweaks

* clean up

* fix types

* docs

* docs tweaks

* docs

* reuse enum
2025-08-16 10:20:33 -05:00
Josh Hawkins
6dc36fcbb4 Audio transcription support (#18398)
* install new packages for transcription support

* add config options

* audio maintainer modifications to support transcription

* pass main config to audio process

* embeddings support

* api and transcription post processor

* embeddings maintainer support for post processor

* live audio transcription with sherpa and faster-whisper

* update dispatcher with live transcription topic

* frontend websocket

* frontend live transcription

* frontend changes for speech events

* i18n changes

* docs

* mqtt docs

* fix linter

* use float16 and small model on gpu for real-time

* fix return value and use requestor to embed description instead of passing embeddings

* run real-time transcription in its own thread

* tweaks

* publish live transcriptions on their own topic instead of tracked_object_update

* config validator and docs

* clarify docs
2025-08-16 10:20:33 -05:00
Martin Weinelt
4d4d54d030
Fix various typing issues (#18187)
* Fix the `Any` typing hint treewide

There has been confusion between the Any type[1] and the any function[2]
in typing hints.

[1] https://docs.python.org/3/library/typing.html#typing.Any
[2] https://docs.python.org/3/library/functions.html#any

* Fix typing for various frame_shape members

Frame shapes are most likely defined by height and width, so a single int
cannot express that.

* Wrap gpu stats functions in Optional[]

These can return `None`, so they need to be `Type | None`, which is what
`Optional` expresses very nicely.

* Fix return type in get_latest_segment_datetime

Returns a datetime object, not an integer.

* Make the return type of FrameManager.write optional

This is necessary since the SharedMemoryFrameManager.write function can
return None.

* Fix total_seconds() return type in get_tz_modifiers

The function returns a float, not an int.

https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds

* Account for floating point results in to_relative_box

Because the function uses division the return types may either be int or
float.

* Resolve ruff deprecation warning

The config has been split into formatter and linter, and the global
options are deprecated.
2025-05-13 08:27:20 -06:00
Josh Hawkins
1e079027ec
Assign requestor correctly (#17944) 2025-04-28 21:52:37 -05:00
Nicolas Mowen
4b4053d174
Add face and lpr to tracked object update topic (#17940)
* Send tracked object updates for face and license_plate objects

* Update docs

* Add to type enum

* Add camera to object description update

* Formatting

* Consolidate yue-Hant

* Add missing
2025-04-28 16:43:03 -06:00
Josh Hawkins
14a32a6472
LPR tweaks (#17783)
* clarify docs

* improve debugging messages

* don't run any lpr postprocessing

* wording
2025-04-18 07:45:37 -06:00
Josh Hawkins
4edf0d8cd3
LPR bugfix (#17384)
* ensure image is numpy array

* clean up debugging

* clean up postprocessor

* process raw input as img
2025-03-26 07:41:00 -05:00
Nicolas Mowen
0cc5d66e5b
Refactor sub label api (#17079)
* Use event metadata updater to handle sub label operations

* Use event metadata publisher for sub label setting

* Formatting

* fix tests

* Cleanup
2025-03-10 16:29:29 -06:00
Josh Hawkins
447f26e1b9
Fix lpr metrics and add yolov9 plate detection metric (#16827) 2025-02-26 07:29:34 -07:00
Josh Hawkins
60b34bcfca
Refactor processors and add LPR postprocessing (#16722)
* recordings data pub/sub

* function to process recording stream frames

* model runner

* lpr model runner

* refactor to mixin class and use model runner

* separate out realtime and post processors

* move model and mixin folders

* basic postprocessor

* clean up

* docs

* postprocessing logic

* clean up

* return none if recordings are disabled

* run postprocessor handle_requests too

* tweak expansion

* add put endpoint

* postprocessor tweaks with endpoint
2025-02-21 06:51:37 -07:00
Nicolas Mowen
9dc85d4a76 Processing refactor (#15935)
* Refactor post processor to be real time processor

* Build out generic API for post processing

* Cleanup

* Fix
2025-02-08 12:47:01 -06:00