Commit Graph

238 Commits

Author SHA1 Message Date
Nicolas Mowen
0a02c665fc
Improve the tablet layout (#19320)
* Improve the tablet layout

* Update imports sort

* Fix more imports
2025-07-29 09:11:52 -06:00
Nicolas Mowen
21d3476bd9
Require setting process priority for FrigateProcess (#19207) 2025-07-18 12:23:06 -05:00
Nicolas Mowen
47a0097e95
Handle SIGINT with forkserver (#18860)
* Pass stopevent from main start

* Share stop event across processes

* preload modules

* remove explicit os._exit call

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-06-24 11:41:11 -06:00
Nicolas Mowen
a22e24f24b Fix process name 2025-06-23 15:55:57 -06:00
Nicolas Mowen
c3ba834a1c
Fix go2rtc init (#18708)
* Cleanup process handling

* Adjust process name
2025-06-13 12:09:51 -05:00
Nicolas Mowen
8485023442
Use Fork-Server As Spawn Method (#18682)
* Set runtime

* Use count correctly

* Don't assume camera sizes

* Use separate zmq proxy for object detection

* Correct order

* Use forkserver

* Only store PID instead of entire process reference

* Cleanup

* Catch correct errors

* Fix typing

* Remove before_run from process util

The before_run never actually ran because:

You're right to suspect an issue with before_run not being called and a potential deadlock. The way you've implemented the run_wrapper using __getattribute__ for the run method of BaseProcess is a common pitfall in Python's multiprocessing, especially when combined with how multiprocessing.Process works internally.

Here's a breakdown of why before_run isn't being called and why you might be experiencing a deadlock:

The Problem: __getattribute__ and Process Serialization
When you create a multiprocessing.Process object and call start(), the multiprocessing module needs to serialize the process object (or at least enough of it to re-create the process in the new interpreter). It then pickles this serialized object and sends it to the newly spawned process.

The issue with your __getattribute__ implementation for run is that:

run is retrieved during serialization: When multiprocessing tries to pickle your Process object to send to the new process, it will likely access the run attribute. This triggers your __getattribute__ wrapper, which then tries to bind run_wrapper to self.
run_wrapper is bound to the parent process's self: The run_wrapper closure, when created in the parent process, captures the self (the Process instance) from the parent's memory space.
Deserialization creates a new object: In the child process, a new Process object is created by deserializing the pickled data. However, the run_wrapper method that was pickled still holds a reference to the self from the parent process. This is a subtle but critical distinction.
The child's run is not your wrapped run: When the child process starts, it internally calls its own run method. Because of the serialization and deserialization process, the run method that's ultimately executed in the child process is the original multiprocessing.Process.run or the Process.run if you had directly overridden it. Your __getattribute__ magic, which wraps run, isn't correctly applied to the Process object within the child's context.

* Cleanup

* Logging bugfix (#18465)

* use mp Manager to handle logging queues

A Python bug (https://github.com/python/cpython/issues/91555) was preventing logs from the embeddings maintainer process from printing. The bug is fixed in Python 3.14, but a viable workaround is to use the multiprocessing Manager, which better manages mp queues and causes the logging to work correctly.

* consolidate

* fix typing

* Fix typing

* Use global log queue

* Move to using process for logging

* Convert camera tracking to process

* Add more processes

* Finalize process

* Cleanup

* Cleanup typing

* Formatting

* Remove daemon

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-06-12 12:12:34 -06:00
Nicolas Mowen
6f16ecdd48
Dynamic Management of Cameras (#18671)
* Add base class for global config updates

* Add or remove camera states

* Move camera process management to separate thread

* Move camera management fully to separate class

* Cleanup

* Stop camera processes when stop command is sent

* Start processes dynamically when needed

* Adjust

* Leave extra room in tracked object queue for two cameras

* Dynamically set extra config pieces

* Add some TODOs

* Fix type check

* Simplify config updates

* Improve typing

* Correctly handle indexed entries

* Cleanup

* Create out SHM

* Use ZMQ for signaling object detectoin is completed

* Get camera correctly created

* Cleanup for updating the cameras config

* Cleanup

* Don't enable audio if no cameras have audio transcription

* Use exact string so similar camera names don't interfere

* Add ability to update config via json body to config/set endpoint

Additionally, update the config in a single rather than multiple calls for each updated key

* fix autotracking calibration to support new config updater function

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-06-11 12:25:30 -05:00
Nicolas Mowen
5dd30b273a
Dynamically update masks and zones for cameras (#18359)
* Include config publisher in api

* Call update topic for passed topics

* Update zones dynamically

* Update zones internally

* Support zone and mask reset

* Handle updating objects config

* Don't put status for needing to restart Frigate

* Cleanup http tests

* Fix tests
2025-05-22 20:51:23 -06:00
Nicolas Mowen
559af44682 Dynamic Config Updates (#18353)
* Create classes to handle publishing and subscribing config updates

* Cleanup

* Use config updater

* Update handling for enabled config

* Cleanup

* Recording config updates

* Birdseye config updates

* Handle notifications

* handle review

* Update motion
2025-05-22 12:24:34 -06:00
Martin Weinelt
4d4d54d030
Fix various typing issues (#18187)
* Fix the `Any` typing hint treewide

There has been confusion between the Any type[1] and the any function[2]
in typing hints.

[1] https://docs.python.org/3/library/typing.html#typing.Any
[2] https://docs.python.org/3/library/functions.html#any

* Fix typing for various frame_shape members

Frame shapes are most likely defined by height and width, so a single int
cannot express that.

* Wrap gpu stats functions in Optional[]

These can return `None`, so they need to be `Type | None`, which is what
`Optional` expresses very nicely.

* Fix return type in get_latest_segment_datetime

Returns a datetime object, not an integer.

* Make the return type of FrameManager.write optional

This is necessary since the SharedMemoryFrameManager.write function can
return None.

* Fix total_seconds() return type in get_tz_modifiers

The function returns a float, not an int.

https://docs.python.org/3/library/datetime.html#datetime.timedelta.total_seconds

* Account for floating point results in to_relative_box

Because the function uses division the return types may either be int or
float.

* Resolve ruff deprecation warning

The config has been split into formatter and linter, and the global
options are deprecated.
2025-05-13 08:27:20 -06:00
Nicolas Mowen
eb4433162c
Fix video MP typing (#17880)
* Fix video typing

* Fix typo
2025-04-23 17:59:02 -06:00
Nicolas Mowen
aa7899e9dc
Docs / typing updates (#17878)
* Mention support for multiple rknn detectors

* Improve typing

* Fix mypy change

* Fix typing
2025-04-23 17:06:06 -06:00
Nicolas Mowen
15fe79178b
Improve async object detector support (#17712)
* Move object detection to folder

* Add input store type

* Add hwnc

* Add hwcn

* Fix test
2025-04-15 08:55:38 -05:00
Josh Hawkins
2920127ada
Dedicated LPR improvements (#17453)
* remove license plate from attributes for dedicated lpr cameras

* ensure we always have a color

* use frigate+ models with dedicated lpr cameras

* docs

* docs clarity

* docs enrichments

* use license_plate as object type
2025-03-30 07:43:24 -06:00
Nicolas Mowen
d4d5c4aac8
Disabled cameras fixing (#17273)
* Fix case where objects are returned as null

* Fix enabled status not being persisted

* Use config as source of truth when refreshed

* Ensure camera always have config object updated

* Cleanup typing
2025-03-20 11:20:44 -05:00
Josh Hawkins
4806d30406
Bugfixes (#17112)
* ensure disabled and offline messages don't overlay each other

* clean up enabled logic

* fix on demand recording button

* clean up
2025-03-12 06:09:09 -06:00
Josh Hawkins
ad0e89e147
Ensure disabling a camera also disables audio detection (#16961)
* Ensure disabling a camera also disables audio detection

* fix enabled state

* fix path
2025-03-05 06:30:23 -07:00
Josh Hawkins
531042467a
Dynamically enable/disable cameras (#16894)
* config options

* metrics

* stop and restart ffmpeg processes

* dispatcher

* frontend websocket

* buttons for testing

* don't recreate log pipe

* add/remove cam from birdseye when enabling/disabling

* end all objects and send empty camera activity

* enable/disable switch in ui

* disable buttons when camera is disabled

* use enabled_in_config for some frontend checks

* tweaks

* handle settings pane with disabled cameras

* frontend tweaks

* change to debug log

* mqtt docs

* tweak

* ensure all ffmpeg processes are initially started

* clean up

* use zmq

* remove camera metrics

* remove camera metrics

* tweaks

* frontend tweaks
2025-03-03 08:30:52 -07:00
Nicolas Mowen
c7db0f479d
Only match on exact config name in some cases (#16704)
* Only match on exact config name in some cases

* Cleanup
2025-02-20 10:04:06 -06:00
Josh Hawkins
4ef6214029
Tracking improvements (#16484)
* norfair tracker config per object type

* change default R back to 3.4

* separate trackers for static and autotracking cameras

* tweak params and fix debug draw

* ensure all trackers are correctly updated even when there are no detections

* basic reid with histograms

* check mp value

* check mp value again

* stationary objects won't have embeddings

* don't switch trackers when autotracking is toggled after startup

* improve motion detection during autotracking

* use helper function

* get histogram in tracker instead of detect
2025-02-11 09:37:58 -06:00
Nicolas Mowen
198d067e25
Implement support for YOLOv9 via ONNX (#16459)
* WIP yolov9

* Implement post processing for yolov9

* Cleanup detection

* Update docs to make note of supported yolov9

* Move post processing to separate utility

* Add note about other models
2025-02-10 15:00:12 -06:00
Josh Hawkins
78a013d63a
Add "frame" to shm frame names to avoid camera name issues (#15615) 2024-12-20 08:46:40 -06:00
Nicolas Mowen
c95bc9fe44
Handle case where camera name ends in number (#15296) 2024-12-01 12:33:10 -07:00
Nicolas Mowen
474c248c9d
Cleanup correctly (#15043) 2024-11-17 16:57:58 -06:00
Nicolas Mowen
5b1b6b5be0
Fix round robin (#15035)
* Move camera SHM frame creation to main process

* Don't reset frame index

* Don't fail if shm exists

* Set more types
2024-11-17 11:25:49 -06:00
Nicolas Mowen
45e9030358
Round robin SHM management (#15027)
* Output frame name to frames processor

* Finish implementing round robin

* Formatting
2024-11-16 16:00:19 -07:00
Nicolas Mowen
8ac4b001a2
Various fixes (#14410)
* Fix access

* Reorganize tracked object for imports

* Separate out rockchip build

* Formatting

* Use original ffmpeg build

* Fix build

* Update default search type value
2024-10-17 11:02:27 -05:00
Nicolas Mowen
e725730982
Fix shared memory frames being stuck when a camera capture crashed (#14140)
* Fix shared memory frames being stuck when a camera capture crashed

* Update ffmpeg build
2024-10-03 09:31:07 -06:00
Nicolas Mowen
3cff0df0ce
Bug fixes (#14092)
* Fix path of selected GPU

* Fix selection bug

* Simplify
2024-10-01 07:54:27 -06:00
Nicolas Mowen
15fa55c223
Refactor attribute saving (#14090)
* Refactor attribute saving

* Ensure sub label is not overwritten

* Formatting

* Fix unused
2024-10-01 07:31:03 -06:00
Nicolas Mowen
38d398c967
Dynamic attributes config (#14035)
* Add config for attribute map and generate all labels from the map

* Update docs

* Formatting

* Use the dynamic label map

* Fix check

* Fix docs typo
2024-09-28 07:49:04 -06:00
gtsiam
c0bd3b362c
Custom classes for Process and Metrics (#13950)
* Subclass Process for audio_process

* Introduce custom mp.Process subclass

In preparation to switch the multiprocessing startup method away from
"fork", we cannot rely on os.fork cloning the log state at fork time.
Instead, we have to set up logging before we run the business logic of
each process.

* Make camera_metrics into a class

* Make ptz_metrics into a class

* Fixed PtzMotionEstimator.ptz_metrics type annotation

* Removed pointless variables

* Do not start audio processor when no audio cameras are configured
2024-09-27 07:53:23 -05:00
gtsiam
dc54981784
Move more things out of FrigateApp (#13897)
* Moved FrigateApp.init_config() into FrigateConfig.load()

* Move frigate config loading into main

* Store PlusApi in FrigateConfig

* Register SIGTERM handler in main

* Ensure logging is setup during config parsing

* Removed pointless try

* Moved config initialization out of FrigateApp

* Made FrigateApp.shm_frame_count into a function

* Removed log calls from signal handlers

python's logging calls are not re-entrant, which caused at least one of
these to deadlock randomly.

* Reopen stdout/err on process fork

This helps avoid deadlocks (https://github.com/python/cpython/issues/91776).

* Make mypy happy

* Whoops. I might have forgotten to save.

Truly an amateur mistake.

* Always call FrigateApp.stop()
2024-09-24 07:07:47 -05:00
Nicolas Mowen
ceb7aa8b36
Improve handling of empty frames (#13651) 2024-09-10 08:39:37 -05:00
Nicolas Mowen
9afa1354da
Limit shm frame count (#12363)
* Limited shm frame count (#12346)

* Only keep 2x detect fps frames in SHM

* Don't delete previous shm frames in output

* Catch case where images do not exist

* Ensure files are closed

* Clear out all frames when shutting down

* Correct the number of frames saved

* Simplify empty shm error handling

* Improve frame safety

* Add handler logs when frame is None

* Don't fail on cleanup

* Cleanup logging

* Update docs

* Update calculation

* Restore condition

* Fix case where thumbnail is saved without frame

* Adjust debug logs

* Calculate best shm frame count

* Fix shm count calculation

* Catch missing frame

* Formatting

* Clarify docs

* Catch none frame in autotracking
2024-09-03 10:22:30 -06:00
Nicolas Mowen
f44df9fe61
Revert "Limited shm frame count (#12346)" (#12362)
This reverts commit 34812b7439.
2024-07-09 11:49:08 -05:00
Nicolas Mowen
34812b7439
Limited shm frame count (#12346)
* Only keep 2x detect fps frames in SHM

* Don't delete previous shm frames in output

* Catch case where images do not exist

* Ensure files are closed

* Clear out all frames when shutting down

* Correct the number of frames saved

* Simplify empty shm error handling

* Improve frame safety
2024-07-09 06:44:53 -06:00
Blake Blackshear
30b86271ea
move clip.mp4 backend to clips folder (#11834)
* move clip.mp4 backend to clips folder

* improve caching

* fix check
2024-06-09 13:45:26 -05:00
Blake Blackshear
b4384a1be3
Shutdown hang (#11793)
* intentionally handle queues during shutdown and carefully manage shutdown order

* more carefully manage shutdown to avoid threadlocks

* use debug for signal logging

* ensure disabled cameras dont break shutdown

* typo
2024-06-06 18:54:38 -05:00
Nicolas Mowen
313e7c016c
fix motion updating (#11083) 2024-04-23 09:57:29 -05:00
Aida Syoko
3d612e510c
chore: fix some typos in comments (#11028)
Signed-off-by: alongdate <alongyear@outlook.com>
2024-04-20 06:16:43 -05:00
Nicolas Mowen
dba21b606d
Convert config updating and video/audio queues to use zmq (#9893)
* Add config pub / sub pattern

* remove recording from feature metrics

* remove audio and feature metrics

* Check for updates from all cameras

* remove birdseye from camera metrics

* remove motion and detection camera metrics

* Ensure that all processes are stopped

* Stop communicators

* Detections

* Cleanup video output queue

* Use select for time sensitive polls

* Use ipc instead of tcp
2024-02-19 07:26:59 -06:00
Nicolas Mowen
dd3dc7949a
Use zmq for inter process communication (#9309)
* Use zmq for inter process communication

* Use localhost for reply and request

* Use pyobj instead of json and Need to use separate requestors for each audio listener

* Cleanup port defining
2024-02-14 18:24:36 -06:00
Nicolas Mowen
89dd114da1
Write recording segments to cache with timezone info (#8468)
* Store recording segments with timezone info

* Don't use _

* Use different separator due to timezone
2023-11-05 14:30:29 -06:00
Nicolas Mowen
63233a5830
Periodically sync for stale recordings (#8433)
* Periodically cleanup recordings files / DB

* Make automatic sync limited ot last 36 hours
2023-11-04 02:21:29 +00:00
Nicolas Mowen
e89dafa82e
Handle recording checks in utc (#8379)
* Handle recording checks in utc

* Formatting
2023-10-30 20:25:21 -04:00
Josh Hawkins
b9649de327
Don't generate region boxes from motion when autotracking (#8306)
* no region boxes from motion boxes when ptz moving

* debug contours and calibration

* remove debugging

* clarifying comment
2023-10-24 19:25:22 -04:00
Nicolas Mowen
0b858419d1
re-enable init delay (#8283) 2023-10-23 20:50:22 -04:00
Nicolas Mowen
2fb7200fb7
Revamp object consolidation logic (#8289)
* Separate object reduction to own function and reduce confidence of boxes on edge of region

* Add tests for different scenarios

* Formatting
2023-10-23 20:20:21 -04:00
Nicolas Mowen
a3c0e30502
Use existing bounding box for region when object is stationary (#8248) 2023-10-20 19:21:34 -04:00