Commit Graph

19 Commits

Author SHA1 Message Date
Nicolas Mowen
8485023442
Use Fork-Server As Spawn Method (#18682)
* Set runtime

* Use count correctly

* Don't assume camera sizes

* Use separate zmq proxy for object detection

* Correct order

* Use forkserver

* Only store PID instead of entire process reference

* Cleanup

* Catch correct errors

* Fix typing

* Remove before_run from process util

The before_run never actually ran because:

You're right to suspect an issue with before_run not being called and a potential deadlock. The way you've implemented the run_wrapper using __getattribute__ for the run method of BaseProcess is a common pitfall in Python's multiprocessing, especially when combined with how multiprocessing.Process works internally.

Here's a breakdown of why before_run isn't being called and why you might be experiencing a deadlock:

The Problem: __getattribute__ and Process Serialization
When you create a multiprocessing.Process object and call start(), the multiprocessing module needs to serialize the process object (or at least enough of it to re-create the process in the new interpreter). It then pickles this serialized object and sends it to the newly spawned process.

The issue with your __getattribute__ implementation for run is that:

run is retrieved during serialization: When multiprocessing tries to pickle your Process object to send to the new process, it will likely access the run attribute. This triggers your __getattribute__ wrapper, which then tries to bind run_wrapper to self.
run_wrapper is bound to the parent process's self: The run_wrapper closure, when created in the parent process, captures the self (the Process instance) from the parent's memory space.
Deserialization creates a new object: In the child process, a new Process object is created by deserializing the pickled data. However, the run_wrapper method that was pickled still holds a reference to the self from the parent process. This is a subtle but critical distinction.
The child's run is not your wrapped run: When the child process starts, it internally calls its own run method. Because of the serialization and deserialization process, the run method that's ultimately executed in the child process is the original multiprocessing.Process.run or the Process.run if you had directly overridden it. Your __getattribute__ magic, which wraps run, isn't correctly applied to the Process object within the child's context.

* Cleanup

* Logging bugfix (#18465)

* use mp Manager to handle logging queues

A Python bug (https://github.com/python/cpython/issues/91555) was preventing logs from the embeddings maintainer process from printing. The bug is fixed in Python 3.14, but a viable workaround is to use the multiprocessing Manager, which better manages mp queues and causes the logging to work correctly.

* consolidate

* fix typing

* Fix typing

* Use global log queue

* Move to using process for logging

* Convert camera tracking to process

* Add more processes

* Finalize process

* Cleanup

* Cleanup typing

* Formatting

* Remove daemon

---------

Co-authored-by: Josh Hawkins <32435876+hawkeye217@users.noreply.github.com>
2025-06-12 12:12:34 -06:00
Nicolas Mowen
8409100623
Classification Model Metrics (#18595)
* Add speed and rate metrics for custom classification models

* Use metrics for classification models

* Use keys

* Cast to list
2025-06-06 11:29:44 -05:00
Nicolas Mowen
38ffe21b69
Add support for rockchip GPU stats (#17812)
* Set rockchip GPU name

* Add support for calculating rockchip GPU load

* Format consistently

* Fix access error
2025-04-19 16:34:05 -05:00
Nicolas Mowen
80a13e43e9
Add support for NPU statistics in metrics page (#17806)
* Add npu usages as a statistic

* Support showing NPU stats in dashboard

* Add sys volume mount for npu usages

* Fix type

* Simplify check

* Cleanup

* Cleanup
2025-04-19 08:20:22 -06:00
Nicolas Mowen
15fe79178b
Improve async object detector support (#17712)
* Move object detection to folder

* Add input store type

* Add hwnc

* Add hwcn

* Fix test
2025-04-15 08:55:38 -05:00
Nicolas Mowen
9e8b85a957
Implement enchrichments events per second graph (#17436)
* Cleanup existing naming

* Add face recognitions per second

* Add lpr fps

* Add all eps

* Clean up line graph

* Translations

* Change wording

* Fix incorrect access

* Don't require plates

* Add comment

* Fix
2025-03-28 18:13:35 -05:00
Josh Hawkins
8d2f461350
Embeddings tweaks (#16864)
* make semantic search optional

* config

* frontend metrics

* docs

* tweak

* fixes

* also check genai cameras for embeddings context
2025-02-28 11:43:08 -07:00
Josh Hawkins
447f26e1b9
Fix lpr metrics and add yolov9 plate detection metric (#16827) 2025-02-26 07:29:34 -07:00
Nicolas Mowen
9dc85d4a76 Processing refactor (#15935)
* Refactor post processor to be real time processor

* Build out generic API for post processing

* Cleanup

* Fix
2025-02-08 12:47:01 -06:00
Nicolas Mowen
88686c44fe Generalize postprocessing (#15931)
* Actually send result to face registration

* Define postprocessing api and move face processing to fit

* Standardize request handling

* Standardize handling of processors

* Rename processing metrics

* Cleanup

* Standardize object end

* Update to newer formatting

* One more

* One more
2025-02-08 12:47:01 -06:00
Nicolas Mowen
a6ae208fe7 Add metrics page for embeddings and face / license plate processing times (#15818)
* Get stats for embeddings inferences

* cleanup embeddings inferences

* Enable UI for feature metrics

* Change threshold

* Fix check

* Update python for actions

* Set python version

* Ignore type for now
2025-02-08 12:47:01 -06:00
Nicolas Mowen
458ca4a983 Add support for SR-IOV GPU stats (#15796)
* Add option to treat GPU as SRIOV in order for stats to work correctly

* Add to intel docs

* fix tests
2025-02-08 12:47:01 -06:00
Nicolas Mowen
b149828c9f
Catch OS error (#15590) 2024-12-18 17:45:08 -06:00
Nicolas Mowen
eca504cb07
More bug fixes (#14593)
* Adjust mqtt logging behavior

* Set disconnect

* Only consider intel gpu stats error if None is returned
2024-10-25 09:45:11 -05:00
gtsiam
c0bd3b362c
Custom classes for Process and Metrics (#13950)
* Subclass Process for audio_process

* Introduce custom mp.Process subclass

In preparation to switch the multiprocessing startup method away from
"fork", we cannot rely on os.fork cloning the log state at fork time.
Instead, we have to set up logging before we run the business logic of
each process.

* Make camera_metrics into a class

* Make ptz_metrics into a class

* Fixed PtzMotionEstimator.ptz_metrics type annotation

* Removed pointless variables

* Do not start audio processor when no audio cameras are configured
2024-09-27 07:53:23 -05:00
gtsiam
edababa88e
Work through most of the cspell warnings in python (#13794) 2024-09-17 10:41:46 -05:00
Nicolas Mowen
2e724291db
Catch case where github sends bad json data (#13077) 2024-08-14 20:41:41 -05:00
Nicolas Mowen
f36e86feb6
Cleanup error gpu (#11130) 2024-04-27 11:26:51 -05:00
Nicolas Mowen
33c77d03c7
Reorganize stats (#9960)
* Reorganize stats

* Fix tests
2024-02-21 14:10:28 -06:00