Refactor recordings retention to be based on review items (#13355)

* Refactor recordings config to be based off of review items

* Update object processing logic for when an event is created

* Migrate to deciding recording retention based on review items

* Refactor recording expiration to be based off of review items

* Remove remainder of recording events access

* Handle migration automatically

* Update version and cleanup

* Update docs

* Clarify docs

* Cleanup

* Target camera config

* Safely access all fields

docs/docs/configuration/record.md

@@ -13,7 +13,7 @@ H265 recordings can be viewed in Chrome 108+, Edge and Safari only. All other br
 
 ### Most conservative: Ensure all video is saved
 
-For users deploying Frigate in environments where it is important to have contiguous video stored even if there was no detectable motion, the following config will store all video for 3 days. After 3 days, only video containing motion and overlapping with events will be retained until 30 days have passed.
+For users deploying Frigate in environments where it is important to have contiguous video stored even if there was no detectable motion, the following config will store all video for 3 days. After 3 days, only video containing motion and overlapping with alerts or detections will be retained until 30 days have passed.
 
 ```yaml
 record:
@@ -21,9 +21,13 @@ record:
   retain:
     days: 3
     mode: all
-  events:
+  alerts:
     retain:
-      default: 30
+      days: 30
+      mode: motion
+  detections:
+    retain:
+      days: 30
       mode: motion
 ```
 
@@ -37,25 +41,28 @@ record:
   retain:
     days: 3
     mode: motion
-  events:
+  alerts:
     retain:
-      default: 30
+      days: 30
+      mode: motion
+  detections:
+    retain:
+      days: 30
       mode: motion
 ```
 
-### Minimum: Events only
+### Minimum: Alerts only
 
-If you only want to retain video that occurs during an event, this config will discard video unless an event is ongoing.
+If you only want to retain video that occurs during an event, this config will discard video unless an alert is ongoing.
 
 ```yaml
 record:
   enabled: True
   retain:
     days: 0
-    mode: all
+  alerts:
-  events:
     retain:
-      default: 30
+      days: 30
       mode: motion
 ```
 
@@ -86,19 +93,22 @@ record:
 
 Continuous recording supports different retention modes [which are described below](#what-do-the-different-retain-modes-mean)
 
-### Event Recording
+### Object Recording
 
-If you only used clips in previous versions with recordings disabled, you can use the following config to get the same behavior. This is also the default behavior when recordings are enabled.
+The number of days to record review items can be specified for review items classified as alerts as well as events.
 
 ```yaml
 record:
   enabled: True
-  events:
+  alerts:
     retain:
-      default: 10 # <- number of days to keep event recordings
+      days: 10 # <- number of days to keep alert recordings
+  detections:
+    retain:
+      days: 10 # <- number of days to keep detections recordings
 ```
 
-This configuration will retain recording segments that overlap with events and have active tracked objects for 10 days. Because multiple events can reference the same recording segments, this avoids storing duplicate footage for overlapping events and reduces overall storage needs.
+This configuration will retain recording segments that overlap with alerts and detections for 10 days. Because multiple events can reference the same recording segments, this avoids storing duplicate footage for overlapping events and reduces overall storage needs.
 
 **WARNING**: Recordings still must be enabled in the config. If a camera has recordings disabled in the config, enabling via the methods listed above will have no effect.
 
@@ -112,11 +122,7 @@ Let's say you have Frigate configured so that your doorbell camera would retain
 - With the `motion` option the only parts of those 48 hours would be segments that Frigate detected motion. This is the middle ground option that won't keep all 48 hours, but will likely keep all segments of interest along with the potential for some extra segments.
 - With the `active_objects` option the only segments that would be kept are those where there was a true positive object that was not considered stationary.
 
-The same options are available with events. Let's consider a scenario where you drive up and park in your driveway, go inside, then come back out 4 hours later.
+The same options are available with alerts and detections, except it will only save the recordings when it overlaps with a review item of that type.
-
-- With the `all` option all segments for the duration of the event would be saved for the event. This event would have 4 hours of footage.
-- With the `motion` option all segments for the duration of the event with motion would be saved. This means any segment where a car drove by in the street, person walked by, lighting changed, etc. would be saved.
-- With the `active_objects` it would only keep segments where the object was active. In this case the only segments that would be saved would be the ones where the car was driving up, you going inside, you coming outside, and the car driving away. Essentially reducing the 4 hours to a minute or two of event footage.
 
 A configuration example of the above retain modes where all `motion` segments are stored for 7 days and `active objects` are stored for 14 days would be as follows:
 
@@ -126,33 +132,18 @@ record:
   retain:
     days: 7
     mode: motion
-  events:
+  alerts:
     retain:
-      default: 14
+      days: 14
+      mode: active_objects
+  detections:
+    retain:
+      days: 14
       mode: active_objects
 ```
 
 The above configuration example can be added globally or on a per camera basis.
 
-### Object Specific Retention
-
-You can also set specific retention length for an object type. The below configuration example builds on from above but also specifies that recordings of dogs only need to be kept for 2 days and recordings of cars should be kept for 7 days.
-
-```yaml
-record:
-  enabled: True
-  retain:
-    days: 7
-    mode: motion
-  events:
-    retain:
-      default: 14
-      mode: active_objects
-      objects:
-        dog: 2
-        car: 7
-```
-
 ## Can I have "continuous" recordings, but only at certain times?
 
 Using Frigate UI, HomeAssistant, or MQTT, cameras can be automated to only record in certain situations or at certain times.

docs/docs/configuration/reference.md

@@ -419,32 +419,46 @@ record:
     # Optional: Quality of recording preview (default: shown below).
     # Options are: very_low, low, medium, high, very_high
     quality: medium
-  # Optional: Event recording settings
+  # Optional: alert recording settings
-  events:
+  alerts:
-    # Optional: Number of seconds before the event to include (default: shown below)
+    # Optional: Number of seconds before the alert to include (default: shown below)
     pre_capture: 5
-    # Optional: Number of seconds after the event to include (default: shown below)
+    # Optional: Number of seconds after the alert to include (default: shown below)
     post_capture: 5
-    # Optional: Objects to save recordings for. (default: all tracked objects)
+    # Optional: Retention settings for recordings of alerts
-    objects:
-      - person
-    # Optional: Retention settings for recordings of events
     retain:
-      # Required: Default retention days (default: shown below)
+      # Required: Retention days (default: shown below)
-      default: 10
+      days: 14
       # Optional: Mode for retention. (default: shown below)
-      #   all - save all recording segments for events regardless of activity
+      #   all - save all recording segments for alerts regardless of activity
-      #   motion - save all recordings segments for events with any detected motion
+      #   motion - save all recordings segments for alerts with any detected motion
-      #   active_objects - save all recording segments for event with active/moving objects
+      #   active_objects - save all recording segments for alerts with active/moving objects
+      #
+      # NOTE: If the retain mode for the camera is more restrictive than the mode configured
+      #       here, the segments will already be gone by the time this mode is applied.
+      #       For example, if the camera retain mode is "motion", the segments without motion are
+      #       never stored, so setting the mode to "all" here won't bring them back.
+      mode: motion
+  # Optional: detection recording settings
+  detections:
+    # Optional: Number of seconds before the detection to include (default: shown below)
+    pre_capture: 5
+    # Optional: Number of seconds after the detection to include (default: shown below)
+    post_capture: 5
+    # Optional: Retention settings for recordings of detections
+    retain:
+      # Required: Retention days (default: shown below)
+      days: 14
+      # Optional: Mode for retention. (default: shown below)
+      #   all - save all recording segments for detections regardless of activity
+      #   motion - save all recordings segments for detections with any detected motion
+      #   active_objects - save all recording segments for detections with active/moving objects
       #
       # NOTE: If the retain mode for the camera is more restrictive than the mode configured
       #       here, the segments will already be gone by the time this mode is applied.
       #       For example, if the camera retain mode is "motion", the segments without motion are
       #       never stored, so setting the mode to "all" here won't bring them back.
       mode: motion
-      # Optional: Per object retention days
-      objects:
-        person: 15
 
 # Optional: Configuration for the jpg snapshots written to the clips directory for each event
 # NOTE: Can be overridden at the camera level

frigate/config.py

@@ -296,12 +296,9 @@ class RetainModeEnum(str, Enum):
     active_objects = "active_objects"
 
 
-class RetainConfig(FrigateBaseModel):
+class RecordRetainConfig(FrigateBaseModel):
-    default: float = Field(default=10, title="Default retention period.")
+    days: float = Field(default=0, title="Default retention period.")
-    mode: RetainModeEnum = Field(default=RetainModeEnum.motion, title="Retain mode.")
+    mode: RetainModeEnum = Field(default=RetainModeEnum.all, title="Retain mode.")
-    objects: Dict[str, float] = Field(
-        default_factory=dict, title="Object retention period."
-    )
 
 
 class EventsConfig(FrigateBaseModel):
@@ -309,18 +306,9 @@ class EventsConfig(FrigateBaseModel):
         default=5, title="Seconds to retain before event starts.", le=MAX_PRE_CAPTURE
     )
     post_capture: int = Field(default=5, title="Seconds to retain after event ends.")
-    objects: Optional[List[str]] = Field(
+    retain: RecordRetainConfig = Field(
-        None,
+        default_factory=RecordRetainConfig, title="Event retention settings."
-        title="List of objects to be detected in order to save the event.",
     )
-    retain: RetainConfig = Field(
-        default_factory=RetainConfig, title="Event retention settings."
-    )
-
-
-class RecordRetainConfig(FrigateBaseModel):
-    days: float = Field(default=0, title="Default retention period.")
-    mode: RetainModeEnum = Field(default=RetainModeEnum.all, title="Retain mode.")
 
 
 class RecordExportConfig(FrigateBaseModel):
@@ -355,8 +343,11 @@ class RecordConfig(FrigateBaseModel):
     retain: RecordRetainConfig = Field(
         default_factory=RecordRetainConfig, title="Record retention settings."
     )
-    events: EventsConfig = Field(
+    detections: EventsConfig = Field(
-        default_factory=EventsConfig, title="Event specific settings."
+        default_factory=EventsConfig, title="Detection specific retention settings."
+    )
+    alerts: EventsConfig = Field(
+        default_factory=EventsConfig, title="Alert specific retention settings."
     )
     export: RecordExportConfig = Field(
         default_factory=RecordExportConfig, title="Recording Export Config"
@@ -924,6 +915,14 @@ class CameraFfmpegConfig(FfmpegConfig):
         return v
 
 
+class RetainConfig(FrigateBaseModel):
+    default: float = Field(default=10, title="Default retention period.")
+    mode: RetainModeEnum = Field(default=RetainModeEnum.motion, title="Retain mode.")
+    objects: Dict[str, float] = Field(
+        default_factory=dict, title="Object retention period."
+    )
+
+
 class SnapshotsConfig(FrigateBaseModel):
     enabled: bool = Field(default=False, title="Snapshots enabled.")
     clean_copy: bool = Field(
@@ -1278,10 +1277,19 @@ def verify_recording_retention(camera_config: CameraConfig) -> None:
     if (
         camera_config.record.retain.days != 0
         and rank_map[camera_config.record.retain.mode]
-        > rank_map[camera_config.record.events.retain.mode]
+        > rank_map[camera_config.record.alerts.retain.mode]
     ):
         logger.warning(
-            f"{camera_config.name}: Recording retention is configured for {camera_config.record.retain.mode} and event retention is configured for {camera_config.record.events.retain.mode}. The more restrictive retention policy will be applied."
+            f"{camera_config.name}: Recording retention is configured for {camera_config.record.retain.mode} and alert retention is configured for {camera_config.record.alerts.retain.mode}. The more restrictive retention policy will be applied."
+        )
+
+    if (
+        camera_config.record.retain.days != 0
+        and rank_map[camera_config.record.retain.mode]
+        > rank_map[camera_config.record.detections.retain.mode]
+    ):
+        logger.warning(
+            f"{camera_config.name}: Recording retention is configured for {camera_config.record.retain.mode} and detection retention is configured for {camera_config.record.detections.retain.mode}. The more restrictive retention policy will be applied."
         )
 
 
@@ -1429,7 +1437,7 @@ class FrigateConfig(FrigateBaseModel):
         default_factory=TimestampStyleConfig,
         title="Global timestamp style configuration.",
     )
-    version: Optional[float] = Field(default=None, title="Current config version.")
+    version: Optional[str] = Field(default=None, title="Current config version.")
 
     def runtime_config(self, plus_api: PlusApi = None) -> FrigateConfig:
         """Merge camera config with globals."""

frigate/events/cleanup.py

@@ -68,7 +68,10 @@ class EventCleanup(threading.Thread):
     def expire(self, media_type: EventCleanupType) -> list[str]:
         ## Expire events from unlisted cameras based on the global config
         if media_type == EventCleanupType.clips:
-            retain_config = self.config.record.events.retain
+            expire_days = max(
+                self.config.record.alerts.retain.days,
+                self.config.record.detections.retain.days,
+            )
             file_extension = None  # mp4 clips are no longer stored in /clips
             update_params = {"has_clip": False}
         else:
@@ -82,7 +85,11 @@ class EventCleanup(threading.Thread):
         # loop over object types in db
         for event in distinct_labels:
             # get expiration time for this label
-            expire_days = retain_config.objects.get(event.label, retain_config.default)
+            if media_type == EventCleanupType.snapshots:
+                expire_days = retain_config.objects.get(
+                    event.label, retain_config.default
+                )
+
             expire_after = (
                 datetime.datetime.now() - datetime.timedelta(days=expire_days)
             ).timestamp()
@@ -132,7 +139,10 @@ class EventCleanup(threading.Thread):
         ## Expire events from cameras based on the camera config
         for name, camera in self.config.cameras.items():
             if media_type == EventCleanupType.clips:
-                retain_config = camera.record.events.retain
+                expire_days = max(
+                    camera.record.alerts.retain.days,
+                    camera.record.detections.retain.days,
+                )
             else:
                 retain_config = camera.snapshots.retain
 
@@ -142,9 +152,11 @@ class EventCleanup(threading.Thread):
             # loop over object types in db
             for event in distinct_labels:
                 # get expiration time for this label
+                if media_type == EventCleanupType.snapshots:
                     expire_days = retain_config.objects.get(
                         event.label, retain_config.default
                     )
+
                 expire_after = (
                     datetime.datetime.now() - datetime.timedelta(days=expire_days)
                 ).timestamp()

frigate/events/maintainer.py

@@ -5,7 +5,7 @@ from multiprocessing.synchronize import Event as MpEvent
 from typing import Dict
 
 from frigate.comms.events_updater import EventEndPublisher, EventUpdateSubscriber
-from frigate.config import EventsConfig, FrigateConfig
+from frigate.config import FrigateConfig
 from frigate.events.types import EventStateEnum, EventTypeEnum
 from frigate.models import Event
 from frigate.util.builtin import to_relative_box
@@ -128,16 +128,13 @@ class EventProcessor(threading.Thread):
         if should_update_db(self.events_in_process[event_data["id"]], event_data):
             updated_db = True
             camera_config = self.config.cameras[camera]
-            event_config: EventsConfig = camera_config.record.events
             width = camera_config.detect.width
             height = camera_config.detect.height
             first_detector = list(self.config.detectors.values())[0]
 
-            start_time = event_data["start_time"] - event_config.pre_capture
+            start_time = event_data["start_time"]
             end_time = (
-                None
+                None if event_data["end_time"] is None else event_data["end_time"]
-                if event_data["end_time"] is None
-                else event_data["end_time"] + event_config.post_capture
             )
             # score of the snapshot
             score = (

frigate/object_processing.py

@@ -1070,25 +1070,27 @@ class TrackedObjectProcessor(threading.Thread):
         if obj.obj_data["position_changes"] == 0:
             return False
 
-        # If there are required zones and there is no overlap
+        # If the object is not considered an alert or detection
         review_config = self.config.cameras[camera].review
-        required_zones = (
+        if not (
-            review_config.alerts.required_zones
+            (
-            + review_config.detections.required_zones
+                obj.obj_data["label"] in review_config.alerts.labels
+                and (
+                    not review_config.alerts.required_zones
+                    or set(obj.entered_zones) & set(review_config.alerts.required_zones)
                 )
-        if len(required_zones) > 0 and not set(obj.entered_zones) & set(required_zones):
-            logger.debug(
-                f"Not creating clip for {obj.obj_data['id']} because it did not enter required zones"
             )
-            return False
+            or (
-
+                not review_config.detections.labels
-        # If the required objects are not present
+                or obj.obj_data["label"] in review_config.detections.labels
-        if (
+            )
-            record_config.events.objects is not None
+            and (
-            and obj.obj_data["label"] not in record_config.events.objects
+                not review_config.detections.required_zones
+                or set(obj.entered_zones) & set(review_config.alerts.required_zones)
+            )
         ):
             logger.debug(
-                f"Not creating clip for {obj.obj_data['id']} because it did not contain required objects"
+                f"Not creating clip for {obj.obj_data['id']} because it did not qualify as an alert or detection"
             )
             return False
 

frigate/record/cleanup.py

@@ -12,7 +12,7 @@ from playhouse.sqlite_ext import SqliteExtDatabase
 
 from frigate.config import CameraConfig, FrigateConfig, RetainModeEnum
 from frigate.const import CACHE_DIR, CLIPS_DIR, MAX_WAL_SIZE, RECORD_DIR
-from frigate.models import Event, Previews, Recordings, ReviewSegment
+from frigate.models import Previews, Recordings, ReviewSegment
 from frigate.record.util import remove_empty_directories, sync_recordings
 from frigate.util.builtin import clear_and_unlink, get_tomorrow_at_time
 
@@ -61,8 +61,42 @@ class RecordingCleanup(threading.Thread):
             db.execute_sql("PRAGMA wal_checkpoint(TRUNCATE);")
             db.close()
 
+    def expire_review_segments(self, config: CameraConfig, now: datetime) -> None:
+        """Delete review segments that are expired"""
+        alert_expire_date = (
+            now - datetime.timedelta(days=config.record.alerts.retain.days)
+        ).timestamp()
+        detection_expire_date = (
+            now - datetime.timedelta(days=config.record.detections.retain.days)
+        ).timestamp()
+        expired_reviews: ReviewSegment = (
+            ReviewSegment.select(ReviewSegment.id)
+            .where(
+                ReviewSegment.camera == config.name
+                and (
+                    (
+                        ReviewSegment.severity == "alert"
+                        and ReviewSegment.end_time < alert_expire_date
+                    )
+                    or (
+                        ReviewSegment.severity == "detection"
+                        and ReviewSegment.end_time < detection_expire_date
+                    )
+                )
+            )
+            .namedtuples()
+        )
+
+        max_deletes = 100000
+        deleted_reviews_list = list(map(lambda x: x[0], expired_reviews))
+        logger.info(f"the list is {deleted_reviews_list}")
+        for i in range(0, len(deleted_reviews_list), max_deletes):
+            ReviewSegment.delete().where(
+                ReviewSegment.id << deleted_reviews_list[i : i + max_deletes]
+            ).execute()
+
     def expire_existing_camera_recordings(
-        self, expire_date: float, config: CameraConfig, events: Event
+        self, expire_date: float, config: CameraConfig, reviews: ReviewSegment
     ) -> None:
         """Delete recordings for existing camera based on retention config."""
         # Get the timestamp for cutoff of retained days
@@ -86,47 +120,47 @@ class RecordingCleanup(threading.Thread):
             .iterator()
         )
 
-        # loop over recordings and see if they overlap with any non-expired events
+        # loop over recordings and see if they overlap with any non-expired reviews
         # TODO: expire segments based on segment stats according to config
-        event_start = 0
+        review_start = 0
         deleted_recordings = set()
         kept_recordings: list[tuple[float, float]] = []
         for recording in recordings:
             keep = False
+            mode = None
             # Now look for a reason to keep this recording segment
-            for idx in range(event_start, len(events)):
+            for idx in range(review_start, len(reviews)):
-                event: Event = events[idx]
+                review: ReviewSegment = reviews[idx]
 
-                # if the event starts in the future, stop checking events
+                # if the review starts in the future, stop checking reviews
                 # and let this recording segment expire
-                if event.start_time > recording.end_time:
+                if review.start_time > recording.end_time:
                     keep = False
                     break
 
-                # if the event is in progress or ends after the recording starts, keep it
+                # if the review is in progress or ends after the recording starts, keep it
-                # and stop looking at events
+                # and stop looking at reviews
-                if event.end_time is None or event.end_time >= recording.start_time:
+                if review.end_time is None or review.end_time >= recording.start_time:
                     keep = True
+                    mode = (
+                        config.record.alerts.retain.mode
+                        if review.severity == "alert"
+                        else config.record.detections.retain.mode
+                    )
                     break
 
-                # if the event ends before this recording segment starts, skip
+                # if the review ends before this recording segment starts, skip
-                # this event and check the next event for an overlap.
+                # this review and check the next review for an overlap.
-                # since the events and recordings are sorted, we can skip events
+                # since the review and recordings are sorted, we can skip review
                 # that end before the previous recording segment started on future segments
-                if event.end_time < recording.start_time:
+                if review.end_time < recording.start_time:
-                    event_start = idx
+                    review_start = idx
 
             # Delete recordings outside of the retention window or based on the retention mode
             if (
                 not keep
-                or (
+                or (mode == RetainModeEnum.motion and recording.motion == 0)
-                    config.record.events.retain.mode == RetainModeEnum.motion
+                or (mode == RetainModeEnum.active_objects and recording.objects == 0)
-                    and recording.motion == 0
-                )
-                or (
-                    config.record.events.retain.mode == RetainModeEnum.active_objects
-                    and recording.objects == 0
-                )
             ):
                 Path(recording.path).unlink(missing_ok=True)
                 deleted_recordings.add(recording.id)
@@ -202,65 +236,6 @@ class RecordingCleanup(threading.Thread):
                 Previews.id << deleted_previews_list[i : i + max_deletes]
             ).execute()
 
-        review_segments: list[ReviewSegment] = (
-            ReviewSegment.select(
-                ReviewSegment.id,
-                ReviewSegment.start_time,
-                ReviewSegment.end_time,
-                ReviewSegment.thumb_path,
-            )
-            .where(
-                ReviewSegment.camera == config.name,
-                ReviewSegment.end_time < expire_date,
-            )
-            .order_by(ReviewSegment.start_time)
-            .namedtuples()
-            .iterator()
-        )
-
-        # expire review segments
-        recording_start = 0
-        deleted_segments = set()
-        for segment in review_segments:
-            keep = False
-            # look for a reason to keep this segment
-            for idx in range(recording_start, len(kept_recordings)):
-                start_time, end_time = kept_recordings[idx]
-
-                # if the recording starts in the future, stop checking recordings
-                # and let this segment expire
-                if start_time > segment.end_time:
-                    keep = False
-                    break
-
-                # if the recording ends after the segment starts, keep it
-                # and stop looking at recordings
-                if end_time >= segment.start_time:
-                    keep = True
-                    break
-
-                # if the recording ends before this segment starts, skip
-                # this recording and check the next recording for an overlap.
-                # since the kept recordings and segments are sorted, we can skip recordings
-                # that end before the current segment started
-                if end_time < segment.start_time:
-                    recording_start = idx
-
-            # Delete segments without any relevant recordings
-            if not keep:
-                Path(segment.thumb_path).unlink(missing_ok=True)
-                deleted_segments.add(segment.id)
-
-        # expire segments
-        logger.debug(f"Expiring {len(deleted_segments)} segments")
-        # delete up to 100,000 at a time
-        max_deletes = 100000
-        deleted_segments_list = list(deleted_segments)
-        for i in range(0, len(deleted_segments_list), max_deletes):
-            ReviewSegment.delete().where(
-                ReviewSegment.id << deleted_segments_list[i : i + max_deletes]
-            ).execute()
-
     def expire_recordings(self) -> None:
         """Delete recordings based on retention config."""
         logger.debug("Start expire recordings.")
@@ -302,30 +277,31 @@ class RecordingCleanup(threading.Thread):
         logger.debug("Start all cameras.")
         for camera, config in self.config.cameras.items():
             logger.debug(f"Start camera: {camera}.")
+            now = datetime.datetime.now()
+
+            self.expire_review_segments(config, now)
 
             expire_days = config.record.retain.days
-            expire_date = (
+            expire_date = (now - datetime.timedelta(days=expire_days)).timestamp()
-                datetime.datetime.now() - datetime.timedelta(days=expire_days)
-            ).timestamp()
 
-            # Get all the events to check against
+            # Get all the reviews to check against
-            events: Event = (
+            reviews: ReviewSegment = (
-                Event.select(
+                ReviewSegment.select(
-                    Event.start_time,
+                    ReviewSegment.start_time,
-                    Event.end_time,
+                    ReviewSegment.end_time,
+                    ReviewSegment.severity,
                 )
                 .where(
-                    Event.camera == camera,
+                    ReviewSegment.camera == camera,
-                    # need to ensure segments for all events starting
+                    # need to ensure segments for all reviews starting
                     # before the expire date are included
-                    Event.start_time < expire_date,
+                    ReviewSegment.start_time < expire_date,
-                    Event.has_clip,
                 )
-                .order_by(Event.start_time)
+                .order_by(ReviewSegment.start_time)
                 .namedtuples()
             )
 
-            self.expire_existing_camera_recordings(expire_date, config, events)
+            self.expire_existing_camera_recordings(expire_date, config, reviews)
             logger.debug(f"End camera: {camera}.")
 
         logger.debug("End all cameras.")

frigate/record/maintainer.py

@@ -28,7 +28,7 @@ from frigate.const import (
     MAX_SEGMENTS_IN_CACHE,
     RECORD_DIR,
 )
-from frigate.models import Event, Recordings
+from frigate.models import Recordings, ReviewSegment
 from frigate.util.services import get_video_properties
 
 logger = logging.getLogger(__name__)
@@ -159,25 +159,27 @@ class RecordingMaintainer(threading.Thread):
             ):
                 self.audio_recordings_info[camera].pop(0)
 
-            # get all events with the end time after the start of the oldest cache file
+            # get all reviews with the end time after the start of the oldest cache file
             # or with end_time None
-            events: Event = (
+            reviews: ReviewSegment = (
-                Event.select(
+                ReviewSegment.select(
-                    Event.start_time,
+                    ReviewSegment.start_time,
-                    Event.end_time,
+                    ReviewSegment.end_time,
-                    Event.data,
+                    ReviewSegment.data,
                 )
                 .where(
-                    Event.camera == camera,
+                    ReviewSegment.camera == camera,
-                    (Event.end_time == None)
+                    (ReviewSegment.end_time == None)
-                    | (Event.end_time >= recordings[0]["start_time"].timestamp()),
+                    | (
-                    Event.has_clip,
+                        ReviewSegment.end_time
+                        >= recordings[0]["start_time"].timestamp()
+                    ),
                 )
-                .order_by(Event.start_time)
+                .order_by(ReviewSegment.start_time)
             )
 
             tasks.extend(
-                [self.validate_and_move_segment(camera, events, r) for r in recordings]
+                [self.validate_and_move_segment(camera, reviews, r) for r in recordings]
             )
 
         recordings_to_insert: list[Optional[Recordings]] = await asyncio.gather(*tasks)
@@ -189,10 +191,11 @@ class RecordingMaintainer(threading.Thread):
         )
 
     async def validate_and_move_segment(
-        self, camera: str, events: list[Event], recording: dict[str, any]
+        self, camera: str, reviews: list[ReviewSegment], recording: dict[str, any]
     ) -> None:
         cache_path = recording["cache_path"]
         start_time = recording["start_time"]
+        record_config = self.config.cameras[camera].record
 
         # Just delete files if recordings are turned off
         if (
@@ -232,10 +235,10 @@ class RecordingMaintainer(threading.Thread):
         ):
             # if the cached segment overlaps with the events:
             overlaps = False
-            for event in events:
+            for review in reviews:
                 # if the event starts in the future, stop checking events
                 # and remove this segment
-                if event.start_time > end_time.timestamp():
+                if review.start_time > end_time.timestamp():
                     overlaps = False
                     Path(cache_path).unlink(missing_ok=True)
                     self.end_time_cache.pop(cache_path, None)
@@ -243,12 +246,16 @@ class RecordingMaintainer(threading.Thread):
 
                 # if the event is in progress or ends after the recording starts, keep it
                 # and stop looking at events
-                if event.end_time is None or event.end_time >= start_time.timestamp():
+                if review.end_time is None or review.end_time >= start_time.timestamp():
                     overlaps = True
                     break
 
             if overlaps:
-                record_mode = self.config.cameras[camera].record.events.retain.mode
+                record_mode = (
+                    record_config.alerts.retain.mode
+                    if review.severity == "alert"
+                    else record_config.detections.retain.mode
+                )
                 # move from cache to recordings immediately
                 return await self.move_segment(
                     camera,
@@ -257,12 +264,14 @@ class RecordingMaintainer(threading.Thread):
                     duration,
                     cache_path,
                     record_mode,
-                    event.data["type"] == "api",
                 )
             # if it doesn't overlap with an event, go ahead and drop the segment
             # if it ends more than the configured pre_capture for the camera
             else:
-                pre_capture = self.config.cameras[camera].record.events.pre_capture
+                pre_capture = max(
+                    record_config.alerts.pre_capture,
+                    record_config.detections.pre_capture,
+                )
                 camera_info = self.object_recordings_info[camera]
                 most_recently_processed_frame_time = (
                     camera_info[-1][0] if len(camera_info) > 0 else 0
@@ -349,12 +358,11 @@ class RecordingMaintainer(threading.Thread):
         duration: float,
         cache_path: str,
         store_mode: RetainModeEnum,
-        manual_event: bool = False,  # if this segment is being moved due to a manual event
     ) -> Optional[Recordings]:
         segment_info = self.segment_stats(camera, start_time, end_time)
 
         # check if the segment shouldn't be stored
-        if not manual_event and segment_info.should_discard_segment(store_mode):
+        if segment_info.should_discard_segment(store_mode):
             Path(cache_path).unlink(missing_ok=True)
             self.end_time_cache.pop(cache_path, None)
             return
@@ -427,8 +435,7 @@ class RecordingMaintainer(threading.Thread):
                     Recordings.duration.name: duration,
                     Recordings.motion.name: segment_info.motion_count,
                     # TODO: update this to store list of active objects at some point
-                    Recordings.objects.name: segment_info.active_object_count
+                    Recordings.objects.name: segment_info.active_object_count,
-                    + (1 if manual_event else 0),
                     Recordings.regions.name: segment_info.region_count,
                     Recordings.dBFS.name: segment_info.average_dBFS,
                     Recordings.segment_size.name: segment_size,

frigate/record/record.py

@@ -11,7 +11,7 @@ from playhouse.sqliteq import SqliteQueueDatabase
 from setproctitle import setproctitle
 
 from frigate.config import FrigateConfig
-from frigate.models import Event, Recordings
+from frigate.models import Recordings, ReviewSegment
 from frigate.record.maintainer import RecordingMaintainer
 from frigate.util.services import listen
 
@@ -41,7 +41,7 @@ def manage_recordings(config: FrigateConfig) -> None:
         },
         timeout=max(60, 10 * len([c for c in config.cameras.values() if c.enabled])),
     )
-    models = [Event, Recordings]
+    models = [ReviewSegment, Recordings]
     db.bind(models)
 
     maintainer = RecordingMaintainer(

frigate/test/test_config.py

@@ -381,9 +381,7 @@ class TestConfig(unittest.TestCase):
     def test_motion_mask_relative_matches_explicit(self):
         config = {
             "mqtt": {"host": "mqtt"},
-            "record": {
+            "record": {"alerts": {"retain": {"days": 20}}},
-                "events": {"retain": {"default": 20, "objects": {"person": 30}}}
-            },
             "cameras": {
                 "explicit": {
                     "ffmpeg": {
@@ -555,9 +553,7 @@ class TestConfig(unittest.TestCase):
     def test_inherit_clips_retention(self):
         config = {
             "mqtt": {"host": "mqtt"},
-            "record": {
+            "record": {"alerts": {"retain": {"days": 20}}},
-                "events": {"retain": {"default": 20, "objects": {"person": 30}}}
-            },
             "cameras": {
                 "back": {
                     "ffmpeg": {
@@ -577,15 +573,17 @@ class TestConfig(unittest.TestCase):
         assert config == frigate_config.model_dump(exclude_unset=True)
 
         runtime_config = frigate_config.runtime_config()
-        assert (
+        assert runtime_config.cameras["back"].record.alerts.retain.days == 20
-            runtime_config.cameras["back"].record.events.retain.objects["person"] == 30
-        )
 
     def test_roles_listed_twice_throws_error(self):
         config = {
             "mqtt": {"host": "mqtt"},
             "record": {
-                "events": {"retain": {"default": 20, "objects": {"person": 30}}}
+                "alerts": {
+                    "retain": {
+                        "days": 20,
+                    }
+                }
             },
             "cameras": {
                 "back": {
@@ -609,7 +607,11 @@ class TestConfig(unittest.TestCase):
         config = {
             "mqtt": {"host": "mqtt"},
             "record": {
-                "events": {"retain": {"default": 20, "objects": {"person": 30}}}
+                "alerts": {
+                    "retain": {
+                        "days": 20,
+                    }
+                }
             },
             "cameras": {
                 "back": {
@@ -633,7 +635,11 @@ class TestConfig(unittest.TestCase):
         config = {
             "mqtt": {"host": "mqtt"},
             "record": {
-                "events": {"retain": {"default": 20, "objects": {"person": 30}}}
+                "alerts": {
+                    "retain": {
+                        "days": 20,
+                    }
+                }
             },
             "cameras": {
                 "back": {
@@ -664,7 +670,11 @@ class TestConfig(unittest.TestCase):
         config = {
             "mqtt": {"host": "mqtt"},
             "record": {
-                "events": {"retain": {"default": 20, "objects": {"person": 30}}}
+                "alerts": {
+                    "retain": {
+                        "days": 20,
+                    }
+                }
             },
             "cameras": {
                 "back": {
@@ -695,37 +705,6 @@ class TestConfig(unittest.TestCase):
             frigate_config.cameras["back"].zones["relative"].contour,
         )
 
-    def test_clips_should_default_to_global_objects(self):
-        config = {
-            "mqtt": {"host": "mqtt"},
-            "record": {
-                "events": {"retain": {"default": 20, "objects": {"person": 30}}}
-            },
-            "objects": {"track": ["person", "dog"]},
-            "cameras": {
-                "back": {
-                    "ffmpeg": {
-                        "inputs": [
-                            {"path": "rtsp://10.0.0.1:554/video", "roles": ["detect"]}
-                        ]
-                    },
-                    "detect": {
-                        "height": 1080,
-                        "width": 1920,
-                        "fps": 5,
-                    },
-                    "record": {"events": {}},
-                }
-            },
-        }
-        frigate_config = FrigateConfig(**config)
-        assert config == frigate_config.model_dump(exclude_unset=True)
-
-        runtime_config = frigate_config.runtime_config()
-        back_camera = runtime_config.cameras["back"]
-        assert back_camera.record.events.objects is None
-        assert back_camera.record.events.retain.objects["person"] == 30
-
     def test_role_assigned_but_not_enabled(self):
         config = {
             "mqtt": {"host": "mqtt"},

frigate/util/config.py

@@ -13,7 +13,7 @@ from frigate.util.services import get_video_properties
 
 logger = logging.getLogger(__name__)
 
-CURRENT_CONFIG_VERSION = 0.14
+CURRENT_CONFIG_VERSION = "0.15-0"
 
 
 def migrate_frigate_config(config_file: str):
@@ -29,7 +29,7 @@ def migrate_frigate_config(config_file: str):
     with open(config_file, "r") as f:
         config: dict[str, dict[str, any]] = yaml.load(f)
 
-    previous_version = config.get("version", 0.13)
+    previous_version = str(config.get("version", "0.13"))
 
     if previous_version == CURRENT_CONFIG_VERSION:
         logger.info("frigate config does not need migration...")
@@ -38,12 +38,12 @@ def migrate_frigate_config(config_file: str):
     logger.info("copying config as backup...")
     shutil.copy(config_file, os.path.join(CONFIG_DIR, "backup_config.yaml"))
 
-    if previous_version < 0.14:
+    if previous_version < "0.14":
         logger.info(f"Migrating frigate config from {previous_version} to 0.14...")
         new_config = migrate_014(config)
         with open(config_file, "w") as f:
             yaml.dump(new_config, f)
-        previous_version = 0.14
+        previous_version = "0.14"
 
         logger.info("Migrating export file names...")
         for file in os.listdir(EXPORT_DIR):
@@ -55,6 +55,13 @@ def migrate_frigate_config(config_file: str):
                 os.path.join(EXPORT_DIR, file), os.path.join(EXPORT_DIR, new_name)
             )
 
+    if previous_version < "0.15-0":
+        logger.info(f"Migrating frigate config from {previous_version} to 0.15-0...")
+        new_config = migrate_015_0(config)
+        with open(config_file, "w") as f:
+            yaml.dump(new_config, f)
+        previous_version = "0.15-0"
+
     logger.info("Finished frigate config migration...")
 
 
@@ -141,7 +148,99 @@ def migrate_014(config: dict[str, dict[str, any]]) -> dict[str, dict[str, any]]:
 
         new_config["cameras"][name] = camera_config
 
-    new_config["version"] = 0.14
+    new_config["version"] = "0.14"
+    return new_config
+
+
+def migrate_015_0(config: dict[str, dict[str, any]]) -> dict[str, dict[str, any]]:
+    """Handle migrating frigate config to 0.15-0"""
+    new_config = config.copy()
+
+    # migrate record.events to record.alerts and record.detections
+    global_record_events = config.get("record", {}).get("events")
+    if global_record_events:
+        alerts_retention = {"retain": {}}
+        detections_retention = {"retain": {}}
+
+        if global_record_events.get("pre_capture"):
+            alerts_retention["pre_capture"] = global_record_events["pre_capture"]
+
+        if global_record_events.get("post_capture"):
+            alerts_retention["post_capture"] = global_record_events["post_capture"]
+
+        if global_record_events.get("retain", {}).get("default"):
+            alerts_retention["retain"]["days"] = global_record_events["retain"][
+                "default"
+            ]
+
+        # decide logical detections retention based on current detections config
+        if not config.get("review", {}).get("alerts", {}).get(
+            "required_zones"
+        ) or config.get("review", {}).get("detections"):
+            if global_record_events.get("pre_capture"):
+                detections_retention["pre_capture"] = global_record_events[
+                    "pre_capture"
+                ]
+
+            if global_record_events.get("post_capture"):
+                detections_retention["post_capture"] = global_record_events[
+                    "post_capture"
+                ]
+
+            if global_record_events.get("retain", {}).get("default"):
+                detections_retention["retain"]["days"] = global_record_events["retain"][
+                    "default"
+                ]
+        else:
+            detections_retention["retain"]["days"] = 0
+
+        new_config["record"]["alerts"] = alerts_retention
+        new_config["record"]["detections"] = detections_retention
+
+        del new_config["record"]["events"]
+
+    for name, camera in config.get("cameras", {}).items():
+        camera_config: dict[str, dict[str, any]] = camera.copy()
+
+        record_events: dict[str, any] = camera_config.get("record", {}).get("events")
+
+        if record_events:
+            alerts_retention = {"retain": {}}
+            detections_retention = {"retain": {}}
+
+            if record_events.get("pre_capture"):
+                alerts_retention["pre_capture"] = record_events["pre_capture"]
+
+            if record_events.get("post_capture"):
+                alerts_retention["post_capture"] = record_events["post_capture"]
+
+            if record_events.get("retain", {}).get("default"):
+                alerts_retention["retain"]["days"] = record_events["retain"]["default"]
+
+            # decide logical detections retention based on current detections config
+            if not camera_config.get("review", {}).get("alerts", {}).get(
+                "required_zones"
+            ) or camera_config.get("review", {}).get("detections"):
+                if record_events.get("pre_capture"):
+                    detections_retention["pre_capture"] = record_events["pre_capture"]
+
+                if record_events.get("post_capture"):
+                    detections_retention["post_capture"] = record_events["post_capture"]
+
+                if record_events.get("retain", {}).get("default"):
+                    detections_retention["retain"]["days"] = record_events["retain"][
+                        "default"
+                    ]
+            else:
+                detections_retention["retain"]["days"] = 0
+
+            camera_config["record"]["alerts"] = alerts_retention
+            camera_config["record"]["detections"] = detections_retention
+            del camera_config["record"]["events"]
+
+        new_config["cameras"][name] = camera_config
+
+    new_config["version"] = "0.15-0"
     return new_config