diff --git a/website/docs/reference/impact-metrics.mdx b/website/docs/reference/impact-metrics.mdx new file mode 100644 index 0000000000..1f0b92a555 --- /dev/null +++ b/website/docs/reference/impact-metrics.mdx @@ -0,0 +1,189 @@ +--- +title: Impact metrics +--- + +import SearchPriority from '@site/src/components/SearchPriority'; + + + +
+ Impact metrics is an early access feature + + Impact metrics and automated release progression are early access features. Functionality may change. We are actively looking for feedback. Share your experience in the Unleash community Slack or email beta@getunleash.io. + + During this early access period, Impact Metrics are available for Unleash Hosted customers. + +
+ + +## Overview + +Impact metrics are lightweight, application-level time-series metrics stored and visualized directly inside Unleash. +They allow you to connect specific application data, such as request counts, error rates, or memory usage, to your feature flags and release plans. + +Use impact metrics to validate feature impact and automate your release process. For example, you can monitor usage patterns or performance to see if a feature is meeting its goals. + +By combining impact metrics with [release templates](/reference/release-templates), you can reduce manual release operations. +Unleash can monitor your rollout and automatically progress to the next milestone or trigger safeguards that pause the release based on the health of your metrics. + +![Example of impact metrics displaying](/img/impact-metrics.png) + +Impact metrics support three types of data: +- **Counters**: Cumulative values that only increase. These are suitable for request counts, error counts, or event counters. +- **Gauges**: Values that fluctuate up or down, such as memory usage or the number of active users. +- **Histograms**: Distribution of values, useful for measuring things like request duration or response size. Supports percentiles (p50, p95, p99). + +You can use these metrics in two primary areas: +- **Charts**: Visualize data in the Impact Metrics section of the Admin UI. +- **Release management**: Drive automatic milestone progression or trigger safeguards to stop a rollout. + +## Key use cases + +Impact metrics provide real-time data about your features. This enables two primary workflows: [automated releases with safeguards](#automated-releases-with-safeguards) and [feature impact validation](#validate-feature-impact-and-improve-iteratively). + +### Automated releases with safeguards + +Impact metrics integrate directly with release templates so Unleash can automatically progress milestones or pause a rollout when metrics fall outside your defined thresholds. This reduces the manual effort required during releases, especially when teams are shipping more frequently or outside working hours. + +Instead of tracking dashboards or waiting for alerts, you can use counters, gauges, or histograms to define what “healthy” looks like and let Unleash manage the rollout based on objective data. + +Examples: + +- Progress from 25% → 50% only if error rates remain below a threshold. +- Automatically pause when request latency increases during a rollout. + +See [Automate release progression](#automate-release-progression) and [Configure safeguards](#configure-safeguards) for more information. + +### Validate feature impact and improve iteratively + +Beyond automation, impact metrics help you understand whether the features you build are actually solving the right problems. You can track usage patterns, performance trends, or flag-correlated outcomes over time to evaluate whether a feature is meeting expectations. This supports both short-term decisions during a rollout and longer-term decisions about iteration, refinement, or deprecation. + +Examples: + +- Track whether a new feature is being used as expected after rollout. +- Monitor error counts, traffic patterns, or operational health for a feature or its variants. +- Identify features that need follow-up work, optimisation, or removal. + +See [Create impact metrics charts](#create-impact-metrics-charts) for more information. + +## Define and record metrics in the SDK + +:::info +Impact metrics are currently supported by the Node SDK. To request support for additional SDKs, please contact [beta@getunleash.io](mailto:beta@getunleash.io). +::: + +To visualize a metric in Unleash, you must first define the metric in the SDK and then count or record values for it. + +The SDK automatically attaches the following context labels to your metrics to ensure they are queryable in the UI: `appName`, `environment`, `origin` (for example, `origin=sdk` or `origin=Edge`). + +### Counters + +Use counters for cumulative values that only increase, such as the total number of requests or errors. + +```javascript +// 1. Define the counter +unleash.impactMetrics.defineCounter( + 'request_count', + 'Total number of HTTP requests processed' +); + +// 2. Increment the counter +unleash.impactMetrics.incrementCounter('request_count'); +``` + +### Gauges + +Use gauges for values that can go up and down, such as current memory usage or active thread count. + +```javascript +// 1. Define the gauge +unleash.impactMetrics.defineGauge( + 'heap_memory_total', + 'Current heap memory usage in bytes' +); + +// 2. Update the gauge value +const currentHeap = process.memoryUsage().heapUsed; +unleash.impactMetrics.updateGauge('heap_memory_total', currentHeap); +``` + +### Histograms + +Use histograms to measure the distribution of values, such as request duration or response size. Unleash automatically calculates percentiles (p50, p95, p99). + +```javascript +// 1. Define the histogram +unleash.impactMetrics.defineHistogram( + 'request_time_ms', + 'Time taken to process a request in milliseconds' +); + +// 2. Record a value +const duration = 125; +unleash.impactMetrics.observeHistogram('request_time_ms', duration); +``` + +## Create impact metrics charts + +You can visualize your defined metrics in the Unleash Admin UI. + +1. Go to **Impact Metrics** in the sidebar. +2. Click **New Chart**. +3. In the **Add New Chart** dialog, configure the following: + - Data series: Select your counter or gauge (for example, `request_count`). + - Time: Select a window (for example, Last 24 hours). + - Mode (aggregation type): Choose how to display the data. + - For Counters: + - Rate per second + - Count + - For Gauges: + - Sum + - Average + - For Histograms: + - p50 + - p95 + - p99 + - Filters: Optionally filter by `appName`, `environment`, `origin`. + +4. Click **Add chart**. + +![Creating a new impact metrics chart in the Admin UI](/img/add-new-impact-metrics-chart.png) + +The chart will display the data based on your configuration. Note that there is a 1–2 minute delay between data generation and visualization due to the scrape and ingestion cycle. + +## Automate release progression + +Impact metrics integrate with [release templates](/reference/release-templates) to automate the rollout process. Instead of manually updating milestones (for example, moving from 10% to 50%), Unleash can handle this for you. + +To configure automatic progression: + +1. Open a feature flag that [uses a release template](https://docs.getunleash.io/reference/release-templates#apply-a-release-template-to-a-feature-flag). +2. Select a milestone and click **Add automation**. +3. Define the conditions: + - Time: The minimum duration the milestone must run. For example, proceed after 24 hours. +4. Click **Save**. + +When the time conditions are satisfied, Unleash automatically advances the release to the next milestone. + +## Configure safeguards + +Safeguards act as a safety net for your releases. They can automatically pause a rollout if metrics indicate system instability. + +:::info + +To experiment with safeguards during the early access phase, please reach out to beta@getunleash.io for configuration guidance. + +::: + + +## Technical implementation details + +### Ingestion and batching + +Impact metrics are batched and sent on the same interval as [regular SDK metrics](/reference/impression-data). They are ingested via the regular metrics endpoint. + +### Unleash Edge behavior + +Unleash Edge forwards impact metrics received from SDKs to the Unleash API. The origin of the label appears as `origin=Edge`. Daisy-chaining Edge instances is not supported. + +If an Edge instance accumulates a large batch of metrics (e.g., due to a temporary network disconnect), it will send them as a single bulk send upon reconnection. This will appear as a large, sudden spike in your counter graphs, rather than a smooth distribution over time. This is expected behavior. \ No newline at end of file diff --git a/website/docs/understanding-unleash/the-anatomy-of-unleash.mdx b/website/docs/understanding-unleash/the-anatomy-of-unleash.mdx index 5527ec061b..2006916084 100644 --- a/website/docs/understanding-unleash/the-anatomy-of-unleash.mdx +++ b/website/docs/understanding-unleash/the-anatomy-of-unleash.mdx @@ -46,7 +46,7 @@ Each project must always have **at least one** active environment. Environments are adjacent to [feature flags](../reference/feature-toggles) in Unleash: neither one contains the other, but they come together to let you define activation strategies. -![Constraints and activation strategies](/img/anatomy-of-unleash-new-feature-rollout.png) +![Constraints and activation strategies](/img/anatomy-of-unleash-environments.png) :::info Environments and API keys @@ -64,10 +64,10 @@ When creating a feature flag, you must assign a unique (across your Unleash inst ## Activation strategies -![An environment containing a feature flag configuration with an activation strategy](/img/anatomy-of-unleash-strategy.png) - [**Activation strategies**](../reference/activation-strategies) (or just **strategies** for short) are the part of feature flags that tell Unleash **who should get a feature**. An activation strategy is assigned to **one **feature flag in **one **environment. +![An environment containing a feature flag configuration with an activation strategy](/img/anatomy-of-unleash-strategy.png) + When you check a [feature flag](../reference/feature-toggles) in an application, the following decides the result: 1. Is the flag active in the current environment? If not, it will be disabled. @@ -122,6 +122,32 @@ Segments are only available to [Pro](/availability#plans) and [Enterprise](https ![An example of segments](/img/anatomy-of-unleash-segments.png) +## Release templates + +[Release templates](/reference/release-templates) provide a way to standardize how you roll out features across your application. While activation strategies define who gets a feature, release templates define how that access expands over time. + +![Illustration showing showing a release template is made up of milestones](/img/anatomy-of-unleash-release-template.png) + +A release template is a blueprint for a rollout. It consists of sequential stages called **milestones**. Each milestone contains one or more **activation strategies** or **segments**. For example, a template might look like this: + +- Milestone 1: Internal users only. +- Milestone 2: Beta users. +- Milestone 3: 50% of all users. +- Milestone 4: 100% of all users. + +![Illustration showing showing that a release template can be applied to a feature flag](/img/anatomy-of-unleash-release-template-apply.png) + +When you apply a release template to a specific feature flag in an environment, it creates a **release plan**. This plan lets you progress through the milestones step-by-step. Release templates ensure that your rollouts are consistent and safe, removing the need to manually configure strategies from scratch every time you launch a new feature. + +![Illustration showing showing a release template applied to a feature flag in an environment](/img/anatomy-of-unleash-release-template-applied.png) + +### Automatic milestone progression and safeguards + +You can further automate your release plans using [impact metrics](/reference/impact-metrics). Impact metrics are data points sent from your application (such as error counts, memory usage, or response latency) that act as real-time health indicators for your feature. + +By defining safeguards based on these metrics, you can allow Unleash to manage the rollout for you. If metrics remain healthy for a set duration, Unleash automatically advances to the next milestone. +If a metric crosses a safety threshold (for example, error rate spikes), Unleash immediately pauses the rollout to prevent incidents. + ## Variants and feature flag payloads By default, a [feature flag](../reference/feature-toggles) in Unleash only tells you whether a feature is enabled or disabled, but you can also add more information to your flags by using [**feature flag variants**](../reference/feature-toggle-variants). Variants also allow you to run [A/B testing experiments](../../feature-flag-tutorials/use-cases/a-b-testing). @@ -132,19 +158,11 @@ When you create new variants for a feature, they must be given a name and a **we You can use the variant payload to attach arbitrary data to a variant. Variants can have different kinds of payloads. -A feature flag can have as many variants as you want. - -### Variants and environments - -Prior to 4.21, variants were independent of [environments](../reference/environments). In other words: if you're on 4.19 or lower, you’ll always have the exact same variants with the exact same weightings and the exact same payloads in all environments. - -![Variants prior to Unleash 4.21](/img/anatomy-of-unleash-variants.png) - -As of version 4.21, a feature can have different variants in different environments. For instance, a development environment might have no variants, while a production environment has 2 variants. Payloads, weightings, and anything else can also differ between environments. +A feature can have different variants in different environments. For instance, a development environment might have no variants, while a production environment has 2 variants. Payloads, weightings, and anything else can also differ between environments. ![Variants after version 4.21](/img/anatomy-of-unleash-variants-environment.png) -## Use case: changing website colors {#use-case} +## Use case: changing website colors Using the concepts we have looked at in the previous sections, let’s create a hypothetical case and see how Unleash would solve it. @@ -205,4 +223,4 @@ if (theme === “green”) { } ``` -Now users that are included in the gradual rollout will get one of the three themes. Users that aren’t included get the old theme. +Now users that are included in the gradual rollout will get one of the three themes. Users that aren’t included get the old theme. \ No newline at end of file diff --git a/website/sidebars.ts b/website/sidebars.ts index 498fdaffad..3233ded4c7 100644 --- a/website/sidebars.ts +++ b/website/sidebars.ts @@ -97,7 +97,19 @@ const sidebars: SidebarsConfig = { 'reference/segments', 'reference/unleash-context', 'reference/stickiness', + ], + }, + { + label: 'Release management', + collapsed: true, + type: 'category', + link: { + type: 'doc', + id: 'reference/release-templates', + }, + items: [ 'reference/release-templates', + 'reference/impact-metrics', ], }, { diff --git a/website/static/img/add-new-impact-metrics-chart.png b/website/static/img/add-new-impact-metrics-chart.png new file mode 100644 index 0000000000..526799fa6b Binary files /dev/null and b/website/static/img/add-new-impact-metrics-chart.png differ diff --git a/website/static/img/anatomy-of-unleash-constraint.png b/website/static/img/anatomy-of-unleash-constraint.png index 9ed15f3b08..2c82e8604b 100644 Binary files a/website/static/img/anatomy-of-unleash-constraint.png and b/website/static/img/anatomy-of-unleash-constraint.png differ diff --git a/website/static/img/anatomy-of-unleash-customer-tiers.png b/website/static/img/anatomy-of-unleash-customer-tiers.png index 391110b061..734c862b8a 100644 Binary files a/website/static/img/anatomy-of-unleash-customer-tiers.png and b/website/static/img/anatomy-of-unleash-customer-tiers.png differ diff --git a/website/static/img/anatomy-of-unleash-environment.png b/website/static/img/anatomy-of-unleash-environment.png index 785b88c17d..1f8a22f7ae 100644 Binary files a/website/static/img/anatomy-of-unleash-environment.png and b/website/static/img/anatomy-of-unleash-environment.png differ diff --git a/website/static/img/anatomy-of-unleash-environments-strategies.png b/website/static/img/anatomy-of-unleash-environments-strategies.png index 90c5b90ddc..4d760e1aa9 100644 Binary files a/website/static/img/anatomy-of-unleash-environments-strategies.png and b/website/static/img/anatomy-of-unleash-environments-strategies.png differ diff --git a/website/static/img/anatomy-of-unleash-environments-strategies2.png b/website/static/img/anatomy-of-unleash-environments-strategies2.png index 3b4d505096..5e28330819 100644 Binary files a/website/static/img/anatomy-of-unleash-environments-strategies2.png and b/website/static/img/anatomy-of-unleash-environments-strategies2.png differ diff --git a/website/static/img/anatomy-of-unleash-environments.png b/website/static/img/anatomy-of-unleash-environments.png new file mode 100644 index 0000000000..098420dfdb Binary files /dev/null and b/website/static/img/anatomy-of-unleash-environments.png differ diff --git a/website/static/img/anatomy-of-unleash-features.png b/website/static/img/anatomy-of-unleash-features.png index 1a38108223..598c6f0dcf 100644 Binary files a/website/static/img/anatomy-of-unleash-features.png and b/website/static/img/anatomy-of-unleash-features.png differ diff --git a/website/static/img/anatomy-of-unleash-new-feature-rollout.png b/website/static/img/anatomy-of-unleash-new-feature-rollout.png deleted file mode 100644 index 3f3370f6e4..0000000000 Binary files a/website/static/img/anatomy-of-unleash-new-feature-rollout.png and /dev/null differ diff --git a/website/static/img/anatomy-of-unleash-release-template-applied.png b/website/static/img/anatomy-of-unleash-release-template-applied.png new file mode 100644 index 0000000000..0a3514323a Binary files /dev/null and b/website/static/img/anatomy-of-unleash-release-template-applied.png differ diff --git a/website/static/img/anatomy-of-unleash-release-template-apply.png b/website/static/img/anatomy-of-unleash-release-template-apply.png new file mode 100644 index 0000000000..c70d1e8aaa Binary files /dev/null and b/website/static/img/anatomy-of-unleash-release-template-apply.png differ diff --git a/website/static/img/anatomy-of-unleash-release-template.png b/website/static/img/anatomy-of-unleash-release-template.png new file mode 100644 index 0000000000..65bd97a790 Binary files /dev/null and b/website/static/img/anatomy-of-unleash-release-template.png differ diff --git a/website/static/img/anatomy-of-unleash-segments.png b/website/static/img/anatomy-of-unleash-segments.png index 5aa6fe68b4..f320169535 100644 Binary files a/website/static/img/anatomy-of-unleash-segments.png and b/website/static/img/anatomy-of-unleash-segments.png differ diff --git a/website/static/img/anatomy-of-unleash-strategy.png b/website/static/img/anatomy-of-unleash-strategy.png index 9f3f094bc5..8a901a2883 100644 Binary files a/website/static/img/anatomy-of-unleash-strategy.png and b/website/static/img/anatomy-of-unleash-strategy.png differ diff --git a/website/static/img/anatomy-of-unleash-variants-environment.png b/website/static/img/anatomy-of-unleash-variants-environment.png index e4f0cc7421..cf8096b8df 100644 Binary files a/website/static/img/anatomy-of-unleash-variants-environment.png and b/website/static/img/anatomy-of-unleash-variants-environment.png differ diff --git a/website/static/img/anatomy-of-unleash-variants.png b/website/static/img/anatomy-of-unleash-variants.png index 6ff7f952ee..eecd9f0d70 100644 Binary files a/website/static/img/anatomy-of-unleash-variants.png and b/website/static/img/anatomy-of-unleash-variants.png differ diff --git a/website/static/img/impact-metrics.png b/website/static/img/impact-metrics.png new file mode 100644 index 0000000000..0b6d30e27e Binary files /dev/null and b/website/static/img/impact-metrics.png differ