mirror of
https://github.com/Unleash/unleash.git
synced 2025-01-25 00:07:47 +01:00
docs: Remove old A/B testing doc (#8258)
<!-- Thanks for creating a PR! To make it easier for reviewers and everyone else to understand what your changes relate to, please add some relevant content to the headings below. Feel free to ignore or delete sections that you don't think are relevant. Thank you! ❤️ --> ## About the changes <!-- Describe the changes introduced. What are they and why are they being introduced? Feel free to also add screenshots or steps to view the changes if they're visual. --> Now that we are building out our Use Case tutorials to include A/B testing, we can remove old content that is associated. <!-- Does it close an issue? Multiple? --> Closes # <!-- (For internal contributors): Does it relate to an issue on public roadmap? --> <!-- Relates to [roadmap](https://github.com/orgs/Unleash/projects/10) item: # --> ### Important files <!-- PRs can contain a lot of changes, but not all changes are equally important. Where should a reviewer start looking to get an overview of the changes? Are any files particularly important? --> ## Discussion points <!-- Anything about the PR you'd like to discuss before it gets merged? Got any questions or doubts? -->
This commit is contained in:
Nnenna Ndukwe
GitHub
parent
aec888a160
commit
41f9cac758
@ -1,25 +1,26 @@
|
||||
---
|
||||
title: How to capture impression data
|
||||
---
|
||||
import ApiRequest from '@site/src/components/ApiRequest'
|
||||
import VideoContent from '@site/src/components/VideoContent.jsx'
|
||||
|
||||
import ApiRequest from "@site/src/components/ApiRequest";
|
||||
import VideoContent from "@site/src/components/VideoContent.jsx";
|
||||
|
||||
:::info Placeholders
|
||||
Placeholders in the code samples below are delimited by angle brackets (i.e. `<placeholder-name>`). You will need to replace them with the values that are correct in _your_ situation for the code samples to run properly.
|
||||
:::
|
||||
|
||||
Unleash allows you to gather [**impression data**](../reference/impression-data.md) from your feature flags, giving you complete visibility into who checked what flags and when. What you do with this data is entirely up to you, but a common use case is to send it off to an aggregation and analytics service such as [Posthog](https://posthog.com/) or [Google Analytics](https://analytics.google.com/), either just for monitoring purposes or to facilitate [A/B testing](../topics/a-b-testing.md).
|
||||
Unleash allows you to gather [**impression data**](../reference/impression-data.md) from your feature flags, giving you complete visibility into who checked what flags and when. What you do with this data is entirely up to you, but a common use case is to send it off to an aggregation and analytics service such as [Posthog](https://posthog.com/) or [Google Analytics](https://analytics.google.com/), either just for monitoring purposes or to facilitate [A/B testing](../feature-flag-tutorials/use-cases/a-b-testing.md).
|
||||
|
||||
This guide will take you through everything you need to do in Unleash to facilitate such a workflow. It will show you how to send data to Posthog as an example sink, but the exact same principles will apply to any other service of the same kind.
|
||||
|
||||
<VideoContent videoUrls={["https://www.youtube.com/embed/bxYdeMb9ffw"]}/>
|
||||
<VideoContent videoUrls={["https://www.youtube.com/embed/bxYdeMb9ffw"]} />
|
||||
|
||||
## Prerequisites
|
||||
|
||||
We will assume that you have the necessary details for your third-party service:
|
||||
|
||||
- **where to send the data to**. We'll refer to this in the code samples below as **`<sink-url>`**.
|
||||
- **what format the data needs to be in**. This will determine how you transform the event data before you send it.
|
||||
- **where to send the data to**. We'll refer to this in the code samples below as **`<sink-url>`**.
|
||||
- **what format the data needs to be in**. This will determine how you transform the event data before you send it.
|
||||
|
||||
In addition, you'll need to have a flag to record impression data for and an [Unleash client SDK](../reference/sdks/index.md) that supports impression data. This guide will use the [JavaScript proxy SDK](/docs/generated/sdks/client-side/javascript-browser.md).
|
||||
|
||||
@ -37,7 +38,12 @@ When you create a new feature flag via the UI, there's an option at the end of t
|
||||
|
||||
To create a feature flag with impression data enabled, set the `impressionData` option to `true` in the request payload, as seen below. For more options, check the [reference docs on creating features](/reference/api/legacy/unleash/admin/features-v2.md#create-toggle).
|
||||
|
||||
<ApiRequest verb="post" payload={{name: "<feature-flag-name>", impressionData: true}} url="api/admin/projects/<project-id>/features" title="Create a feature flag with impression data enabled."/>
|
||||
<ApiRequest
|
||||
verb="post"
|
||||
payload={{ name: "<feature-flag-name>", impressionData: true }}
|
||||
url="api/admin/projects/<project-id>/features"
|
||||
title="Create a feature flag with impression data enabled."
|
||||
/>
|
||||
|
||||
### Enabling impression data for existing feature flags {#step-1-existing-toggles}
|
||||
|
||||
@ -47,8 +53,12 @@ To enable impression data for an existing flag, go to the "Settings" tab of that
|
||||
|
||||
To enable impression data for an existing flag, use the [API's flag patching functionality](/reference/api/legacy/unleash/admin/features-v2.md#patch-toggle):
|
||||
|
||||
<ApiRequest verb="patch" payload={[{op: "replace", path: "/impressionData", value: true}]} url="api/admin/projects/<project-id>/features/<feature-flag-name>" title="Enable impression data on an existing flag."/>
|
||||
|
||||
<ApiRequest
|
||||
verb="patch"
|
||||
payload={[{ op: "replace", path: "/impressionData", value: true }]}
|
||||
url="api/admin/projects/<project-id>/features/<feature-flag-name>"
|
||||
title="Enable impression data on an existing flag."
|
||||
/>
|
||||
|
||||
## Step 2: Capture impression events in your client {#step-2}
|
||||
|
||||
@ -58,7 +68,7 @@ The steps below will use extracts from said documentation.
|
||||
|
||||
After initializing the library, you'll probably also want to identify the current user, so that events can be correlated properly:
|
||||
|
||||
``` js title="Identify the user."
|
||||
```js title="Identify the user."
|
||||
posthog.identify(userId);
|
||||
```
|
||||
|
||||
@ -66,10 +76,9 @@ posthog.identify(userId);
|
||||
|
||||
Attach an event listener to Unleash that listens for `"impression"` events. Inside the listener, transform the event data to the format you want and send it to your analytics service.
|
||||
|
||||
``` js title="Capture, transform, and send impression data."
|
||||
```js title="Capture, transform, and send impression data."
|
||||
// listen for impression events
|
||||
unleash.on("impression", (event) => {
|
||||
|
||||
// capture and transform events
|
||||
posthog.capture(event.eventType, {
|
||||
...event.context,
|
||||
@ -78,7 +87,6 @@ unleash.on("impression", (event) => {
|
||||
enabled: event.enabled,
|
||||
variant: event.variant,
|
||||
});
|
||||
|
||||
});
|
||||
```
|
||||
|
||||
|
||||
@ -1,25 +1,24 @@
|
||||
---
|
||||
title: The Anatomy of Unleash
|
||||
---
|
||||
import Figure from '@site/src/components/Figure/Figure.tsx'
|
||||
|
||||
import Figure from "@site/src/components/Figure/Figure.tsx";
|
||||
|
||||
This guide's purpose is to give you a conceptual overview of how Unleash works. It covers the various components that exist within our system and how they interact with each other and with external applications. The diagrams help you understand the fundamental building blocks, such as [projects](../reference/projects.md), [environments](../reference/environments.md), [variants](../reference/feature-toggle-variants.md) and of course, [feature flags](../reference/feature-toggles.mdx).
|
||||
|
||||
The end of this guide presents a [short use case, explaining how you might configure Unleash](#use-case) to start working with feature flags.
|
||||
|
||||
|
||||
## The root level
|
||||
|
||||
Some things in Unleash are configured and defined on the root level. These options apply across the entire Unleash instance. The most important root configuration options for day-to-day operations are:
|
||||
|
||||
- [API access tokens](../reference/api-tokens-and-client-keys.mdx)
|
||||
- [Projects](../reference/projects.md)
|
||||
- [Segments](../reference/segments.mdx)
|
||||
- [Strategy types](../reference/activation-strategies.md) (including [custom activation strategy types](../reference/custom-activation-strategies.md))
|
||||
- [Tag types](../reference/tags.md)
|
||||
- [Unleash context](../reference/unleash-context.md) fields (including [custom context fields](../reference/unleash-context.md#custom-context-fields))
|
||||
- Users, [user groups](../reference/rbac.md#user-groups), and [roles](../reference/rbac.md)
|
||||
|
||||
- [API access tokens](../reference/api-tokens-and-client-keys.mdx)
|
||||
- [Projects](../reference/projects.md)
|
||||
- [Segments](../reference/segments.mdx)
|
||||
- [Strategy types](../reference/activation-strategies.md) (including [custom activation strategy types](../reference/custom-activation-strategies.md))
|
||||
- [Tag types](../reference/tags.md)
|
||||
- [Unleash context](../reference/unleash-context.md) fields (including [custom context fields](../reference/unleash-context.md#custom-context-fields))
|
||||
- Users, [user groups](../reference/rbac.md#user-groups) and [roles](../reference/rbac.md)
|
||||
|
||||
## Projects
|
||||
|
||||
@ -29,11 +28,18 @@ All Unleash instances must have at least one project at any given time. New inst
|
||||
|
||||
Pro and Enterprise customers can create, rename, and delete projects as they wish (as long as there is always **at least one project**). Open-source users, on the other hand, only get access to the Default project.
|
||||
|
||||
<Figure caption="Unleash projects contain one or more environments." alt="A square labeled 'project' containing another square, labeled 'environment'." img="/img/anatomy-of-unleash-environment.png"/>
|
||||
<Figure
|
||||
caption="Unleash projects contain one or more environments."
|
||||
alt="A square labeled 'project' containing another square, labeled 'environment'."
|
||||
img="/img/anatomy-of-unleash-environment.png"
|
||||
/>
|
||||
|
||||
## Environments and project environments
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-customer-tiers.png" caption="Feature flags can be activated or deactivated independently in different environments. For instance, a feature flag can be active in the development environment and deactivated in the production environment. Even if their configuration is otherwise identical, deactivated feature flags will never be considered enabled."/>
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-customer-tiers.png"
|
||||
caption="Feature flags can be activated or deactivated independently in different environments. For instance, a feature flag can be active in the development environment, and deactivated in the production environment. Even if their configuration is otherwise identical, deactivated feature flags will never be considered enabled."
|
||||
/>
|
||||
|
||||
[**Environments**](../reference/environments.md) in Unleash let you change how a feature flag works in your application’s different environments. For instance, while you are developing a feature, it’s likely that you’ll want it to be available in your development environment, but not in your production environment: environments let you do that. You might also want to enable a feature for only some users in your development environment, but no users in your production environment: environments let you do that.
|
||||
|
||||
@ -45,7 +51,10 @@ Enterprise users can create and remove environments. Open-source and Pro custome
|
||||
|
||||
Environments are adjacent to [feature flags](../reference/feature-toggles.mdx) in Unleash: neither one contains the other, but they come together to let you define activation strategies.
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-new-feature-rollout.png" caption="You can use different activation strategies and constraints in different environments. For instance, you can show a feature only to select user IDs in development, but roll it out to 25% of your user base in production."/>
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-new-feature-rollout.png"
|
||||
caption="You can use different activation strategies and constraints in different environments. For instance, you can show a feature only to select user IDs in development, but roll it out to 25% of your user base in production."
|
||||
/>
|
||||
|
||||
:::info Environments and API keys
|
||||
|
||||
@ -53,22 +62,30 @@ When connecting an SDK to Unleash, it's the **API key that decides which environ
|
||||
|
||||
:::
|
||||
|
||||
|
||||
## Features (feature flags)
|
||||
## Features (feature flags)
|
||||
|
||||
[**Feature flags**](../reference/feature-toggles.mdx) are at the heart of Unleash’s functionality. Feature flags belong to projects and live next to project environments. In and of itself, a feature flag doesn’t do anything. You must assign [**activation strategies**](../reference/activation-strategies.md) to it for it to start taking effect.
|
||||
|
||||
When creating a feature flag, you must assign a unique (across your Unleash instance) name, a [feature flag type](../reference/feature-toggle-types.md), a [project](../reference/projects.md) it belongs to, and an optional description. Everything except for the name can be changed later.
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-features.png" caption="Feature flag states are evaluated independently in each environment." alt="A hierarchy showing a project containing an environment containing a feature flag configuration."/>
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-features.png"
|
||||
caption="Feature flag states are evaluated independently in each environment."
|
||||
alt="A hierarchy showing a project containing an environment containing a feature flag configuration."
|
||||
/>
|
||||
|
||||
## Activation strategies
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-strategy.png" caption="Activation strategies are applied to feature flags on a per-environment basis and decide whether a feature is enabled or not." alt="A hierarchy displaying an environment containing a feature flag configuration with an activation strategy."/>
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-strategy.png"
|
||||
caption="Activation strategies are applied to feature flags on a per-environment basis and decide whether a feature is enabled or not."
|
||||
alt="A hierarchy displaying an environment containing a feature flag configuration with an activation strategy."
|
||||
/>
|
||||
|
||||
[**Activation strategies**](../reference/activation-strategies.md) (or just **strategies** for short) are the part of feature flags that tell Unleash **who should get a feature**. An activation strategy is assigned to **one **feature flag in **one **environment.
|
||||
|
||||
When you check a [feature flag](../reference/feature-toggles.mdx) in an application, the following decides the result:
|
||||
|
||||
1. Is the flag active in the current environment? If not, it will be disabled.
|
||||
2. If the flag **is** active in the current environment, the flag’s strategies decide the result. As long as **at least one** of a flag’s strategies resolves to `true` for the current context (user or application), then the flag will be considered enabled. In other words, if you have a hundred strategies and ninety-nine of them resolve to false, but one of them resolves to true, then the flag is enabled.
|
||||
|
||||
@ -76,10 +93,15 @@ Activation strategies tie feature flags and [environments](../reference/environm
|
||||
|
||||
Unleash comes with a number of [built-in strategies](../reference/activation-strategies.md). You can also create your own [custom activation strategies](../reference/custom-activation-strategies.md). All strategies can be further augmented by [**strategy constraints**](../reference/strategy-constraints.md).
|
||||
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-environments-strategies.png"
|
||||
caption="Feature flags exist across environments and can have different activation strategies in each environment."
|
||||
/>
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-environments-strategies.png" caption="Feature flags exist across environments and can have different activation strategies in each environment."/>
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-environments-strategies2.png" caption="Feature flag activation strategies can be copied between environments. You can also create new strategies in each environment."/>
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-environments-strategies2.png"
|
||||
caption="Feature flag activation strategies can be copied between environments. You can also create new strategies in each environment."
|
||||
/>
|
||||
|
||||
## Strategy constraints
|
||||
|
||||
@ -96,19 +118,23 @@ An activation strategy can have as many constraints as you want. When an activat
|
||||
Feature flag strategies are **permissive**: As long as **one** strategy resolves to true, the feature is considered enabled. On the other hand, constraints are **restrictive**: for a given strategy, **all** constraints must be met for it to resolve to true.
|
||||
|
||||
We can exemplify this difference with the logical operators AND and OR:
|
||||
- For a feature flag, if Strategy1 OR Strategy2 OR .. OR StrategyN is true, **then the feature is enabled**.
|
||||
- For a strategy, it can be evaluated **if and only if** Constraint1 AND Constraint2 AND .. AND ConstraintN are met.
|
||||
|
||||
- For a feature flag, if Strategy1 OR Strategy2 OR .. OR StrategyN is true, **then the feature is enabled**.
|
||||
- For a strategy, it can be evaluated **if and only if** Constraint1 AND Constraint2 AND .. AND ConstraintN are met.
|
||||
|
||||
Note that even if all the constraints are met, the strategy itself might not resolve to true: that will depend on the strategy and the provided context.
|
||||
|
||||
:::
|
||||
|
||||
|
||||
You can define constraints on whatever properties you want in your [Unleash context](../reference/unleash-context.md).
|
||||
|
||||
Constraints are applied to individual strategies and do not stay in sync with each other. When you need to have the same constraints applied to multiple strategies and need those constraints to stay in sync, use [**segments**](../reference/segments.mdx).
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-constraint.png" caption="Constraints can be applied to strategies, allowing you to narrow a feature's audience." alt="A hierarchy drawing showing a constraint applied to an activation strategy."/>
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-constraint.png"
|
||||
caption="Constraints can be applied to strategies, allowing you to narrow a feature's audience."
|
||||
alt="A hierarchy drawing showing a constraint applied to an activation strategy."
|
||||
/>
|
||||
|
||||
## Segments
|
||||
|
||||
@ -120,11 +146,14 @@ You can apply multiple segments to a strategy. Much like with constraints, **eve
|
||||
|
||||
Segments are only available to Pro and Enterprise users.
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-segments.png" caption="Segments are reusable lists of constraints that can be applied to a strategy instead of or in addition to constraints." />
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-segments.png"
|
||||
caption="Segments are reusable lists of constraints that can be applied to a strategy instead of or in addition to constraints."
|
||||
/>
|
||||
|
||||
## Variants and feature flag payloads
|
||||
|
||||
By default, a [feature flag](../reference/feature-toggles.mdx) in Unleash only tells you whether a feature is enabled or disabled, but you can also add more information to your flags by using [**feature flag variants**](../reference/feature-toggle-variants.md). Variants also allow you to run [A/B testing experiments](/topics/a-b-testing.md).
|
||||
By default, a [feature flag](../reference/feature-toggles.mdx) in Unleash only tells you whether a feature is enabled or disabled, but you can also add more information to your flags by using [**feature flag variants**](../reference/feature-toggle-variants.md). Variants also allow you to run [A/B testing experiments](../feature-flag-tutorials/use-cases/a-b-testing.md).
|
||||
|
||||
Feature flags are designed to let you decide which users get access to a feature. Variants are designed to let you decide **which version** of the feature a user gets access to. For instance, if user A is part of your beta testing program and gets access to a new beta feature, then you can use variants to decide whether they should get the red version or the green version of the feature.
|
||||
|
||||
@ -138,12 +167,17 @@ A feature flag can have as many variants as you want.
|
||||
|
||||
Prior to 4.21, variants were independent of [environments](../reference/environments.md). In other words: if you're on 4.19 or lower, you’ll always have the exact same variants with the exact same weightings and the exact same payloads in all environments.
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-variants.png" caption="Before Unleash 4.21, feature flag variants were the same for all environments."/>
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-variants.png"
|
||||
caption="Before Unleash 4.21, feature flag variants were the same for all environments."
|
||||
/>
|
||||
|
||||
As of version 4.21, a feature can have different variants in different environments. For instance, a development environment might have no variants, while a production environment has 2 variants. Payloads, weightings, and anything else can also differ between environments.
|
||||
|
||||
<Figure img="/img/anatomy-of-unleash-variants-environment.png" caption="From Unleash 4.21 on, a feature flag can have different variants in each environment."/>
|
||||
|
||||
<Figure
|
||||
img="/img/anatomy-of-unleash-variants-environment.png"
|
||||
caption="From Unleash 4.21 on, a feature flag can have different variants in each environment."
|
||||
/>
|
||||
|
||||
## Use case: changing website colors {#use-case}
|
||||
|
||||
@ -175,8 +209,7 @@ if (unleash.isEnabled(“new-color-scheme”)) {
|
||||
}
|
||||
```
|
||||
|
||||
And with that, the new color scheme is now live in your development environment. Because there aren’t any strategies defined in the production environment yet, the feature is not active, and everything is as it was.
|
||||
|
||||
And with that, the new color scheme is now live in your development environment. Because there aren’t any strategies defined in the production environment yet, the feature is not active, and everything is as it was.
|
||||
|
||||
### Rolling out the feature to users
|
||||
|
||||
@ -186,7 +219,6 @@ You decide to add a _gradual rollout_ strategy to the new-color-scheme feature i
|
||||
|
||||
As soon as you enable the production environment, the feature gets rolled out to 5% of your users (assuming you’ve deployed the code to production).
|
||||
|
||||
|
||||
### Adding variants
|
||||
|
||||
While you were developing the new color scheme, you also dabbled a bit with other colors in addition to blue: green and purple might be nice too! So you decide to create two extra color schemes that you’re happy with. But you’d like to hear what your users think too, so you need to roll it out to them somehow.
|
||||
|
||||
@ -82,7 +82,6 @@ module.exports = {
|
||||
label: 'Migrating from homegrown feature management solutions',
|
||||
id: 'topics/feature-flag-migration/feature-flag-migration-best-practices',
|
||||
},
|
||||
'topics/a-b-testing',
|
||||
],
|
||||
},
|
||||
{
|
||||
|
||||
Loading…
Reference in New Issue
Block a user