If apiTokens are enabled breaks middleware chain with a 401 if no token
is found for requests to client and frontend apis. Previously the
middleware allowed the chain to process.
Removes the regex search for multiple slashes, and instead configures
the apiTokenMiddleware to reject unauthorized requests.
After a Team Retro, one of our squads felt like we needed more data on
our test suites. This is the first effort to make our test results
easier to grab. It uses the test-reporter action to add a github check
to our main build and PR builds with our test results.
This at least should make it easier to parse which tests are failing.
However, it does not give us trends. So it does not yet make it easier
to decide which tests are flaky just from a quick view.
---------
Co-authored-by: Gastón Fournier <gaston@getunleash.io>
This PR attempts to improve the error handling introduced in #3607.
## About the changes
## **tl;dr:**
- Make `UnleashError` constructor protected
- Make all custom errors inherit from `UnleashError`.
- Add tests to ensure that all special error cases include their
relevant data
- Remove `PasswordMismatchError` and `BadRequestError`. These don't
exist.
- Add a few new error types: `ContentTypeError`, `NotImplementedError`,
`UnauthorizedError`
- Remove the `...rest` parameter from error constructor
- Add an unexported `GenericUnleashError` class
- Move OpenAPI conversion function to `BadDataError` clas
- Remove explicit `Error.captureStackTrace`. This is done automatically.
- Extract `getPropFromString` function and add tests
### **In a more verbose fashion**
The main thing is that all our internal errors now inherit
from`UnleashError`. This allows us to simplify the `UnleashError`
constructor and error handling in general while still giving us the
extra benefits we added to that class. However, it _does_ also mean that
I've had to update **all** existing error classes.
The constructor for `UnleashError` is now protected and all places that
called that constructor directly have been updated. Because the base
error isn't available anymore, I've added three new errors to cover use
cases that we didn't already have covered: `NotImplementedError`,
`UnauthorizedError`, `ContentTypeError`. This is to stay consistent in
how we report errors to the user.
There is also an internal class, `GenericUnleashError` that inherits
from the base error. This class is only used in conversions for cases
where we don't know what the error is. It is not exported.
In making all the errors inherit, I've also removed the `...rest`
parameter from the `UnleashError` constructor. We don't need this
anymore.
Following on from the fixes with missing properties in #3638, I have
added tests for all errors that contain extra data.
Some of the error names that were originally used when creating the list
don't exist in the backend. `BadRequestError` and
`PasswordMismatchError` have been removed.
The `BadDataError` class now contains the conversion code for OpenAPI
validation errors. In doing so, I extracted and tested the
`getPropFromString` function.
### Main files
Due to the nature of the changes, there's a lot of files to look at. So
to make it easier to know where to turn your attention:
The changes in `api-error.ts` contain the main changes: protected
constructor, removal of OpenAPI conversion (moved into `BadDataError`.
`api-error.test.ts` contains tests to make sure that errors work as
expected.
Aside from `get-prop-from-string.ts` and the tests, everything else is
just the required updates to go through with the changes.
## Discussion points
I've gone for inheritance of the Error type over composition. This is in
large part because throwing actual Error instances instead of just
objects is preferable (because they collect stack traces, for instance).
However, it's quite possible that we could solve the same thing in a
more elegant fashion using composition.
## For later / suggestions for further improvements
The `api-error` files still contain a lot of code. I think it might be
beneficial to break each Error into a separate folder that includes the
error, its tests, and its schema (if required). It would help decouple
it a bit.
We don't currently expose the schema anywhere, so it's not available in
the openapi spec. We should look at exposing it too.
Finally, it would be good to go through each individual error message
and update each one to be as helpful as possible.
This PR implements the first version of a suggested unification (and
documentation) of the errors that we return from the API today.
The goal is for this to be the first step towards the error type defined
in this internal [linear
task](https://linear.app/unleash/issue/1-629/define-the-error-type
'Define the new API error type').
## The state of things today
As things stand, we currently have no (or **very** little) documentation
of the errors that are returned from the API. We mention error codes,
but never what the errors may contain.
Second, there is no specified format for errors, so what they return is
arbitrary, and based on ... Who knows? As a result, we have multiple
different errors returned by the API depending on what operation you're
trying to do. What's more, with OpenAPI validation in the mix, it's
absolutely possible for you to get two completely different error
objects for operations to the same endpoint.
Third, the errors we do return are usually pretty vague and don't really
provide any real help to the user. "You don't have the right
permissions". Great. Well what permissions do I need? And how would I
know? "BadDataError". Sick. Why is it bad?
... You get it.
## What we want to achieve
The ultimate goal is for error messages to serve both humans and
machines. When the user provides bad data, we should tell them what
parts of the data are bad and what they can do to fix it. When they
don't have the right permissions, we should tell them what permissions
they need.
Additionally, it would be nice if we could provide an ID for each error
instance, so that you (or an admin) can look through the logs and locate
he incident.
## What's included in **this** PR?
This PR does not aim to implement everything above. It's not intended to
magically fix everything. Its goal is to implement the necessary
**breaking** changes, so that they can be included in v5. Changing error
messages is a slightly grayer area than changing APIs directly, but
changing the format is definitely something I'd consider breaking.
So this PR:
- defines a minimal version of the error type defined in the [API error
definition linear
task](https://linear.app/unleash/issue/1-629/define-the-error-type).
- aims to catch all errors we return today and wrap them in the error
type
- updates tests to match the new expectations.
An important point: because we are cutting v5 very soon and because work
for this wasn't started until last week, the code here isn't necessarily
very polished. But it doesn't need to be. The internals can be as messy
as we want, as long as the API surface is stable.
That said, I'm very open to feedback about design and code completeness,
etc, but this has intentionally been done quickly.
Please also see my inline comments on the changes for more specific
details.
### Proposed follow-ups
As mentioned, this is the first step to implementing the error type. The
public API error type only exposes `id`, `name`, and `message`. This is
barely any more than most of the previous messages, but they are now all
using the same format. Any additional properties, such as `suggestion`,
`help`, `documentationLink` etc can be added as features without
breaking the current format. This is an intentional limitation of this
PR.
Regarding additional properties: there are some error responses that
must contain extra properties. Some of these are documented in the types
of the new error constructor, but not all. This includes `path` and
`type` properties on 401 errors, `details` on validation errors, and
more.
Also, because it was put together quickly, I don't yet know exactly how
we (as developers) would **prefer** to use these new error messages
within the code, so the internal API (the new type, name, etc), is just
a suggestion. This can evolve naturally over time if (based on feedback
and experience) without changing the public API.
## Returning multiple errors
Most of the time when we return errors today, we only return a single
error (even if many things are wrong). AJV, the OpenAPI integration we
use does have a setting that allows it to return all errors in a request
instead of a single one. I suggest we turn that on, but that we do it in
a separate PR (because it updates a number of other snapshots).
When returning errors that point to `details`, the objects in the
`details` now contain a new `description` property. This "deprecates"
the `message` property. Due to our general deprecation policy, this
should be kept around for another full major and can be removed in v6.
```json
{
"name": "BadDataError",
"message": "Something went wrong. Check the `details` property for more information."
"details": [{
"message": "The .params property must be an object. You provided an array.",
"description": "The .params property must be an object. You provided an array.",
}]
}
```
## About the changes
Introduce a snapshot version of instanceStats inside
instance-stats-service to provide a cached state of the statistics
without compromising the DB.
### Important notes
Some rule-of-thumb applied in the PR that can be changed:
1. The snapshot refresh time
2. The threshold to report appName with the metrics
## Discussion points
1. The snapshot could be limited to just the information needed (things
like `hasOIDC` don't change until there's a restart), to optimize the memory usage
3. metrics.ts (used to expose Prometheus metrics) has a [refresh
interval of
2hs](2d16730cc2/src/lib/metrics.ts (L189-L195)),
but with this implementation, we could remove that background task and
rely on the snapshot
4. We could additionally update the snapshot every time someone queries
the DB to fetch stats (`getStats()` method), but it may increase
complexity without a significant benefit
Co-authored-by: Mateusz Kwasniewski <kwasniewski.mateusz@gmail.com>
Co-authored-by: Simon Hornby <liquidwicked64@gmail.com>
* This PR adds a configurable maxAge header to the CORS middleware. This
allows the preflight request to be cached so that we can reduce the
request load on our end for the frontend clients starting to utilise the
frontend api.
* Middleware first version
* Middleware tests
* Add tests
* Finish middleware tests
* Add type for request
* Add flagresolver
* Fix snapshot
* Update flags and tests
* Put it back as default
* Update snapshot
* feat: use unleash flags for embedded proxy
* feat: add a separate flag for the proxy frontend
* fix: setup unleash in dev
* fix: check flagResolver on each request
* fix: remove unleash client setup
* refactor: update frontend routes snapshot
* refactor: make batchMetrics flag dynamic
* fix: always check dynamic CORS origins config
* fix: make conditionalMiddleware work with the OpenAPI schema generation
Co-authored-by: olav <mail@olav.io>
* fix: use the frontend dir from the backend
* Build is now working
* Fix workflows
* Fix workflows
* Fix build PRs
* Test coverage workflow
* Test coverage
* Test coverage run
* Fix jest report
* refactor: add missing frontend build
* refactor: ignore frontend dir for coverage
* refactor: run frontend build in PRs
* refactor: run backend tests in PRs
* Revert "refactor: run backend tests in PRs"
This reverts commit 22cabddfd1.
* refactor: remove unused frontend build file
* refactor: test workflows in PR
* refactor: use a prepare script for the frontend
* refactor: simplify yarn build scripts
* refactor: fix check-release script
* Revert "refactor: test workflows in PR"
This reverts commit 496ae19404.
* refactor: remove unused gitignore lines
* refactor: remove renovate config from the frontend repo
* refactor: remove frontend repo license
* refactor: remove frontend repo changelog
* refactor: update frontend repo readme
* refactor: add frontend node_modules to dockerignore
* refactor: update the docker yarn.lock snapshot
Co-authored-by: olav <mail@olav.io>
* fix: remove unused exp flag
* fix: remove unused flag
* fix: add support for external flag resolver
* fix: rename flagsresolver to flagresolver
* fix: disable external flag resolver
* fix: refactor a bit
* fix: stop using unleash in server-dev
* fix: remove userGroups flag
* fix: revert bumping frontend
* feat: add OpenAPI validation to a few endpoints (2)
* refactor: use package version as the OpenAPI version
* refactor: keep the existing OpenAPI page for now
* refactor: add snapshots tests for the OpenAPI output
* refactor: validate Content-Type by default
* refactor: update vulnerable deps
* refactor: fix documentation URL to match schema
* refactor: improve external type declaration
* refactor: remove unused package resolutions
* refactor: try express-openapi fork
* Update package.json
* Update src/lib/services/openapi-service.ts
* Update src/lib/types/openapi.d.ts
* Update src/lib/types/openapi.d.ts
Co-authored-by: Ivar Conradi Østhus <ivarconr@gmail.com>
Adds environment support
This PR adds environments as a first-class concept in Unleash.
It necessitated a full rewrite on how we connect feature <-> strategy, as well as a rethink on which levels environments makes sense.
This enables PUTs on strategy configurations for a feature, since all strategies now have ids.
This also updates export/import format. The importer handles both formats, but export is no longer possible in version 1 of the export format, only in version 2, with strategy configurations for a feature as a separate object.
Co-authored-by: Christopher Kolstad <chriswk@getunleash.ai>
Co-authored-by: Fredrik Oseberg <fredrik.no@gmail.com>
Co-authored-by: Ivar Conradi Østhus <ivarconr@gmail.com>
Unleash will listen for 'SIGINT' & 'SIGTERM' and close background tasks and db connections before shutting down.
Co-authored-by: Christopher Kolstad <chriswk@getunleash.ai>
Co-authored-by: Christopher Kolstad <chriswk@getunleash.ai>
* fix: catch all route only for baseUriPath
* Update src/lib/app.ts
Co-authored-by: Fredrik Strand Oseberg <fredrik.no@gmail.com>
Co-authored-by: Fredrik Strand Oseberg <fredrik.no@gmail.com>
* chore: update changelog
* chore: update changelog
* feat: format asset paths and insert baseUri in html
* feat: add tests
* feat: pass dependencies to pre router hook
Co-authored-by: Ivar Conradi Østhus <ivarconr@gmail.com>
feat: options are now typed
- This makes it easier to know what to send to unleash.start / unleash.create
- Using a Partial to instantiate the config, then melding it with defaults to get a config object with all fields set either to their defaults or to whatever is passed in.
Co-authored-by: Fredrik Strand Oseberg <fredrik.no@gmail.com>
Co-authored-by: Ivar Conradi Østhus <ivarconr@gmail.com>
feat: Add Reset token functionality
This allows admin users to create a reset token for other users. Thus allowing resetting their password.
Co-authored-by: Fredrik Oseberg <fredrik.no@gmail.com>
fixes: #778
This PR Introduces first steps towards RBAC according to our specifications. Rbac will assume users to exist in the Unleash user table with a unique id. This is required to make correct mappings between users and roles.
