I've held an internal knowledge sharing session on this already. If
someone can think of a better phrasing for the background, I'm all ears.
I think it's just nice to have this documented, so people remember that
our logging framework already has a good way to format errors when you
use the API `logger.error("<message>", e)`
Having old ADRs makes it harder for newcomers, that have to navigate
through many ADRs and spot the ones that were superseded by others. We
can save the mental burden by just moving the old ones to a deprecated
folder
## Key Updates in the ADR
- **Separation of Migrations in PRs**: Migrations are now required to be
carried out in separate pull requests. This change is intended to
improve the monitoring and management of database schema changes during
deployment.
- **Primary Key Requirement for New Tables**: A new paragraph mandates
the inclusion of primary keys in all new tables, emphasizing the
importance of data integrity, efficient data retrieval, and supporting
table relationships. Additionally, by adding primary keys, we resolve
the issue of migrations failing during upgrades in replicated database
setups, as we are not using PostgreSQL replica identities. Exceptions to
this rule require a compelling justification.
Also added better structuring and styling to ADR for better readability.
https://linear.app/unleash/issue/2-1680/adr-specify-tables-when-querying-columns
This PR includes my suggestion of a "Specificity in database column
references" ADR.
The ADR proposes a standard for explicitly specifying the full table
name or alias for each column in our SQL queries, especially for queries
with joins between multiple tables. This decision is a direct response
to recent ambiguity errors encountered due to a database migration,
emphasizing the need for clearer SQL query standards in complex joins
and schema changes.
---------
Co-authored-by: Mateusz Kwasniewski <kwasniewski.mateusz@gmail.com>
See [Linear issue
2-1627](https://linear.app/unleash/issue/2-1627/db-permissions-update-docs-and-migration-guides).
Since we now use functions, we need more permissions than just GRANT ALL
PERMISSIONS ON DATBASE. In addition we need to be allowed to create
functions in the schema that's actually being used. This PR takes the
easy way out and say that we need OWNER privileges on the database. That
guarantees that we can do all we do in our migration scripts.
### Discussion points
We might encounter some pushback on this, if so, we'll need to say that
we need
`GRANT ALL PRIVILEGES ON ALL FUNCTIONS IN <schema>` in addition to GRANT
ALL PRIVILEGES ON DATABASE, where <schema> is the schema selected in
their configuration.
We've been caught out here, we added a migration that used PG>=11
syntax, but our docs still state that we support v10, when it reached
EoL Nov 2022. This updates our docs to state v14 and newer, v14 came out
in 2021 and as such won't be EoL until 2026 according to [Postgres 5
year versioning scheme
support](https://www.postgresql.org/support/versioning/)
## About the changes
While working on Terraform I identified some issues with how our backend
handled some requests. This happened because the Terraform client was
unaware of new fields and was not sending them. This resulted in bad
behavior as some of those missing fields were treated in the backend as
`null` and removed existing data from the DB.
This ADR aims to shed some light on the problem and specifies how our
server should handle these requests.
---------
Co-authored-by: Nuno Góis <github@nunogois.com>
https://linear.app/unleash/issue/2-943/adr-separate-request-and-response-types-in-apis
During the updating of our OpenAPI documentation, we've seen several
times that our schemas are either way too wide for a response or way to
strict for a request. This is usually due to us reusing the same schema
for both request and response. We should write an ADR where we reason
about the usefulness of code duplication and keeping separate response
and request schemas.
Based on our needs, this PR adds my suggested ADR.
---------
Co-authored-by: Simon Hornby <liquidwicked64@gmail.com>
# Simplify package scripts
This PR's purpose is to raise a discussion surrounding our current
package scripts.
It includes some suggestions that aim to simplify the scripts and
hopefully bring a much more straightforward approach to developing and
contributing to Unleash.
Building (prod) should only happen **explicitly** and when needed.
## Before PR (current behavior)
- Clone the project;
- Open 2 terminals: One for `unleash` and another for
`unleash/frontend`;
- On `unleash`:
- Run `yarn` (which will also build, for some reason?);
- Run `yarn start:dev` to start backend in dev mode (`tsc-watch`);
- On `unleash/frontend`:
- Run `yarn` (which will also build, for some reason?);
- Run `yarn start` to start frontend in dev mode (`vite`);
So it seems to me like we build unnecessarily every time we install
dependencies. Neither dev scripts need to build the project, as backend
uses `tsc-watch` and frontend uses `vite`. I'm unsure why this is the
case, as building can take a very long time.
There's also some complexity in the way we need to split the terminal to
`cd` into `frontend` and treat it as a different project. The fact that
we have different script names is also confusing (`yarn start`, `yarn
start:dev`, etc).
## After PR
- Clone the project;
- Run `yarn` to install all dependencies;
- Run `yarn dev` to get started developing Unleash;
Running `yarn` should take care of everything needed to start
developing. This includes installing dependencies for frontend as well.
It should not build projects if we are not being explicit about it,
especially since we don't need to build them at this stage.
Running `yarn dev` should start the project in dev mode. This means
running both projects in `dev` mode, which for `backend` means running
`tsc-watch` and for `frontend` means running `vite`.
Here this PR attempts to provide a better DX by using
[concurrently](https://www.npmjs.com/package/concurrently) and
[wait-on](https://www.npmjs.com/package/wait-on) - This means both tasks
are ran simultaneously, stdout is labeled accordingly, and are stopped
together. It also means that `frontend` waits for `backend` to be
serving at `4242` before starting, since `frontend` starts pretty much
immediately with `vite` and `backend` takes a bit longer. Of course,
when the `backend` is hot-reloading you may still find some
`ECONNREFUSED`s on `frontend` stdout while it recompiles.
No more splitting your terminal and treating `frontend` as a separate
project.
## Discussion points
Maybe there's a better alternative to `tsc-watch`? I briefly explored
some alternatives and while they had a much faster starting speed,
hot-reload was sometimes slower. IMO we should aspire to run
`src/server-dev.ts` directly and only compile when needed.
Running `dev:backend` still serves a version of the frontend (at 4242).
**Why? Can we remove that behavior?**
I can't imagine a scenario in dev where we wouldn't want to run the
latest version of the frontend with `vite`.
~~**Note:** This PR removes all other out-of-scope scripts to focus on
this revamp. If we decide to merge it, we should evaluate what other
existing scripts we still want to include. May be a good opportunity to
clean up unused ones and only include the ones we really use. This
includes scripts that our GH actions rely on.~~
**Update:** In an effort to minimize impact surface of this PR and make
it a bit more ready for merging:
- It updates some docs in
2a4ff805e8
and
1bbc488251
to reflect our new simplified flow;
- It includes the old package scripts for now in
039bc04699;
- It updates some of our GH actions to reflect the new scripts in
7782cb9b12;
Given its current status I'll promote the PR to "ready for review".
I still think we should have a second look at our existing scripts and
GH actions to see what we really need and/or should adapt, but it should
be a team effort so we have a broader context. Maybe on a follow-up PR.
Does this require any changes to related projects (e.g. Enterprise)?
---------
Co-authored-by: Gastón Fournier <gaston@getunleash.io>
## About the changes
Update `passord` documentation with `password`. Note this was not a typo
but just Norwegian:
https://dictionary.cambridge.org/dictionary/english-norwegian/password
```shell
grep passord * -R -l | grep -v .git | grep -v dist | grep -v v3 | xargs sed -i 's/passord/password/g'
```
The script above avoids updating v3 because of legacy reasons
Related to #1265
This PR puts our contributing guidelines in the sidebar of the unleash
documentation. Currently there was no way of navigating to them easily,
which made our contribution guides and ADRs less useful. This PR adds
them to the sidebar as their own category, and adds an ADR for domain
centric language.
Co-authored-by: Thomas Heartman <thomas@getunleash.ai>
* docs: add ADRs
* docs/adrs
* fix: update developer guide
* fix: add space
* Update website/docs/contributing/backend/overview.md
Co-authored-by: Ivar Conradi Østhus <ivar@getunleash.ai>
* docs: remove auto-generated sidebar
This should've been in .gitignore, but has only been ignored to the
ignore file for the website subdirectory. (This has been fixed on main.)
* docs: delete empty file
* Revert "docs: delete empty file"
This reverts commit 2435f173ff.
* docs: add frontmatter to new dev docs
* Docs(fix): add quotes around page titles
In yaml, the colon is a special character, so we need to use quotes.
* docs: fix remaining titles
* Update website/docs/contributing/backend/overview.md
Co-authored-by: Ivar Conradi Østhus <ivar@getunleash.ai>
* fix: update empty ADR
* fix: update text to reflect postgres 12
* fix: update backend overview
* fix: remove link
* fix: add form ADR
Co-authored-by: Ivar Conradi Østhus <ivar@getunleash.ai>
Co-authored-by: Thomas Heartman <thomas@getunleash.ai>
