1
0
mirror of https://github.com/Unleash/unleash.git synced 2024-10-28 19:06:12 +01:00
unleash.unleash/website/docs/contributing/backend/overview.md

132 lines
3.6 KiB
Markdown
Raw Normal View History

---
title: Back end
---
2023-02-03 16:04:11 +01:00
The backend is written in nodejs/typescript. It's written as a REST API following a CSR (controller, service, repository/store) pattern. The following ADRs are defined for the backend:
## ADRs
We have created a set of ADRs to help guide the development of the backend:
* [Naming](../ADRs/back-end/naming.md)
* [Preferred export](../ADRs/back-end/preferred-export.md)
## Requirements
Before developing on this project you will need two things:
- PostgreSQL 14.x or newer
- Node.js 20.x or newer
```sh
corepack enable
yarn install
chore: simplify package scripts (#3736) # Simplify package scripts This PR's purpose is to raise a discussion surrounding our current package scripts. It includes some suggestions that aim to simplify the scripts and hopefully bring a much more straightforward approach to developing and contributing to Unleash. Building (prod) should only happen **explicitly** and when needed. ## Before PR (current behavior) - Clone the project; - Open 2 terminals: One for `unleash` and another for `unleash/frontend`; - On `unleash`: - Run `yarn` (which will also build, for some reason?); - Run `yarn start:dev` to start backend in dev mode (`tsc-watch`); - On `unleash/frontend`: - Run `yarn` (which will also build, for some reason?); - Run `yarn start` to start frontend in dev mode (`vite`); So it seems to me like we build unnecessarily every time we install dependencies. Neither dev scripts need to build the project, as backend uses `tsc-watch` and frontend uses `vite`. I'm unsure why this is the case, as building can take a very long time. ![image](https://github.com/Unleash/unleash/assets/14320932/5ecb7df1-e5b4-4d70-ba7e-97119f5d1116) There's also some complexity in the way we need to split the terminal to `cd` into `frontend` and treat it as a different project. The fact that we have different script names is also confusing (`yarn start`, `yarn start:dev`, etc). ## After PR - Clone the project; - Run `yarn` to install all dependencies; - Run `yarn dev` to get started developing Unleash; Running `yarn` should take care of everything needed to start developing. This includes installing dependencies for frontend as well. It should not build projects if we are not being explicit about it, especially since we don't need to build them at this stage. ![image](https://github.com/Unleash/unleash/assets/14320932/614e42fc-3467-432f-91fc-624b1b35c7c1) Running `yarn dev` should start the project in dev mode. This means running both projects in `dev` mode, which for `backend` means running `tsc-watch` and for `frontend` means running `vite`. Here this PR attempts to provide a better DX by using [concurrently](https://www.npmjs.com/package/concurrently) and [wait-on](https://www.npmjs.com/package/wait-on) - This means both tasks are ran simultaneously, stdout is labeled accordingly, and are stopped together. It also means that `frontend` waits for `backend` to be serving at `4242` before starting, since `frontend` starts pretty much immediately with `vite` and `backend` takes a bit longer. Of course, when the `backend` is hot-reloading you may still find some `ECONNREFUSED`s on `frontend` stdout while it recompiles. ![image](https://github.com/Unleash/unleash/assets/14320932/8bde8ee2-3cad-4e3f-a0db-9eed60cfb04d) No more splitting your terminal and treating `frontend` as a separate project. ## Discussion points Maybe there's a better alternative to `tsc-watch`? I briefly explored some alternatives and while they had a much faster starting speed, hot-reload was sometimes slower. IMO we should aspire to run `src/server-dev.ts` directly and only compile when needed. Running `dev:backend` still serves a version of the frontend (at 4242). **Why? Can we remove that behavior?** I can't imagine a scenario in dev where we wouldn't want to run the latest version of the frontend with `vite`. ~~**Note:** This PR removes all other out-of-scope scripts to focus on this revamp. If we decide to merge it, we should evaluate what other existing scripts we still want to include. May be a good opportunity to clean up unused ones and only include the ones we really use. This includes scripts that our GH actions rely on.~~ **Update:** In an effort to minimize impact surface of this PR and make it a bit more ready for merging: - It updates some docs in https://github.com/Unleash/unleash/pull/3736/commits/2a4ff805e87b65d9c1256effaa189ddcccba15fb and https://github.com/Unleash/unleash/pull/3736/commits/1bbc4882519b5a82e3116f0be255ad24a6f3ce53 to reflect our new simplified flow; - It includes the old package scripts for now in https://github.com/Unleash/unleash/pull/3736/commits/039bc04699ac880e491fd3ce01f9bcd6f97a94b9; - It updates some of our GH actions to reflect the new scripts in https://github.com/Unleash/unleash/pull/3736/commits/7782cb9b12e37ee844507e41ef2b7137eaf55666; Given its current status I'll promote the PR to "ready for review". I still think we should have a second look at our existing scripts and GH actions to see what we really need and/or should adapt, but it should be a team effort so we have a broader context. Maybe on a follow-up PR. Does this require any changes to related projects (e.g. Enterprise)? --------- Co-authored-by: Gastón Fournier <gaston@getunleash.io>
2023-05-12 12:23:22 +02:00
yarn dev
```
## PostgreSQL {#postgresql}
To run and develop unleash, you need to have PostgreSQL database (PostgreSQL v14.x or newer) locally.
Unleash currently also work with PostgreSQL v14+, but this might change in a future feature release, and we have stopped running automatic integration tests below PostgreSQL v12. The current recommendation is to use a role with Owner privileges since Unleash uses Postgres functions to simplify our database usage.
### Create a local unleash databases in postgres {#create-a-local-unleash-databases-in-postgres}
```bash
$ psql postgres <<SQL
CREATE USER unleash_user WITH PASSWORD 'password';
CREATE DATABASE unleash WITH OWNER unleash_user;
CREATE DATABASE unleash_test WITH OWNER unleash_user;
ALTER DATABASE unleash_test SET timezone TO 'UTC';
SQL
```
Then set env vars:
(Optional as unleash will assume these as default values).
```
export DATABASE_URL=postgres://unleash_user:password@localhost:5432/unleash
export TEST_DATABASE_URL=postgres://unleash_user:password@localhost:5432/unleash_test
```
## PostgreSQL with docker {#postgresql-with-docker}
If you don't want to install PostgreSQL locally, you can spin up an Docker instance. We have created a script to ease this process: `scripts/docker-postgres.sh`
## Start the application {#start-the-application}
In order to start the application you will need Node.js v14.x or newer installed locally.
```
// Install dependencies
yarn install
chore: simplify package scripts (#3736) # Simplify package scripts This PR's purpose is to raise a discussion surrounding our current package scripts. It includes some suggestions that aim to simplify the scripts and hopefully bring a much more straightforward approach to developing and contributing to Unleash. Building (prod) should only happen **explicitly** and when needed. ## Before PR (current behavior) - Clone the project; - Open 2 terminals: One for `unleash` and another for `unleash/frontend`; - On `unleash`: - Run `yarn` (which will also build, for some reason?); - Run `yarn start:dev` to start backend in dev mode (`tsc-watch`); - On `unleash/frontend`: - Run `yarn` (which will also build, for some reason?); - Run `yarn start` to start frontend in dev mode (`vite`); So it seems to me like we build unnecessarily every time we install dependencies. Neither dev scripts need to build the project, as backend uses `tsc-watch` and frontend uses `vite`. I'm unsure why this is the case, as building can take a very long time. ![image](https://github.com/Unleash/unleash/assets/14320932/5ecb7df1-e5b4-4d70-ba7e-97119f5d1116) There's also some complexity in the way we need to split the terminal to `cd` into `frontend` and treat it as a different project. The fact that we have different script names is also confusing (`yarn start`, `yarn start:dev`, etc). ## After PR - Clone the project; - Run `yarn` to install all dependencies; - Run `yarn dev` to get started developing Unleash; Running `yarn` should take care of everything needed to start developing. This includes installing dependencies for frontend as well. It should not build projects if we are not being explicit about it, especially since we don't need to build them at this stage. ![image](https://github.com/Unleash/unleash/assets/14320932/614e42fc-3467-432f-91fc-624b1b35c7c1) Running `yarn dev` should start the project in dev mode. This means running both projects in `dev` mode, which for `backend` means running `tsc-watch` and for `frontend` means running `vite`. Here this PR attempts to provide a better DX by using [concurrently](https://www.npmjs.com/package/concurrently) and [wait-on](https://www.npmjs.com/package/wait-on) - This means both tasks are ran simultaneously, stdout is labeled accordingly, and are stopped together. It also means that `frontend` waits for `backend` to be serving at `4242` before starting, since `frontend` starts pretty much immediately with `vite` and `backend` takes a bit longer. Of course, when the `backend` is hot-reloading you may still find some `ECONNREFUSED`s on `frontend` stdout while it recompiles. ![image](https://github.com/Unleash/unleash/assets/14320932/8bde8ee2-3cad-4e3f-a0db-9eed60cfb04d) No more splitting your terminal and treating `frontend` as a separate project. ## Discussion points Maybe there's a better alternative to `tsc-watch`? I briefly explored some alternatives and while they had a much faster starting speed, hot-reload was sometimes slower. IMO we should aspire to run `src/server-dev.ts` directly and only compile when needed. Running `dev:backend` still serves a version of the frontend (at 4242). **Why? Can we remove that behavior?** I can't imagine a scenario in dev where we wouldn't want to run the latest version of the frontend with `vite`. ~~**Note:** This PR removes all other out-of-scope scripts to focus on this revamp. If we decide to merge it, we should evaluate what other existing scripts we still want to include. May be a good opportunity to clean up unused ones and only include the ones we really use. This includes scripts that our GH actions rely on.~~ **Update:** In an effort to minimize impact surface of this PR and make it a bit more ready for merging: - It updates some docs in https://github.com/Unleash/unleash/pull/3736/commits/2a4ff805e87b65d9c1256effaa189ddcccba15fb and https://github.com/Unleash/unleash/pull/3736/commits/1bbc4882519b5a82e3116f0be255ad24a6f3ce53 to reflect our new simplified flow; - It includes the old package scripts for now in https://github.com/Unleash/unleash/pull/3736/commits/039bc04699ac880e491fd3ce01f9bcd6f97a94b9; - It updates some of our GH actions to reflect the new scripts in https://github.com/Unleash/unleash/pull/3736/commits/7782cb9b12e37ee844507e41ef2b7137eaf55666; Given its current status I'll promote the PR to "ready for review". I still think we should have a second look at our existing scripts and GH actions to see what we really need and/or should adapt, but it should be a team effort so we have a broader context. Maybe on a follow-up PR. Does this require any changes to related projects (e.g. Enterprise)? --------- Co-authored-by: Gastón Fournier <gaston@getunleash.io>
2023-05-12 12:23:22 +02:00
// Start Unleash in development
yarn dev
// Unleash UI
chore: simplify package scripts (#3736) # Simplify package scripts This PR's purpose is to raise a discussion surrounding our current package scripts. It includes some suggestions that aim to simplify the scripts and hopefully bring a much more straightforward approach to developing and contributing to Unleash. Building (prod) should only happen **explicitly** and when needed. ## Before PR (current behavior) - Clone the project; - Open 2 terminals: One for `unleash` and another for `unleash/frontend`; - On `unleash`: - Run `yarn` (which will also build, for some reason?); - Run `yarn start:dev` to start backend in dev mode (`tsc-watch`); - On `unleash/frontend`: - Run `yarn` (which will also build, for some reason?); - Run `yarn start` to start frontend in dev mode (`vite`); So it seems to me like we build unnecessarily every time we install dependencies. Neither dev scripts need to build the project, as backend uses `tsc-watch` and frontend uses `vite`. I'm unsure why this is the case, as building can take a very long time. ![image](https://github.com/Unleash/unleash/assets/14320932/5ecb7df1-e5b4-4d70-ba7e-97119f5d1116) There's also some complexity in the way we need to split the terminal to `cd` into `frontend` and treat it as a different project. The fact that we have different script names is also confusing (`yarn start`, `yarn start:dev`, etc). ## After PR - Clone the project; - Run `yarn` to install all dependencies; - Run `yarn dev` to get started developing Unleash; Running `yarn` should take care of everything needed to start developing. This includes installing dependencies for frontend as well. It should not build projects if we are not being explicit about it, especially since we don't need to build them at this stage. ![image](https://github.com/Unleash/unleash/assets/14320932/614e42fc-3467-432f-91fc-624b1b35c7c1) Running `yarn dev` should start the project in dev mode. This means running both projects in `dev` mode, which for `backend` means running `tsc-watch` and for `frontend` means running `vite`. Here this PR attempts to provide a better DX by using [concurrently](https://www.npmjs.com/package/concurrently) and [wait-on](https://www.npmjs.com/package/wait-on) - This means both tasks are ran simultaneously, stdout is labeled accordingly, and are stopped together. It also means that `frontend` waits for `backend` to be serving at `4242` before starting, since `frontend` starts pretty much immediately with `vite` and `backend` takes a bit longer. Of course, when the `backend` is hot-reloading you may still find some `ECONNREFUSED`s on `frontend` stdout while it recompiles. ![image](https://github.com/Unleash/unleash/assets/14320932/8bde8ee2-3cad-4e3f-a0db-9eed60cfb04d) No more splitting your terminal and treating `frontend` as a separate project. ## Discussion points Maybe there's a better alternative to `tsc-watch`? I briefly explored some alternatives and while they had a much faster starting speed, hot-reload was sometimes slower. IMO we should aspire to run `src/server-dev.ts` directly and only compile when needed. Running `dev:backend` still serves a version of the frontend (at 4242). **Why? Can we remove that behavior?** I can't imagine a scenario in dev where we wouldn't want to run the latest version of the frontend with `vite`. ~~**Note:** This PR removes all other out-of-scope scripts to focus on this revamp. If we decide to merge it, we should evaluate what other existing scripts we still want to include. May be a good opportunity to clean up unused ones and only include the ones we really use. This includes scripts that our GH actions rely on.~~ **Update:** In an effort to minimize impact surface of this PR and make it a bit more ready for merging: - It updates some docs in https://github.com/Unleash/unleash/pull/3736/commits/2a4ff805e87b65d9c1256effaa189ddcccba15fb and https://github.com/Unleash/unleash/pull/3736/commits/1bbc4882519b5a82e3116f0be255ad24a6f3ce53 to reflect our new simplified flow; - It includes the old package scripts for now in https://github.com/Unleash/unleash/pull/3736/commits/039bc04699ac880e491fd3ce01f9bcd6f97a94b9; - It updates some of our GH actions to reflect the new scripts in https://github.com/Unleash/unleash/pull/3736/commits/7782cb9b12e37ee844507e41ef2b7137eaf55666; Given its current status I'll promote the PR to "ready for review". I still think we should have a second look at our existing scripts and GH actions to see what we really need and/or should adapt, but it should be a team effort so we have a broader context. Maybe on a follow-up PR. Does this require any changes to related projects (e.g. Enterprise)? --------- Co-authored-by: Gastón Fournier <gaston@getunleash.io>
2023-05-12 12:23:22 +02:00
http://localhost:3000
// API:
chore: simplify package scripts (#3736) # Simplify package scripts This PR's purpose is to raise a discussion surrounding our current package scripts. It includes some suggestions that aim to simplify the scripts and hopefully bring a much more straightforward approach to developing and contributing to Unleash. Building (prod) should only happen **explicitly** and when needed. ## Before PR (current behavior) - Clone the project; - Open 2 terminals: One for `unleash` and another for `unleash/frontend`; - On `unleash`: - Run `yarn` (which will also build, for some reason?); - Run `yarn start:dev` to start backend in dev mode (`tsc-watch`); - On `unleash/frontend`: - Run `yarn` (which will also build, for some reason?); - Run `yarn start` to start frontend in dev mode (`vite`); So it seems to me like we build unnecessarily every time we install dependencies. Neither dev scripts need to build the project, as backend uses `tsc-watch` and frontend uses `vite`. I'm unsure why this is the case, as building can take a very long time. ![image](https://github.com/Unleash/unleash/assets/14320932/5ecb7df1-e5b4-4d70-ba7e-97119f5d1116) There's also some complexity in the way we need to split the terminal to `cd` into `frontend` and treat it as a different project. The fact that we have different script names is also confusing (`yarn start`, `yarn start:dev`, etc). ## After PR - Clone the project; - Run `yarn` to install all dependencies; - Run `yarn dev` to get started developing Unleash; Running `yarn` should take care of everything needed to start developing. This includes installing dependencies for frontend as well. It should not build projects if we are not being explicit about it, especially since we don't need to build them at this stage. ![image](https://github.com/Unleash/unleash/assets/14320932/614e42fc-3467-432f-91fc-624b1b35c7c1) Running `yarn dev` should start the project in dev mode. This means running both projects in `dev` mode, which for `backend` means running `tsc-watch` and for `frontend` means running `vite`. Here this PR attempts to provide a better DX by using [concurrently](https://www.npmjs.com/package/concurrently) and [wait-on](https://www.npmjs.com/package/wait-on) - This means both tasks are ran simultaneously, stdout is labeled accordingly, and are stopped together. It also means that `frontend` waits for `backend` to be serving at `4242` before starting, since `frontend` starts pretty much immediately with `vite` and `backend` takes a bit longer. Of course, when the `backend` is hot-reloading you may still find some `ECONNREFUSED`s on `frontend` stdout while it recompiles. ![image](https://github.com/Unleash/unleash/assets/14320932/8bde8ee2-3cad-4e3f-a0db-9eed60cfb04d) No more splitting your terminal and treating `frontend` as a separate project. ## Discussion points Maybe there's a better alternative to `tsc-watch`? I briefly explored some alternatives and while they had a much faster starting speed, hot-reload was sometimes slower. IMO we should aspire to run `src/server-dev.ts` directly and only compile when needed. Running `dev:backend` still serves a version of the frontend (at 4242). **Why? Can we remove that behavior?** I can't imagine a scenario in dev where we wouldn't want to run the latest version of the frontend with `vite`. ~~**Note:** This PR removes all other out-of-scope scripts to focus on this revamp. If we decide to merge it, we should evaluate what other existing scripts we still want to include. May be a good opportunity to clean up unused ones and only include the ones we really use. This includes scripts that our GH actions rely on.~~ **Update:** In an effort to minimize impact surface of this PR and make it a bit more ready for merging: - It updates some docs in https://github.com/Unleash/unleash/pull/3736/commits/2a4ff805e87b65d9c1256effaa189ddcccba15fb and https://github.com/Unleash/unleash/pull/3736/commits/1bbc4882519b5a82e3116f0be255ad24a6f3ce53 to reflect our new simplified flow; - It includes the old package scripts for now in https://github.com/Unleash/unleash/pull/3736/commits/039bc04699ac880e491fd3ce01f9bcd6f97a94b9; - It updates some of our GH actions to reflect the new scripts in https://github.com/Unleash/unleash/pull/3736/commits/7782cb9b12e37ee844507e41ef2b7137eaf55666; Given its current status I'll promote the PR to "ready for review". I still think we should have a second look at our existing scripts and GH actions to see what we really need and/or should adapt, but it should be a team effort so we have a broader context. Maybe on a follow-up PR. Does this require any changes to related projects (e.g. Enterprise)? --------- Co-authored-by: Gastón Fournier <gaston@getunleash.io>
2023-05-12 12:23:22 +02:00
http://localhost:3000/api/
// Execute tests in all packages:
yarn test
```
## Database changes {#database-changes}
We use database migrations to track database changes. Never change a migration that has been merged to main. If you need to change a migration, create a new migration that reverts the old one and then creates the new one.
### Making a schema change {#making-a-schema-change}
To run migrations, you will set the environment variable for DATABASE_URL
`export DATABASE_URL=postgres://unleash_user:password@localhost:5432/unleash`
Use db-migrate to create new migrations file.
```bash
> yarn run db-migrate create YOUR-MIGRATION-NAME
```
All migrations require one `up` and one `down` method. There are some migrations that will maintain the database integrity, but not the data integrity and may not be safe to run on a production database.
Example of a typical migration:
```js
/* eslint camelcase: "off" */
'use strict';
exports.up = function(db, cb) {
db.createTable(
'examples',
{
id: { type: 'int', primaryKey: true, notNull: true },
created_at: { type: 'timestamp', defaultValue: 'now()' },
},
cb,
);
};
exports.down = function(db, cb) {
return db.dropTable('examples', cb);
};
```
Test your migrations:
```bash
> yarn run db-migrate up
> yarn run db-migrate down
```
## Publishing / Releasing new packages {#publishing--releasing-new-packages}
Please run `yarn test` checks before publishing.
Run `npm run publish` to start the publishing process.
`npm run publish:dry`