diff --git a/Docker.md b/Docker.md index d5231bf..a889607 100644 --- a/Docker.md +++ b/Docker.md @@ -20,25 +20,87 @@ pipeline/coverage status table below: ## Services -| Service | Port | Profile | Purpose | -|-----------------|---------|---------|---------------------------------| -| `ca` | | | Self-signed CA generation | -| `memcached` | | | Live memcached service | -| `redis` | `16379` | | Live Redis service | -| `mariadb` | `13306` | | Live MariaDB service | -| `mariadb_init` | | | MariaDB initialization | -| `git` | `2222` | | AUR Git repository | -| `smartgit` | | | HTTP Git frontend | -| `cgit-php` | `13000` | | CGit backend (PHP) | -| `cgit-fastapi` | `13001` | | CGit backend (FastAPI) | -| `php-fpm` | `19000` | | Live php-fpm service | -| `fastapi` | `18000` | | Live `$FASTAPI_BACKEND` service | -| `nginx` | `8443` | | Live nginx frontend to php-fpm | -| `nginx` | `8444` | | Live nginx frontend to fastapi | -| `sharness` | | `dev` | Run sharness test suites | -| `pytest-mysql` | | `dev` | Run pytest suites with mariadb | -| `pytest-sqlite` | | `dev` | Run pytest suites with sqlite | -| `test` | | `dev` | Run all tests and linters | +| Service | Host Bind | Purpose | +|-----------------|-------------------|---------------------------------| +| `ca` | | Self-signed CA generation | +| `memcached` | | Live memcached service | +| `redis` | `127.0.0.1:16379` | Live Redis service | +| `mariadb` | `127.0.0.1:13306` | Live MariaDB service | +| `mariadb_test` | `127.0.0.1:13307` | Live test MariaDB service | +| `mariadb_init` | | `mariadb` initialization | +| `git` | `0.0.0.0:2222` | AUR Git SSH interface | +| `smartgit` | | HTTP Git frontend | +| `cgit-php` | `127.0.0.1:13000` | CGit backend (PHP) | +| `cgit-fastapi` | `127.0.0.1:13001` | CGit backend (FastAPI) | +| `php-fpm` | `127.0.0.1:19000` | Live php-fpm service | +| `fastapi` | `127.0.0.1:18000` | Live `$FASTAPI_BACKEND` service | +| `nginx` | `127.0.0.1:8443` | Live nginx frontend to php-fpm | +| `nginx` | `127.0.0.1:8444` | Live nginx frontend to fastapi | +| `sharness` | | Run sharness test suites | +| `pytest-mysql` | | Run pytest suites with mariadb | +| `pytest-sqlite` | | Run pytest suites with sqlite | +| `test` | | Run all tests and linters | + +## Profiles + +We have two kinds of profiles: + +- [default](#profile-default) (Development) +- [aur-dev](#profile-aur-dev) (Production) + +#### Profile: default + +This profile provides support for the following environment variables: + +- `FASTAPI_BACKEND` + - Default: `uvicorn` + - Choices: `uvicorn`, `hypercorn`, `gunicorn` + - Controls the backend used for the `fastapi` service. In production, + `gunicorn` is recommended. +- `FASTAPI_WORKERS` + - Default: `2` + - Only used when `FASTAPI_BACKEND=gunicorn`; controls the number of + workers used. +- `AURWEB_PHP_PREFIX` + - Default: `https://localhost:8443` + - Used to populate `aur_location` in the `php-fpm` service. + - Used to populate `git_clone_uri_anon` in the `php-fpm` service. +- `AURWEB_FASTAPI_PREFIX` + - Default: `https://localhost:8444` + - Used to populate `aur_location` in the `fastapi` service. + - Used to populate `git_clone_uri_anon` in the `fastapi` service. +- `AURWEB_SSHD_PREFIX` + - Default: `ssh://aur@localhost:2222` + - Used to populate `git_clone_uri_priv`. +- `TEST_RECURSION_LIMIT` + - Default: `10000` + - This should be left alone. It is used to configure the recursion + limit used for tests. +- `COMMIT_HASH` + - Default: empty + - When set, used to produce a link to the specified commit hash + in the https://gitlab.archlinux.org/archlinux/aurweb repository. + +#### Profile: aur-dev + +To use this profile, execute docker-compose with the following: + + $ docker-compose -f docker-compose.yml -f docker-compose.aur-dev.yml ... + +This profile provides support for the following environment variables, +along with any inherited from [default](#profile-default): + +- `MARIADB_SOCKET_DIR` + - Default: `/var/run/mysqld` + - Bind mount a host mariadb instance socket directory. +- `GIT_DATA_DIR` + - Default: `./aur.git` + - Bind mount a Git repository on the host filesystem. + +This profile **does not** bind mount aurweb repository components +for live reloading like the [default](#profile-default) profile does. +This means that you'll have to rebuild the Docker image for any +local filesystem changes to take effect. ## Getting Started @@ -56,12 +118,6 @@ Users will notice initially starting up a service can take some time, especially $ docker-compose run test # First run with no up'd services, takes a while. $ docker-compose run test # Starts instantly. -## Continued Imaging - -After building, users do not need to rebuild the image for code changes in the `aurweb` package. Our `fastapi` Docker service takes advantage of hot reload and has shared volume access to `$aurwebdir/aurweb/`. As a user makes changes to the `aurweb` package, the `fastapi` service automatically reloads with the new code. See the `fastapi` service in `docker-compose.yml` for a complete list of shared local directory volumes. - -**NOTE** However, an image rebuild is necessary whenever Docker infrastructure changes. This includes `poetry` dependencies or `$aurwebdir/docker/` modifications. - ## Development During development, users may wish to run instances of FastAPI and PHP side by side. For this, users can depend on the `nginx` service, which hosts PHP at https://localhost:8443 and FastAPI at https://localhost:8444. @@ -75,19 +131,14 @@ For an all in one testing strategy, the `test` service should be preferred, as i Our test suites also provide coverage data via a shared volume directory at `$aurwebdir/cache/`. After tests are complete, users can copy coverage data output by Docker into their local setup by executing `./util/fix-coverage ./cache/.coverage`. After this is done, users can continue on with standard coverage execution: $ docker-compose run pytest-mysql - ... - $ ./util/fix-coverage ./cache/.coverage - $ coverage report - $ coverage html - $ htmlcov/index.html -Test services which provide coverage data: `pytest-mysql`, `pytest-sqlite`, and `test`. +Test services which provide coverage data: `pytest-mysql` and `test`. ## Production -For production, the [docker-compose.prod.yml](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/docker-compose.prod.yml) should be used in unison with the standard `docker-compose.yml`: +For production, the [docker-compose.aur-dev.yml](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/docker-compose.aur-dev.yml) profile config should be used in unison with the standard `docker-compose.yml`: - $ docker-compose -f docker-compose.yml -f docker-compose.prod.yml up -d + $ docker-compose -f docker-compose.yml -f docker-compose.aur-dev.yml up -d This provides service overrides which mainly affect volume behavior. In production, volumes will not be shared with docker from the host. @@ -142,30 +193,12 @@ host to container. In production, the `cgit-php` and `cgit-fastapi` services should be configured to use the intended `clone-prefix`. -- `CGIT_CLONE_PREFIX_PHP` +- `AURWEB_PHP_PREFIX` - Defaulted to `https://localhost:8443` -- `CGIT_CLONE_PREFIX_FASTAPI` +- `AURWEB_FASTAPI_PREFIX` - Defaulted to `https://localhost:8444` - -#### git-bridge (Work In Progress) - -In production, hosts should be able to run a instance of sshd in -front of Docker, which allows standard authentication to the host -in addition to customized authentication and shell for a particular -user (`aur` by default). - -For this reason, we must provide a persistent endpoint in a Docker -service which bridges to `aurweb-git-auth`. This bridge must customize -output when it is invoking the script via the `git-bridge` RPC API. - -This way, the Docker host could configure sshd's `AuthorizedKeysCommand` -to contact this endpoint for auth negotiation, which would tunnel through -to `aurweb-git-serve` when successful. - -This feature is being worked on and has had success thus far, however, -it is not yet available and must be simplified as it stands. This section -will be updated when the feature comes to completion and can be used -by production. +- `AURWEB_SSHD_PREFIX` + - Defaulted to `ssh://aur@localhost:2222` #### fastapi @@ -200,7 +233,7 @@ The `nginx` service should **not be used in production**. When running aurweb in a production Docker instance, users should configure an instance in front of Docker to use exposed services as backends: -- `cgit-php` - port `13000` -- `php-fpm` - port `19000` -- `cgit-fastapi` - port `13001` -- `fastapi` - port `18000` +- `cgit-php` @ `127.0.0.1:13000` +- `php-fpm` @ `127.0.0.1:19000` +- `cgit-fastapi` @ `127.0.0.1:13001` +- `fastapi` @ `127.0.0.1:18000` \ No newline at end of file