With this commit, it is advised to `rm ./data/root_ca.crt ./data/*.pem`,
as new certificates and a root CA will be generated while utilizing the
step volume.
Closes#367
Signed-off-by: Kevin Morris <kevr@0cost.org>
This change brings some new additions to our archives:
- SHA-256 .sha256 hexdigests
- We construct our archives in a tmpdir now and move them to the
archive destination when all are completed. This removes some
corrupted downloading when archiving is in-process.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This variable allows users to override the logging.conf used
for Python logging configuration. By default, this is set
to logging.conf, which is a production config. LOG_CONFIG
is treated relative to [options] aurwebdir.
This patch allows us to specify the logging config as opposed
to copying over logging.conf in our test docker and gitlab
test scripts, as well as ease-of-testing as a developer.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Other ports we use are locked to 127.0.0.1. The `git` service,
however, already promotes security in its sshd service and
can't really be abused from an external source. This simplifies
the need to forward to localhost if deploy targets want the sshd
to be available.
Signed-off-by: Kevin Morris <kevr@0cost.org>
For the `git` service, ./data is always used to provide an
optional overriding of ssh host keys. In aur-dev production
containers, most services which use the data mount use an
internal Docker `data` volume instead.
Signed-off-by: Kevin Morris <kevr@0cost.org>
A user that can create databases is now required for tests,
we use the 'root' user in Docker.
Added docker services:
---------------------
- mariadb_test - host localhost:13307
Signed-off-by: Kevin Morris <kevr@0cost.org>
Normally, these scripts are used to update official providers
in the aurweb database along with archives that can be retrieved.
Run both of these scripts in a 5 minute cron job, to both reflect
the live instance database and production load.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Previously, we restricted this to gunicorn to get it working on aur-dev.
This change makes it usable through any backend, and also no-op if
PROMETHEUS_MULTIPROC_DIR is not defined.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Previously CGIT_CLONE_PREFIX_(PHP|FASTAPI), we found that we could
use the same env var in multiple places, including non-cgit-clone-prefix
areas.
So, they were renamed, and one additional prefix was added.
- CGIT_CLONE_PREFIX_PHP -> AURWEB_PHP_PREFIX
- Used for cgit's clone prefix and AUR_CONFIG's aur_location for PHP
- CGIT_CLONE_PREFIX_FASTAPI -> AURWEB_FASTAPI_PREFIX
- Used for cgit's clone prefix and AUR_CONFIG's aur_location for FastAPI
- AURWEB_SSHD_PREFIX
- Used for aurweb's sshd clone prefix shown on package pages
Signed-off-by: Kevin Morris <kevr@0cost.org>
Additionally, for now, no-op usage of the MARIADB_SOCKET_DIR
environment variable. This is particularly useful for a serious
production setup, but we don't currently use that.
Signed-off-by: Kevin Morris <kevr@0cost.org>
The default recursion limit used by Docker's archlinux:base-devel
Python package becomes problematic in some cases when running tests
against our FastAPI application using starlette.testclient.TestClient
(aliased to fastapi.testclient.TestClient). starlette ends up with
test failures because it exceeds the recursion limit, but this only
happens when using the `TestClient`. When the ASGI servers are run,
this is not an issue and so in that case, the recursion limit has
not been touched.
This change uses a `TEST_RECURSION_LIMIT` environment variable to
modify the recursion limit of the FastAPI application. This variable
is, by default, only supplied when running pytests in Docker, but
can be force-supplied by the user.
TEST_RECURSION_LIMIT=10000 has been added to `.env` and `.gitlab-ci.yml`.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This change exposes the uwsgi daemon we use for cgit on:
- PHP: docker-host:13000
- FastAPI: docker-host:13001
These ports can then be used to take advantage of cgit on
a production server that hosts nginx in front of Docker.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Additionally, clone-prefix is now configurable via environment variables:
- CGIT_CLONE_PREFIX_PHP
- CGIT_CLONE_PREFIX_FASTAPI
These vars can be used by production to customize the clone prefix.
Signed-off-by: Kevin Morris <kevr@0cost.org>
The new `aurweb-image` service does not perform any purpose
other than providing a build definition for 'aurweb:latest'.
With this, `docker-compose build` now just runs once for the
`aurweb-image` service, which builds the image used by all
other services.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Supply FASTAPI_BACKEND=gunicorn and FASTAPI_WORKERS=<threads_num> to
docker-compose up to use the gunicorn backend.
This is defaulted in production to gunicorn, but FASTAPI_WORKERS
should definitely be configured by any production deployment.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Provides a single source of truth for mariadb database
initialization. Previously, php-fpm and fastapi were
racing against each other; while this wasn't an issue,
it was very messy.
Signed-off-by: Kevin Morris <kevr@0cost.org>
These weren't needed at all and provided false negatives in
general. Removed them to let Docker deal with them.
Additionally. 'exit 0' -> 'echo' for ca's command; 'exit 0'
happens to depend on the shell running Docker (it seems).
echo is quite a bit more agnostic.
Moreso, added mariadb deps to php-fpm and fastapi.
Signed-off-by: Kevin Morris <kevr@0cost.org>
In addition, remove some unneeded dependencies on tests.
Though, in the future we _should_ craft tests that use these.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This script purely removes any existing sqlite and is
used before tests are run. This causes the test flow
to run `aurweb.initdb` again (if ever).
Signed-off-by: Kevin Morris <kevr@0cost.org>
This was completely bugged out. This commit fixes git, provides
two separate cgit servers for the different URL bases and also
supplies a smartgit service for $AURWEB_URL/repo.git interaction.
Docker image needs to be rebuilt with this change:
$ docker build -t aurweb:latest .
Signed-off-by: Kevin Morris <kevr@0cost.org>
This caused a bug where generated locale would not be used.
Also, removed appending to /etc/hosts which was bugging out
on Mac OS X. archlinux:base-devel seems to come with a valid
/etc/hosts.
Additionally, remove AUR_CONFIG from Dockerfile. We don't
set it up; just use the defaults during installation.
Signed-off-by: Kevin Morris <kevr@0cost.org>
By default we now use uvicorn because it has a much
better developer feedback out of the box. We'll work
on hypercorn logging, but for now, hypercorn is usable
via: `docker-compose --env-file docker/hypercorn.env up nginx`.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Before, docker build was the only way to transfer new code
over to the docker image. This allows users to execute code
in their working directory.
Signed-off-by: Kevin Morris <kevr@0cost.org>
