Mariadb has scheduled to remove the deprecated mysql drop-in interface.
Let's adapt which also removes a lot of warnings while spinning up the
service.
This is a very useful stack for local development as well, by allowing
to easily access a local grafana instance and look at the accessed
endpoints, query usage and durations etc.
As a nice side effect this also makes sure we have an easy way to
actually test any changes to the opentelemetry integration in an actual
environment instead of just listening to a raw socket.
We want the dependent services to wait until the initialization service
of mariadb finishes, but also properly accept if it already exited
before a leaf service gets picked up and put into created state. By
using the service_completed_successfully signal, we can ensure precisely
this, without being racy and leading to none booted services.
While at it, remove the compose version identifiers as docker-compose
deprecated them and always warned about when running docker-compose.
With this commit, it is advised to `rm ./data/root_ca.crt ./data/*.pem`,
as new certificates and a root CA will be generated while utilizing the
step volume.
Closes#367
Signed-off-by: Kevin Morris <kevr@0cost.org>
This change brings some new additions to our archives:
- SHA-256 .sha256 hexdigests
- We construct our archives in a tmpdir now and move them to the
archive destination when all are completed. This removes some
corrupted downloading when archiving is in-process.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This variable allows users to override the logging.conf used
for Python logging configuration. By default, this is set
to logging.conf, which is a production config. LOG_CONFIG
is treated relative to [options] aurwebdir.
This patch allows us to specify the logging config as opposed
to copying over logging.conf in our test docker and gitlab
test scripts, as well as ease-of-testing as a developer.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Other ports we use are locked to 127.0.0.1. The `git` service,
however, already promotes security in its sshd service and
can't really be abused from an external source. This simplifies
the need to forward to localhost if deploy targets want the sshd
to be available.
Signed-off-by: Kevin Morris <kevr@0cost.org>
For the `git` service, ./data is always used to provide an
optional overriding of ssh host keys. In aur-dev production
containers, most services which use the data mount use an
internal Docker `data` volume instead.
Signed-off-by: Kevin Morris <kevr@0cost.org>
A user that can create databases is now required for tests,
we use the 'root' user in Docker.
Added docker services:
---------------------
- mariadb_test - host localhost:13307
Signed-off-by: Kevin Morris <kevr@0cost.org>
Normally, these scripts are used to update official providers
in the aurweb database along with archives that can be retrieved.
Run both of these scripts in a 5 minute cron job, to both reflect
the live instance database and production load.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Previously, we restricted this to gunicorn to get it working on aur-dev.
This change makes it usable through any backend, and also no-op if
PROMETHEUS_MULTIPROC_DIR is not defined.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Previously CGIT_CLONE_PREFIX_(PHP|FASTAPI), we found that we could
use the same env var in multiple places, including non-cgit-clone-prefix
areas.
So, they were renamed, and one additional prefix was added.
- CGIT_CLONE_PREFIX_PHP -> AURWEB_PHP_PREFIX
- Used for cgit's clone prefix and AUR_CONFIG's aur_location for PHP
- CGIT_CLONE_PREFIX_FASTAPI -> AURWEB_FASTAPI_PREFIX
- Used for cgit's clone prefix and AUR_CONFIG's aur_location for FastAPI
- AURWEB_SSHD_PREFIX
- Used for aurweb's sshd clone prefix shown on package pages
Signed-off-by: Kevin Morris <kevr@0cost.org>
Additionally, for now, no-op usage of the MARIADB_SOCKET_DIR
environment variable. This is particularly useful for a serious
production setup, but we don't currently use that.
Signed-off-by: Kevin Morris <kevr@0cost.org>
The default recursion limit used by Docker's archlinux:base-devel
Python package becomes problematic in some cases when running tests
against our FastAPI application using starlette.testclient.TestClient
(aliased to fastapi.testclient.TestClient). starlette ends up with
test failures because it exceeds the recursion limit, but this only
happens when using the `TestClient`. When the ASGI servers are run,
this is not an issue and so in that case, the recursion limit has
not been touched.
This change uses a `TEST_RECURSION_LIMIT` environment variable to
modify the recursion limit of the FastAPI application. This variable
is, by default, only supplied when running pytests in Docker, but
can be force-supplied by the user.
TEST_RECURSION_LIMIT=10000 has been added to `.env` and `.gitlab-ci.yml`.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This change exposes the uwsgi daemon we use for cgit on:
- PHP: docker-host:13000
- FastAPI: docker-host:13001
These ports can then be used to take advantage of cgit on
a production server that hosts nginx in front of Docker.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Additionally, clone-prefix is now configurable via environment variables:
- CGIT_CLONE_PREFIX_PHP
- CGIT_CLONE_PREFIX_FASTAPI
These vars can be used by production to customize the clone prefix.
Signed-off-by: Kevin Morris <kevr@0cost.org>
The new `aurweb-image` service does not perform any purpose
other than providing a build definition for 'aurweb:latest'.
With this, `docker-compose build` now just runs once for the
`aurweb-image` service, which builds the image used by all
other services.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Supply FASTAPI_BACKEND=gunicorn and FASTAPI_WORKERS=<threads_num> to
docker-compose up to use the gunicorn backend.
This is defaulted in production to gunicorn, but FASTAPI_WORKERS
should definitely be configured by any production deployment.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Provides a single source of truth for mariadb database
initialization. Previously, php-fpm and fastapi were
racing against each other; while this wasn't an issue,
it was very messy.
Signed-off-by: Kevin Morris <kevr@0cost.org>
These weren't needed at all and provided false negatives in
general. Removed them to let Docker deal with them.
Additionally. 'exit 0' -> 'echo' for ca's command; 'exit 0'
happens to depend on the shell running Docker (it seems).
echo is quite a bit more agnostic.
Moreso, added mariadb deps to php-fpm and fastapi.
Signed-off-by: Kevin Morris <kevr@0cost.org>
In addition, remove some unneeded dependencies on tests.
Though, in the future we _should_ craft tests that use these.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This script purely removes any existing sqlite and is
used before tests are run. This causes the test flow
to run `aurweb.initdb` again (if ever).
Signed-off-by: Kevin Morris <kevr@0cost.org>
This was completely bugged out. This commit fixes git, provides
two separate cgit servers for the different URL bases and also
supplies a smartgit service for $AURWEB_URL/repo.git interaction.
Docker image needs to be rebuilt with this change:
$ docker build -t aurweb:latest .
Signed-off-by: Kevin Morris <kevr@0cost.org>
This caused a bug where generated locale would not be used.
Also, removed appending to /etc/hosts which was bugging out
on Mac OS X. archlinux:base-devel seems to come with a valid
/etc/hosts.
Additionally, remove AUR_CONFIG from Dockerfile. We don't
set it up; just use the defaults during installation.
Signed-off-by: Kevin Morris <kevr@0cost.org>
