We heavily attempt to provide easy use of poetry virtualenvs
with aurweb in this revision of the INSTALL file. Added a
section about cron jobs and updated the nginx config example
with a lot more detail and locations for other parts of
the AUR infrastructure.
Signed-off-by: Kevin Morris <kevr@0cost.org>
As the new-age Python package manager, Poetry brings a lot
of good additions to the table. It allows us to more easily
deal with virtualenvs for the project and resolve dependencies.
As of this commit, `requirements.txt` is replaced by Poetry,
configured at `pyproject.toml`.
In Docker and GitLab, we currently use Poetry in a root fashion.
We should work toward purely using virtualenvs in Docker, but,
for now we'd like to move forward with other things. The project
can still be installed to a virtualenv and used on a user's system
through Poetry; it is just not yet doing so in Docker.
Modifications:
* docker/scripts/install-deps.sh
* Remove python dependencies.
* conf/config.defaults
* Script paths have been updated to use '/usr/bin'.
* docker/git-entrypoint.sh
* Use '/usr/bin/aurweb-git-auth' instead of
'/usr/local/bin/aurweb-git-auth'.
Additions:
* docker/scripts/install-python-deps.sh
* A script used purely to install Python dependencies with Poetry.
This has to be used within the aurweb project directory and
requires system-wide dependencies are installed beforehand.
* Also upgrades system-wide pip.
Signed-off-by: Kevin Morris <kevr@0cost.org>
python-orjson speeds up a lot of JSON serialization steps,
so we choose to use it over the standard library json module.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This includes the addition of the python-fakeredis package,
used for stubbing python-redis when a user does not have a
configured cache.
Signed-off-by: Kevin Morris <kevr@0cost.org>
+ Mounted static files (at web/html) to /static.
+ Added AURWEB_VERSION to aurweb.config (this is used around HTML
to refer back to aurweb's release on git.archlinux.org), so we
need it easily accessible in the Python codebase.
+ Implemented basic Jinja2 partials to put together whole aurweb
pages. This may be missing some things currently and is a WIP
until this set is ready to be merged.
+ Added config [options] aurwebdir = YOUR_AUR_ROOT; this configuration
option should specify the root directory of the aurweb project.
It is used by various parts of the FastAPI codebase to target
project directories.
Added routes via aurweb.routers.html:
* POST /language: Set your session language.
* GET /favicon.ico: Redirect to /static/images/favicon.ico.
* Some browsers always look for $ROOT/favicon.ico to get an icon
for the page being loaded, regardless of a specified "shortcut
icon" given in a <link> directive.
* GET /: Home page; WIP.
* Updated aurweb.routers.html.language passes query parameters to
its next redirection.
When calling aurweb.templates.render_template, the context passed should
be formed via the aurweb.templates.make_context. See
aurweb.routers.html.index for an example of this.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Additionally, we now ask for two more favors from contributors:
1. All source modified or added within a patchset **must** maintain
equivalent or increased coverage by providing tests that use the
functionality.
2. Please keep your source within an 80 column width.
PS: Sneak a few test Makefile and gitlab fixes.
Signed-off-by: Kevin Morris <kevr@0cost.org>
uvicorn is subjectively nicer to play with for local dev work, but
hypercorn is required in order to do HTTP/2 which is fairly
performance-important.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Co-authored-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
The new schema was generated with sqlacodegen and then manually adjusted
to fit schema/aur-schema.sql faithfully, both in the organisation of the
code and in the SQL generated by SQLAlchemy.
Initializing the database now requires the new tool aurweb.initdb.
References to aur-schema.sql have been updated and the old schema
dropped.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
In the process, rename config.proto to config.defaults (because that is
what it is now).
Also use dict.get('key', default_value) when querying os.environ, rather
than an if block, as it is more pythonic/readable/concise, and reduces
the number of dict lookups.
This change allows aurweb configuration to be done via either:
- copying config.defaults to config and modifying values
- creating a new config only containing modified values, next to a
config.defaults containing unmodified values
The motivation for this change is to enable ansible configuration in our
flagship deployment by storing only changed values, and deferring to
config.defaults otherwise.
A side benefit is, it is easier to see what has changed by inspecting
only the site configuration file.
If a config.defaults file does not exist next to $AUR_CONFIG or in
$AUR_CONFIG_DEFAULTS, it is ignored and *all* values are expected to
live in the modified config file.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Change the defines to config_get and add one cache option and one option
to define memcache_servers. Mention the required dependency to get
memcached working in the INSTALL file.
Signed-off-by: Jelle van der Waa <jelle@vdwaa.nl>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
To people unfamiliar with the code, it is not obvious that
the pdo_* PHP extensions must be enabled.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Add installation instructions for python-bleach and python-markdown
which are required for the rendercomment script.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
AUR_PRIVILEGED allows people with privileged AUR accounts to evade the
block on non-fast-forward commits. While valid in this case, we should
not do so by default, since in at least one case a TU did this without
realizing there was an existing package.
( https://aur.archlinux.org/packages/rtmidi/ )
Switch to using allow_overwrite to check for destructive actions.
Use .ssh/config "SendEnv" on the TU's side and and sshd_config
"AcceptEnv" in the AUR server to specifically request overwrite access.
TUs should use: `AUR_OVERWRITE=1 git push --force`
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Add instructions to test aurweb's web interface via the PHP built-in web
server.
Signed-off-by: Mark Weiman <mark.weiman@markzz.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Add a note to the Git/SSH interface documentation that we recommend to
disable automatic garbage collection and use a maintenance script to
cleanup and optimize the Git repository instead.
Also, add a reference to the Git/SSH interface documentation to the Git
repository setup instructions in INSTALL.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Since d4fe77a (Reorganize Git interface scripts, 2016-10-08), the key
components of the aurweb SSH interface are installed system-wide. Update
the default configuration path to point to a central location.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Move the Git interface scripts from git-interface/ to aurweb/git/. Use
setuptools to automatically create wrappers which can be installed using
`python3 setup.py install`. Update the configuration files, the test
suite as well as the INSTALL and README files to reflect these changes.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
In 002d348 (Describe how to omit "have" lines, 2015-11-14), we added
instructions on how to omit "have" lines originating from other package
repositories. Fix those instructions such that the HEAD ref of the
repository is transferred properly.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
python-srcinfo is a more transparent and simpler library for parsing
SRCINFO files.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
A new feature in Git allows for omitting "have" lines corresponding to
refs outside the current Git namespace. Explain how to enable this
feature in the INSTALL instructions and in the Git interface
documentation.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Using uWSGI for the Smart HTTP protocol caused some issues, see e.g.
FS#45428. Suggest using fcgiwrap instead which is more lightweight, has
better documentation and is easier to debug.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Specifically check for URIs git-http-backend(1) can handle. This also
allows us to make the ".git" suffix optional.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
To make sure we never lose any history, non-fast-forwards are forbidden.
Instead of relying on receive.denyNonFastForwards, add a simple check to
the update hook. This has the added benefit of more flexibility.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Instead of using one Git repository per package, use a single large
object storage for space efficiency. The refs of the individual package
bases are divided using gitnamespaces(7) which allows for exposing each
namespace as an independent repository easily. Also, git-serve is
modified to create a branch for each package, allowing to browse the
large repository with cgit.
Helped-by: Florian Pritz <bluewind@xinu.at>
Helped-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Use the latest version of Damien Miller's patch to extend the parameters
to the AuthorizedKeysCommand.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Rename the project to help differentiate between the software providing
access to the Arch User Repository and the collection of source packages
itself.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Do not waste disk space by copying dozens of unneeded sample hooks. Use
a custom template directory that only includes the git-update hook.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
* Add more details on how to configure the web server.
* Clarify that the OpenSSH patch is build against OpenSSH 6.7p1.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
There aren't any other subdirectories in support/. Reduce the nesting
depth by moving schema/ to the top-level source directory.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
