The install method in Python’s gettext API aliases the translator’s
gettext method to an application-global _(). We don’t use that anywhere,
and it’s clear from aurweb’s Translator interface that we want to
translate a piece of text without affecting any global namespace.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Developers can go to /sso/login to get redirected to the SSO. On
successful login, the ID token is displayed.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
aurweb.spawn used to launch only PHP’s built-in server. Now it spawns a
dummy FastAPI application too. Since both stacks spawn their own HTTP
server, aurweb.spawn also spawns nginx as a reverse proxy to mount them
under the same base URL, defined by aur_location in the configuration.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
This program makes it easier for developers to spawn the PHP server
since it fetches automatically what it needs from the configuration
file, rather than having the user explicitly pass arguments to the php
executable.
When the setup gets more complicated as we introduce Python,
aurweb.spawn will keep providing the same interface, while under the
hood it is planned to support running multiple sub-processes.
Its Python interface provides an way for the test suite to spawn the
test server when it needs to perform HTTP requests to the test server.
The current implementation is somewhat weak as it doesn’t detect when a
child process dies, but this is not supposed to happen often, and it is
only meant for aurweb developers.
In the long term, aurweb.spawn will eventually become obsolete, and
replaced by Docker or Flask’s tools.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
When using the sendmail() function with smtplib.SMTP or
smtplib.SMTP_SSL, the list of actual recipients for the email (to be
translated to RCPT commands) has to be provided as a parameter.
Update the notification script and add all Cc recipients to that
parameter.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
After f7a57c8 (Localize notification emails, 2018-05-17), the
server.sendmail line was not updated to now send the to the email
address but instead sends to (email, 'en') and as sendmail accepts an
iterable an email is also send to 'en'.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
The new schema was generated with sqlacodegen and then manually adjusted
to fit schema/aur-schema.sql faithfully, both in the organisation of the
code and in the SQL generated by SQLAlchemy.
Initializing the database now requires the new tool aurweb.initdb.
References to aur-schema.sql have been updated and the old schema
dropped.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Add more options to configure the smtplib implementation for sending
notification emails.
The port can be changed using the new smtp-port option.
Encryption can be configured using smtp-use-ssl and smtp-use-starttls.
Keep in mind that you usually also need to change the port when enabling
either of these options.
Authentication can be configured using smtp-user and smtp-password.
Authentication is disabled if either of these values is empty.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Support mail delivery without a local MTA. Instead, an SMTP server can
now be configured using the smtp-server option in the [notifications]
section. In order to use this option, the value of the sendmail option
must be empty.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
First, this gets rid of the deprecation warnings Python displayed.
Second, this fixes the case where a link contained a pair of
underscores, which used to be interpreted as an emphasis because the
linkify processor ran after the emphasis processor.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
When an FS#123 is part of a code block, it must not be converted into a
link. FS#123 may also appear inside an URL, in which case regular
linkifaction of URLs must take precedence.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Fixes a few edge cases:
- URLs within code blocks used to get redundant <> added, breaking bash
code snippets like `curl https://...` into `curl <https://...>`.
- Links written with markdown's <https://...> syntax also used to get an
extra pair of brackets.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Turn the git-commits markdown processor into an inline processor, which
is smart enough not to convert Git hashes contained in code blocks or
links.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Support secondary email addresses that can be used to recover an account
in case access to the primary email address is lost. Reset keys for an
account are always sent to both the primary and the backup email
address.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Since commit eeaa1c3 (Separate text from footer in notification emails,
2020-01-04), information about unsubscribing from notifications is added
in a signature block. However, the code to format the email body trimmed
the RFC 3676 signature delimiter, replacing "-- " by "--". Fix this by
adding a special case for signature delimiters.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
We support multiple database backends. Don't require Python modules for
all backends to be installed.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Verify that the update hook exists and is executable before running Git
to prevent from broken repositories when permissions are broken.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Will no longer send notifications about "a orphan request", but determine
whether to use a/an based on the first character of the request type.
Signed-off-by: Lars Rustand <rustand.lars@gmail.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
After creating a new account, users need to verify their email address
and set an initial password. Without setting a password, users cannot
use their account on the web interface. However, when logging in via
SSH, we did not check whether the account is verified.
Fix this by only allowing SSH access once a password is set.
Reported-by: Pat Hogan <pathtofile@gmail.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Reverse the order of deletion and addition so that deletion comes first.
This prevents corner cases such as failing unique key constraints when a
provided package changes from lower case to upper case and the old name
is not yet gone.
Helped-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
In commit f3b4c5c (Refactor the notification script, 2018-05-17), the
parameters of the adopt, disown, comaintainer-add and
comaintainer-remove notification modules were accidentally pushed around
without changing the order in the callers. The notify script now expects
to see the userid followed by additional arguments like the pkgbase id.
As a result, some random userid with the same id as the pkgbase, got
sent a notification regarding some package with the same id as the real
user's id.
Fix this by changing the order in every invocation of the aforementioned
modules.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Add a new configuration option to specify the locale directory to use.
This allows the Python scripts to find the translations, even when not
being run from the source code checkout. At the same time, multiple
parallel aurweb setups can still use different sets of translations.
Fixes FS#59278.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
In commit 840ee20 (Rename translation resources from aur to aurweb,
2018-07-07) the translations file was renamed but we never actually
switched to using the renamed translations.
As a result, every single push to the AUR contains the following
traceback:
remote: Traceback (most recent call last):
remote: File "/usr/bin/aurweb-notify", line 11, in <module>
remote: load_entry_point('aurweb==4.7.0', 'console_scripts', 'aurweb-notify')()
remote: File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/scripts/notify.py", line 541, in main
remote: File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/scripts/notify.py", line 69, in send
remote: File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/scripts/notify.py", line 56, in get_body_fmt
remote: File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/scripts/notify.py", line 192, in get_body
remote: File "/usr/lib/python3.6/site-packages/aurweb-4.7.0-py3.6.egg/aurweb/l10n.py", line 14, in translate
remote: File "/usr/lib/python3.6/gettext.py", line 514, in translation
remote: raise OSError(ENOENT, 'No translation file found for domain', domain)
remote: FileNotFoundError: [Errno 2] No translation file found for domain: 'aur'
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Mention both the package base name and the request type in the subject
of request closure notification.
Implements FS#41607.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Currently we hardcode the architectures the official repos historically
supported, which seems both inefficient because of hardcoding, and
simply wrong, because many packages support various ARM platforms too.
If we were to say "only officially supported arches will be supported in
the AUR" we'd have to disable i686, which seems silly and arbitrarily
restrictive. Also there's better places to implement such a blacklist
(via die_commit in the main loop, via a config option to list supported
arches, would make much more sense in terms of logic).
As for the metadata extraction itself, there's no reason to hardcode the
arches to check for at all. We can get this information too, from the
.SRCINFO itself. Detecting this dynamically is not incompatible with a
blacklist, should we ever decide to implement such a thing.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
User modern Python format() strings with curly braces. Also, convert all
placeholders to named arguments. This allows translators to reorder
messages.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Add support for translating notification emails and send localized
notifications, based on the user's language preferences. Also, update
the translations Makefile to add strings from the notification script
to the message catalog.
Implements FS#31850.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Reimplement most of the notification script logic. Create a separate
class for each notification type. Each class provides methods for
generating the list of recipients, the message subject, the message
body, the references to add at the end of the message and the message
headers. Additionally, a method for sending notification emails is
provided.
One major benefit of the new implementation is that both the generation
of recipients and message contents are much more flexible. For example,
it is now easily possible to make user-specific adjustments to every
single notification of a batch.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Add a script to periodically remove old IP addresses from the users
database.
The login IP addresses are stored for spam protection and to prevent
from abuse. It is quite unlikely that we ever need the IP address of a
user whose last login is more than a week old. It makes sense to remove
such IP addresses to protect our users' privacy.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
In the process, rename config.proto to config.defaults (because that is
what it is now).
Also use dict.get('key', default_value) when querying os.environ, rather
than an if block, as it is more pythonic/readable/concise, and reduces
the number of dict lookups.
This change allows aurweb configuration to be done via either:
- copying config.defaults to config and modifying values
- creating a new config only containing modified values, next to a
config.defaults containing unmodified values
The motivation for this change is to enable ansible configuration in our
flagship deployment by storing only changed values, and deferring to
config.defaults otherwise.
A side benefit is, it is easier to see what has changed by inspecting
only the site configuration file.
If a config.defaults file does not exist next to $AUR_CONFIG or in
$AUR_CONFIG_DEFAULTS, it is ignored and *all* values are expected to
live in the modified config file.
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
When sanitizing rendered comments, keep <hr> tags and <br> tags. The
former are generated when using "---" in Markdown comments, the latter
are used when putting two spaces at the end of a line.
Fixes FS#56649.
git/auth is run as an AutherizedKeysCommand which does not get the
environment variables passed to it, so AUR_OVERWRITE always got
hard-set to '0' by it. Instead we need to perform the actual privilege
check in git/update instead.
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Automatically detect references to Flyspray bug reports in comments and
convert them to links to the Arch Linux bug tracker.
Implements FS#52008.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Use `/usr/bin/env python3` instead of `/usr/bin/python3` in the shebang
of Python scripts. This adds support for non-standard Python interpreter
paths such as the paths used in virtualenv environments.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Add a missing space to the SQL statement performing the disown
operation.
Fixes FS#55068.
Note that the broken query was not discovered by the test suite since
SQLite parses "?AND" inside prepared statements gracefully while MySQL
does not.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
In addition to the packages list and the package base list, also create
a list of registered user names.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
AUR_PRIVILEGED allows people with privileged AUR accounts to evade the
block on non-fast-forward commits. While valid in this case, we should
not do so by default, since in at least one case a TU did this without
realizing there was an existing package.
( https://aur.archlinux.org/packages/rtmidi/ )
Switch to using allow_overwrite to check for destructive actions.
Use .ssh/config "SendEnv" on the TU's side and and sshd_config
"AcceptEnv" in the AUR server to specifically request overwrite access.
TUs should use: `AUR_OVERWRITE=1 git push --force`
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
