Commit graph

3628 commits

Author SHA1 Message Date
Kevin Morris
ebb333565e
update INSTALL: asgi-driven aurweb direction
We heavily attempt to provide easy use of poetry virtualenvs
with aurweb in this revision of the INSTALL file. Added a
section about cron jobs and updated the nginx config example
with a lot more detail and locations for other parts of
the AUR infrastructure.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-17 14:18:28 -08:00
Kevin Morris
e5dfd53b9a
examples: poetry-driven git scripts & aurweb service
This introduces examples of a gunicorn systemd service in
addition to git script wrappers that can be used for poetry
virtualenv-driven installations.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-17 14:05:39 -08:00
Kevin Morris
290ef1a2ed
fix(gitlab-ci): remove logging.prod.conf copy
No longer needed; logging.conf, which is the default config
used, is now setup for production INFO logging.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-17 12:27:06 -08:00
Kevin Morris
bf4662e26f
change(logging): restrict logging.conf & add logging.test.conf
We'll override logging.conf with logging.test.conf for debug logging
needed for tests now, so we can rely on the default logging.conf
for production use.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-17 12:11:08 -08:00
Kevin Morris
cce9385fb1
fix(db): remove debug logging of dbname
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-17 12:11:02 -08:00
Kevin Morris
c07c40bcb6
fix: clean up package action templates (merge, delete, disown)
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-17 11:53:05 -08:00
Kevin Morris
d94e2dc9d7
feat(poetry): add srcinfo
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-16 18:16:42 -08:00
Kevin Morris
eb59cbaa39
change(python): use transaction query in BasicAuthBackend
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-16 02:01:52 -08:00
Kevin Morris
64069b9b5d
change(python): use a transaction query in get_pkg_or_base
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-16 02:00:41 -08:00
Kevin Morris
9441f4f904
change(python): move test_requests tests to their own suite
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-15 21:52:53 -08:00
Kevin Morris
3e3706911c
change(python): move test_pkgbase tests to their own suite
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-15 21:43:23 -08:00
Kevin Morris
42aa12d075
fix(docker): unrestrict --forwarded-allow-ips on (uvi|hyper)corn
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-15 21:04:47 -08:00
Kevin Morris
b092e247fc
fix(docker): update keyring before installing deps
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-15 20:21:40 -08:00
Kevin Morris
34a29df1a8
fix(docker): remove fastapi rewrite rule
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-15 20:19:58 -08:00
Kevin Morris
0f4ead759c
fix(docker): correct proxy configuration
- On non-localhost communication, this whitelists forwarded headers
  on all remote ips
- Add more headers
- Force https X-Forwarded-Proto
- Unset Forwarded header and rely on X-Forwarded-*

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-15 20:11:51 -08:00
Kevin Morris
ec3295ffd4
fix(docker): update archlinux-keyring prior to -Syu
When the Docker image is outdated, we need to fetch updated
archlinux-keyring keys to perform an -Syu without problems.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-15 15:18:23 -08:00
Kevin Morris
88cb1096c0
feat(docker): add more cron scripts
Added the rest:
- aurweb-pkgmaint
- aurweb-usermaint
- aurweb-tuvotereminder
- aurweb-popupdate

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-14 01:02:00 -08:00
Kevin Morris
b4495a49bf
fix(rpc): improve type=search performance
This patch brings in the use of .with_entities on our
RPC search query. This primarily fixes performance issues
we were seeing with large queries.

That being said, we do see a bit of a slowdown on
large record count rpc queries, but it's quite negligible
at this point.

We still do aim to perform better than the older PHP
implementation, so this is not a finishing patch by
any means.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-14 00:50:39 -08:00
Kevin Morris
d31a51742b
fix(gitlab-ci): compile asciidoc
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-13 22:16:17 -08:00
Kevin Morris
c4ea1171cd
fix(docker): compile doc during image build
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-13 22:16:17 -08:00
Kevin Morris
43b7fdb61d
fix(rpc): display rpc doc when no query string is provided
Closes #255

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-13 22:16:13 -08:00
Kevin Morris
60ae676075
fix(util): catch homepage validation exceptions
We were allowing erroneous URLs through, raising exceptions,
from e.g. `http://[localhost:8444/blah`. This patch catches
any ValueErrors raised during the parse process and returns
False, indicating that the validation failed.

This patch also adds testing specifically for `util.valid_homepage`.
We didn't have specific testing for this before; this will allow us
to catch regressions in this area.

Closes #250

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-13 19:47:36 -08:00
Kevin Morris
1ee8d177b4
fix(docker): rewrite trailing slashes to non-trailing in nginx config
Without this rewriting, we've been running into conversing with
HTTP over HTTPS (400 Bad Request).

TODO: Refactor this entire nginx config to something a bit more
simple and clean.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-10 14:49:53 -08:00
Kevin Morris
6d4e8028eb
change(gitlab-ci): explicitly down containers before upping
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-10 00:26:11 -08:00
Kevin Morris
4edae5015a
change(docker): remove ca dependencies on php-fpm/fastapi
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-10 00:18:09 -08:00
Kevin Morris
9f9b1c1732
change(docker): host fastapi over plain http
We don't need the https certificates being dealt with in the fastapi
service; we will define our certificates in any frontend nginx
running on top.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-10 00:12:01 -08:00
Kevin Morris
d675c0dc26
feat(python): catch all exceptions thrown through fastapi route paths
This commit does quite a bit:
- Catches unhandled exceptions raised in the route handler and
  produces a 500 Internal Server Error Arch-themed response.
- Each unhandled exception causes a notification to be sent to new
  `notifications.postmaster` email with a "Traceback ID."
- Traceback ID is logged to the server along with the traceback which
  caused the 500: `docker-compose logs fastapi | grep '<traceback_id>'`
- If `options.traceback` is set to `1`, traceback is displayed in
  the new 500.html template.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-09 23:10:02 -08:00
Kevin Morris
c775e8a692
feat(templates): add version to make_context
Prioritizes COMMIT_HASH environment variable and uses
`aurweb.config.AURWEB_VERSION` as a fallback.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-09 22:39:11 -08:00
Kevin Morris
e6679e4c4e
change(poetry): update fastapi to 0.71.0 release
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-08 13:54:54 -08:00
Kevin Morris
6f6f067597
feat: add aurweb-adduser console script
Originally left at util/adduser.py, this script allows administrators
to simply add a user to the configured aurweb database.

See --help for options.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-08 13:40:38 -08:00
Kevin Morris
9e7ae5904f
feat(python): handle RuntimeErrors raised through routes
This gets raised when a client closes a connection before receiving
a valid response; this is not controllable from our side.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-07 18:21:23 -08:00
Kevin Morris
bf371c447f
change(poetry): move fastapi to tiangolo/fastapi@2b10ca1
After two months, this finally got merged by somebody else.
Still largely considering moving away from FastAPI in the
long run, but this is better than relying on kevr's fork
for starlette 0.17.1 compat.

Other packages have also been upgraded and locked to versions.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-07 02:20:38 -08:00
Kevin Morris
a6faf9bd2e
feat(docker): perform migrations when starting the fastapi service
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-06 22:11:03 -08:00
Kevin Morris
b5ff8581f3
feat(migrations): add upgrade_voteinfo_integers ref
This migration modifies the Yes, No, Abstain and ActiveTUs columns
of the TUVoteInfo table from unsigned TINYINT to unsigned INTEGER.

TINYINT supports a total of 1 byte (up to 255 trusted users). This
is quite limited and we don't spend too much more by storing a
standard 4-byte INTEGER.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-06 20:27:23 -08:00
Kevin Morris
6e27f62e1b
fix(routers.trusted_user): set ActiveTUs on vote creation
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-06 20:26:45 -08:00
Kevin Morris
efd61979f7
fix(models.tu_voteinfo): default vote-count related columns to 0
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-06 20:25:30 -08:00
Kevin Morris
d49886f44f
fix(web/html/addvote): convert quorum to str using strval
Previous conversion was causing a straight up bug when submitting
new proposals. This patch fixes that issue.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-06 15:29:40 -08:00
Kevin Morris
059733cb8c
fix(routers.trusted_user): use creds to determine authorization
Closes #237

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 22:09:49 -08:00
Kevin Morris
9d221604b4
fix(routers.trusted_user): fix proposal participation percentage
Closes #238

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 20:17:47 -08:00
Kevin Morris
902c4d7a9c
fix(routers.packages): fix repeatead user joins
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 19:46:06 -08:00
Kevin Morris
0df57debb8
fix(routers.trusted_user): only display Voters on ended proposals
In addition, we display the Voters partial regardless of them existing
or not; with no voters, an empty Voters list is displayed.

Closes #236

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 17:51:57 -08:00
Kevin Morris
ae7621fb54
fix(routers.trusted_user): fix missing submitter link on /tu/{id}
Closes #235

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 17:18:19 -08:00
Kevin Morris
0988415931
fix(models.package_relation): add RelTypeID to PKs
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 14:37:03 -08:00
Kevin Morris
8ffff6261b
fix(models.package_dependency): add DepTypeID to PKs
This was stopping us from using numerous records for each dep type.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 14:36:57 -08:00
Kevin Morris
2cb9de0800
fix(models.package_group): add backref cascade
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 14:36:49 -08:00
Kevin Morris
1af61b0c50
fix(routers.packages): fix /packages/{name} relation ordering
Conflicts, Provides and Replaces did not have consistent
ordering with PHP. This patch fixes that issue.

Closes #228

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-03 22:58:48 -08:00
Kevin Morris
b0eea00181
fix(pkgbase.util): filter pending requests
Closes #229

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-03 22:33:31 -08:00
Kevin Morris
71e73ca654
fix(routers.pkgbase): fix next argument for merge redirection
This was redirecting us to the package which we merged, leading
us into a 404. This fixes that issue by instead redirecting us
into the target we merge into.

Closes #231

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-03 22:25:35 -08:00
Kevin Morris
83dc26ccde
fix(packages.request): fix autogenerated merge closure
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-03 21:16:54 -08:00
Kevin Morris
6c6eb2c21b
test: add tests to check various 404 paths and 503
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-03 18:22:10 -08:00