Commit graph

694 commits

Author SHA1 Message Date
Kevin Morris
b0b5e4c9d1
fix(fastapi): use secrets module to generate random strings
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-03 17:58:58 -08:00
Steven Guikal
75ad2fb53d fix(FastAPI): cleanup auth_required decorator
Signed-off-by: Steven Guikal <void@fluix.one>
2021-12-03 14:07:47 -05:00
Kevin Morris
81f8c23265
fix(fastapi): log out IntegrityError from failed SID generation
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-02 23:42:13 -08:00
Kevin Morris
806a19b91a
feat(fastapi): render a 500 html response when unique SID generation fails
We've seen a bug in the past where unique SID generation fails and
still ends up raising an exception.

This commit reworks how we deal with database exceptions internally,
tries for 36 iterations to set a fresh unique SID, and raises a 500
HTTPException if we were unable to.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-02 23:26:42 -08:00
Kevin Morris
abfd41f31e
change(fastapi): centralize HTTPException
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-02 23:23:27 -08:00
Steven Guikal
42701514e7 fix(FastAPI): Use HTTPStatus instead of raw number
Signed-off-by: Steven Guikal <void@fluix.one>
2021-12-01 21:15:49 +00:00
Kevin Morris
c09784d58f
fix(auth.auth_required): remove unused keyword arguments
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-01 11:56:44 -08:00
Kevin Morris
112837e0e9
fix(test_auth): cover mismatched referer situation
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-01 11:53:43 -08:00
Kevin Morris
a0e1a1641d
fix(fastapi): support UsersID and User columns in the Session model
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-01 00:30:20 -08:00
Steven Guikal
0b30216229 fix(FastAPI): remove unnecessary arguments to auth_required
Signed-off-by: Steven Guikal <void@fluix.one>
2021-12-01 03:11:01 -05:00
Steven Guikal
429d8059e1 fix(FastAPI): remove login and redirect parameters from auth_required
Signed-off-by: Steven Guikal <void@fluix.one>
2021-12-01 02:57:23 -05:00
Steven Guikal
a10f8663fd fix(FastAPI): reorganize credential checkin into dedicated file
Signed-off-by: Steven Guikal <void@fluix.one>
2021-12-01 02:03:02 -05:00
Steven Guikal
125b244f44 fix(FastAPI): use account type vars instead of strings
Signed-off-by: Steven Guikal <void@fluix.one>
2021-11-30 16:33:34 -05:00
Steven Guikal
ecbab8546b fix(FastAPI): access AccountType ID directly
Signed-off-by: Steven Guikal <void@fluix.one>
2021-11-30 16:33:34 -05:00
Kevin Morris
a6ac5f0dbf fix(rpc): fix ordering of related records
They were being ordered by IDs; they should be ordered by Names.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-30 16:33:34 -05:00
Kevin Morris
9bfe2b07ba
fix(fastapi): render Logged-in as page on authenticated /login
This was missed during the initial porting of the /login route.

Modifications:
-------------
- A form is now used for the [Logout] link and some css was
  needed to deal with positioning.

Closes #186

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-29 19:40:55 -08:00
Kevin Morris
69eb17cb0d
change(fastapi): remove the GET /logout route; replaced with POST
Had to add some additional CSS in to style a form button the same
as <a> links are styled.

Closes #188

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-29 16:52:10 -08:00
Kevin Morris
436d742017
fix(fastapi): use CRED_TU_LIST_VOTES for "Trusted User" navigation item
Closes #189

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-29 14:08:00 -08:00
Kevin Morris
bc1cf8b1f6
fix(rendercomment): markdown.util.etree -> xml.etree.ElementTree
This removes a deprecation warning.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:12 -08:00
Kevin Morris
2d0e09cd63
change(rendercomment): converted to use aurweb.db ORM
- Added aurweb.util.git_search.
    - Decoupled away from rendercomment for easier testability.
- Added aurweb.testing.git.GitRepository.
- Added templates/testing/{PKGBUILD,SRCINFO}.j2.
- Added aurweb.testing.git.GitRepository + `git` pytest fixture

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:11 -08:00
Kevin Morris
4b0cb0721d
fix(conftest): use synchronization locks for setup_database
We were running into data race issues where the `fn.is_file()`
check would occur twice before writing the file in the `else`
clause. For this reason, a new aurweb.lock.Lock class has been
added which doubles as a thread and process lock. We can use
this elsewhere in the future, but we are also able to use it
to solve this kind of data race issue.

That being said, we still need the lock file state to tell us
when the first caller acquired the lock.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:11 -08:00
Kevin Morris
d8e3ca1abb
change(notify): converted to use aurweb.db ORM
- Removed notify sharness test

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:10 -08:00
Kevin Morris
9fb1fbe32c
feat(testing): add email testing utilities
Changes:
- util/sendmail now populates email files in the 'test-emails' directory.
    - util/sendmail does this in a serialized fashion based off of
      the test suite and name retrieved from PYTEST_CURRENT_TEST
      in the format: `<test_suite>_<test_function>.n.txt` where n
      is increased by one every time sendmail is run.
- pytest conftest fixtures have been added for test email setup;
  it wipes out old emails for the particular test function being run.
- New aurweb.testing.email.Email class allows developers to test
  against emails stored by util/sendmail. Simple pass the serial
  you want to test against, starting at serial = 1; e.g. Email(serial).

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:10 -08:00
Kevin Morris
b72bd38f76
change(pkgmaint): converted to use aurweb.db ORM
- Replaced time.time() usage with datetime.utcnow().timestamp()
- Removed pkgmaint sharness test

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:10 -08:00
Kevin Morris
f4ef02fa5b
fix(fastapi): fix Package's PackageBase backref cascade
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:09 -08:00
Kevin Morris
d097799b34
change(usermaint): converted to use aurweb.db ORM
- Removed usermaint sharness test

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:09 -08:00
Kevin Morris
8d5683d3f1
change(tuvotereminder): converted to use aurweb.db ORM
- Removed tuvotereminder sharness test.
- Added [tuvotereminder] section to config.defaults.
- Added `range_start` option to config.defaults [tuvotereminder].
- Added `range_end` option to config.defaults [tuvotereminder].

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:09 -08:00
Kevin Morris
29c2d0de6b
change(mkpkglists): converted to use aurweb.db ORM
- Improved speed dramatically
- Removed mkpkglists sharness

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:08 -08:00
Kevin Morris
c59acbf6d6
add noop testing utility
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:08 -08:00
Kevin Morris
29989b7fdb
change(aurblup): converted to use aurweb.db ORM
Introduces:
- aurweb.testing.alpm.AlpmDatabase
    - Used to mock up and manage a remote repository.
- templates/testing/alpm_package.j2
    - Used to generate a single ALPM package desc.
- Removed aurblup sharness test

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:08 -08:00
Kevin Morris
3efb9a57b5
change(popupdate): converted to use aurweb.db ORM
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:07 -08:00
Kevin Morris
0e938209af
feat(aurweb-config): add unset action and simplify
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-27 22:34:15 -08:00
Kevin Morris
199622c53f
fix(fastapi): refresh records when fetching updated packages
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-27 21:35:48 -08:00
Kevin Morris
47feb72f48
fix(fastapi): fix SessionID (and ResetKey) generation
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-27 20:19:40 -08:00
Kevin Morris
d658627e99
fix(fastapi): don't redirect to login on authed /login
Closes #184

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-27 19:14:32 -08:00
Kevin Morris
759f18ea75
feat: add aurweb-config console script
This can be used to update config values for the entirety
of a config. When config values are set through this tool,
$AUR_CONFIG is overridden with a copy of the config file
with all sections and options found in $AUR_CONFIG
+ $AUR_CONFIG_DEFAULTS.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-27 16:44:56 -08:00
Kevin Morris
e558e979ff
fix(fastapi): check ssh key prefixes against configured valid-keytypes
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-24 21:29:28 -08:00
Kevin Morris
19191fa8b5
fix: update nginx config in aurweb.spawn
Host a specific FastAPI nginx frontend as well as a PHP
nginx frontend, configurable by the (PHP|FASTAPI)_NGINX_PORT
environment variables.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-20 15:47:25 -08:00
Kevin Morris
47d0df76e6
feat: support gunicorn in aurweb.spawn
This also comes with a -w|--workers argument that allows
the caller to set the number of gunicorn workers.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-20 15:47:24 -08:00
Kevin Morris
82ca4ad9a0
feat: check php configuration in aurweb.spawn
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-20 15:47:24 -08:00
Kevin Morris
191198ca41
housekeep(fastapi): simplify aurweb.spawn.stop()
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-20 15:47:24 -08:00
Kevin Morris
008a8824ce
housekeep(fastapi): simplify package_base_comaintainers_post
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-20 13:20:46 -08:00
Kevin Morris
7f981b9ed7
fix(fastapi): utilize auto_{orphan,deletion}_age
Didn't get this in when the initial request port went down;
here it is.

Auto-accept orphan requests when the package has been out of
date for longer than auto_orphan_age.

Auto-accept deletion requests by the package's maintainer
if the package has been uploaded within auto_deletion_age
seconds ago.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-18 21:27:03 -08:00
Kevin Morris
2892d21ff1
remove global aurweb.models flake8 F401 ignore
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 06:00:12 -08:00
Kevin Morris
303585cdbf
change(fastapi): decouple update logic from account edit
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 06:00:07 -08:00
Kevin Morris
94972841d6
change(fastapi): decouple error logic from process_account_form
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 05:58:08 -08:00
Kevin Morris
abe8c0630c
fix(rpc): improve type=info performance
Now, we use an equivalent query to PHP's query, yet we grab
every piece of data we need for all packages asked for in one
database query.

At this time, local benchmarks have shown a slight performance
improvement when compared to PHP.

fastapi 262 requests/sec
php 250 requests/sec

Extras:

- Moved RPCError to the aurweb.exceptions module

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 02:30:06 -08:00
Kevin Morris
fb92fb509b
change(fastapi): use sys.getrecursionlimit() + 1000 as default
Without the increment, we've seen tests failed due to recursion
errors caused by starlette's base middleware. Just make it safe
in case nobody supplies TEST_RECURSION_LIMIT.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 01:36:19 -08:00
Kevin Morris
fa43f6bc3e
change(aurweb): add parallel tests and improve aurweb.db
This change utilizes pytest-xdist to perform a multiproc test
run and reworks aurweb.db's code. We no longer use a global
engine, session or Session, but we now use a memo of engines
and sessions as they are requested, based on the PYTEST_CURRENT_TEST
environment variable, which is available during testing.

Additionally, this change strips several SQLite components
out of the Python code-base.

SQLite is still compatible with PHP and sharness tests, but
not with our FastAPI implementation.

More changes:
------------
- Remove use of aurweb.db.session global in other code.
- Use new aurweb.db.name() dynamic db name function in env.py.
- Added 'addopts' to pytest.ini which utilizes multiprocessing.
    - Highly recommended to leave this be or modify `-n auto` to
      `-n {cpu_threads}` where cpu_threads is at least 2.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 01:34:59 -08:00
Kevin Morris
07aac768d6
change(fastapi): remove sqlite support
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 01:34:59 -08:00