Commit graph

525 commits

Author SHA1 Message Date
Kevin Morris
7f981b9ed7
fix(fastapi): utilize auto_{orphan,deletion}_age
Didn't get this in when the initial request port went down;
here it is.

Auto-accept orphan requests when the package has been out of
date for longer than auto_orphan_age.

Auto-accept deletion requests by the package's maintainer
if the package has been uploaded within auto_deletion_age
seconds ago.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-18 21:27:03 -08:00
Kevin Morris
2df7187514
fix global test_ssh_pub_key E501 flake8 violation
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 06:00:12 -08:00
Kevin Morris
94972841d6
change(fastapi): decouple error logic from process_account_form
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 05:58:08 -08:00
Kevin Morris
ccf50cbdf5
change: rework test_rpc's TestClient usage into a fixture
This is the first step on our path to reworking the test
suite in general.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 02:30:06 -08:00
Kevin Morris
fa43f6bc3e
change(aurweb): add parallel tests and improve aurweb.db
This change utilizes pytest-xdist to perform a multiproc test
run and reworks aurweb.db's code. We no longer use a global
engine, session or Session, but we now use a memo of engines
and sessions as they are requested, based on the PYTEST_CURRENT_TEST
environment variable, which is available during testing.

Additionally, this change strips several SQLite components
out of the Python code-base.

SQLite is still compatible with PHP and sharness tests, but
not with our FastAPI implementation.

More changes:
------------
- Remove use of aurweb.db.session global in other code.
- Use new aurweb.db.name() dynamic db name function in env.py.
- Added 'addopts' to pytest.ini which utilizes multiprocessing.
    - Highly recommended to leave this be or modify `-n auto` to
      `-n {cpu_threads}` where cpu_threads is at least 2.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 01:34:59 -08:00
Kevin Morris
07aac768d6
change(fastapi): remove sqlite support
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-17 01:34:59 -08:00
Kevin Morris
e3fff9e357
Merge branch 'feat-csrf-login-check' into pu 2021-11-15 12:00:07 -08:00
Kevin Morris
4103ab49c9
housekeep(fastapi): rework aurweb.db session API
Changes:
-------
- Add aurweb.db.get_session()
    - Returns aurweb.db's global `session` instance
    - Provides us a way to change the implementation of the session
      instance without interrupting user code.
- Use aurweb.db.get_session() in session API methods
- Add docstrings to session API methods
- Refactor aurweb.db.delete
    - Normalize aurweb.db.delete to an alias of session.delete
- Refresh instances in places we depend on their non-PK columns
  being up to date.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-14 16:15:50 -08:00
Kevin Morris
686c032290
feat(fastapi): add id="licenses" to package details licenses
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 19:55:04 -08:00
Kevin Morris
7aa959150e
feat(fastapi): add id="conflicts" to package details conflicts
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 19:54:30 -08:00
Kevin Morris
a33e9bd571
feat(fastapi): add Replaces field to package details
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 19:15:29 -08:00
Kevin Morris
50a9690c2d
feat(fastapi): add Provides field in package details
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 19:15:03 -08:00
Kevin Morris
363afff332
feat(fastapi): add /pkgbase/{name}/keywords (post)
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 17:36:08 -08:00
Kevin Morris
0c57c53da1
fix(sharness): fix AUR_CONFIG generation for mkpkglists test
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 07:39:23 -08:00
Kevin Morris
f3f662c696
fix(mkpkglists): improve package meta archive
The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.

Two command line arguments are available:

    --extended | Include License, Keywords, Groups, relations
                 and dependencies.

When --extended is passed, the script will create a
packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.

Archive JSON is in the following format: line-separated package objects
enclosed in a list:

    [
    {...},
    {...},
    {...}
    ]

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-09 02:06:50 -08:00
Kristian Klausen
f606140050
feat(PHP): Add packages dump file with more metadata 2021-11-09 02:04:58 -08:00
Kevin Morris
85ebc72e8a
fix(fastapi): only elevated users are allowed to suspend accounts
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-08 18:51:39 -08:00
Kevin Morris
51fb24ab73
fix(mkpkglists): improve package meta archive
The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.

Two command line arguments are available:

    --extended | Include License, Keywords, Groups, relations
                 and dependencies.

When --extended is passed, the script will create a
packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.

Archive JSON is in the following format: line-separated package objects
enclosed in a list:

    [
    {...},
    {...},
    {...}
    ]

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-06 16:19:34 -07:00
Kristian Klausen
69773a5b58
feat(PHP): Add packages dump file with more metadata 2021-11-06 16:15:55 -07:00
Steven Guikal
020409ef46 fix(FastAPI): prevent CSRF forging login requests
Signed-off-by: Steven Guikal <void@fluix.one>
2021-11-04 14:34:14 -04:00
Kevin Morris
dc397f6bd8
fix(fastapi): utilize PROMETHEUS_MULTIPROC_DIR in our own /metrics
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-01 13:23:48 -07:00
Kevin Morris
61f3cb938c
feat(rpc): support the If-None-Match request header
If the If-None-Match header is supplied with a previously
obtained ETag from the same query, a 304 Not Modified is
returned with no content.

This allows clients to completely leverage the ETag header.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-31 01:22:54 -07:00
Kevin Morris
2cc44e8f28
fix(rpc): perform regex match against callback name
Since we're in the hot path, a constant re.compiled
JSONP_EXPR is defined for checks against the callback.

Additionally, reorganized `content_type` and `content`
to avoid performing a DB query when we encounter a
regex mismatch.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-31 01:17:16 -07:00
Kevin Morris
12b4269ba8
feat(rpc): support jsonp callbacks
This change introduces alternate rendering of text/javascript
JSONP-compatible callback content. The `examples/jsonp.html`
HTML document can be used to test this functionality against
a running aurweb server.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-31 00:29:19 -07:00
Kevin Morris
05e6cfca62
feat(rpc): add msearch type handler
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 22:56:18 -07:00
Kevin Morris
af2f3694e7
feat(rpc): add search type handler
This commit introduces a PackageSearch-derivative class: `RPCSearch`.
This derivative modifies callback behavior of PackageSearch to
suit RPC searches, including [make|check|opt]depends `by` types.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 19:25:54 -07:00
Kevin Morris
ece25e0499
Merge branch 'pu-rpc-suggest' into pu 2021-10-30 16:57:03 -07:00
Kevin Morris
c28f1695ed
fix(fastapi): support by maintainer search with no keywords
In this case, package search should return orphaned packages.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 16:24:53 -07:00
Kevin Morris
9d6dbaf0ec
feat(rpc): add suggest type handler
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 00:36:21 -07:00
Kevin Morris
6d376fed15
feat(rpc): add ETag header with md5 hash content
The ETag header can be used for client-side caching.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 23:57:33 -07:00
Kevin Morris
9464de108f
feat(fastapi): add /pkgbase/{name}/comments/{id}/edit (get)
This is needed so that users can edit comments when they don't have
Javascript being used in their browser.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 21:37:52 -07:00
Kevin Morris
7f4c011dc3
fix(fastapi): sanitize PP/O parameters for package search
This definitely leaked through in more areas. We'll need to reuse
this new utility function in a few other routes in upcoming commits.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 20:39:55 -07:00
Kevin Morris
691b7b9091
feat(fastapi): add comment actions to /account/{username}/comments
With this change, we've decoupled some partials shared between
`/pkgbase/{name}` and `/account/{username}/comments`. The comment
actions template now resolves its package base via the `comment`
instance instead of requiring `pkgbase`.

We've also modified the existing package comment routes to
support execution from any location using the `next` parameter.
This allows us to reuse code from package comments for
account comments actions.

Moved the majority of comment editing javascript to its own
.js file.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 17:18:49 -07:00
Kevin Morris
adb6252f85
feat(fastapi): add /account/{username}/comments
This commit contains a base template of account comments
in sorted order (based on ColumnTS.desc).

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-28 13:21:16 -07:00
Kevin Morris
65be8b8e07
fix(fastapi): support "Account Type:" input for account edit
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-25 22:05:23 -07:00
Kevin Morris
0d734eb07d
feat(fastapi): add configurable commit hash display
Two new options have been added:

- [devel] commit_url
    - URL including an %s format specifier that can be used to link
      to a webpage for the commit.
- [devel] commit_hash
    - HEAD's commit hash (produced via `git rev-parse HEAD`)

If a `[devel] commit_hash` is configured, a link to the commit based on
`[devel] commit_url` will be displayed in the aurweb footer in
the form: `HEAD@<commit_hash>`. If no `[devel] commit_url` is
configured, a non-linked hash will be displayed.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-24 18:12:19 -07:00
Kevin Morris
5fb75b9614
feat(fastapi): add /pkgbase/{name}/merge (post)
Changes:

- `via` is not required in FastAPI. We deduce the involved
  requests via their PackageBaseName / MergeBaseName columns
  and set them to Accepted when merged.
- When erroneous input is given, the error is now presented
  on the merge page instead of sending the user to the pkgbase
  page.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-23 19:03:32 -07:00
Kevin Morris
60bffa4fb6
feat(FastAPI): add /packages (post) action: 'delete'
Improvements:

- Package deletion now creates a PackageRequest on behalf of
  the deleter if one does not yet exist.
- All package deletions are now logged to keep track of who did what.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-23 18:23:15 -07:00
Kevin Morris
13b344d238
feat(FastAPI): add /packages (post) action: 'disown'
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-22 19:38:14 -07:00
Kevin Morris
f1ad1b9aed
feat(FastAPI): add /packages (post) action: 'adopt'
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-22 19:01:13 -07:00
Kevin Morris
d5520c9ed2
feat(FastAPI): add /packages (post) action: 'unnotify'
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-21 17:38:44 -07:00
Kevin Morris
b277d94e0b
feat(FastAPI): add /packages (post) action: 'notify'
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-21 16:52:04 -07:00
Kevin Morris
fb85cb60a0
feat(FastAPI): add /packages (post) action: 'unflag'
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-21 11:29:55 -07:00
Kevin Morris
65240c8343
feat(rpc): enforce ratelimiting
New configuration options:

- `[ratelimit] cache`
    - A boolean indicating whether we should use configured cache (1)
      or database (0) for ratelimiting.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-21 11:07:36 -07:00
Kevin Morris
a06f4ec19c
fix(fastapi): centralize logging initialization
With this change, we provide a wrapper to `logging.getLogger`
in the `aurweb.logging` module. Modules wishing to log using
logging.conf should get their module-local loggers by calling
`aurweb.logging.getLogger(__name__)`, similar to `logging.getLogger`,
this way initialization with logging.conf is guaranteed.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-21 10:40:52 -07:00
Kevin Morris
7c4fb539d8
change(fastapi): rework /rpc (get)
This reworks the base implementation of the RPC to use a
class called RPC for handling of requests. Took a bit of
a different approach than PHP in terms of exposed methods,
but it does end up achieving the same goal, with one additional
error: "Request type '{type}' is not yet implemented."

For FastAPI development, we'll stick with:

- If the supplied 'type' argument has an alias mapping in
  RPC.ALIASES, we convert the type argument over to its alias
  before doing anything. Example: 'info' is aliased to 'multiinfo',
  so when a user requests type=info, it is converted to type=multiinfo.
- If the type does not exist in RPC.EXPOSED_TYPES, the following
  error is produced: "No request type/data specified."
- If the type **does** exist in RPC.EXPOSED_TYPES, but does not
  have an implemented `RPC._handle_{type}_type` function, the
  following error is produced: "Request type '{type}' is not yet
  implemented."

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-20 22:17:05 -07:00
Kevin Morris
2b9840149e
feat(fastapi): add /pkgbase/{name}/merge (get)
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-20 20:31:44 -07:00
Kevin Morris
990f4d182b
feat(rpc): add 'suggest-pkgbase' type
This feature of RPC is required to take advantage of
javascript typeahead.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-20 20:31:42 -07:00
Kevin Morris
ddc51dd5eb
Merge branch 'fix-trailing-slashes' into pu 2021-10-20 17:46:12 -07:00
Kevin Morris
37f0c352f6
feat(FastAPI): add /pkgbase/{name}/flag-comment (get)
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-19 13:39:00 -07:00