Commit graph

387 commits

Author SHA1 Message Date
Kevin Morris
e3fff9e357
Merge branch 'feat-csrf-login-check' into pu 2021-11-15 12:00:07 -08:00
Kevin Morris
4103ab49c9
housekeep(fastapi): rework aurweb.db session API
Changes:
-------
- Add aurweb.db.get_session()
    - Returns aurweb.db's global `session` instance
    - Provides us a way to change the implementation of the session
      instance without interrupting user code.
- Use aurweb.db.get_session() in session API methods
- Add docstrings to session API methods
- Refactor aurweb.db.delete
    - Normalize aurweb.db.delete to an alias of session.delete
- Refresh instances in places we depend on their non-PK columns
  being up to date.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-14 16:15:50 -08:00
Kevin Morris
f8ba2c5342
cleanup(fastapi): simplify aurweb.routers.accounts.accounts_post
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-14 15:32:23 -08:00
Kevin Morris
cee7512e4d cleanup(fastapi): simplify PackageDependency.is_package()
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-12 20:50:08 -08:00
Kevin Morris
e8e9edbb21
change(fastapi): simplify package details database queries
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 19:30:21 -08:00
Kevin Morris
a33e9bd571
feat(fastapi): add Replaces field to package details
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 19:15:29 -08:00
Kevin Morris
50a9690c2d
feat(fastapi): add Provides field in package details
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 19:15:03 -08:00
Kevin Morris
363afff332
feat(fastapi): add /pkgbase/{name}/keywords (post)
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 17:36:08 -08:00
Kevin Morris
0da11f068b
fix(fastapi): check for prometheus info.response
When this is unchecked, exceptions cause the resulting stack
trace to be oblivious to the original exception thrown.

This commit changes that behavior so that metrics are created
only when info.response exists.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-11 16:24:10 -08:00
Kevin Morris
8788f99005
fix(mkpkglists): restore isort order
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 13:54:18 -08:00
Kevin Morris
6e344ce9da
fix(mkpkglists): default keys to result[1]
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 13:41:53 -08:00
Kevin Morris
daef98080e
fix(fastapi): fix broken official package query
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 13:05:19 -08:00
Kevin Morris
4f7aeafa8d
feat(docker): host gzip archive downloads
- added config option [mkpkglists] archivedir
    - created by mkpkglists

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 07:39:23 -08:00
Kevin Morris
abbecf5194
change(mkpkglists): remove header comments
These comments change every time mkpkglists is run; which
would invalidate the ETag headers disbursed by the gzip
host. This commit removes those changing headers.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 07:39:17 -08:00
Kevin Morris
0155f4ea84
fix(mkpkglists): remove caching
We really need caching for this; however, our current caching
method will cause the script to bypass changes to columns
if they have nothing to do with IDs.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-09 02:07:13 -08:00
Kevin Morris
f3f662c696
fix(mkpkglists): improve package meta archive
The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.

Two command line arguments are available:

    --extended | Include License, Keywords, Groups, relations
                 and dependencies.

When --extended is passed, the script will create a
packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.

Archive JSON is in the following format: line-separated package objects
enclosed in a list:

    [
    {...},
    {...},
    {...}
    ]

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-09 02:06:50 -08:00
Kristian Klausen
f606140050
feat(PHP): Add packages dump file with more metadata 2021-11-09 02:04:58 -08:00
Kevin Morris
10fcf93991
fix(fastapi): use correct official pkg base url
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-09 01:51:23 -08:00
Kevin Morris
85ebc72e8a
fix(fastapi): only elevated users are allowed to suspend accounts
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-08 18:51:39 -08:00
Kevin Morris
446a082352
change(fastapi): refactor database ORM model definitions
We don't want to depend on the database to load up data
about the models we define. We now leverage the existing
`aurweb.schema` module for table definitions and set
__table_args__["autoload"] to False.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-07 17:31:34 -08:00
Steven Guikal
020409ef46 fix(FastAPI): prevent CSRF forging login requests
Signed-off-by: Steven Guikal <void@fluix.one>
2021-11-04 14:34:14 -04:00
Kevin Morris
16e6fa2cdd
fix(fastapi): fix prometheus parsing of HTTPStatus
This wasn't actually casting to int. We shouldn't be providing
HTTPStatus.CONSTANTS directly anyway, but, in case we do, we now
just convert the status to an int before converting it to a string.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-01 14:23:15 -07:00
Kevin Morris
9aa8decf40
fix(fastapi): use metrics in cases where PROMETHEUS_MULTIPROC_DIR is defined
Previously, we restricted this to gunicorn to get it working on aur-dev.
This change makes it usable through any backend, and also no-op if
PROMETHEUS_MULTIPROC_DIR is not defined.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-01 14:18:19 -07:00
Kevin Morris
dc397f6bd8
fix(fastapi): utilize PROMETHEUS_MULTIPROC_DIR in our own /metrics
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-01 13:23:48 -07:00
Kevin Morris
f21765bfe4
feat(fastapi): add prometheus /metrics
This commit provides custom metrics, so we can group requests
into their route paths and not by the arguments given, e.g.
/pkgbase/some-package -> /pkgbase/{name}. We also count RPC
requests as `http_api_requests_total`, split by the RPC
query "type" argument.

- `http_api_requests_total`
    - Labels: ["type", "status"]
- `http_requests_total`
    - Number of HTTP requests in total.
    - Labels: ["method", "path", "status"]

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-01 11:41:39 -07:00
Kevin Morris
451eec0c28
fix(fastapi): remove info-specific fields from search results
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-31 16:22:24 -07:00
Kevin Morris
b7475a5bd0
fix(rpc): fix performance of suggest[-pkgbase]
We were selecting the entire record; we should just select
the Name column as done in this commit.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-31 04:11:42 -07:00
Kevin Morris
61f3cb938c
feat(rpc): support the If-None-Match request header
If the If-None-Match header is supplied with a previously
obtained ETag from the same query, a 304 Not Modified is
returned with no content.

This allows clients to completely leverage the ETag header.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-31 01:22:54 -07:00
Kevin Morris
2cc44e8f28
fix(rpc): perform regex match against callback name
Since we're in the hot path, a constant re.compiled
JSONP_EXPR is defined for checks against the callback.

Additionally, reorganized `content_type` and `content`
to avoid performing a DB query when we encounter a
regex mismatch.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-31 01:17:16 -07:00
Kevin Morris
12b4269ba8
feat(rpc): support jsonp callbacks
This change introduces alternate rendering of text/javascript
JSONP-compatible callback content. The `examples/jsonp.html`
HTML document can be used to test this functionality against
a running aurweb server.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-31 00:29:19 -07:00
Kevin Morris
05e6cfca62
feat(rpc): add msearch type handler
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 22:56:18 -07:00
Kevin Morris
9fef8b0611
fix(rpc): fix search arg check
When by == 'maintainer', we allow an unspecified keyword,
resulting in a search of orphan packages. Fix our search
check so that when no arg is given, it is set to an empty
str().

We already check for valid args when type is not maintainer,
so there's no need to worry about other args falling through.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 22:53:30 -07:00
Kevin Morris
af2f3694e7
feat(rpc): add search type handler
This commit introduces a PackageSearch-derivative class: `RPCSearch`.
This derivative modifies callback behavior of PackageSearch to
suit RPC searches, including [make|check|opt]depends `by` types.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 19:25:54 -07:00
Kevin Morris
ece25e0499
Merge branch 'pu-rpc-suggest' into pu 2021-10-30 16:57:03 -07:00
Kevin Morris
c28f1695ed
fix(fastapi): support by maintainer search with no keywords
In this case, package search should return orphaned packages.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 16:24:53 -07:00
Kevin Morris
9d6dbaf0ec
feat(rpc): add suggest type handler
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-30 00:36:21 -07:00
Kevin Morris
6d376fed15
feat(rpc): add ETag header with md5 hash content
The ETag header can be used for client-side caching.

https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/ETag

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 23:57:33 -07:00
Kevin Morris
b3b31394e8
fix(rpc): simplify json generation complexity
This simply decouples depends and relations population into
their own helper functions.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 22:59:40 -07:00
Kevin Morris
9464de108f
feat(fastapi): add /pkgbase/{name}/comments/{id}/edit (get)
This is needed so that users can edit comments when they don't have
Javascript being used in their browser.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 21:37:52 -07:00
Kevin Morris
01e27fa347
fix(fastapi): sanitize /requests params
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 20:40:00 -07:00
Kevin Morris
7f4c011dc3
fix(fastapi): sanitize PP/O parameters for package search
This definitely leaked through in more areas. We'll need to reuse
this new utility function in a few other routes in upcoming commits.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 20:39:55 -07:00
Kevin Morris
348128fada
fix(fastapi): fix /account/{username} page title
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 17:18:53 -07:00
Kevin Morris
691b7b9091
feat(fastapi): add comment actions to /account/{username}/comments
With this change, we've decoupled some partials shared between
`/pkgbase/{name}` and `/account/{username}/comments`. The comment
actions template now resolves its package base via the `comment`
instance instead of requiring `pkgbase`.

We've also modified the existing package comment routes to
support execution from any location using the `next` parameter.
This allows us to reuse code from package comments for
account comments actions.

Moved the majority of comment editing javascript to its own
.js file.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-29 17:18:49 -07:00
Kevin Morris
adb6252f85
feat(fastapi): add /account/{username}/comments
This commit contains a base template of account comments
in sorted order (based on ColumnTS.desc).

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-28 13:21:16 -07:00
Kevin Morris
9fd07c36eb
fix(fastapi): fix account page title
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-28 12:12:13 -07:00
Kevin Morris
1f2347c6b4
fix(fastapi): fix User.login signature typing
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-28 07:35:20 -07:00
Kevin Morris
034288711b
fix(fastapi): rework cookies - do not re-emit generically
This change removes cookie re-emission of AURLANG and AURTZ,
adds the AURREMEMBER cookie (the state of the "Remember Me"
checkbox on login), and re-emits AURSID based on the AURREMEMBER
cookie.

Previously, re-emission of AURSID was forcefully modifying
the expiration of the AURSID cookie. The introduction of
AURREMEMBER allows us to deduct the correct cookie expiration
timing based on configuration variables. With this addition,
we now re-emit the AURSID cookie with an updated expiration
based on the "Remember Me" checkbox on login.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-28 07:35:14 -07:00
Kevin Morris
d7ac95a707
fix(fastapi): limit cookie migration to whitelisted keys
Whitelisted keys: AURSID, AURTZ, AURLANG

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-26 19:09:39 -07:00
Kevin Morris
65be8b8e07
fix(fastapi): support "Account Type:" input for account edit
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-25 22:05:23 -07:00
Kevin Morris
7e7a1ead88
fix(fastapi): unify homepage cache expiry time to five minutes
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-10-24 19:08:03 -07:00