Starlette 0.16.0 has a pretty bad bug in terms of logging which
has been fixed in the 0.17.0 release. That being said, FastAPI has
not yet merged a request at https://github.com/tiangolo/fastapi/pull/4145
which resolves this dependency resolution so we can use the updated
starlette package.
kevr has forked the pull request in question and we are using it
for now in our poetry dependencies to get ahead of the game.
When FastAPI upstream is updated to support 0.17.0, we'll need
to switch this back to using upstream's source.
Signed-off-by: Kevin Morris <kevr@0cost.org>
We don't want to depend on the database to load up data
about the models we define. We now leverage the existing
`aurweb.schema` module for table definitions and set
__table_args__["autoload"] to False.
Signed-off-by: Kevin Morris <kevr@0cost.org>
We really need caching for this; however, our current caching
method will cause the script to bypass changes to columns
if they have nothing to do with IDs.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Two new archives are available:
- packages-meta-v1.json.gz
- RPC search formatted data for all packages
- ~2.1MB at the time of writing.
- packages-meta-ext-v1.json.gz (via --extended)
- RPC multiinfo formatted data for all packages.
- ~9.8MB at the time of writing.
New dependencies are required for this update:
- `python-orjson`
All archives served out by aur.archlinux.org distribute the Last-Modified
header and support the If-Modified-Since header, which should be
populated with Last-Modified's value. These should be used by clients
to avoid redownloading the archive when unnecessary.
Additionally, the new meta archives contain a format suitable for
streaming the data as the file is retrieved. It is still in JSON
format, however, users can parse package objects line by line after
the first '[' found in the file, until the last ']'; both contained
on their own lines.
Note: This commit is a documentation change and commit body.
Signed-off-by: Kevin Morris <kevr@0cost.org>
The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.
Two command line arguments are available:
--extended | Include License, Keywords, Groups, relations
and dependencies.
When --extended is passed, the script will create a
packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.
Archive JSON is in the following format: line-separated package objects
enclosed in a list:
[
{...},
{...},
{...}
]
Signed-off-by: Kevin Morris <kevr@0cost.org>
This wasn't actually casting to int. We shouldn't be providing
HTTPStatus.CONSTANTS directly anyway, but, in case we do, we now
just convert the status to an int before converting it to a string.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Previously, we restricted this to gunicorn to get it working on aur-dev.
This change makes it usable through any backend, and also no-op if
PROMETHEUS_MULTIPROC_DIR is not defined.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This commit provides custom metrics, so we can group requests
into their route paths and not by the arguments given, e.g.
/pkgbase/some-package -> /pkgbase/{name}. We also count RPC
requests as `http_api_requests_total`, split by the RPC
query "type" argument.
- `http_api_requests_total`
- Labels: ["type", "status"]
- `http_requests_total`
- Number of HTTP requests in total.
- Labels: ["method", "path", "status"]
Signed-off-by: Kevin Morris <kevr@0cost.org>
This is not used anymore in our FastAPI code, however, for
back-compatibility with pre-FastAPI scripts, we need it.
Signed-off-by: Kevin Morris <kevr@0cost.org>
If the If-None-Match header is supplied with a previously
obtained ETag from the same query, a 304 Not Modified is
returned with no content.
This allows clients to completely leverage the ETag header.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Since we're in the hot path, a constant re.compiled
JSONP_EXPR is defined for checks against the callback.
Additionally, reorganized `content_type` and `content`
to avoid performing a DB query when we encounter a
regex mismatch.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This change introduces alternate rendering of text/javascript
JSONP-compatible callback content. The `examples/jsonp.html`
HTML document can be used to test this functionality against
a running aurweb server.
Signed-off-by: Kevin Morris <kevr@0cost.org>
When by == 'maintainer', we allow an unspecified keyword,
resulting in a search of orphan packages. Fix our search
check so that when no arg is given, it is set to an empty
str().
We already check for valid args when type is not maintainer,
so there's no need to worry about other args falling through.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This commit introduces a PackageSearch-derivative class: `RPCSearch`.
This derivative modifies callback behavior of PackageSearch to
suit RPC searches, including [make|check|opt]depends `by` types.
Signed-off-by: Kevin Morris <kevr@0cost.org>
The previous method was super bad. Even if a variable was declared,
if it was empty, we would run into a false-positive. Additionally,
the previous method did not allow us to not specify the COMMIT_HASH
variable; which is problematic for development environments.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This is needed so that users can edit comments when they don't have
Javascript being used in their browser.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This definitely leaked through in more areas. We'll need to reuse
this new utility function in a few other routes in upcoming commits.
Signed-off-by: Kevin Morris <kevr@0cost.org>
With this change, we've decoupled some partials shared between
`/pkgbase/{name}` and `/account/{username}/comments`. The comment
actions template now resolves its package base via the `comment`
instance instead of requiring `pkgbase`.
We've also modified the existing package comment routes to
support execution from any location using the `next` parameter.
This allows us to reuse code from package comments for
account comments actions.
Moved the majority of comment editing javascript to its own
.js file.
Signed-off-by: Kevin Morris <kevr@0cost.org>
This uses the internally defined docker volume `git_data`,
but the variable is configurable for changes in the future.
Signed-off-by: Kevin Morris <kevr@0cost.org>
Previously CGIT_CLONE_PREFIX_(PHP|FASTAPI), we found that we could
use the same env var in multiple places, including non-cgit-clone-prefix
areas.
So, they were renamed, and one additional prefix was added.
- CGIT_CLONE_PREFIX_PHP -> AURWEB_PHP_PREFIX
- Used for cgit's clone prefix and AUR_CONFIG's aur_location for PHP
- CGIT_CLONE_PREFIX_FASTAPI -> AURWEB_FASTAPI_PREFIX
- Used for cgit's clone prefix and AUR_CONFIG's aur_location for FastAPI
- AURWEB_SSHD_PREFIX
- Used for aurweb's sshd clone prefix shown on package pages
Signed-off-by: Kevin Morris <kevr@0cost.org>
Additionally, for now, no-op usage of the MARIADB_SOCKET_DIR
environment variable. This is particularly useful for a serious
production setup, but we don't currently use that.
Signed-off-by: Kevin Morris <kevr@0cost.org>
