Commit graph

66 commits

Author SHA1 Message Date
Hanabishi
2b8c8fc92a fix: make dependency source use superscript tag
Avoid using special characters and use '<sup>' HTML tag instead.
To not rely on user's fonts Unicode coverage.

Closes: #490
Signed-off-by: Hanabishi <1722-hanabishi@users.noreply.gitlab.archlinux.org>
2023-10-18 16:19:58 +00:00
moson
0a7b02956f
feat: Indicate dependency source
Dependencies might reside in the AUR or official repositories.
Add "AUR" as superscript letters to indicate if a package/provider
is present in the AUR.

Signed-off-by: moson <moson@archlinux.org>
2023-09-03 14:17:11 +02:00
moson-mo
c41f2e854a
perf: tweak some search queries
We currently sorting on two columns in different tables which is quite
expensive in terms of performance:
MariaDB is first merging the data into some temporary table to apply the
sorting and record limiting.

We can tweak a couple of these queries by changing the "order by" clause
such that they refer to columns within the same table (PackageBases).
So instead performing the second sorting on "Packages.Name", we do
this on "PackageBases.Name" instead.
This should still be "good enough" to produce properly sorted results.

Signed-off-by: moson-mo <mo-son@mailbox.org>
2023-07-02 13:21:11 +02:00
moson-mo
b9df7541b3
fix: add comments in email for direct deletion/merge
TUs and Devs can delete and merge packages directly.
Currently the comments they enter, don't end up in the ML notification.

Include the comment in the notifications for direct deletion / merge

Signed-off-by: moson-mo <mo-son@mailbox.org>
2023-03-14 11:17:45 +01:00
moson-mo
7d06c9ab97
fix: encode package name in URL for source files
Package(Base) names might include characters that require url-encoding

Signed-off-by: moson-mo <mo-son@mailbox.org>
2023-03-01 18:04:20 +01:00
moson-mo
0c5b4721d6
fix: include package data without "Last Packager"
Data for packages that do not have a "Last Packager"
(e.g. because the user account was deleted)
should still be available from the /rpc and metadata archives.

Signed-off-by: moson-mo <mo-son@mailbox.org>
2023-02-21 11:19:02 +01:00
moson-mo
becce1aac4
fix: occasional errors when loading package details
Fixes errors that might occur when loading the package details page.

Problem:
We are querying a list of "Required by" packages.
This list is loaded with all details for a "PackageDependency" record.

Now we also have a reference to some attributes from the
related package (PackageDependency.Package.xxx)

This will effectively trigger the ORM to run another query (lazyload),
to fetch the missing Package data (for each PackageDependency record).

At that point it might have happened that a referenced package
got deleted / updated so that we can't retrieve this data anymore and
our dep.Package object is "None"

Fix:
We can force our query to include Package data right away.
Thus we can avoid running a separate query (per "required by"...)

As a side-effect we get better performance.

Signed-off-by: moson-mo <mo-son@mailbox.org>
2023-01-25 22:34:19 +01:00
moson-mo
7a9448a3e5
perf: improve packages search-query
Improves performance for queries with large result sets.

The "group by" clause can be removed for all search types but the keywords.

Signed-off-by: moson-mo <mo-son@mailbox.org>
2023-01-05 22:00:32 +01:00
moson-mo
bcd808ddc1
feat(rpc): add "by" parameter - comaintainers
Add "by" parameter: comaintainers

Signed-off-by: moson-mo <mo-son@mailbox.org>
2022-11-11 11:32:39 +01:00
moson-mo
efd20ed2c7
feat(rpc): add "by" parameter - keywords
Add "by" parameter: keywords

Signed-off-by: moson-mo <mo-son@mailbox.org>
2022-11-11 11:32:31 +01:00
moson-mo
5484e68b42
feat(rpc): add "by" parameter - submitter
Add "by" parameter: submitter

Signed-off-by: moson-mo <mo-son@mailbox.org>
2022-11-11 11:32:19 +01:00
moson-mo
0583f30a53
feat(rpc): add "by" parameter - groups
Adding "by" parameter to search by "groups"

Signed-off-by: moson-mo <mo-son@mailbox.org>
2022-11-11 11:32:01 +01:00
moson-mo
50287cb066
feat(rpc): add "by" parameters - package relations
This adds new "by" search-parameters: provides, conflicts and replaces

Signed-off-by: moson-mo <mo-son@mailbox.org>
2022-11-11 11:30:44 +01:00
Leonidas Spyropoulos
7e06823e58
refactor: remove redundand parenthesis when return tuple
Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>
2022-10-28 13:43:32 +01:00
Leonidas Spyropoulos
9c0f8f053e
chore: rename logging.py and redis.py to avoid circular imports
Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>
2022-10-22 18:51:38 +01:00
Kevin Morris
ec3152014b
fix: retry transactions who fail due to deadlocks
In my opinion, this kind of handling of transactions is pretty ugly.
The being said, we have issues with running into deadlocks on aur.al,
so this commit works against that immediate bug.

An ideal solution would be to deal with retrying transactions through
the `db.begin()` scope, so we wouldn't have to explicitly annotate
functions as "retry functions," which is what this commit does.

Closes #376

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-09-13 12:54:08 -07:00
Joakim Saario
9c6c13b78a
style: Run pre-commit 2022-08-22 22:40:45 +02:00
Jelle van der Waa
a509e40474 fix(python): use standard dict/list type annotation
Since Python 3.9 list/dict can be used as type hint.
2022-08-02 12:06:58 +00:00
Kevin Morris
ed41a4fe19
feat: add paging to package depends & required by
This patch does not include a javascript implementating, but
provides a pure HTML/HTTP method of paging through these lists.

Also fixes erroneous limiting. We now use a hardcoded limit of 20
by default.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-03-30 17:07:40 -07:00
Colin Woodbury
3aa8d523f5
change(rpc): search module reformatting 2022-02-21 16:56:10 -08:00
Kevin Morris
14347232fd
fix: treat all keywords as lowercase when updating
In addition, treat package search by keywords as lowercase.

Closes #296, #297, #298, #301

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-18 16:58:02 -08:00
Kevin Morris
1d86b3e210
fix: use a transaction for package query; remove refresh
Closes #284

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-18 15:58:42 -08:00
Kevin Morris
e3864d4b7c
fix: set RequestTS when autogenerating requests
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-17 15:54:04 -08:00
Kevin Morris
957803a70b
fix: M/c search with multiple keywords
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-07 16:22:21 -08:00
Kevin Morris
26f0b014f9
fix: /packages search by keywords
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-06 23:29:09 -08:00
Kevin Morris
750653361f
fix: remove /packages search count limit
...took this opportunity to use the new options.max_search_results
tunable for a PP upper-bound.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-06 16:21:56 -08:00
Kevin Morris
39d6f927e6
fix: Maintainer, Co-maintainer /package search
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-05 03:47:16 -08:00
Kevin Morris
28549b47bb
fix: /packages search by co-maintainer
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-05 03:47:15 -08:00
Kevin Morris
b7bf83c5f0
fix: prioritize local db record in pkgname_link
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-04 20:13:24 -08:00
Kevin Morris
c783ce17be
fix: remove erroneous official pkg check
This causes an issue that should have been obvious from the get-go:
if a package request is up in the AUR, but the package has already
been picked up by an official repository, we would end up returning
a 404 here, leading a TU to not be able to perform an action for
a request's target.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-04 19:57:59 -08:00
Kevin Morris
7f6c23d4cb
housekeep: centralize datetime generation
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-18 07:31:04 -08:00
Kevin Morris
64069b9b5d
change(python): use a transaction query in get_pkg_or_base
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-16 02:00:41 -08:00
Kevin Morris
b4495a49bf
fix(rpc): improve type=search performance
This patch brings in the use of .with_entities on our
RPC search query. This primarily fixes performance issues
we were seeing with large queries.

That being said, we do see a bit of a slowdown on
large record count rpc queries, but it's quite negligible
at this point.

We still do aim to perform better than the older PHP
implementation, so this is not a finishing patch by
any means.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-14 00:50:39 -08:00
Kevin Morris
902c4d7a9c
fix(routers.packages): fix repeatead user joins
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-05 19:46:06 -08:00
Kevin Morris
83dc26ccde
fix(packages.request): fix autogenerated merge closure
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-03 21:16:54 -08:00
Steven Guikal
e126d431d7
fix(FastAPI): add custom error templates for certain exceptions
Signed-off-by: Steven Guikal <void@fluix.one>
2022-01-03 18:22:03 -08:00
Kevin Morris
a1f46611e1
change(python): move request & pkgbase request routes
Move package request routes and related routes to their
respective routers. In addition, move some utility used
for requests over from `aurweb.packages`.

Introduced routers:
- `aurweb.routers.requests`

Introduced package:
- `aurweb.requests`

Introduced module:
- `aurweb.requests.util`

Changes:
- Moved `aurweb.packages.validate` to `aurweb.pkgbase.validate`
- Moved requests listing & request closure routes to
  `aurweb.routers.requests`
- Moved pkgbase request creation route to `aurweb.routers.pkgbase`
- Moved `get_pkgreq_by_id` from `aurweb.packages.util` to
  `aurweb.requests.util` and fixed its return type hint.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-02 01:44:36 -08:00
Kevin Morris
a77d44e919
change(python): move comaint routes to pkgbase router
Also brings over comaint utility functions to the pkgbase
package.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-02 00:33:31 -08:00
Kevin Morris
8f8929f324
fix(routers.packages): handle package source display
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-30 23:10:00 -08:00
Kevin Morris
be7a96076e
fix: handle broken packages which have valid provides
Closes #226

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-30 19:49:49 -08:00
Kevin Morris
6fdaeee026
change(packages.util): handle queried record links via .is_official
This removes an unneeded query from our path.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-30 19:49:41 -08:00
Kevin Morris
fc229d755b
change(python): refactor & centralize comaintainer management
This commit centralizes comaintainer management with a few new
functions and uses them more appropriately within routes:

- aurweb.packages.util.latest_priority
- aurweb.packages.util.remove_comaintainer
- aurweb.packages.util.remove_comaintainers
- aurweb.packages.util.add_comaintainer
- aurweb.packages.util.add_comaintainers
- aurweb.packages.util.rotate_comaintainers

Closes #117

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-29 19:50:17 -08:00
Kevin Morris
9d3e77bab1
fix(packages.util.pkg_required): correct type hints and docstring
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-29 14:48:12 -08:00
Kevin Morris
3a771fc807
fix(packages.requests): disown as maintainer does not need handle_requests
As a maintainer, we don't deal with requests at all.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-28 18:31:21 -08:00
Kevin Morris
56bd60559c
fix(packages.search): fix default ordering & improve performance
- Use queries more closely aligned to PHP's implementation; removes
  the need for separate vote/notification queries.
- Default sort by popularity

Closes #214

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-25 11:15:20 -08:00
Kevin Morris
50eec96dd0
fix(routers.packages): fix related package metadata
Closes #218

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-21 18:02:37 -08:00
Kevin Morris
26b1674c9e
fix(requests): rework handling of requests
This commit changes several things about how we were handling
package requests.

Modifications (requests):
-------------
- `/requests/{id}/close` no longer provides an Accepted selection.
  All manual request closures will cause a rejection.
- Relevent `pkgbase` actions now trigger request closures:
  `/pkgbase/{name}/delete` (deletion), `/pkgbase/{name}/merge` (merge)
  and `/pkgbase/{name}/disown` (orphan).
- Comment fields have been added to
  `/pkgbase/{name}/{delete,merge,disown}`, which is used to set the
  `PackageRequest.ClosureComment` on pending requests. If the comment
  field is left blank, a closure comment is autogenerated.
- Autogenerated request notifications are only sent out once
  as a closure notification.
- Some markup has been fixed.

Modifications (disown/orphan):
-----------------------------
- Orphan requests are now handled through the same path as
  deletion/merge.
- We now check for due date when disowning as non-maintainer;
  previously, this was only done for display and not functionally.
  This check applies to Trusted Users' disowning of a package.

This style of notification flow does reduce our visibility, but
accounting can still be done via the close request; it includes
the action, pkgbase name and the user who accepted it.

Closes #204

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-09 19:09:51 -08:00
Kevin Morris
a9a0adaead
fix(python): fix package_link check
This was failing when it matched more than one record.
This fixes that issue by using an EXISTS query.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-12-07 07:44:56 -08:00
Kevin Morris
d8e3ca1abb
change(notify): converted to use aurweb.db ORM
- Removed notify sharness test

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:10 -08:00
Kevin Morris
199622c53f
fix(fastapi): refresh records when fetching updated packages
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-27 21:35:48 -08:00