Commit graph

2424 commits

Author SHA1 Message Date
Kevin Morris
efe99dc16f Support conjunctive keyword search in RPC interface
Newly supported API Version 6 modifies `type=search` for _by_ type
`name-desc`: it now behaves the same as `name-desc` search through the
https://aur.archlinux.org/packages/ search page.

Search for packages containing the literal keyword `blah blah` AND `haha`:
https://aur.archlinux.org/rpc/?v=6&type=search&arg="blah blah"%20haha

Search for packages containing the literal keyword `abc 123`:
https://aur.archlinux.org/rpc/?v=6&type=search&arg="abc 123"

The following example searches for packages that contain `blah` AND `abc`:
https://aur.archlinux.org/rpc/?v=6&type=search&arg=blah%20abc

The legacy method still searches for packages that contain `blah abc`:
https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc
https://aur.archlinux.org/rpc/?v=5&type=search&arg=blah%20abc

API Version 6 is currently only considered during a `search` of `name-desc`.

Note: This change was written as a solution to
https://bugs.archlinux.org/task/49133.

PS: + Some spacing issues fixed in comments.

Signed-off-by: Kevin Morris <kevr.gtalk@gmail.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
239988def7 Build a translation facility for FastAPI
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
e323156947 SSO: Port account suspension
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
0e08b151e5 SSO: Port IP ban checking
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
357dba87b3 Save id_token for the SSO logout
As far as I can see, Keycloak ignores it entirely. I can login in as SSO
user A, then disconnect from the SSO directly and reconnect as user B,
but when I disconnect user A from AUR, Keycloak disconnects B even
though AUR passed it an ID token for A.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
4d0f2d2279 Implement SSO logout
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
d12ea08fca SSO: Add an SSO option in the login page
We’ll probably change the whole login page in the future, but this makes
development easier.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
4bf8228324 SSO: Explain the rationale behind prompt=login
We might reconsider it in the future.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
8d5244d0c0 Fix typos in CONTRIBUTING.md
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
42f8f160b6 Open AUR sessions from SSO
Only the core functionality is implemented here. See the TODOs.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
c77e9d1de0 Integrate SQLAlchemy into FastAPI
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
a5554c19a9 Add SSO account ID in table Users
This column holds a user ID issed by the single sign-on provider. For
Keycloak, it is an UUID. For more flexibility, we will be using a
standardly-sized VARCHAR field.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
3f31d149a6 aurweb.l10n: Translate without side effects
The install method in Python’s gettext API aliases the translator’s
gettext method to an application-global _(). We don’t use that anywhere,
and it’s clear from aurweb’s Translator interface that we want to
translate a piece of text without affecting any global namespace.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
2b439b8199 Guide to setting up Keycloak for the SSO
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
3b347d3989 Crude OpenID Connect client using Authlib
Developers can go to /sso/login to get redirected to the SSO. On
successful login, the ID token is displayed.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
b1300117ac aurweb.spawn: Fix isort errors
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
0e3bd8b596 Remove the FastAPI /hello test route
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
8c868e088c Introduce conf/config.dev for development
conf/config.dev’s purpose is to provide a lighter configuration template
for developers, and split development-specific options off the default
configuration file.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
5be07a8a9e aurweb.spawn: Integrate FastAPI and nginx
aurweb.spawn used to launch only PHP’s built-in server. Now it spawns a
dummy FastAPI application too. Since both stacks spawn their own HTTP
server, aurweb.spawn also spawns nginx as a reverse proxy to mount them
under the same base URL, defined by aur_location in the configuration.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Filipe Laíns
d4abe0b72d Add CONTRIBUTING.md
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Filipe Laíns
41a8493411 pre-commit: add initial config
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Filipe Laíns
8f47b8d731 isort: add initial config
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Filipe Laíns
4cf94816ae flake8: add initial config
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Filipe Laíns
8d1be7ea8a Refactor code to comply with flake8 and isort
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Filipe Laíns
48b58b1c2f ci: remove Travis CI
We are are moving to Gitlab CI.

Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Frédéric Mangano-Tarumi
8a13500535 Create aurweb.spawn for spawing the test server
This program makes it easier for developers to spawn the PHP server
since it fetches automatically what it needs from the configuration
file, rather than having the user explicitly pass arguments to the php
executable.

When the setup gets more complicated as we introduce Python,
aurweb.spawn will keep providing the same interface, while under the
hood it is planned to support running multiple sub-processes.

Its Python interface provides an way for the test suite to spawn the
test server when it needs to perform HTTP requests to the test server.

The current implementation is somewhat weak as it doesn’t detect when a
child process dies, but this is not supposed to happen often, and it is
only meant for aurweb developers.

In the long term, aurweb.spawn will eventually become obsolete, and
replaced by Docker or Flask’s tools.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Filipe Laíns
23f6dd16a7 ci: add cache to gitlab ci
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Filipe Laíns
db02227cc4 ci: add gitlab ci
Signed-off-by: Filipe Laíns <lains@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:24:30 -05:00
Eli Schwartz
71740a75a2 rewrite query to support both mysql/sqlite
Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:22:11 -05:00
Eli Schwartz
be5197a5fe prevent running mysql-specific query in sqlite
We usually guard such queries and have both mysql and sqlite branches.
But I have not implemented the sqlite branch. Given sqlite is typically
used for local dev setups, the fact that "users with more than the
configured max simultaneous logins" can avoid getting some logins
annulled is probably not a huge risk.

And this always *used* to fail on sqlite, silently. Now, in php 8, it
raises PDOException, which prevents running the test server

Document this as a FIXME for now, until someone reimplements the query
for sqlite.

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:22:11 -05:00
Eli Schwartz
d92dd69aa3 fix broken SQL query that always failed
Due to missing whitespace at the end of strings during joining, we ended
up with the query fragment

"DelTS IS NULLAND NOT PinnedTS"

which should be

"DelTS IS NULL AND NOT PinnedTS"

So the check for pinned comments > 5 likely always failed.

In php 7, a completely broken query that raises exceptions in the
database engine was silently ignored... in php 8, it raises

Uncaught PDOException: SQLSTATE[HY000]: General error: 1 near "PinnedTS": syntax error in <file>

and aborts the page building. End result: users with permission to pin
comments cannot see any comments, or indeed page content below the first
comment header

Signed-off-by: Eli Schwartz <eschwartz@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-02-20 11:22:11 -05:00
Frederik Schwan
d5e308550a Fix requests not being sent to the Cc recipients
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-10-13 20:26:51 -04:00
Morten Linderud
613364b773 pkg_search_page: Limit number of results on package search
The current package search query is quite poorly optimized and becomes a
resource hog when the offsets gets large enough. This DoSes the service.

A quick fix is to just ensure we have some limit to the number of hits
we return. The current hardcoding of 2500 is based on the following:

    * 250 hits per page max
    * 10 pages

We can maybe consider having it lower, but it seems easier to just have
this a multiple of 250 in the first iteration.

Signed-off-by: Morten Linderud <morten@linderud.pw>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-09-05 10:21:16 -04:00
Lukas Fleischer
c4f4ac510b Deliver emails to Cc in smtplib code path
When using the sendmail() function with smtplib.SMTP or
smtplib.SMTP_SSL, the list of actual recipients for the email (to be
translated to RCPT commands) has to be provided as a parameter.

Update the notification script and add all Cc recipients to that
parameter.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-08-27 07:19:57 -04:00
Jelle van der Waa
03a6fa2f7e Call sendmail with to, not recipient
After f7a57c8 (Localize notification emails, 2018-05-17), the
server.sendmail line was not updated to now send the to the email
address but instead sends to (email, 'en') and as sendmail accepts an
iterable an email is also send to 'en'.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-08-26 08:32:32 -04:00
Lukas Fleischer
169607f153 Fix PHP notices in the account form
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-04-05 15:59:56 -04:00
Lukas Fleischer
1369eb87b3 Fix invalid session ID check
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-04-05 15:59:55 -04:00
Lukas Fleischer
853ed9a950 Release 5.0.0
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-03-27 08:51:15 -04:00
Lukas Fleischer
279d8042e3 Add new upgrade instructions
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-03-27 08:49:34 -04:00
Lukas Fleischer
a09c4d8168 Translation updates from Transifex
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-03-27 08:31:46 -04:00
Frédéric Mangano-Tarumi
31a5b40b5c Map BIGINT to INTEGER for SQLite
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-03-22 16:50:10 -04:00
Frédéric Mangano-Tarumi
28ba3f77dc Write test/README.md to help working with tests
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-29 14:57:46 +01:00
Frédéric Mangano-Tarumi
bf7c49158c test/Makefile: Run tests with prove when available
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-29 14:56:47 +01:00
Frédéric Mangano-Tarumi
90c0a361b5 Support running tests from any directory
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-29 14:56:47 +01:00
Frédéric Mangano-Tarumi
e374a91feb Change the extension of TAP test suites to .t
This is the common convention for TAP, and makes harnesses like prove
automatically detect them. Plus, test suites don’t have to be shell
scripts anymore.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00
Frédéric Mangano-Tarumi
81d55e70ee Disable Alembic support on test databases
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00
Frédéric Mangano-Tarumi
e4cbe264cf Create an initial Alembic migration
This way the database will get stamped, and Git will create the
`versions` directory without which Alembic won’t work.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00
Frédéric Mangano-Tarumi
a8a1f74a92 Set up Alembic for database migrations
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00
Frédéric Mangano-Tarumi
7188743fc3 Migrate the database schema to SQLAlchemy
The new schema was generated with sqlacodegen and then manually adjusted
to fit schema/aur-schema.sql faithfully, both in the organisation of the
code and in the SQL generated by SQLAlchemy.

Initializing the database now requires the new tool aurweb.initdb.
References to aur-schema.sql have been updated and the old schema
dropped.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00
Lukas Fleischer
4b2102ceb2 Properly escape passwords in the account edit form
Addresses FS#65639.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00