Commit graph

132 commits

Author SHA1 Message Date
Kevin Morris
c94793b0b1 add user registration routes
* Added /register get and post routes.
+ Added default attributes to AnonymousUser, including a new
  AnonymousList which behaves like an sqlalchemy relationship
  list.
+ aurweb.util: Added validation functions for various user fields
  used throughout registration.
+ test_accounts_routes: Added get|post register route tests.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
19b4a896f1 add openssh to test dependencies
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
df0a637d2b add aurweb.captcha, a CAPTCHA utility module
This CAPTCHA workflow is the same workflow used by our current
PHP implementation of account registration.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
9052688ed2 add aurweb.time module
This module includes timezone-based utilities for a FastAPI request.
This commit introduces use of the AURTZ cookie within get_request_timezone.
This cookie should be set to the user or session's timezone.

* `make_context` has been modified to parse the request's timezone
  and include the "timezone" and "timezones" variables, along with
  a timezone specified "now" date.
+ Added `Timezone` attribute to aurweb.testing.requests.Request.user.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
07d5907ecd aurweb.auth: add user credentials and matcher functions
This clones the behavior already present in the PHP implementation,
but it uses a global dict with credential constant keys to
validation functions to determine if a given user has a credential.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
670f711b59 add SSHPubKey ORM model
Includes `aurweb.models.ssh_pub_key.get_fingerprint(pubkey)` helper.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
9fdbe3f775 add authenticated User LangPreference tracking
+ Use User.LangPreference when there is no set AURSID
  if request.user.is_authenticated is true.
+ Updated post /language to update LangPreference when
  request.user.is_authenticated.
+ Restore language during test where we change it.
+ Added the user attribute to aurweb.testing.requests.Request.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
a33d076d8b add passreset routes
Introduced `get|post` `/passreset` routes. These routes mimic the
behavior of the existing PHP implementation, with the exception of
HTTP status code returns.

Routes added:
    GET /passreset
    POST /passreset

Routers added:
    aurweb.routers.accounts

* On an unknown user or mismatched resetkey (where resetkey must ==
  user.resetkey), return HTTP status NOT_FOUND (404).
* On another error in the request, return HTTP status BAD_REQUEST (400).

Both `get|post` routes requires that the current user is **not**
authenticated, hence `@auth_required(False, redirect="/")`.

+ Added auth_required decorator to aurweb.auth.
+ Added some more utility to aurweb.models.user.User.
+ Added `partials/error.html` template.
+ Added `passreset.html` template.
+ Added aurweb.db.ConnectionExecutor functor for paramstyle logic.
  Decoupling the executor logic from the database connection logic
  is needed for us to easily use the same logic with a fastapi
  database session, when we need to use aurweb.scripts modules.

At this point, notification configuration is now required to complete
tests involved with notifications properly, like passreset.
`conf/config.dev` has been modified to include [notifications] sendmail,
sender and reply-to overrides. Dockerfile and .gitlab-ci.yml have been
updated to setup /etc/hosts and start postfix before running tests.

* setup.cfg: ignore E741, C901 in aurweb.routers.accounts

These two warnings (shown in the commit) are not dangerous and a bi-product
of maintaining compatibility with our current code flow.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
5d4a5deddf implement login + logout routes and templates
+ Added route: GET `/login` via `aurweb.routers.auth.login_get`
+ Added route: POST `/login` via `aurweb.routers.auth.login_post`
+ Added route: GET `/logout` via `aurweb.routers.auth.logout`
+ Added route: POST `/logout` via `aurweb.routers.auth.logout_post`
* Modify archdev-navbar.html template to toggle displays on auth state
+ Added login.html template

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
56f2798279 add aurweb.auth and authentication to User
+ Added aurweb.auth.AnonymousUser
    * An instance of this model is returned as the request user
      when the request is not authenticated
+ Added aurweb.auth.BasicAuthBackend
+ Add starlette's AuthenticationMiddleware to app middleware,
  which uses our BasicAuthBackend facility
+ Added User.is_authenticated()
+ Added User.authenticate(password)
+ Added User.login(request, password)
+ Added User.logout(request)
+ Added repr(User(...)) representation
+ Added aurweb.auth.auth_required decorator.

This change uses the same AURSID logic in the PHP implementation.

Additionally, introduce a few helpers for authentication,
one of which being `User.update_password(password, rounds = 12)`
where `rounds` is a configurable number of salt rounds.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
1922e5380d add aurweb.models.session.Session ORM database object
+ Added aurweb.util module.
    - Added make_random_string function.
+ Added aurweb.db.make_random_value function.
    - Takes a model and a column and introspects them to figure out the
      proper column length to create a random string for; then creates
      a unique string for that column.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
adc9fccb7d add aurweb.models.ban.Ban ORM mapping
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
a836892cde aurweb.db: add query, create, delete helpers
Takes sqlalchemy kwargs or stanzas:

query(Model, Model.Column == value)
query(Model, and_(Model.Column == value, Model.Column != "BAD!"))

Updated tests to reflect the new utility and a comment about upcoming
function deprecation is added to get_account_type().

From here on, phase out the use of get_account_type().

+ aurweb.db: Added create utility function
+ aurweb.db: Added delete utility function

The `delete` function can be used to delete a record by search
kwargs directly.

Example:
    delete(User, User.ID == 6)

All three functions added in this commit are typically useful to
perform these operations without having to import aurweb.db.session.
Removes a bit of redundancy overall.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
5185df629e move aurweb.testing to its own package
+ Added aurweb.testing.setup_test_db(*tables)
+ Added aurweb.testing.models.make_user(**kwargs)
+ Added aurweb.testing.models.make_session(**kwargs)
+ Added aurweb.testing.requests.Client
+ Added aurweb.testing.requests.Request
* Updated test_l10n.py to use our new Request

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Leonidas Spyropoulos
64bc93926f Add support for configuring database with port instead of socket
Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
2021-06-05 20:11:17 -07:00
Kevin Morris
e0eb6b0e76 test_db: remove use of mkdtemp and os.removedirs
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:56 -07:00
Marcus Andersson
1d5827007f Adding route tests
Removing status code from 404 title

Removing status code from 503 title

Adding id to 503 error box

Indatation fix
2021-06-05 19:52:56 -07:00
Kevin Morris
02311eab76 add test_initdb.py
IMPORTANT: This test completely wipes out the database it's using.
Make sure you've got AUR_CONFIG set to a test database configuration!

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:42 -07:00
Kevin Morris
8a47afd2ea add aurweb.models.user.User
+ Added aurweb.models.user.User class. This is the first example
  of an sqlalchemy ORM model. We can search for users via for example:
  `session.query(User).filter(User.ID==1).first()`, where `session` is
  a configured `aurweb.db.session` object.
+ Along with the User class, defined the AccountType class.
  Each User maintains a relationship to its AccountType via User.AccountType.
+ Added AccountType.users backref.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:42 -07:00
Kevin Morris
32f2893095 add aurweb.models.account_type.AccountType
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:42 -07:00
Kevin Morris
4238a9fc68 add aurweb.db.session
+ Added Session class and global session object to aurweb.db,
  these are sessions created by sqlalchemy ORM's sessionmaker
  and will allow us to use declarative/imperative models.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:40 -07:00
Kevin Morris
2df90ce280 port over base HTML layout from PHP to FastAPI+Jinja2
+ Mounted static files (at web/html) to /static.
+ Added AURWEB_VERSION to aurweb.config (this is used around HTML
  to refer back to aurweb's release on git.archlinux.org), so we
  need it easily accessible in the Python codebase.
+ Implemented basic Jinja2 partials to put together whole aurweb
  pages. This may be missing some things currently and is a WIP
  until this set is ready to be merged.
+ Added config [options] aurwebdir = YOUR_AUR_ROOT; this configuration
  option should specify the root directory of the aurweb project.
  It is used by various parts of the FastAPI codebase to target
  project directories.

Added routes via aurweb.routers.html:
    * POST /language: Set your session language.
    * GET /favicon.ico: Redirect to /static/images/favicon.ico.
        * Some browsers always look for $ROOT/favicon.ico to get an icon
          for the page being loaded, regardless of a specified "shortcut
          icon" given in a <link> directive.
    * GET /: Home page; WIP.

* Updated aurweb.routers.html.language passes query parameters to
  its next redirection.

When calling aurweb.templates.render_template, the context passed should
be formed via the aurweb.templates.make_context. See
aurweb.routers.html.index for an example of this.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:50:51 -07:00
Kevin Morris
c1e29e90ca aurweb: Globalize a Translator instance, add more utility
+ Added SUPPORTED_LANGUAGES, a global constant dictionary of
  language => display pairs for languages we support.
+ Add Translator.get_translator, a function used to retrieve a
  translator after initializing it (if needed). Use `fallback=True`
  while creating languages, in case we setup a language that we
  don't have a translation for, it will noop the translation.
  This is particularly useful for "en," since we do not translate
  it, but doing this will allow us to go through our normal translation
  flow in any case.
+ Added typing.
+ Added get_request_language, a function that grabs the language for
  a request session, defaulting to aurweb.config [options] default_lang.
+ Added get_raw_translator_for_request, a function that retrieves
  the concrete translation object for a given language.
+ Added tr, a jinja2 contextfilter that can be used to inline translate
  strings in jinja2 templates.
+ Added `python-jinja` dep to .gitlab-ci.yml. This needs to be
  included in documentation before this set is merged in.
+ Introduce pytest units (test_l10n.py) in `test` along with
  __init__.py, which marks `test` as a test package.
+ Additionally, fix up notify.py to use the global translator. Also
  reduce its source width to <= 80 by newlining some code.
+ Additionally, prepare locale in .gitlab-ci.yml and add
  aurweb.config [options] localedir to config.dev with YOUR_AUR_ROOT
  like others.

Signed-off-by: Kevin Morris <kevr@0cost.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2021-06-05 19:49:42 -07:00
Kevin Morris
52ab056e18 update documentation for FastAPI tests and deps.
Additionally, we now ask for two more favors from contributors:

1. All source modified or added within a patchset **must** maintain
   equivalent or increased coverage by providing tests that use the
   functionality.
2. Please keep your source within an 80 column width.

PS: Sneak a few test Makefile and gitlab fixes.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:49:19 -07:00
Kevin Morris
6d08789ac1 add test_popupdate.py
We had no coverage over aurweb.scripts.popupdate. This test covers
all of its functionality.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:49:19 -07:00
Kevin Morris
4b7609681d add test_exceptions.py
This helps gain coverage over aurweb.exceptions regardless
of their actual use in the testing base.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:49:19 -07:00
Kevin Morris
e800cefe95 Makefile: run pytest units
Important note: Python tests will repeatedly clear out tables
that they test against; for this reason, one should always run
the shell tests first. The __init__.py file is necessary for
coverage to collect data from the tests being run.

At this point in FastAPI development, I'd like to encourage a
few things going forward:

1. Any time you contribute to the FastAPI codebase, you **must**
   maintain equal or increased coverage on the overall source.
   Developers are highly appreciated for adding tests in your
   specific domain of addition or modification that may be missing
   coverage. Our goal is 100% coverage, and all newly added files
   **must** have 100% coverage through tests.
2. All source should be formatted with the autopep8 tool and
   kept within an 80 column width, with the exception of HTML
   templates.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:49:19 -07:00
Lukas Fleischer
c4f4ac510b Deliver emails to Cc in smtplib code path
When using the sendmail() function with smtplib.SMTP or
smtplib.SMTP_SSL, the list of actual recipients for the email (to be
translated to RCPT commands) has to be provided as a parameter.

Update the notification script and add all Cc recipients to that
parameter.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-08-27 07:19:57 -04:00
Frédéric Mangano-Tarumi
28ba3f77dc Write test/README.md to help working with tests
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-29 14:57:46 +01:00
Frédéric Mangano-Tarumi
bf7c49158c test/Makefile: Run tests with prove when available
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-29 14:56:47 +01:00
Frédéric Mangano-Tarumi
90c0a361b5 Support running tests from any directory
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-29 14:56:47 +01:00
Frédéric Mangano-Tarumi
e374a91feb Change the extension of TAP test suites to .t
This is the common convention for TAP, and makes harnesses like prove
automatically detect them. Plus, test suites don’t have to be shell
scripts anymore.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00
Frédéric Mangano-Tarumi
81d55e70ee Disable Alembic support on test databases
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00
Frédéric Mangano-Tarumi
7188743fc3 Migrate the database schema to SQLAlchemy
The new schema was generated with sqlacodegen and then manually adjusted
to fit schema/aur-schema.sql faithfully, both in the organisation of the
code and in the SQL generated by SQLAlchemy.

Initializing the database now requires the new tool aurweb.initdb.
References to aur-schema.sql have been updated and the old schema
dropped.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-27 16:44:36 +01:00
Frédéric Mangano-Tarumi
e15d5c8180 rendercomment: use python-markdown's new registration API
First, this gets rid of the deprecation warnings Python displayed.

Second, this fixes the case where a link contained a pair of
underscores, which used to be interpreted as an emphasis because the
linkify processor ran after the emphasis processor.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-02 20:49:16 +01:00
Frédéric Mangano-Tarumi
81faab9978 rendercomment: test headings lowering
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-02 20:49:16 +01:00
Frédéric Mangano-Tarumi
127bb4c84c rendercomment: safer Flyspray task linkification
When an FS#123 is part of a code block, it must not be converted into a
link. FS#123 may also appear inside an URL, in which case regular
linkifaction of URLs must take precedence.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-02 20:49:16 +01:00
Frédéric Mangano-Tarumi
199f34e42e rendercomment: safer auto-linkification of URLs
Fixes a few edge cases:

- URLs within code blocks used to get redundant <> added, breaking bash
  code snippets like `curl https://...` into `curl <https://...>`.

- Links written with markdown's <https://...> syntax also used to get an
  extra pair of brackets.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-02 12:12:43 +01:00
Frédéric Mangano-Tarumi
0fc69e96bd rendercomment: add a test for Git commit links
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-02-02 12:12:43 +01:00
Lukas Fleischer
d0e5c3db69 t2500: fix test cases
Since commit eeaa1c3 (Separate text from footer in notification emails,
2020-01-04), information about unsubscribing from notifications is added
in a signature block. Fix the test cases accordingly.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2020-01-30 13:25:15 +01:00
Lukas Fleischer
ee959c9907 t2500: fix test case for orphan request notifications
Since commit a66c7fa (notify.py: Use a/an correctly when sending request
notifications, 2019-08-09), the body of notification emails sent when
filing orphan requests refers to "an orphan request" instead of "a
orphan request".

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2019-11-23 16:23:00 -05:00
Lukas Fleischer
2422fb020b Store timestamp and user ID when closing requests
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2019-11-23 12:30:46 -05:00
Lukas Fleischer
882c011e74 Upgrade Sharness to 1.1.0
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2019-11-23 12:25:23 -05:00
Lukas Fleischer
8a2f13f8c2 t2500: add test for disown notifications
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2018-08-12 13:41:56 +02:00
Lukas Fleischer
0ae1ca15e9 t2500: use unique identifiers
Use disjoint sets of IDs for users, package bases, package comments and
package requests to ensure the notification script expects the
parameters in the same order we pass them.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2018-08-12 13:40:51 +02:00
Lukas Fleischer
bf5a79da6b Initialize locale directory for tests
Since commit a7865ef (Make the locale directory configurable,
2018-07-22), we need to specify the locale directory in the
configuration file.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2018-08-12 13:40:51 +02:00
Lukas Fleischer
b70f048bc3 Add package base name in request close notifications
Mention both the package base name and the request type in the subject
of request closure notification.

Implements FS#41607.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2018-05-20 16:55:58 +02:00
Lukas Fleischer
f7a57c82bc Localize notification emails
Add support for translating notification emails and send localized
notifications, based on the user's language preferences. Also, update
the translations Makefile to add strings from the notification script
to the message catalog.

Implements FS#31850.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2018-05-17 22:58:54 +02:00
Lukas Fleischer
fec253a65d t2500: Add test cases for all notifications
Check that for all kinds of notifications, the generated messages match
what we expect.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2018-05-17 22:05:33 +02:00
Lukas Fleischer
ce93360257 Erase login IP addresses after seven days
Add a script to periodically remove old IP addresses from the users
database.

The login IP addresses are stored for spam protection and to prevent
from abuse. It is quite unlikely that we ever need the IP address of a
user whose last login is more than a week old. It makes sense to remove
such IP addresses to protect our users' privacy.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2018-05-10 21:38:25 +02:00