Commit graph

2673 commits

Author SHA1 Message Date
Kevin Morris
f8a6049de2 aurweb.db.session: Use autoflush=True for Sessions
We'd like SQLAlchemy to automatically maintain flushes for us.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
794868b20f aurweb.testing.setup_test_db: Expunge objects
This is needed to avoid redundant objects in SQLAlchemy's
IdentityMap, since we pass a direct .execute to delete
the tables passed in. Additionally, remove our engine.connect()
call in favor of relying on the already-established Session.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
b692b11f62 add Group SQLAlchemy ORM model
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
e1ab02c2bf Fix database initialization in test_term.py
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
718fa48a5c add AcceptedTerm SQLAlchemy ORM model
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
29db2ee513 add Term SQLAlchemy ORM model
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
fb21015811 add PackageKeyword SQLAlchemy ORM model
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
a7e5498197 add PackageBase SQLAlchemy ORM model
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
822905be7d bugfix: relax next verification
AUR renders its own 404 Not Found page when a bad route
is encountered. Introducing the previous verification
caused an error in this case when setting a language
while viewing the Not Found page. So, instead of checking
through routes, just make sure that the next parameter
starts with a '/' character, which removes the possibility
of any cross attacks.

+ Removed aurweb.asgi.routes; no longer needed.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Leonidas Spyropoulos
32abdbafae fastapi: Jinja contextfilter renamed to pass_context
Closes: #23

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
2021-06-05 20:11:17 -07:00
Kevin Morris
4f928b4577 add account (view) route
+ Added get /account/{username} route.
+ Added account/show.html template which shows a single use

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
4e9ef6fb00 add account edit (settings) routes
* Added account_url filter to jinja2 environment. This produces a path
  to the user's account url (/account/{username}).
* Updated archdev-navbar to link to new edit route.
+ Added migrate_cookies(request, response) to aurweb.util, a function
  that simply migrates the request cookies to response and returns it.
+ Added account_edit tests to test_accounts_routes.py.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
d323c1f95b add python-lxml to dependencies
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
c94793b0b1 add user registration routes
* Added /register get and post routes.
+ Added default attributes to AnonymousUser, including a new
  AnonymousList which behaves like an sqlalchemy relationship
  list.
+ aurweb.util: Added validation functions for various user fields
  used throughout registration.
+ test_accounts_routes: Added get|post register route tests.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
19b4a896f1 add openssh to test dependencies
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
df0a637d2b add aurweb.captcha, a CAPTCHA utility module
This CAPTCHA workflow is the same workflow used by our current
PHP implementation of account registration.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
7a6a38592e add python-email-validator dependency
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
a5be6fc9be aurweb.templates: add make_variable_context
A new make_context wrapper which additionally includes either
query parameters (get) or form data (post) in the context.

Use this to simplify setting context variables for form data
in particular.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
9052688ed2 add aurweb.time module
This module includes timezone-based utilities for a FastAPI request.
This commit introduces use of the AURTZ cookie within get_request_timezone.
This cookie should be set to the user or session's timezone.

* `make_context` has been modified to parse the request's timezone
  and include the "timezone" and "timezones" variables, along with
  a timezone specified "now" date.
+ Added `Timezone` attribute to aurweb.testing.requests.Request.user.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
07d5907ecd aurweb.auth: add user credentials and matcher functions
This clones the behavior already present in the PHP implementation,
but it uses a global dict with credential constant keys to
validation functions to determine if a given user has a credential.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
670f711b59 add SSHPubKey ORM model
Includes `aurweb.models.ssh_pub_key.get_fingerprint(pubkey)` helper.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
9fdbe3f775 add authenticated User LangPreference tracking
+ Use User.LangPreference when there is no set AURSID
  if request.user.is_authenticated is true.
+ Updated post /language to update LangPreference when
  request.user.is_authenticated.
+ Restore language during test where we change it.
+ Added the user attribute to aurweb.testing.requests.Request.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
a33d076d8b add passreset routes
Introduced `get|post` `/passreset` routes. These routes mimic the
behavior of the existing PHP implementation, with the exception of
HTTP status code returns.

Routes added:
    GET /passreset
    POST /passreset

Routers added:
    aurweb.routers.accounts

* On an unknown user or mismatched resetkey (where resetkey must ==
  user.resetkey), return HTTP status NOT_FOUND (404).
* On another error in the request, return HTTP status BAD_REQUEST (400).

Both `get|post` routes requires that the current user is **not**
authenticated, hence `@auth_required(False, redirect="/")`.

+ Added auth_required decorator to aurweb.auth.
+ Added some more utility to aurweb.models.user.User.
+ Added `partials/error.html` template.
+ Added `passreset.html` template.
+ Added aurweb.db.ConnectionExecutor functor for paramstyle logic.
  Decoupling the executor logic from the database connection logic
  is needed for us to easily use the same logic with a fastapi
  database session, when we need to use aurweb.scripts modules.

At this point, notification configuration is now required to complete
tests involved with notifications properly, like passreset.
`conf/config.dev` has been modified to include [notifications] sendmail,
sender and reply-to overrides. Dockerfile and .gitlab-ci.yml have been
updated to setup /etc/hosts and start postfix before running tests.

* setup.cfg: ignore E741, C901 in aurweb.routers.accounts

These two warnings (shown in the commit) are not dangerous and a bi-product
of maintaining compatibility with our current code flow.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
4423326cec add the request parameter to render_template
This allows us to inspect things about the request we're rendering from.

* Use render_template(request, ...) in aurweb.routers.auth

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
5d4a5deddf implement login + logout routes and templates
+ Added route: GET `/login` via `aurweb.routers.auth.login_get`
+ Added route: POST `/login` via `aurweb.routers.auth.login_post`
+ Added route: GET `/logout` via `aurweb.routers.auth.logout`
+ Added route: POST `/logout` via `aurweb.routers.auth.logout_post`
* Modify archdev-navbar.html template to toggle displays on auth state
+ Added login.html template

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
56f2798279 add aurweb.auth and authentication to User
+ Added aurweb.auth.AnonymousUser
    * An instance of this model is returned as the request user
      when the request is not authenticated
+ Added aurweb.auth.BasicAuthBackend
+ Add starlette's AuthenticationMiddleware to app middleware,
  which uses our BasicAuthBackend facility
+ Added User.is_authenticated()
+ Added User.authenticate(password)
+ Added User.login(request, password)
+ Added User.logout(request)
+ Added repr(User(...)) representation
+ Added aurweb.auth.auth_required decorator.

This change uses the same AURSID logic in the PHP implementation.

Additionally, introduce a few helpers for authentication,
one of which being `User.update_password(password, rounds = 12)`
where `rounds` is a configurable number of salt rounds.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
137c050f99 add python-bcrypt dependency
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
1922e5380d add aurweb.models.session.Session ORM database object
+ Added aurweb.util module.
    - Added make_random_string function.
+ Added aurweb.db.make_random_value function.
    - Takes a model and a column and introspects them to figure out the
      proper column length to create a random string for; then creates
      a unique string for that column.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
adc9fccb7d add aurweb.models.ban.Ban ORM mapping
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
a836892cde aurweb.db: add query, create, delete helpers
Takes sqlalchemy kwargs or stanzas:

query(Model, Model.Column == value)
query(Model, and_(Model.Column == value, Model.Column != "BAD!"))

Updated tests to reflect the new utility and a comment about upcoming
function deprecation is added to get_account_type().

From here on, phase out the use of get_account_type().

+ aurweb.db: Added create utility function
+ aurweb.db: Added delete utility function

The `delete` function can be used to delete a record by search
kwargs directly.

Example:
    delete(User, User.ID == 6)

All three functions added in this commit are typically useful to
perform these operations without having to import aurweb.db.session.
Removes a bit of redundancy overall.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Kevin Morris
5185df629e move aurweb.testing to its own package
+ Added aurweb.testing.setup_test_db(*tables)
+ Added aurweb.testing.models.make_user(**kwargs)
+ Added aurweb.testing.models.make_session(**kwargs)
+ Added aurweb.testing.requests.Client
+ Added aurweb.testing.requests.Request
* Updated test_l10n.py to use our new Request

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 20:11:17 -07:00
Leonidas Spyropoulos
64bc93926f Add support for configuring database with port instead of socket
Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
2021-06-05 20:11:17 -07:00
Kristian Klausen
ac31f520ea Add coverage report for "Test Coverage Visualization"[1]
[1] https://docs.gitlab.com/ee/user/project/merge_requests/test_coverage_visualization.html
2021-06-05 20:11:17 -07:00
Leonidas Spyropoulos
7b7c3abbe2 Conditionally apply SSOAccountId migration to support misaligned databases
Closes: #34

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
2021-06-05 20:11:17 -07:00
Leonidas Spyropoulos
72f755817c Adds Alembic migration for DB/Tables conversion to utf8mb4
MySql defaults to `utf8` and case insensitive collation so migrate these to case sensitive and `utf8mb4`

Closes #21

Signed-off-by: Leonidas Spyropoulos <artafinde@gmail.com>
2021-06-05 20:11:04 -07:00
Kevin Morris
66189c4460 alembic: restore logging, fix pytest conflicts
In this case, when running pytests, we do not allow alembic to
configure loggers.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:56 -07:00
Kevin Morris
3f1f03e03c aurweb.db: only pass check_same_thread with sqlite
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:56 -07:00
Kevin Morris
e0eb6b0e76 test_db: remove use of mkdtemp and os.removedirs
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:56 -07:00
Marcus Andersson
1d5827007f Adding route tests
Removing status code from 404 title

Removing status code from 503 title

Adding id to 503 error box

Indatation fix
2021-06-05 19:52:56 -07:00
Marcus Andersson
f6744d3e39 Adding error 503 catcher 2021-06-05 19:52:56 -07:00
Marcus Andersson
cdf75ced9a Adding error 404 catcher 2021-06-05 19:52:56 -07:00
Kevin Morris
82f3871a83 Support SQLAlchemy 1.4 URL.create recommendation
This fixes a deprecating warning when using SQLAlchemy 1.4.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:56 -07:00
Kevin Morris
81856f3b64 Fix incorrect construction of MySQL SQLAlchemy URL
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:49 -07:00
Kevin Morris
02311eab76 add test_initdb.py
IMPORTANT: This test completely wipes out the database it's using.
Make sure you've got AUR_CONFIG set to a test database configuration!

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:42 -07:00
Kevin Morris
8a47afd2ea add aurweb.models.user.User
+ Added aurweb.models.user.User class. This is the first example
  of an sqlalchemy ORM model. We can search for users via for example:
  `session.query(User).filter(User.ID==1).first()`, where `session` is
  a configured `aurweb.db.session` object.
+ Along with the User class, defined the AccountType class.
  Each User maintains a relationship to its AccountType via User.AccountType.
+ Added AccountType.users backref.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:42 -07:00
Kevin Morris
e860d828b6 add aurweb.testing, a module with testing utilities
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:42 -07:00
Kevin Morris
32f2893095 add aurweb.models.account_type.AccountType
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:42 -07:00
Kevin Morris
4238a9fc68 add aurweb.db.session
+ Added Session class and global session object to aurweb.db,
  these are sessions created by sqlalchemy ORM's sessionmaker
  and will allow us to use declarative/imperative models.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:52:40 -07:00
Kevin Morris
7c65604dad move off env.py's active code to __name__ == "__main__"
* Moved migrations/env.py's logging initialization and migration execution
  into a `__name__ == "__main__"` stanza so it doesn't immediately happen
  when imported by another module.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:50:51 -07:00
Kevin Morris
2df90ce280 port over base HTML layout from PHP to FastAPI+Jinja2
+ Mounted static files (at web/html) to /static.
+ Added AURWEB_VERSION to aurweb.config (this is used around HTML
  to refer back to aurweb's release on git.archlinux.org), so we
  need it easily accessible in the Python codebase.
+ Implemented basic Jinja2 partials to put together whole aurweb
  pages. This may be missing some things currently and is a WIP
  until this set is ready to be merged.
+ Added config [options] aurwebdir = YOUR_AUR_ROOT; this configuration
  option should specify the root directory of the aurweb project.
  It is used by various parts of the FastAPI codebase to target
  project directories.

Added routes via aurweb.routers.html:
    * POST /language: Set your session language.
    * GET /favicon.ico: Redirect to /static/images/favicon.ico.
        * Some browsers always look for $ROOT/favicon.ico to get an icon
          for the page being loaded, regardless of a specified "shortcut
          icon" given in a <link> directive.
    * GET /: Home page; WIP.

* Updated aurweb.routers.html.language passes query parameters to
  its next redirection.

When calling aurweb.templates.render_template, the context passed should
be formed via the aurweb.templates.make_context. See
aurweb.routers.html.index for an example of this.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-06-05 19:50:51 -07:00