external/aurweb
0
0
Fork 0
mirror of https://gitlab.archlinux.org/archlinux/aurweb.git synced 2025-02-03 10:43:03 +01:00
Code Issues Projects Releases Packages Wiki Activity Actions

Update Testing Guide

Kevin Morris 2021-10-27 08:09:16 +00:00
parent 17a26ea9db
commit fea6f02c7f
1 changed files with 36 additions and 373 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

409
Testing-Guide.md

@ -18,59 +18,6 @@ Branches containing FastAPI are located in the pipeline/coverage status table be
|--------------|
| ![pipeline](https://gitlab.archlinux.org/archlinux/aurweb/badges/pu/pipeline.svg?key_text=build) ![coverage](https://gitlab.archlinux.org/archlinux/aurweb/badges/pu/coverage.svg) |
## Locations
Testers should choose one of the following methods:
- [aur-dev.archlinux.org Testing](#aur-devarchlinuxorg-testing)
- [Docker Testing](#docker-testing)
**Too long, didn't read?** Running a local Docker instance is much more flexible
in terms of testing -- testers have complete access to the MariaDB database
used by both the `php-fpm` and `fastapi` services. For users who do not wish
to interact with Docker, the `aur-dev.archlinux.org` instance can be used
to perform any testing.
See [Accounts](#accounts) for an explanation on obtaining accounts
in either system.
### aur-dev.archlinux.org Testing
A Docker instance of the `pu` branch is hosted at https://aur-dev.archlinux.org
and can be used by anybody to test the code-base.
### Docker Testing
The following table lists out requirements needed to run our `docker-compose`
services:
| Package | Distribution | Method |
|------------------|--------------|-----------------------------------------|
| docker | Arch Linux | `pacman -S docker` |
| docker-compose | Arch Linux | `pacman -S docker-compose` |
| poetry | Arch Linux | `pacman -S python-poetry` |
| docker | Other | https://docs.docker.com/engine/install/ |
| docker-compose | Other | https://docs.docker.com/engine/install/ |
| poetry | Other | https://python-poetry.org/docs/ |
## Poetry (Python Dependencies)
Within the `pu` branch, aurweb now uses the `python-poetry` Python
package manager. See [Requirements](#requirements) above on installation
methods.
If poetry is configured to create virtualenvs with
`poetry config virtualenvs.create true --local` (recommended), users can install
aurweb dependencies and the aurweb project into their poetry virtualenv:
$ poetry update # Update/fetch all dependencies in the virtualenv.
$ poetry install # Install project-specific package and scripts.
After installing dependencies (via `update`) and the project (via `install`),
users can then run commands within the virtualenv:
$ poetry run pytest test/
## Introduction
Welcome to the FastAPI testing guide. This article was written to
@ -85,33 +32,33 @@ UX and API (RPC) access. Details of the implementations may be changed,
but the end user experience should either result in an equal and/or
improved state.
A compiled changelog will be produced when the port is released
(version 6.0.0); before that, users and developers can take advantage
of `git blame` and `git log` to identify commits which ported features
or made changes. The involved commits **should** lay out the features,
fixes or changes in detail. In some cases this is not done so well; if
you experience a case like this, please do report it as an issue
in the project repository so we can address it.
The aurweb project now maintains a
[Dockerfile](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/Dockerfile)
and [docker-compose.yml](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/docker-compose.yml).
The Dockerfile builds a very basic image, and the docker-compose services
use that image as a base.
For a complete rundown on all of the services available, see the
[aurweb Docker wiki page](https://gitlab.archlinux.org/archlinux/aurweb/-/wikis/Docker).
As a bit of a guide to the guide, I'd like to help you navigate around
a bit:
1. Read [Accounts](#accounts)
1. Read [Testing Methods](#testing-methods)
2. Read [User Experience](#user-experience)
3. Read [Edge Cases](#edge-cases)
4. Read [Getting Started](#getting-started)
- Follow the link to [Development Servers](#development-servers) at the end
5. Go back and focus on [User Experience](#user-experience) or any of the
other needs specified in [What We Need Tested](#what-we-need-tested) and [What Else?](#what-else)
4. Go back and focus on [User Experience](#user-experience) or any of the
other needs specified in [What We Need Tested](#what-we-need-tested)
and [What Else?](#what-else)
## Testing Methods
There are two methods which can be utilized for testing purposes.
**Users wishing to test and provide feedback should read the entirety of
their respective choice's page.** They contain specific information
about the handling of user accounts, account types, and toggling between
PHP and FastAPI.
The following list links to the relevant wiki pages for each supported method:
- [aur-dev.archlinux.org Testing](https://gitlab.archlinux.org/archlinux/aurweb/-/wikis/aur-dev.archlinux.org-Testing)
- Available for users who do not wish to dabble with our Docker
infrastructure; it can be used to test features which do not
require database access.
- [Docker Testing](https://gitlab.archlinux.org/archlinux/aurweb/-/wikis/Docker-Testing)
- In all cases, our [Docker](https://gitlab.archlinux.org/archlinux/aurweb/-/wikis/Docker)
infrastructure allows users to spin up an instance of aurweb which
they fully control. With this method, you have access to the database
used and all services which are utilized by FastAPI.
## What We Need Tested
@ -123,310 +70,26 @@ a bit:
- [ ] [Benchmarks](#benchmarks) - <small>Medium Priority</small>
- [ ] [Code Review](#code-review) - <small>High Priority</small>
## Environment Variables
If you use aurweb locally, you'll need to create a valid `AUR_CONFIG` by copying
[conf/config.dev](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/conf/config.dev)
to `conf/config` (or anywhere else) and configuring the copy for your host
development environment:
$ export AUR_CONFIG="$(pwd)/conf/config"
$ export AUR_CONFIG_DEFAULTS="$(pwd)/conf/config.defaults"
Suggestion: Craft a second config with Docker details, so that you can
test directly against Docker databases or manipulate the production DB
from the host.
## Getting Started
First, checkout to and pull the latest `pu` branch revision. This branch
contains new code not located in `live` or `master`.
$ git remote add upstream ssh://git@gitlab.archlinux.org:222/archlinux/aurweb.git
$ git checkout pu
$ git pull upstream pu
Build the `aurweb:latest` Docker image:
$ docker-compose build
At this point, we have two choices:
1. Run a test command
- `docker-compose run sharness`
- `docker-compose run pytest-mysql`
- `docker-compose run pytest-sqlite`
- `docker-compose run test`
2. Run the development servers (FastAPI and PHP, side-by-side)
- `docker-compose up -d nginx` (see [Development Servers](#development-servers))
- See [Add User](#add-user) (optional)
- See [Generating Dummy Data](#generating-dummy-data)
## Development Servers
**Note** it is important to read each service section linked in the list
to get a fluent understanding of what each component offers us for
testability.
When a user runs `docker-compose up -d nginx`, a few things happen
<small>(some services are omitted)</small>:
1. The [ca](#certificate-authority) service starts
2. The [mariadb](#mariadb) service starts
3. The [git](#git) service starts
4. The [redis](#redis) service starts
5. The `memcached` service starts
6. The [fastapi](#fastapi) service starts
7. The [php-fpm](#php-fpm) service starts
8. The [nginx](#nginx) service starts
###### Notes
The `fastapi` and `php-fpm` services live update when you change
source files in your tree.
The first step of starting services takes several minutes on Docker
currently; however, if you load up and rely on hot reload, you can
do work and restart specific services if need be without concern
(at least, until you add a dependency).
* During checkouts to completely different branches, it is sometimes
necessary to restart the fastapi service: `docker-compose restart fastapi`.
#### Certificate Authority
The `ca` service will produce a `cache/ca.root.pem` file, which is used to
sign the `cache/localhost.cert.pem` certificate. Users can import our
`ca.root.pem` as a Certificate Authority into browsers or ca-certificates.
#### mariadb
The `mariadb` service will bind to `localhost:13306` on the host machine. This
allows users to inspect the database used by the `fastapi` and `php-fpm`
services:
$ mysql -h127.0.0.1 -P13306 -uaur -paur aurweb \
-e "SELECT * FROM Packages"
**NOTE**: On my personal system, specifying `-h127.0.0.1` is important,
as the mysql client does something other than reach the mariadb server with
`localhost` (assumed some type of confusion with localhost, didn't look much
further).
###### Generating Dummy Data
<small>
<b>NOTE</b>: This step requires Python dependencies to be accessible from
your shell's python3. See
[pyproject.toml](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/pyproject.toml)
for a quite rudimentary [poetry](https://python-poetry.org/) configuration.</small>
Users can generate configurable amounts of dummy data with
[schema/gendummydata.py](https://gitlab.archlinux.org/archlinux/aurweb):
$ export MAX_USERS=5000
$ export MAX_PKGS=10000
# With python3
$ python3 schema/gendummydata.py dummy.sql
# With poetry
$ poetry run python3 schema/gendummydata.py dummy.sql
Configurable env variables:
- `MAX_USERS`: The maximum number of users to generate (default: 38000)
- `MAX_PKGS`: The maximum number of packages to generate (default: 32000)
- `OPEN_PROPOSALS`: Number of open proposals to generate (default: 15)
- `CLOSE_PROPOSALS`: Number of close proposals to generate (default: 50)
- `FORTUNE_FILE`: Path to fortune cookie file (default: '/usr/share/fortune/cookie')
Every user generated by the dummy file will have a password which matches
its username (i.e. username: `kevin`, password: `kevin`).
#### git
The `git` service runs sshd using the aurweb-git-auth backend and
aurweb-git-update hook. It is accessible at `ssh://aur@localhost:2222` on
the host system. Users can use this local sshd just like a production
AUR instance for git management.
Note that in order for you to do anything with the sshd, you'll need
to either login to a dummy user or register an account and setup
your public SSH key like you would on aur.archlinux.org.
#### redis
The `redis` service is used as the Redis backend for the `fastapi` service.
You can converse with the service on `localhost:16379`:
$ redis-cli -p 16379
...
#### fastapi
The `fastapi` service runs the aurweb FastAPI implementation through
an ASGI server. Users can configure this with the `FASTAPI_BACKEND`
environment variable: `uvicorn` or `hypercorn`. It enables hot
reloading of the server for changes in all directories in your tree,
which automatically reflects on the [nginx](#nginx) service.
This service is exposed on `localhost:18000`.
#### php-fpm
This service is exposed on `localhost:19000` and majorly depends on
[./web](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/web).
#### nginx
The `nginx` service will host two web servers (denoted below), both signed
by the Root CA certificate `ca.root.pem`.
| Backend | URL |
|---------------------|------------------------|
| [php-fpm](#php-fpm) | https://localhost:8443 |
| [fastapi](#fastapi) | https://localhost:8444 |
## Accounts
If you have not created or requested an account, users can login
to all users generated by `gendummydata.py` (most of the users on
aur-dev.archlinux.org) using a password which is exactly the same as
the account's username.
Example:
Username: some-generated-user
Password: some-generated-user
Test users are completely free to change the password of a dummy
user, but this is highly discouraged in favor of requesting an
account of your own (see below).
#### aur-dev.archlinux.org Accounts
`aur-dev.archlinux.org` does not expose its database to users. For this
reason, we don't allow users to freely create accounts on their own.
For users testing Trusted User or Developer-specific routes, requests
for a Trusted User account can be made to aur-dev@lists.archlinux.org.
Send an email to aur-dev@lists.archlinux.org titled
`[Test Account Request] your_username` with the following details:
Username: username_you_want
Email: email@youwant.org
Type: account_type_you_want
Where `account_type_you_want` is one of `User` or `Trusted User`. We will not be giving out `Developer` accounts.
Upon receiving account requests, DevOps will be able to generate
an account for you in `aur-dev.archlinux.org`'s database, as long
as there is no existing email/username conflict.
You will receive an email at the email you've requested for your account, including the login details (username and password).
This will be done in quite an insecure manner, so it is highly recommended that you change your password as soon as possible to avoid any problems. The `aur-dev.archlinux.org` database is not connected to `aur.archlinux.org`'s database.
#### Local Docker Accounts
Users who take advantage of a local Docker instance for testing have full
access to the Docker database and can use the [adduser.py](#add-user) script
described below.
###### Add User
You can create a user account in the database using the
[util/adduser.py](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/util/adduser.py) script.
Users can run `adduser.py` from the Docker host without installing any
host dependencies:
`docker exec -it <fastapi_container> python /aurweb/util/adduser.py ...`.
See `docker exec -it <fastapi_container> python /aurweb/util/adduser.py --help`
for details on command line arguments.
## User Experience
Some of the most important testing we could do for aurweb at this
point would be testing that FastAPI's user experience is
feature-equivalent to PHP's user experience.
We have, however, changed some things which we believe benefits users (and implements tasks requested by bug reports, etc).
We have, however, changed some things which we believe benefits users
(and implements tasks requested by bug reports, etc).
If something seems to be not working as you would expect it should,
please report these problems in the form of issues at
https://gitlab.archlinux.org/archlinux/aurweb/-/issues.
Additionally, if you could, please provide some thoughts about
your experience with the FastAPI code in an issue prefixed with
`[Feedback]` using the [Feedback issue template](https://gitlab.archlinux.org/archlinux/aurweb/-/issues/new?issueable_template=Feedback). This is what we'll need to collect data about issues
to move forward with production.
Additionally, if you could, please provide some overall thoughts about
your experience with the FastAPI code in an issue prefixed with `[Feedback]`.
[Issue #161](https://gitlab.archlinux.org/archlinux/aurweb/-/issues/161)
is used to track the overall progress of these tests and links out to
issues specific to each test we need.
Below, we provide a table of routes that we are really looking for UX
testing on, and people wishing to contribute with UX testers should
stay up to date with it:
| Route | Status | Description | Account type |
|--------------------------------------------------------------|--------|-------------------------|-------------------|
| https://localhost:8444/ | #138 | Homepage | User |
| https://localhost:8444/register | #155 | Account Registration | |
| https://localhost:8444/login | #139 | User Login | User |
| https://localhost:8444/logout | #140 | User Logout | User |
| https://localhost:8444/passreset ([Notes](#passreset-notes)) | #156 | User Password Reset | Logged out User |
| https://localhost:8444/account/{username} | #141 | Account View | User |
| https://localhost:8444/account/{username}/edit | #142 | Account Edit | User |
| https://localhost:8444/accounts | #143 | Account Search | Trusted User |
| https://localhost:8444/packages | #144 | Package Search | User |
| https://localhost:8444/packages | #145 | Package Search actions | User |
| https://localhost:8444/(pkgbase\|packages)/{name} | #146 | Comment testing | User |
| https://localhost:8444/packages/{name} | #147 | Package View | User |
| https://localhost:8444/packages/{name} | #148 | Package Base actions | User / TU[D] |
| https://localhost:8444/pkgbase/{name} | #149 | Package Base View | User |
| https://localhost:8444/pkgbase/{name}/comaintainers | #150 | Comaintainer Edit | Maintainer |
| https://localhost:8444/pkgbase/{name}/flag | #151 | Package Flagging | User |
| https://localhost:8444/pkgbase/{name}/request | #152 | Package Request Create | User |
| https://localhost:8444/requests | #153 | Package Requests | User / TU[D] |
| https://localhost:8444/requests/{id}/close | #154 | Package Request Closure | Requester / TU[D] |
| https://localhost:8444/tu | #157 | Trusted User page | Trusted User |
| https://localhost:8444/tu/{id} | #158 | Trusted User Proposal | Trusted User |
| https://localhost:8444/addvote | #159 | Trusted User Proposal Create | Trusted User |
Each item has its own issue, linked in the **Status** column. Feedback can be provided in the form of comments to these issues, or through their own issue using the Feedback template.
In any case, take a look at the Feedback template and use the same template for comments you may include on one of the `[User Testing]` issues.
#### Development Branches
The following snippet shows an example workflow of setting up
your tree for testing a development branch:
## Checkout to a devel branch based on pu:
$ git checkout -b devel_branch pu
## Add the Git Fork to a remote:
$ git remote add devel_fork https://gitlab.archlinux.org/user/fork
## Fetch the branch's data and hard reset to its HEAD:
$ git fetch devel_fork devel_branch
$ git reset --hard devel_fork/devel_branch
## Or, you can just pull the branch if your local pu is good:
$ git pull devel_fork devel_branch
## (Required when checking out to a different tree)
## Once you've got the devel branch tree, restart `fastapi`:
$ docker-compose restart fastapi
###### passreset Notes
Currently, external mail notifications are not enabled within Docker.
To utilize the passreset mail, users can get the ResetKey column through
[mariadb](#mariadb):
$ mysql -h127.0.0.1 -P13306 -uaur -paur aurweb \
-e "SELECT ResetKey FROM Users WHERE Username = 'my_user'"
You can then use that key by navigating to:
https://localhost:8444/passreset?resetkey=your_reset_key.
In any case, take a look at the Feedback template and use the same template
for comments you may include on one of the `[User Testing]` issues.
## Edge Cases