diff --git a/Testing-Guide.md b/Testing-Guide.md index 20a756d..4a26e2e 100644 --- a/Testing-Guide.md +++ b/Testing-Guide.md @@ -18,59 +18,6 @@ Branches containing FastAPI are located in the pipeline/coverage status table be |--------------| | ![pipeline](https://gitlab.archlinux.org/archlinux/aurweb/badges/pu/pipeline.svg?key_text=build) ![coverage](https://gitlab.archlinux.org/archlinux/aurweb/badges/pu/coverage.svg) | -## Locations - -Testers should choose one of the following methods: - -- [aur-dev.archlinux.org Testing](#aur-devarchlinuxorg-testing) -- [Docker Testing](#docker-testing) - -**Too long, didn't read?** Running a local Docker instance is much more flexible -in terms of testing -- testers have complete access to the MariaDB database -used by both the `php-fpm` and `fastapi` services. For users who do not wish -to interact with Docker, the `aur-dev.archlinux.org` instance can be used -to perform any testing. - -See [Accounts](#accounts) for an explanation on obtaining accounts -in either system. - -### aur-dev.archlinux.org Testing - -A Docker instance of the `pu` branch is hosted at https://aur-dev.archlinux.org -and can be used by anybody to test the code-base. - -### Docker Testing - -The following table lists out requirements needed to run our `docker-compose` -services: - -| Package | Distribution | Method | -|------------------|--------------|-----------------------------------------| -| docker | Arch Linux | `pacman -S docker` | -| docker-compose | Arch Linux | `pacman -S docker-compose` | -| poetry | Arch Linux | `pacman -S python-poetry` | -| docker | Other | https://docs.docker.com/engine/install/ | -| docker-compose | Other | https://docs.docker.com/engine/install/ | -| poetry | Other | https://python-poetry.org/docs/ | - -## Poetry (Python Dependencies) - -Within the `pu` branch, aurweb now uses the `python-poetry` Python -package manager. See [Requirements](#requirements) above on installation -methods. - -If poetry is configured to create virtualenvs with -`poetry config virtualenvs.create true --local` (recommended), users can install -aurweb dependencies and the aurweb project into their poetry virtualenv: - - $ poetry update # Update/fetch all dependencies in the virtualenv. - $ poetry install # Install project-specific package and scripts. - -After installing dependencies (via `update`) and the project (via `install`), -users can then run commands within the virtualenv: - - $ poetry run pytest test/ - ## Introduction Welcome to the FastAPI testing guide. This article was written to @@ -85,33 +32,33 @@ UX and API (RPC) access. Details of the implementations may be changed, but the end user experience should either result in an equal and/or improved state. -A compiled changelog will be produced when the port is released -(version 6.0.0); before that, users and developers can take advantage -of `git blame` and `git log` to identify commits which ported features -or made changes. The involved commits **should** lay out the features, -fixes or changes in detail. In some cases this is not done so well; if -you experience a case like this, please do report it as an issue -in the project repository so we can address it. - -The aurweb project now maintains a -[Dockerfile](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/Dockerfile) -and [docker-compose.yml](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/docker-compose.yml). -The Dockerfile builds a very basic image, and the docker-compose services -use that image as a base. - -For a complete rundown on all of the services available, see the -[aurweb Docker wiki page](https://gitlab.archlinux.org/archlinux/aurweb/-/wikis/Docker). - -As a bit of a guide to the guide, I'd like to help you navigate around -a bit: - -1. Read [Accounts](#accounts) +1. Read [Testing Methods](#testing-methods) 2. Read [User Experience](#user-experience) 3. Read [Edge Cases](#edge-cases) -4. Read [Getting Started](#getting-started) - - Follow the link to [Development Servers](#development-servers) at the end -5. Go back and focus on [User Experience](#user-experience) or any of the - other needs specified in [What We Need Tested](#what-we-need-tested) and [What Else?](#what-else) +4. Go back and focus on [User Experience](#user-experience) or any of the + other needs specified in [What We Need Tested](#what-we-need-tested) + and [What Else?](#what-else) + +## Testing Methods + +There are two methods which can be utilized for testing purposes. + +**Users wishing to test and provide feedback should read the entirety of +their respective choice's page.** They contain specific information +about the handling of user accounts, account types, and toggling between +PHP and FastAPI. + +The following list links to the relevant wiki pages for each supported method: + +- [aur-dev.archlinux.org Testing](https://gitlab.archlinux.org/archlinux/aurweb/-/wikis/aur-dev.archlinux.org-Testing) + - Available for users who do not wish to dabble with our Docker + infrastructure; it can be used to test features which do not + require database access. +- [Docker Testing](https://gitlab.archlinux.org/archlinux/aurweb/-/wikis/Docker-Testing) + - In all cases, our [Docker](https://gitlab.archlinux.org/archlinux/aurweb/-/wikis/Docker) + infrastructure allows users to spin up an instance of aurweb which + they fully control. With this method, you have access to the database + used and all services which are utilized by FastAPI. ## What We Need Tested @@ -123,310 +70,26 @@ a bit: - [ ] [Benchmarks](#benchmarks) - Medium Priority - [ ] [Code Review](#code-review) - High Priority -## Environment Variables - -If you use aurweb locally, you'll need to create a valid `AUR_CONFIG` by copying -[conf/config.dev](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/conf/config.dev) -to `conf/config` (or anywhere else) and configuring the copy for your host -development environment: - - $ export AUR_CONFIG="$(pwd)/conf/config" - $ export AUR_CONFIG_DEFAULTS="$(pwd)/conf/config.defaults" - -Suggestion: Craft a second config with Docker details, so that you can -test directly against Docker databases or manipulate the production DB -from the host. - -## Getting Started - -First, checkout to and pull the latest `pu` branch revision. This branch -contains new code not located in `live` or `master`. - - $ git remote add upstream ssh://git@gitlab.archlinux.org:222/archlinux/aurweb.git - $ git checkout pu - $ git pull upstream pu - -Build the `aurweb:latest` Docker image: - - $ docker-compose build - -At this point, we have two choices: - -1. Run a test command - - `docker-compose run sharness` - - `docker-compose run pytest-mysql` - - `docker-compose run pytest-sqlite` - - `docker-compose run test` -2. Run the development servers (FastAPI and PHP, side-by-side) - - `docker-compose up -d nginx` (see [Development Servers](#development-servers)) - - See [Add User](#add-user) (optional) - - See [Generating Dummy Data](#generating-dummy-data) - -## Development Servers - -**Note** it is important to read each service section linked in the list -to get a fluent understanding of what each component offers us for -testability. - -When a user runs `docker-compose up -d nginx`, a few things happen -(some services are omitted): - -1. The [ca](#certificate-authority) service starts -2. The [mariadb](#mariadb) service starts -3. The [git](#git) service starts -4. The [redis](#redis) service starts -5. The `memcached` service starts -6. The [fastapi](#fastapi) service starts -7. The [php-fpm](#php-fpm) service starts -8. The [nginx](#nginx) service starts - -###### Notes - -The `fastapi` and `php-fpm` services live update when you change -source files in your tree. - -The first step of starting services takes several minutes on Docker -currently; however, if you load up and rely on hot reload, you can -do work and restart specific services if need be without concern -(at least, until you add a dependency). - -* During checkouts to completely different branches, it is sometimes - necessary to restart the fastapi service: `docker-compose restart fastapi`. - -#### Certificate Authority - -The `ca` service will produce a `cache/ca.root.pem` file, which is used to -sign the `cache/localhost.cert.pem` certificate. Users can import our -`ca.root.pem` as a Certificate Authority into browsers or ca-certificates. - -#### mariadb - -The `mariadb` service will bind to `localhost:13306` on the host machine. This -allows users to inspect the database used by the `fastapi` and `php-fpm` -services: - - $ mysql -h127.0.0.1 -P13306 -uaur -paur aurweb \ - -e "SELECT * FROM Packages" - -**NOTE**: On my personal system, specifying `-h127.0.0.1` is important, -as the mysql client does something other than reach the mariadb server with -`localhost` (assumed some type of confusion with localhost, didn't look much -further). - -###### Generating Dummy Data - - -NOTE: This step requires Python dependencies to be accessible from -your shell's python3. See -[pyproject.toml](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/pyproject.toml) -for a quite rudimentary [poetry](https://python-poetry.org/) configuration. - -Users can generate configurable amounts of dummy data with -[schema/gendummydata.py](https://gitlab.archlinux.org/archlinux/aurweb): - - $ export MAX_USERS=5000 - $ export MAX_PKGS=10000 - - # With python3 - $ python3 schema/gendummydata.py dummy.sql - - # With poetry - $ poetry run python3 schema/gendummydata.py dummy.sql - -Configurable env variables: - -- `MAX_USERS`: The maximum number of users to generate (default: 38000) -- `MAX_PKGS`: The maximum number of packages to generate (default: 32000) -- `OPEN_PROPOSALS`: Number of open proposals to generate (default: 15) -- `CLOSE_PROPOSALS`: Number of close proposals to generate (default: 50) -- `FORTUNE_FILE`: Path to fortune cookie file (default: '/usr/share/fortune/cookie') - -Every user generated by the dummy file will have a password which matches -its username (i.e. username: `kevin`, password: `kevin`). - -#### git - -The `git` service runs sshd using the aurweb-git-auth backend and -aurweb-git-update hook. It is accessible at `ssh://aur@localhost:2222` on -the host system. Users can use this local sshd just like a production -AUR instance for git management. - -Note that in order for you to do anything with the sshd, you'll need -to either login to a dummy user or register an account and setup -your public SSH key like you would on aur.archlinux.org. - -#### redis - -The `redis` service is used as the Redis backend for the `fastapi` service. -You can converse with the service on `localhost:16379`: - - $ redis-cli -p 16379 - ... - -#### fastapi - -The `fastapi` service runs the aurweb FastAPI implementation through -an ASGI server. Users can configure this with the `FASTAPI_BACKEND` -environment variable: `uvicorn` or `hypercorn`. It enables hot -reloading of the server for changes in all directories in your tree, -which automatically reflects on the [nginx](#nginx) service. - -This service is exposed on `localhost:18000`. - -#### php-fpm - -This service is exposed on `localhost:19000` and majorly depends on -[./web](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/web). - -#### nginx - -The `nginx` service will host two web servers (denoted below), both signed -by the Root CA certificate `ca.root.pem`. - -| Backend | URL | -|---------------------|------------------------| -| [php-fpm](#php-fpm) | https://localhost:8443 | -| [fastapi](#fastapi) | https://localhost:8444 | - -## Accounts - -If you have not created or requested an account, users can login -to all users generated by `gendummydata.py` (most of the users on -aur-dev.archlinux.org) using a password which is exactly the same as -the account's username. - -Example: - - Username: some-generated-user - Password: some-generated-user - -Test users are completely free to change the password of a dummy -user, but this is highly discouraged in favor of requesting an -account of your own (see below). - -#### aur-dev.archlinux.org Accounts - -`aur-dev.archlinux.org` does not expose its database to users. For this -reason, we don't allow users to freely create accounts on their own. - -For users testing Trusted User or Developer-specific routes, requests -for a Trusted User account can be made to aur-dev@lists.archlinux.org. - -Send an email to aur-dev@lists.archlinux.org titled -`[Test Account Request] your_username` with the following details: - - Username: username_you_want - Email: email@youwant.org - Type: account_type_you_want - -Where `account_type_you_want` is one of `User` or `Trusted User`. We will not be giving out `Developer` accounts. - -Upon receiving account requests, DevOps will be able to generate -an account for you in `aur-dev.archlinux.org`'s database, as long -as there is no existing email/username conflict. - -You will receive an email at the email you've requested for your account, including the login details (username and password). - This will be done in quite an insecure manner, so it is highly recommended that you change your password as soon as possible to avoid any problems. The `aur-dev.archlinux.org` database is not connected to `aur.archlinux.org`'s database. - -#### Local Docker Accounts - -Users who take advantage of a local Docker instance for testing have full -access to the Docker database and can use the [adduser.py](#add-user) script -described below. - -###### Add User - -You can create a user account in the database using the -[util/adduser.py](https://gitlab.archlinux.org/archlinux/aurweb/-/tree/pu/util/adduser.py) script. - -Users can run `adduser.py` from the Docker host without installing any -host dependencies: -`docker exec -it python /aurweb/util/adduser.py ...`. - -See `docker exec -it python /aurweb/util/adduser.py --help` -for details on command line arguments. - ## User Experience Some of the most important testing we could do for aurweb at this point would be testing that FastAPI's user experience is feature-equivalent to PHP's user experience. -We have, however, changed some things which we believe benefits users (and implements tasks requested by bug reports, etc). +We have, however, changed some things which we believe benefits users +(and implements tasks requested by bug reports, etc). -If something seems to be not working as you would expect it should, -please report these problems in the form of issues at -https://gitlab.archlinux.org/archlinux/aurweb/-/issues. +Additionally, if you could, please provide some thoughts about +your experience with the FastAPI code in an issue prefixed with +`[Feedback]` using the [Feedback issue template](https://gitlab.archlinux.org/archlinux/aurweb/-/issues/new?issueable_template=Feedback). This is what we'll need to collect data about issues +to move forward with production. -Additionally, if you could, please provide some overall thoughts about -your experience with the FastAPI code in an issue prefixed with `[Feedback]`. +[Issue #161](https://gitlab.archlinux.org/archlinux/aurweb/-/issues/161) +is used to track the overall progress of these tests and links out to +issues specific to each test we need. -Below, we provide a table of routes that we are really looking for UX -testing on, and people wishing to contribute with UX testers should -stay up to date with it: - -| Route | Status | Description | Account type | -|--------------------------------------------------------------|--------|-------------------------|-------------------| -| https://localhost:8444/ | #138 | Homepage | User | -| https://localhost:8444/register | #155 | Account Registration | | -| https://localhost:8444/login | #139 | User Login | User | -| https://localhost:8444/logout | #140 | User Logout | User | -| https://localhost:8444/passreset ([Notes](#passreset-notes)) | #156 | User Password Reset | Logged out User | -| https://localhost:8444/account/{username} | #141 | Account View | User | -| https://localhost:8444/account/{username}/edit | #142 | Account Edit | User | -| https://localhost:8444/accounts | #143 | Account Search | Trusted User | -| https://localhost:8444/packages | #144 | Package Search | User | -| https://localhost:8444/packages | #145 | Package Search actions | User | -| https://localhost:8444/(pkgbase\|packages)/{name} | #146 | Comment testing | User | -| https://localhost:8444/packages/{name} | #147 | Package View | User | -| https://localhost:8444/packages/{name} | #148 | Package Base actions | User / TU[D] | -| https://localhost:8444/pkgbase/{name} | #149 | Package Base View | User | -| https://localhost:8444/pkgbase/{name}/comaintainers | #150 | Comaintainer Edit | Maintainer | -| https://localhost:8444/pkgbase/{name}/flag | #151 | Package Flagging | User | -| https://localhost:8444/pkgbase/{name}/request | #152 | Package Request Create | User | -| https://localhost:8444/requests | #153 | Package Requests | User / TU[D] | -| https://localhost:8444/requests/{id}/close | #154 | Package Request Closure | Requester / TU[D] | -| https://localhost:8444/tu | #157 | Trusted User page | Trusted User | -| https://localhost:8444/tu/{id} | #158 | Trusted User Proposal | Trusted User | -| https://localhost:8444/addvote | #159 | Trusted User Proposal Create | Trusted User | - -Each item has its own issue, linked in the **Status** column. Feedback can be provided in the form of comments to these issues, or through their own issue using the Feedback template. - -In any case, take a look at the Feedback template and use the same template for comments you may include on one of the `[User Testing]` issues. - -#### Development Branches - -The following snippet shows an example workflow of setting up -your tree for testing a development branch: - - ## Checkout to a devel branch based on pu: - $ git checkout -b devel_branch pu - - ## Add the Git Fork to a remote: - $ git remote add devel_fork https://gitlab.archlinux.org/user/fork - - ## Fetch the branch's data and hard reset to its HEAD: - $ git fetch devel_fork devel_branch - $ git reset --hard devel_fork/devel_branch - - ## Or, you can just pull the branch if your local pu is good: - $ git pull devel_fork devel_branch - - ## (Required when checking out to a different tree) - ## Once you've got the devel branch tree, restart `fastapi`: - $ docker-compose restart fastapi - -###### passreset Notes - -Currently, external mail notifications are not enabled within Docker. -To utilize the passreset mail, users can get the ResetKey column through -[mariadb](#mariadb): - - $ mysql -h127.0.0.1 -P13306 -uaur -paur aurweb \ - -e "SELECT ResetKey FROM Users WHERE Username = 'my_user'" - -You can then use that key by navigating to: -https://localhost:8444/passreset?resetkey=your_reset_key. +In any case, take a look at the Feedback template and use the same template +for comments you may include on one of the `[User Testing]` issues. ## Edge Cases