This document provides guidance on running tests and development servers in the aurweb (base branch: pu) project.
Status
Branches containing FastAPI are located in the pipeline/coverage status table below:
| pu (fastapi) |
|---|
  |
Introduction
Welcome to the FastAPI testing guide. This article was written to assist developers and testers with spinning up instances of aurweb to perform tests or modify code live. Additionally, it outlines the kind of testing that aurweb is currently looking for.
At this point in the port, the most important kind of contributions would be testing the user experience of FastAPI aurweb. We aim to, during the port, mirror the existing PHP implementations in terms of UX and API (RPC) access. Details of the implementations may be changed, but the end user experience should either result in an equal and/or improved state.
- Read Testing Methods
- Read User Experience
- Read User Experience Feedback
- Read Edge Cases
- Go back and focus on User Experience or any of the other needs specified in What We Need Tested and What Else?
Testing Methods
There are two methods which can be utilized for testing purposes.
Users wishing to test and provide feedback should read the entirety of their respective choice's page. They contain specific information about the handling of user accounts, account types, and toggling between PHP and FastAPI.
The following list links to the relevant wiki pages for each supported method:
- aur-dev.archlinux.org Testing
- Available for users who do not wish to dabble with our Docker infrastructure; it can be used to test features which do not require database access.
- Docker Testing
- In all cases, our Docker infrastructure allows users to spin up an instance of aurweb which they fully control. With this method, you have access to the database used and all services which are utilized by FastAPI.
What We Need Tested
- User Experience - Highest Priority
- Edge Cases - Medium Priority
What Else?
- Benchmarks - Medium Priority
- Code Review - High Priority
User Experience
Some of the most important testing we could do for aurweb at this point would be testing that FastAPI's user experience is feature-equivalent to PHP's user experience.
We have, however, changed some things which we believe benefits users (and implements tasks requested by bug reports, etc).
Issue #161 is used to track the overall progress of these tests and links out to issues specific to each test we need.
User Experience Feedback
If you could, please provide some thoughts about your experience with the
FastAPI code in an issue prefixed with [Feedback] #<issue_id> (where <issue_id> is a [User Testing] issue) using the Feedback issue template. This is what we'll need to collect data about issues to move forward with production.
If you find any extensive bugs, you can file a [Bug] issue using the
Bug issue template,
which would keep things more organized for us in general.
In any case, take a look at the Feedback template and use the same template
for comments you may include on one of the [User Testing] issues.
Edge Cases
In short: try to break the implementation with user input! Do whatever
you can to cause the code to fail from a user's perspective. If you find
any bugs, please produce an issue at
https://gitlab.archlinux.org/archlinux/aurweb/-/issues
with a [FastAPI] prefix in the subject and a description of the bug.
Please include a docker-compose logs fastapi output with the report if
you do.
This should serve as a long-term goal of contributors over time: attempting to harden the project at every opportunity. We like our stuff working, and this is how we keep it working.
We very high appreciate any help, patches, or points that the community can provide us. Thank you!
Benchmarks
One of our sub-goals of this port is to increase performance and efficiency overall across the service. Lower benchmark times indicate that code is performing better overall.
When performing benchmark comparisons between PHP and FastAPI, users should opt
into defining FASTAPI_BACKEND=gunicorn FASTAPI_WORKERS=5 as they docker-compose up -d nginx. Our php-fpm service uses 5 workers by default, and so using gunicorn with 5 workers does the best we can to match core usage with PHP.
To perform benchmarks, the hyperfine tool can be utilized:
$ hyperfine --warmup 20 --min-runs 500 'curl https://localhost:8444'
Or, the oha tool:
$ oha -c 8 -z 1min 'https://localhost:8444'
These kinds of benchmarks matter more on particular routes with specific behaviors. We are primarily concerned with the performance of database query related tasks like package search, package/pkgbase viewing, requests viewing, homepage and dashboard, RPC API, and more.
Code Review
It does not matter how experienced a developer is: another set of eyes and different perspective can often reveal issues that a writer did not intend. It is pretty much impossible for a developer to be "perfect" in every case for the entirety of their life.
That is why it is extremely important that we get multiple unique views on a set of code that may be for a feature or fixing a bug.
All reviews of code are welcome and appreciated, regardless of it being in a particular branch. Please keep your eyes out for problems!
Communication
We are commonly available in the #archlinux-aurweb channel on the Libera.chat network.
You can also reach us via the aur-general and aur-dev mailing lists, as long as the inquiry is topical. If you wish to speak to kevr (main developer of the FastAPI port), you can send him an email at kevr@0cost.org for any reason.
Patches are also welcome via the
aur-dev list with
git send-email.