use aurweb_test for default mysql dev database

This also updates `test/README.md` to be a bit more specific
and precise with our current state of testing.

Signed-off-by: Kevin Morris <kevr@0cost.org>

.gitlab-ci.yml

@@ -20,14 +20,14 @@ before_script:
            python-pytest-asyncio python-coverage python-bcrypt
            python-email-validator openssh python-lxml mariadb
            python-isort flake8
-  - bash -c "echo '127.0.0.1' > /etc/hosts"
+  - bash -c "echo '127.0.0.1 localhost' > /etc/hosts"
-  - bash -c "echo '::1' >> /etc/hosts"
+  - bash -c "echo '::1 localhost' >> /etc/hosts"
   - mariadb-install-db --user=mysql --basedir=/usr --datadir=/var/lib/mysql
   - (cd '/usr' && /usr/bin/mysqld_safe --datadir='/var/lib/mysql') &
   - 'until : > /dev/tcp/127.0.0.1/3306; do sleep 1s; done'
   - mysql -u root -e "CREATE USER 'aur'@'localhost' IDENTIFIED BY 'aur';"
-  - mysql -u root -e "CREATE DATABASE aurweb;"
+  - mysql -u root -e "CREATE DATABASE aurweb_test;"
-  - mysql -u root -e "GRANT ALL PRIVILEGES ON aurweb.* TO 'aur'@'localhost';"
+  - mysql -u root -e "GRANT ALL ON aurweb_test.* TO 'aur'@'localhost';"
   - mysql -u root -e "FLUSH PRIVILEGES;"
   - sed -r "s;YOUR_AUR_ROOT;$(pwd);g" conf/config.dev > conf/config
   - cp conf/config conf/config.sqlite

conf/config.dev

@@ -10,7 +10,7 @@
 backend = mysql
 
 ; If using sqlite, set name to the database file path.
-name = aurweb
+name = aurweb_test
 
 ; MySQL database information. User defaults to root for containerized
 ; testing with mysqldb. This should be set to a non-root user.

test/README.md

@@ -1,5 +1,5 @@
-Running tests
+aurweb Test Collection
--------------
+======================
 
 To run all tests, you may run `make check` under `test/` (alternative targets:
 `make pytest`, `make sh`).
@@ -12,7 +12,8 @@ may receive command-line options to help debugging. See for example sharness's
 documentation for shell test scripts:
 https://github.com/chriscool/sharness/blob/master/README.git
 
-### Dependencies
+Dependencies
+------------
 
 For all the test to run, the following Arch packages should be installed:
 
@@ -30,39 +31,149 @@ For all the test to run, the following Arch packages should be installed:
 - postfix
 - openssh
 
+Test Configuration
+------------------
+
+To perform any tests, we need to supply `aurweb` with a valid
+configuration. For development (and testing) purposes, an example
+[conf/config.dev](../conf/config.dev) can be slightly modified.
+
+Start off by copying `config.dev` to a new configuration.
+
+    $ cp -v conf/config.dev conf/config
+
+First, we must tell `aurweb` where the root of our project
+lives by replacing `YOUR_AUR_ROOT` with the path to the aurweb
+repository.
+
+    $ sed -i "s;YOUR_AUR_ROOT;/path/to/aurweb;g" conf/config
+
+Now, one must decide a database backend to use; see
+[Test Database](#test-database) for details on configuring
+the different supported backends.
+
+Test Database
+-------------
+
+Users may choose to configure one of several backends, including:
+`mysql` and `sqlite`. By default, `conf/config.dev` is configured
+for a the `mysql` backend using a test database named `aurweb_test`.
+
+Users can initialize an empty MySQL database by performing the following:
+
+    $ cat conf/config
+    [database]
+    backend = mysql
+    name = aurweb_test
+    user = aur
+    password = aur
+    socket = /var/run/mysqld/mysqld.sock
+    ...
+
+    # mysql -u root -e "CREATE USER 'aur'@'localhost' IDENTIFIED BY 'aur'"
+    # mysql -u root -e "CREATE DATABASE aurweb_test"
+    # mysql -u root -e "GRANT ALL ON aurweb_test.* TO 'aur'@'localhost'"
+    # mysql -u root -e "FLUSH ALL PRIVILEGES"
+
+    $ export AUR_CONFIG=conf/config
+    $ python3 -m aurweb.initdb
+
+Or more lightweight with `sqlite`:
+
+    $ cat $AUR_CONFIG
+    [database]
+    backend = sqlite
+    name = aurweb.sqlite3
+    ...
+
+    $ export AUR_CONFIG=conf/config
+    $ python3 -m aurweb.initdb
+
+After initializing a fresh test database, users can continue on to
+[Running Tests](#running-tests).
+
 Running tests
 -------------
 
 Recommended method of running tests: `make check`.
 
-First, setup the test configuration:
+Makefile test targets: `sh`, `pytest`.
 
-    $ sed -r 's;YOUR_AUR_ROOT;$(pwd);g' conf/config.dev > conf/config
+Run tests from the project root.
 
-You'll need to make sure that emails can be sent out by aurweb.scripts.notify.
+    $ cd /path/to/aurweb
-If you don't have anything setup, just install postfix and start it before
-running tests.
 
-With those installed, one can run Python tests manually with any AUR config
+Ensure you have the proper `AUR_CONFIG` exported:
-specified by `AUR_CONFIG`:
 
-    $ AUR_CONFIG=conf/config coverage run --append /usr/bin/pytest test/
+    $ export AUR_CONFIG=conf/config
 
-After tests are run (particularly, with `coverage run` included), one can
+To run `sharness` shell test suites (requires Arch Linux):
-produce coverage reports.
+
+    $ make -C test sh
+
+To run `pytest` Python test suites:
+
+    $ make -C test pytest
+
+To produce coverage reports related to Python when running tests manually,
+use the following method:
+
+    $ coverage run --append /path/to/python/file.py
+
+**Note:** Sharness test suites (shell) internally run coverage run.
+
+After tests are run, one can produce coverage reports.
 
     # Print out a CLI coverage report.
     $ coverage report
+
     # Produce an HTML-based coverage report.
     $ coverage html
 
-When running `make -C test`, all tests ran will produce coverage data via
+Writing Python tests (current)
-`coverage run --append`. After running `make -C test`, one can continue with
+------------------------------
-coverage reporting steps above. Running tests through `make` will test and
-cover code ran by both aurweb scripts and our pytest collection.
 
-Writing tests
+Almost all of our `pytest` suites use the database in some way. There
--------------
+are a few particular testing utilities in `aurweb` that one should
+keep aware of to aid testing code:
+
+- `aurweb.testing.setup_init_db(*tables)`
+    - Prepares test database tables to be cleared before a test
+      is run. Be careful not to specify any tables we depend on
+      for constant records, like `AccountTypes`, `DependencyTypes`,
+      `RelationTypes` and `RequestTypes`.
+- `aurweb.testing.requests.Request`
+    - A fake stripped down version of `fastapi.Request` that can
+      be passed to any functions in our codebase which use
+      `fastapi.Request` parameters.
+
+Example code:
+
+    import pytest
+
+    from aurweb import db
+    from aurweb.models.user import User
+    from aurweb.testing import setup_test_db
+    from aurweb.testing.requests import Request
+
+
+    @pytest.fixture(autouse=True)
+    def setup():
+        setup_test_db(User.__tablename__)
+
+    @pytest.fixture
+    def user():
+        yield db.create(User, Passwd="testPassword", ...)
+
+    def test_user_login(user):
+        assert isinstance(user, User) is True
+
+        fake_request = Request()
+        sid = user.login(fake_request, "testPassword")
+        assert sid is not None
+
+Writing Sharness tests (legacy)
+-------------------------------
 
 Shell test scripts must follow the Test Anything Protocol specification:
 http://testanything.org/tap-specification.html
@@ -77,7 +188,7 @@ importable without toying with os.path or PYTHONPATH.
 Tests written in shell should use sharness. In general, new tests should be
 consistent with existing tests unless they have a good reason not to.
 
-Debugging tests
+Debugging Sharness tests
 ---------------
 
 By default, `make -C test` is quiet and does not print out verbose information

test/test_db.py

@@ -119,9 +119,20 @@ def test_sqlalchemy_mysql_port_url():
         aurweb.config.rehash()
 
 
+def test_sqlalchemy_mysql_socket_url():
+    tmpctx, tmp = make_temp_config("conf/config",
+                                   (r"[;]?port = 3306", ";port = 3306"))
+
+    with tmpctx:
+        with mock.patch.dict(os.environ, {"AUR_CONFIG": tmp}):
+            aurweb.config.rehash()
+            assert db.get_sqlalchemy_url()
+        aurweb.config.rehash()
+
+
 def test_sqlalchemy_unknown_backend():
     tmpctx, tmp = make_temp_config("conf/config",
-                                   (r"backend = mysql", "backend = blah"))
+                                   (r"backend = .+", "backend = blah"))
 
     with tmpctx:
         with mock.patch.dict(os.environ, {"AUR_CONFIG": tmp}):
@@ -156,7 +167,7 @@ def test_connection_class_sqlite_without_fail():
 
 def test_connection_class_unsupported_backend():
     tmpctx, tmp = make_temp_config("conf/config",
-                                   (r"backend = mysql", "backend = blah"))
+                                   (r"backend = .+", "backend = blah"))
 
     with tmpctx:
         with mock.patch.dict(os.environ, {"AUR_CONFIG": tmp}):