update INSTALL: asgi-driven aurweb direction

We heavily attempt to provide easy use of poetry virtualenvs
with aurweb in this revision of the INSTALL file. Added a
section about cron jobs and updated the nginx config example
with a lot more detail and locations for other parts of
the AUR infrastructure.

Signed-off-by: Kevin Morris <kevr@0cost.org>

INSTALL

@@ -9,98 +9,131 @@ In particular, the cgit interface will be unusable as well as the ssh+git
 interface. For a detailed description on how to setup a full aurweb server,
 read the instructions below.
 
-1) Clone the aurweb project:
+1) Clone the aurweb project and install it (via `python-poetry`):
 
-   $ cd /srv/http/
+    $ cd /srv/http/
-   $ git clone git://git.archlinux.org/aurweb.git
+    $ git clone git://git.archlinux.org/aurweb.git
+    $ poetry install
 
 2) Setup a web server with PHP and MySQL. Configure the web server to redirect
    all URLs to /index.php/foo/bar/. The following block can be used with nginx:
 
     server {
-        listen       80;
+        # https is preferred and can be done easily with LetsEncrypt
+        # or self-CA signing. Users can still listen over 80 for plain
+        # http, for which the [options] disable_http_login used to toggle
+        # the authentication feature.
+        listen       443 ssl http2;
         server_name  aur.local aur;
 
-        root   /srv/http/aurweb/web/html;
+        # To enable SSL proxy properly, make sure gunicorn and friends
-        index  index.php;
+        # are supporting forwarded headers over 127.0.0.1 or any if
+        # the asgi server is contacted by non-localhost hosts.
+        ssl_certificate /etc/ssl/certs/aur.cert.pem;
+        ssl_certificate_key /etc/ssl/private/aur.key.pem;
 
-        location ~ ^/[^/]+\.php($|/) {
+        # Asset root. This is used to match against gzip archives.
-            fastcgi_pass   unix:/var/run/php-fpm/php-fpm.sock;
+        root   /srv/http/aurweb/web/html;
-            fastcgi_index  index.php;
+
-            fastcgi_split_path_info ^(/[^/]+\.php)(/.*)$;
+        # TU Bylaws redirect.
-            fastcgi_param  SCRIPT_FILENAME  $document_root$fastcgi_script_name;
+        location = /trusted-user/TUbylaws.html {
-            fastcgi_param  PATH_INFO        $fastcgi_path_info;
+            return 301 https://tu-bylaws.aur.archlinux.org;
-            include        fastcgi_params;
         }
 
-        location ~ .* {
+        # smartgit location.
-            rewrite ^/(.*)$ /index.php/$1 last;
+        location ~ "^/([a-z0-9][a-z0-9.+_-]*?)(\.git)?/(git-(receive|upload)-pack|HEAD|info/refs|objects/(info/(http-)?alternates|packs)|[0-9a-f]{2}/[0-9a-f]{38}|pack/pack-[0-9a-f]{40}\.(pack|idx))$" {
+            include      uwsgi_params;
+            uwsgi_pass   smartgit;
+            uwsgi_modifier1 9;
+            uwsgi_param  SCRIPT_FILENAME /usr/lib/git-core/git-http-backend;
+            uwsgi_param  PATH_INFO /aur.git/$3;
+            uwsgi_param  GIT_HTTP_EXPORT_ALL "";
+            uwsgi_param  GIT_NAMESPACE $1;
+            uwsgi_param  GIT_PROJECT_ROOT /srv/http/aurweb;
+        }
+
+        # cgitrc.proto should be configured and located somewhere
+        # of your choosing.
+        location ~ ^/cgit {
+            include uwsgi_params;
+            rewrite ^/cgit/([^?/]+/[^?]*)?(?:\?(.*))?$ /cgit.cgi?url=$1&$2 last;
+            uwsgi_modifier1 9;
+            uwsgi_param CGIT_CONFIG /srv/http/aurweb/conf/cgitrc.proto;
+            uwsgi_pass cgit;
+        }
+
+        # Static archive assets.
+        location ~ \.gz$ {
+            types { application/gzip text/plain }
+            default_type text/plain;
+            add_header Content-Encoding gzip;
+            expires 5m;
+        }
+
+        # For everything else, proxy the http request to (guni|uvi|hyper)corn.
+        # The ASGI server application should allow this request's IP to be
+        # forwarded via the headers used below.
+        # https://docs.gunicorn.org/en/stable/settings.html#forwarded-allow-ips
+        location / {
+            proxy_pass http://127.0.0.1:8000;
+            proxy_set_header Host $http_host;
+            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+            proxy_set_header X-Forwarded-Protocol ssl;
+            proxy_set_header X-Forwarded-Proto $scheme;
+            proxy_set_header X-Forwarded-Ssl on;
         }
     }
 
-    Ensure to enable the pdo_mysql extension in php.ini.
-
 3) Optionally copy conf/config.defaults to /etc/aurweb/. Create or copy
    /etc/aurweb/config (this is expected to contain all configuration settings
    if the defaults file does not exist) and adjust the configuration (pay
    attention to disable_http_login, enable_maintenance and aur_location).
 
-4) Install dependencies.
+4) Install system-wide dependencies:
 
-4a) Install system-wide dependencies:
+    # pacman -S git gpgme cgit curl openssh uwsgi uwsgi-plugin-cgi \
+        python-poetry
 
-    # pacman -S git gpgme cgit pyalpm python-srcinfo curl openssh \
+5) Create a new user:
-                uwsgi uwsgi-plugin-cgi php php-fpm
-
-4b) Install Python dependencies via poetry (required):
-
-**NOTE** Users do not need to install pip or poetry dependencies system-wide.
-You may take advantage of Poetry's virtualenv integration to manage
-dependencies. This is merely a demonstration to show users how to without
-a virtualenv. In Docker and CI, we don't yet use a virtualenv.
-
-    ## Install Poetry dependencies system-wide, if not using a virtualenv.
-    # pacman -S python-pip
-
-    ## Ensure pip is upgraded. Poetry depends on it being up to date.
-    # pip install --upgrade pip
-
-    ## Install Poetry.
-    # curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -
-    # export PATH="$HOME/.poetry/bin:${PATH}"
-
-    ## Use Poetry to install dependencies and the aurweb package.
-    # poetry lock     # Resolve dependencies
-    # poetry update   # Install/update dependencies
-    # poetry build    # Build the aurweb package
-    # poetry install  # Install the aurweb package and scripts
-
-When installing in a virtualenv, config.defaults must contain the correct
-absolute paths to aurweb scripts, which requires modification.
-
-4c) Setup FastAPI Redis cache (optional).
-
-First, install Redis and start its service.
-
-    # pacman -S redis
-    # systemctl enable --now redis
-
-Now that Redis is running, ensure that you configure aurweb to use
-the Redis cache by setting `cache = redis` in your AUR config.
-
-In `conf/config.defaults`, the `redis_address` configuration is set
-to `redis://localhost`. This can be set to point to any Redis server
-and will be used as long as `cache = redis`.
-
-5) Create a new database and a user and import the aurweb SQL schema:
-
-    $ python -m aurweb.initdb
-
-6) Create a new user:
 
     # useradd -U -d /srv/http/aurweb -c 'AUR user' aur
+    # su - aur
 
-7) Initialize the Git repository:
+6a) Install Python dependencies via poetry:
+
+    # Install the package and scripts as the aur user.
+    $ poetry install
+
+6b) Setup Services
+
+aurweb utilizes the following systemd services:
+- mariadb
+- redis (optional, requires [options] cache 'redis')
+- `examples/aurweb.service`
+
+6c) Setup Cron
+
+Using [cronie](https://archlinux.org/packages/core/x86_64/cronie/):
+
+    # su - aur
+    $ crontab -e
+
+The following crontab file uses every script meant to be run on an
+interval:
+
+    AUR_CONFIG='/etc/aurweb/config'
+    */5 * * * * bash -c 'poetry run aurweb-aurblup'
+    */5 * * * * bash -c 'poetry run aurweb-mkpkglists --extended'
+    */5 * * * * bash -c 'poetry run aurweb-pkgmaint'
+    */5 * * * * bash -c 'poetry run aurweb-usermaint'
+    */5 * * * * bash -c 'poetry run aurweb-tuvotereminder'
+    */5 * * * * bash -c 'poetry run aurweb-popupdate'
+
+7) Create a new database and a user and import the aurweb SQL schema:
+
+    $ poetry run python -m aurweb.initdb
+
+8) Initialize the Git repository:
 
     # mkdir /srv/http/aurweb/aur.git/
     # cd /srv/http/aurweb/aur.git/
@@ -108,19 +141,26 @@ and will be used as long as `cache = redis`.
     # git config --local transfer.hideRefs '^refs/'
     # git config --local --add transfer.hideRefs '!refs/'
     # git config --local --add transfer.hideRefs '!HEAD'
-    # ln -s /usr/local/bin/aurweb-git-update hooks/update
     # chown -R aur .
 
+Link to `aurweb-git-update` poetry wrapper provided at
+`examples/aurweb-git-update.sh` which should be installed
+somewhere as executable.
+
+    # ln -s /path/to/aurweb-git-update.sh hooks/update
+
    It is recommended to read doc/git-interface.txt for more information on the
    administration of the package Git repository.
 
-8) Configure sshd(8) for the AUR. Add the following lines at the end of your
+9) Configure sshd(8) for the AUR. Add the following lines at the end of your
-   sshd_config(5) and restart the sshd. Note that OpenSSH 6.9 or newer is
+   sshd_config(5) and restart the sshd.
-   needed!
+
+If using a virtualenv, copy `examples/aurweb-git-auth.sh` to a location
+and call it below:
 
     Match User aur
         PasswordAuthentication no
-        AuthorizedKeysCommand /usr/local/bin/aurweb-git-auth "%t" "%k"
+        AuthorizedKeysCommand /path/to/aurweb-git-auth.sh "%t" "%k"
         AuthorizedKeysCommandUser aur
         AcceptEnv AUR_OVERWRITE