diff --git a/INSTALL b/INSTALL index e14b9f31..1779b1dc 100644 --- a/INSTALL +++ b/INSTALL @@ -9,98 +9,131 @@ In particular, the cgit interface will be unusable as well as the ssh+git interface. For a detailed description on how to setup a full aurweb server, read the instructions below. -1) Clone the aurweb project: +1) Clone the aurweb project and install it (via `python-poetry`): - $ cd /srv/http/ - $ git clone git://git.archlinux.org/aurweb.git + $ cd /srv/http/ + $ git clone git://git.archlinux.org/aurweb.git + $ poetry install 2) Setup a web server with PHP and MySQL. Configure the web server to redirect all URLs to /index.php/foo/bar/. The following block can be used with nginx: server { - listen 80; + # https is preferred and can be done easily with LetsEncrypt + # or self-CA signing. Users can still listen over 80 for plain + # http, for which the [options] disable_http_login used to toggle + # the authentication feature. + listen 443 ssl http2; server_name aur.local aur; - root /srv/http/aurweb/web/html; - index index.php; + # To enable SSL proxy properly, make sure gunicorn and friends + # are supporting forwarded headers over 127.0.0.1 or any if + # the asgi server is contacted by non-localhost hosts. + ssl_certificate /etc/ssl/certs/aur.cert.pem; + ssl_certificate_key /etc/ssl/private/aur.key.pem; - location ~ ^/[^/]+\.php($|/) { - fastcgi_pass unix:/var/run/php-fpm/php-fpm.sock; - fastcgi_index index.php; - fastcgi_split_path_info ^(/[^/]+\.php)(/.*)$; - fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; - fastcgi_param PATH_INFO $fastcgi_path_info; - include fastcgi_params; + # Asset root. This is used to match against gzip archives. + root /srv/http/aurweb/web/html; + + # TU Bylaws redirect. + location = /trusted-user/TUbylaws.html { + return 301 https://tu-bylaws.aur.archlinux.org; } - location ~ .* { - rewrite ^/(.*)$ /index.php/$1 last; + # smartgit location. + location ~ "^/([a-z0-9][a-z0-9.+_-]*?)(\.git)?/(git-(receive|upload)-pack|HEAD|info/refs|objects/(info/(http-)?alternates|packs)|[0-9a-f]{2}/[0-9a-f]{38}|pack/pack-[0-9a-f]{40}\.(pack|idx))$" { + include uwsgi_params; + uwsgi_pass smartgit; + uwsgi_modifier1 9; + uwsgi_param SCRIPT_FILENAME /usr/lib/git-core/git-http-backend; + uwsgi_param PATH_INFO /aur.git/$3; + uwsgi_param GIT_HTTP_EXPORT_ALL ""; + uwsgi_param GIT_NAMESPACE $1; + uwsgi_param GIT_PROJECT_ROOT /srv/http/aurweb; + } + + # cgitrc.proto should be configured and located somewhere + # of your choosing. + location ~ ^/cgit { + include uwsgi_params; + rewrite ^/cgit/([^?/]+/[^?]*)?(?:\?(.*))?$ /cgit.cgi?url=$1&$2 last; + uwsgi_modifier1 9; + uwsgi_param CGIT_CONFIG /srv/http/aurweb/conf/cgitrc.proto; + uwsgi_pass cgit; + } + + # Static archive assets. + location ~ \.gz$ { + types { application/gzip text/plain } + default_type text/plain; + add_header Content-Encoding gzip; + expires 5m; + } + + # For everything else, proxy the http request to (guni|uvi|hyper)corn. + # The ASGI server application should allow this request's IP to be + # forwarded via the headers used below. + # https://docs.gunicorn.org/en/stable/settings.html#forwarded-allow-ips + location / { + proxy_pass http://127.0.0.1:8000; + proxy_set_header Host $http_host; + proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; + proxy_set_header X-Forwarded-Protocol ssl; + proxy_set_header X-Forwarded-Proto $scheme; + proxy_set_header X-Forwarded-Ssl on; } } - Ensure to enable the pdo_mysql extension in php.ini. - 3) Optionally copy conf/config.defaults to /etc/aurweb/. Create or copy /etc/aurweb/config (this is expected to contain all configuration settings if the defaults file does not exist) and adjust the configuration (pay attention to disable_http_login, enable_maintenance and aur_location). -4) Install dependencies. +4) Install system-wide dependencies: -4a) Install system-wide dependencies: + # pacman -S git gpgme cgit curl openssh uwsgi uwsgi-plugin-cgi \ + python-poetry - # pacman -S git gpgme cgit pyalpm python-srcinfo curl openssh \ - uwsgi uwsgi-plugin-cgi php php-fpm - -4b) Install Python dependencies via poetry (required): - -**NOTE** Users do not need to install pip or poetry dependencies system-wide. -You may take advantage of Poetry's virtualenv integration to manage -dependencies. This is merely a demonstration to show users how to without -a virtualenv. In Docker and CI, we don't yet use a virtualenv. - - ## Install Poetry dependencies system-wide, if not using a virtualenv. - # pacman -S python-pip - - ## Ensure pip is upgraded. Poetry depends on it being up to date. - # pip install --upgrade pip - - ## Install Poetry. - # curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python - - # export PATH="$HOME/.poetry/bin:${PATH}" - - ## Use Poetry to install dependencies and the aurweb package. - # poetry lock # Resolve dependencies - # poetry update # Install/update dependencies - # poetry build # Build the aurweb package - # poetry install # Install the aurweb package and scripts - -When installing in a virtualenv, config.defaults must contain the correct -absolute paths to aurweb scripts, which requires modification. - -4c) Setup FastAPI Redis cache (optional). - -First, install Redis and start its service. - - # pacman -S redis - # systemctl enable --now redis - -Now that Redis is running, ensure that you configure aurweb to use -the Redis cache by setting `cache = redis` in your AUR config. - -In `conf/config.defaults`, the `redis_address` configuration is set -to `redis://localhost`. This can be set to point to any Redis server -and will be used as long as `cache = redis`. - -5) Create a new database and a user and import the aurweb SQL schema: - - $ python -m aurweb.initdb - -6) Create a new user: +5) Create a new user: # useradd -U -d /srv/http/aurweb -c 'AUR user' aur + # su - aur -7) Initialize the Git repository: +6a) Install Python dependencies via poetry: + + # Install the package and scripts as the aur user. + $ poetry install + +6b) Setup Services + +aurweb utilizes the following systemd services: +- mariadb +- redis (optional, requires [options] cache 'redis') +- `examples/aurweb.service` + +6c) Setup Cron + +Using [cronie](https://archlinux.org/packages/core/x86_64/cronie/): + + # su - aur + $ crontab -e + +The following crontab file uses every script meant to be run on an +interval: + + AUR_CONFIG='/etc/aurweb/config' + */5 * * * * bash -c 'poetry run aurweb-aurblup' + */5 * * * * bash -c 'poetry run aurweb-mkpkglists --extended' + */5 * * * * bash -c 'poetry run aurweb-pkgmaint' + */5 * * * * bash -c 'poetry run aurweb-usermaint' + */5 * * * * bash -c 'poetry run aurweb-tuvotereminder' + */5 * * * * bash -c 'poetry run aurweb-popupdate' + +7) Create a new database and a user and import the aurweb SQL schema: + + $ poetry run python -m aurweb.initdb + +8) Initialize the Git repository: # mkdir /srv/http/aurweb/aur.git/ # cd /srv/http/aurweb/aur.git/ @@ -108,19 +141,26 @@ and will be used as long as `cache = redis`. # git config --local transfer.hideRefs '^refs/' # git config --local --add transfer.hideRefs '!refs/' # git config --local --add transfer.hideRefs '!HEAD' - # ln -s /usr/local/bin/aurweb-git-update hooks/update # chown -R aur . +Link to `aurweb-git-update` poetry wrapper provided at +`examples/aurweb-git-update.sh` which should be installed +somewhere as executable. + + # ln -s /path/to/aurweb-git-update.sh hooks/update + It is recommended to read doc/git-interface.txt for more information on the administration of the package Git repository. -8) Configure sshd(8) for the AUR. Add the following lines at the end of your - sshd_config(5) and restart the sshd. Note that OpenSSH 6.9 or newer is - needed! +9) Configure sshd(8) for the AUR. Add the following lines at the end of your + sshd_config(5) and restart the sshd. + +If using a virtualenv, copy `examples/aurweb-git-auth.sh` to a location +and call it below: Match User aur PasswordAuthentication no - AuthorizedKeysCommand /usr/local/bin/aurweb-git-auth "%t" "%k" + AuthorizedKeysCommand /path/to/aurweb-git-auth.sh "%t" "%k" AuthorizedKeysCommandUser aur AcceptEnv AUR_OVERWRITE