Commit graph

23 commits

Author SHA1 Message Date
Leonidas Spyropoulos
286834bab1
fix: regression on gzipped filenames from 3dcbee5a
With the 3dcbee5a the filenames inside the .gz archives contained .tmp
at the end. This fixes those by using Gzip Class constructor instead of
the gzip.open method.

Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>
2022-10-31 14:43:31 +00:00
Mario Oenning
6ee34ab3cb feat: add field "CoMaintainers" to metadata-archives 2022-10-31 09:42:56 +00:00
Mario Oenning
333051ab1f feat: add field "Submitter" to metadata-archives 2022-10-28 16:55:16 +00:00
Mario Oenning
3dcbee5a4f fix: make overwriting of archive files atomic 2022-10-28 12:42:50 +00:00
Leonidas Spyropoulos
9c0f8f053e
chore: rename logging.py and redis.py to avoid circular imports
Signed-off-by: Leonidas Spyropoulos <artafinde@archlinux.org>
2022-10-22 18:51:38 +01:00
Kevin Morris
30e72d2db5 feat: archive git repository (experimental)
See doc/git-archive.md for general Git archive specifications
See doc/repos/metadata-repo.md for info and direction related to the new Git metadata archive
2022-09-24 16:51:25 +00:00
Joakim Saario
9c6c13b78a
style: Run pre-commit 2022-08-22 22:40:45 +02:00
Jelle van der Waa
a509e40474 fix(python): use standard dict/list type annotation
Since Python 3.9 list/dict can be used as type hint.
2022-08-02 12:06:58 +00:00
Awal Garg
b119db251b fixup: feat(archives): add .sha256 and construct archives in tmpdir 2022-02-09 07:03:12 +05:30
Kevin Morris
40a0e866e7
feat(archives): add {archive}.sha256 and construct archives in tmpdir
This change brings some new additions to our archives:
- SHA-256 .sha256 hexdigests
- We construct our archives in a tmpdir now and move them to the
archive destination when all are completed. This removes some
corrupted downloading when archiving is in-process.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-08 13:28:39 -08:00
Kevin Morris
33cddb36ff
fix: restore URL field in mkpkglists meta archives
Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-02-07 01:20:42 -08:00
Kevin Morris
211ca5e49c
housekeep: define filters in their own modules
This patch cleans up aurweb.templates and removes direct
module-level initialization of the environment.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2022-01-18 03:06:17 -08:00
Kevin Morris
29c2d0de6b
change(mkpkglists): converted to use aurweb.db ORM
- Improved speed dramatically
- Removed mkpkglists sharness

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-28 19:55:08 -08:00
Kevin Morris
8788f99005
fix(mkpkglists): restore isort order
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 13:54:18 -08:00
Kevin Morris
6e344ce9da
fix(mkpkglists): default keys to result[1]
Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 13:41:53 -08:00
Kevin Morris
4f7aeafa8d
feat(docker): host gzip archive downloads
- added config option [mkpkglists] archivedir
    - created by mkpkglists

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 07:39:23 -08:00
Kevin Morris
abbecf5194
change(mkpkglists): remove header comments
These comments change every time mkpkglists is run; which
would invalidate the ETag headers disbursed by the gzip
host. This commit removes those changing headers.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-10 07:39:17 -08:00
Kevin Morris
0155f4ea84
fix(mkpkglists): remove caching
We really need caching for this; however, our current caching
method will cause the script to bypass changes to columns
if they have nothing to do with IDs.

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-09 02:07:13 -08:00
Kevin Morris
f3f662c696
fix(mkpkglists): improve package meta archive
The SQL logic in this file for package metadata now exactly
reflects RPC's search logic, without searching for specific
packages.

Two command line arguments are available:

    --extended | Include License, Keywords, Groups, relations
                 and dependencies.

When --extended is passed, the script will create a
packages-meta-ext-v1.json.gz, configured via packagesmetaextfile.

Archive JSON is in the following format: line-separated package objects
enclosed in a list:

    [
    {...},
    {...},
    {...}
    ]

Signed-off-by: Kevin Morris <kevr@0cost.org>
2021-11-09 02:06:50 -08:00
Kristian Klausen
f606140050
feat(PHP): Add packages dump file with more metadata 2021-11-09 02:04:58 -08:00
Lukas Fleischer
cb307bf01a Do not hardcode path to the Python interpreter
Use `/usr/bin/env python3` instead of `/usr/bin/python3` in the shebang
of Python scripts. This adds support for non-standard Python interpreter
paths such as the paths used in virtualenv environments.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2017-08-25 07:01:12 +02:00
Lukas Fleischer
d9883ee642 mkpkglists: Generate a list of user names
In addition to the packages list and the package base list, also create
a list of registered user names.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2017-08-01 07:08:29 +02:00
Lukas Fleischer
37188603b5 Make maintenance scripts installable
Add wrappers for the maintenance scripts to the setuptools
configuration.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-10-17 15:23:13 +02:00
Renamed from scripts/mkpkglists.py (Browse further)