Commit graph

66 commits

Author SHA1 Message Date
Dan McGee
6ae2bc514f Add a upload directory transform script
This goes with the previous patch that moves uploads into segmented
subdirectories. To actually run, follow the DRYRUN instructions.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-10 15:23:00 +02:00
Dan McGee
9a79d2105e Segment the upload directory by package name prefix
This implements the following scheme:

* /packages/cower/ --> /packages/co/cower/
* /packages/j/     --> /packages/j/j/
* /packages/zqy/   --> /packages/zq/zqy/

We take up to the first two characters of each package name as a
intermediate subdirectory, and then the full package name lives
underneath that. Shorter named packages live in a single letter
directory.

Why, you ask? Well because earlier today the AUR hit 32,000 entries in
the unsupported/ directory, making new package uploads impossible. While
some might argue we shouldn't have so many damn packages in the repos,
we should be able to handle this case.

Why two characters instead of one? Our two biggest two-char groups, 'pe'
and 'py', both start with 'p', and have nearly 2000 packages each. Go
Python and Perl.

Still needed is a "move the existing data" script, as well as a set of
rewrite rules for those wishing to preserve backward compatible URLs for
any helper programs doing the wrong thing and relying on them.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-10 14:34:07 +02:00
Dan McGee
1e20af9aed scripts/cleanup: use native PHP only
No need to shell out to the system here. Also fix the script so it
actually works.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-10 14:34:06 +02:00
Dan McGee
2e20a7929f aurblup: use implicit make rules and adjust CFLAGS
We don't need to add rules for implicit .c -> .o conversions. Also add
-O2 to the CFLAGS as gcc doesn't print many warnings unless at least
some level of optimization is used.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-28 08:53:07 +02:00
Dan McGee
59c82af185 aurblup: mark all functions and global variables static
These don't need visibility outside of this compilation unit. This also
allows a C compiler to inline and optimize as it sees fit.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-28 08:53:07 +02:00
Dan McGee
0cb493ef2a aurblup: blacklist processing query changes
* Do all list building and freeing outside of the transaction to keep it
  as short as possible.
* Remove ability to blacklist without transactions as we now only
  support InnoDB/transactional engines with proper relations.
* No need to turn autocommit off; BEGIN TRANSACTION operates regardless
  of this setting.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-28 08:53:06 +02:00
Lukas Fleischer
e72f02872f Use a real sync algorithm in aurblup (fixes FS#23039).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-26 20:41:04 +01:00
Lukas Fleischer
c332c0a0f5 Update aurblup "README".
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-11 14:43:24 +01:00
Lukas Fleischer
a64cd12feb Add support for MySQL transactions to aurblup.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-11 14:43:23 +01:00
Lukas Fleischer
13bcc269ff Fix typos in aurblup source and "README".
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-11 14:43:23 +01:00
Lukas Fleischer
ddc54358d0 Add packages' provides and replaces to the blacklist in aurblup.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-11 14:43:23 +01:00
Lukas Fleischer
48957ef5d5 Fix memory leak in aurblup.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-11 14:43:22 +01:00
Lukas Fleischer
400d7845e4 Add AUR package blacklist updater (aurblup).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-11 14:43:22 +01:00
Dan McGee
7f5af61c88 Drop PackageLocations table and references
We don't need this anymore since all packages managed here are
well...managed here. Rip out all of the places we were using this field,
many of which depended on the magic value '2' anyway.

On the display side of things, we had a column that was always showing
'unsupported' that is now gone, and you can no longer sort by this column.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-01 12:41:08 +01:00
Loui Chang
bbc05414cf Update cleanup script to remove non-unsupported files.
Signed-off-by: Loui Chang <louipc.ist@gmail.com>
2009-03-03 11:21:08 -05:00
Loui Chang
8d80ff50af Add a script to clean up files of deleted unsupported packages.
Signed-off-by: Loui Chang <louipc.ist@gmail.com>
2008-11-21 13:52:04 -05:00