Commit graph

590 commits

Author SHA1 Message Date
Dan McGee
9a79d2105e Segment the upload directory by package name prefix
This implements the following scheme:

* /packages/cower/ --> /packages/co/cower/
* /packages/j/     --> /packages/j/j/
* /packages/zqy/   --> /packages/zq/zqy/

We take up to the first two characters of each package name as a
intermediate subdirectory, and then the full package name lives
underneath that. Shorter named packages live in a single letter
directory.

Why, you ask? Well because earlier today the AUR hit 32,000 entries in
the unsupported/ directory, making new package uploads impossible. While
some might argue we shouldn't have so many damn packages in the repos,
we should be able to handle this case.

Why two characters instead of one? Our two biggest two-char groups, 'pe'
and 'py', both start with 'p', and have nearly 2000 packages each. Go
Python and Perl.

Still needed is a "move the existing data" script, as well as a set of
rewrite rules for those wishing to preserve backward compatible URLs for
any helper programs doing the wrong thing and relying on them.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-10 14:34:07 +02:00
Lukas Fleischer
16cda58ff8 Fix empty depends database insert
Do not try to insert a falsey value into the database in case of
"depends=()".

Signed-off-by: Manuel <manutortosa@chakra-project.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-07-29 23:51:29 +02:00
Lukas Fleischer
a32b2f2ef7 Replace split() invocations with explode()
Do not use split(), which has been deprecated as of PHP 5.3.0. As we
don't even require regular expressions here, just use explode() instead.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-28 08:55:47 +02:00
Slavi Pantaleev
2131d3cb8b Honor epoch field in PKGBUILD files.
The epoch field in PKGBUILD files was completely ignored until now,
and the final Version field for a package consisted only of
pkgver and pkgrel (example: 5.0-1)

This means that rpc.php reported the version incorrectly for packages
having epoch > 0.
One case where this was a problem is that it confused AUR helpers
wanting to examine all locally installed packages (with epoch > 0)
and search the AUR for an updated version.

The epoch field is taken into consideration now, and if not 0,
will be prepended to the final Version field (example: 1:5.0-1)

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-25 11:29:56 +02:00
elij
888db089c5 rename *.inc files to *.inc.php and adjust imports and references
Lukas: Add note to "UPGRADING".

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-22 15:15:04 +02:00
elij
023d2a2521 make rss.php use the apc cache instead of a cache file
utilize the apc cache functionality in aur.inc to cache the rss feed
output. the cache will cache on a per-protocol basis (http/https) so
that urls are appropriate regardless of which url people hit.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-22 15:04:23 +02:00
Lukas Fleischer
fa53ca68b0 Cleanup RPC usage output a bit
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-29 16:06:28 +02:00
Lukas Fleischer
21827dfb1c Properly encode ampersands in the RPC usage output
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-29 15:58:46 +02:00
elij
4c9da512af fix two issues (php notice level) with html/rss.php
- Undefined index: HTTPS in rss.php on line 8
- Undefined property: RSSCreator20::$cssStyleSheet in
  feedcreator.class.php on line 591

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-29 15:57:45 +02:00
elij
4a24bca069 restructure the html/rpc.php endpoint
- move request_method test to the top, and catch other request types
  (HEAD, PUT, etc)
- change how html output is handled. instead of building a string, just
  output the html
- set appropriate response header for incorrect request_method.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-29 15:57:45 +02:00
elij
0898f1447a test return value from db_query before assuming it is valid
make the sql query form consistent in usage by cleaning up instances
where db_query's result was not inspected before attempting to fetch row
data from the handle

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-17 10:43:42 +02:00
elij
10ea5f5ff6 create variable before referencing it with .=
fixes php notice level error:
  Undefined variable: whovoted in ../tu.php

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-27 14:50:28 +02:00
Dan McGee
3609cf140e SQL: treat all UID/ID values as numbers, not strings
Ensure we are not quoting these values in any of our SQL queries.

Thanks-to: elij <elij.mx@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-27 14:50:27 +02:00
Dan McGee
fcda6671f3 SQL: use standard LIMIT/OFFSET syntax
Increases compatibility with standard SQL dialect.

Thanks-to: elij <elij.mx@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-27 14:50:27 +02:00
Lukas Fleischer
ed9c95623f pkgsubmit.php: Ensure the session is linked to a valid user.
Prevent race conditions that may occur when either the session or the
user is deleted before we extract the actual user identifier.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-24 14:22:24 +02:00
Lukas Fleischer
9ff30614b8 pkgsubmit.php: Remove redundant uid_from_sid() invocations.
uid_from_sid() is called once at the very beginning of the script,
storing the actual user identifier in "$uid". No need to fire up another
query.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-24 14:00:09 +02:00
Lukas Fleischer
5ea9fc197d Use HTTPs for links in password reset confirmation mails.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-19 23:59:19 +02:00
Dan McGee
0488e8597c rpc: introduce multiinfo query (fixes FS#17583)
The majority of "real world" info requests [1] come in hefty batches. We
would be better served to handle these in one request rather than
multiple by allowing AUR clients to send multiple arguments.

This enables things like this to work:
    http://aur.test/rpc.php?type=multiinfo&arg[]=cups-xerox&arg[]=cups-mc2430dl&arg[]=10673

Note to RPC users: unfortunately due to the asinine design of PHP, you
unfortunately have to use the 'arg[]' syntax if you want more than one
query argument, or you will only get the package satisfying the last arg
you pass.

[1] Rough data from April 11, 2011, with a total hit count of 1,109,163:
     12 /login.php
     13 /rpc.php?type=sarch
     15 /rpc.php?type=msearch
     16 /pingserver.php
     16 /rpc.php
     22 /logout.php
    163 /passreset.php
    335 /account.php
    530 /pkgsubmit.php
    916 /rss2.php
   3838 /index.php
   6752 /rss.php
   9699 /
  42478 /rpc.php?type=search
 184737 /packages.php
 681725 /rpc.php?type=info

That means a whopping 61.5% of our requests were for info over the RPC
interface; package pages are a distant second at only 16.7%.

Lukas: Introduce "multiinfo" query instead of extending "info" (for the
sake of backward compatibility).

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-16 17:49:00 +02:00
Lukas Fleischer
eef5353bde Remove "New Package Notify" option from user account settings.
Do this in preparation for the upcoming notification script removal.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-13 12:22:28 +02:00
Lukas Fleischer
8b4b2d207e Remove File_Find PEAR module from code base.
We removed the code depending on this a long time ago - drop it and add
some note to "UPGRADING".

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-05 11:31:12 +02:00
Lukas Fleischer
d142ca4053 Fix PHP warning when browsing a non-existent package.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-04 23:31:19 +02:00
Dan McGee
7c91c59245 Remove Dummy Package concept
Instead, we just store dependencies directly in the PackageDepends
table. Since we don't use this info anywhere besides the package details
page, there is little value in precalculating what is in the AUR vs.
what is not.

An upgrade path is provided via several SQL statements in the UPGRADING
document. There should be no user-visible change from this, but the DB
schema gets a bit more sane and we no longer have loads of junk packages
in our tables that are never shown to the end user. This should also
help the MySQL query planner in several cases as we no longer have to be
careful to exclude dummy packages on every query.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-03 22:59:32 +02:00
Dan McGee
1f252eba64 Always set ModifiedTS including new packages
Set it equal to the SubmittedTS field, which will be our indication the
package is new when we show the logo on the front page of the AUR.

This results in the ability to remove the use of the unindexable
GREATEST() function from the AUR code everywhere we had to use it before
to handle the 0 timestamp case.

Note that there is no race condition here in calling UNIX_TIMESTAMP()
twice- it always returns the time at the beginning of statment
execution:

    mysql> select unix_timestamp(), sleep(2), unix_timestamp();
    +------------------+----------+------------------+
    | unix_timestamp() | sleep(2) | unix_timestamp() |
    +------------------+----------+------------------+
    |       1300851746 |        0 |       1300851746 |
    +------------------+----------+------------------+
    1 row in set (2.00 sec)

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-03 22:59:31 +02:00
Dan McGee
1128489bd0 Submission process code refactor
We had a ton of duplicate code shared between the insert and update
cases. Do a refactor so we can pull this stuff out below the if/else
block and only need it there once, saving some headaches.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-03 22:59:31 +02:00
Lukas Fleischer
a427bd72a7 Be more restrictive with source tarball contents.
Reject tarballs containing more than one directory or files outside a
directory.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-30 20:49:21 +02:00
Lukas Fleischer
746c2b72b5 Fix PHP notice when submitting an empty file.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-30 16:16:27 +02:00
Lukas Fleischer
e4130fcca6 Fix strict standards warnings in "web/html/pkgsubmit.php".
end() expects a reference but we pass a function return value here.
Using list() is a bit hacky as well as it expects a 0-based array
whereas unpack() returns a 1-based array - thus we use "list(, $foo)".

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-30 12:28:17 +02:00
Lukas Fleischer
47faf88f14 Check if submitted files are in GZIP format.
This is quite hacky but this way we can ensure users get comprehensible
error messages when trying to upload ".tar.xz" or ".tar.bz2" files.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-30 11:55:32 +02:00
Lukas Fleischer
2eb45e7d9e Fix XSS vulnerabilities in "web/html/voters.php".
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-30 11:10:16 +02:00
Lukas Fleischer
55eb55a75f Fix PHP undefined notice in "web/html/packages.php".
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-30 11:06:20 +02:00
Lukas Fleischer
7f9e498e48 Fix broken XHTML.
Fix a lot of invalid XHTML in the templates and actions. There might
still be some legacy code left, but this should cover most of it.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-11 19:00:50 +01:00
Dan McGee
80401c6afc More PHP Notice undefined fixups
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-10 18:15:59 +01:00
Dan McGee
cdc01130cf Fix PHP notices in account pages
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-04 10:29:17 +01:00
Dan McGee
d1d0288598 Add action lookup helper function
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-04 10:29:15 +01:00
Dan McGee
984ce9529c Improve cookie handling
* Remove comment that is mostly bogus- the domain is automatically set.
* When logging out, don't delete the language cookie.
* Make the language cookie persistent.
* Use the minimal time possible to expire cookies; no need to compute
  anything.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-04 10:29:13 +01:00
Dan McGee
90485e8f42 Fix potential injection vulnerability
We trusted the values we pulled out of the IDs array and never coerced
them to integers, passing them to the backend unescaped and uncasted.
Ensure they are treated as integers only and validate the resulting
value is > 0.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-01 20:27:49 +01:00
Lukas Fleischer
1e3fa38de5 Define "Packages.SubmitterUID" and "Packages.MaintainerUID" as "NULL".
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-27 19:46:19 +01:00
Lukas Fleischer
2e1ce869ec Reject blacklisted packages on initial submission only.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-25 11:18:02 +01:00
Lukas Fleischer
09d8128f99 Protect users against ZIP bombs (fixes FS#22991).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-23 11:17:27 +01:00
Lukas Fleischer
4b2b8afb8a Reject packages with subdirectories (fixes FS#22995).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-21 19:19:53 +01:00
Lukas Fleischer
31b0b68b47 Automatically adopt when updating an orphan package (fixes FS#22992).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-21 17:56:19 +01:00
Lukas Fleischer
bc207d25cd Use move_uploaded_file() instead of rename() in "pkgsubmit.php".
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-21 17:07:51 +01:00
PyroPeter
3d6c6cc98a Support for langauges written right-to-left
Signed-off-by: PyroPeter <abi1789@googlemail.com>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-18 13:52:38 +01:00
PyroPeter
346e82716b pkg_search_results: replace blind-table with floating divs
* I tried to remove errors in the sgml-structure
  e.g.: <div>
          <?php if (foo) { ?>
            </div>
          <?php } ?>
* I did not remove or add code (except the <table> and <div> stuff, of cause).
  I only changed the order of the html/php-tags.
* The bottom and top of the script are now properly indented.
  I did not indent the middle part (table of search results) because that would
  render the diff completely useless.

Signed-off-by: PyroPeter <abi1789@googlemail.com>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-18 13:52:35 +01:00
PyroPeter
1fdecbd5a4 pkg_search_results: rewrite of pagination
* Most of the PHP-code was moved to pkgfuncs.php to keep the template simple.

Signed-off-by: PyroPeter <abi1789@googlemail.com>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-17 18:54:26 +01:00
Lukas Fleischer
b69f548065 Add a package name blacklist.
Can be used to blacklist package names for normal users. TUs and
developers are not affected. This is especially useful if used together
with a cron job that updates the blacklist periodically, e.g. to reject
packages which are available in the binary repos (FS#12902).

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-11 14:43:22 +01:00
Lukas Fleischer
e3bde91f6d Minor variable parser bug fix (cf. commits 492c8c66, 7a58e99e).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-02 17:18:27 +01:00
Lukas Fleischer
8e32767c70 Parse versioned deps correctly when using "<" or ">" (fixes FS#22679).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-02 17:05:58 +01:00
Lukas Fleischer
366ea03118 Remove "FSPath" column from "Packages" table.
This field is not used anymore, so drop it from the table and remove all
references.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-01 15:01:56 +01:00
Lukas Fleischer
7a58e99eab Improve PKGBUILD variable parser correctness (cf. commit 492c8c66).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-02-01 12:41:15 +01:00