Commit graph

550 commits

Author SHA1 Message Date
Dan McGee
45a50d4a66 Remove unnecessary atype and uid lookup from package_details()
These were never used in the function. Where they are used is in the
pkg_details.php template, so move them closer to their actual usage so
as not to confuse poor programmers such as myself.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-11 16:17:09 +02:00
Dan McGee
34fe0bec4c Use 'true' instead of 'True' everywhere
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-11 16:17:09 +02:00
Dan McGee
0ac74ed733 pkgfuncs.inc.php: allow all DB funcs to take handle arg
Allows handle reuse if one is available.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-11 16:17:08 +02:00
Dan McGee
242f7afb96 aur.inc.php: allow all functions using DB to take handle as arg
This prevents needless calls to db_connect() if we already have a
reference to a handle we can pass into a function. Although the current
underlying implementation using mysql_connect() will return the same
connection if all parameters are the same, this might not be true if we
switch to a more modern (e.g. PDO) interface.

In the face of safe transactions, it is extremely important all actions
are being taken over the same connection to the database.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-11 16:17:08 +02:00
Lukas Fleischer
f481645e64 Add "mergepkgid" argument to pkg_delete()
This allows for merging comments and votes of deleted packages into
another one which is useful if a package needs to be renamed.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
2011-08-11 16:17:07 +02:00
Lukas Fleischer
0ac8970b62 Rename package_exists() to pkgid_from_name()
Describe what this function actually does: Return the ID of a package
with a given name and return NULL if such a package doesn't exist.

The function name is chosen in a fashion similar to other functions from
"pkgfuncs.inc.php" (pkgname_from_id(), pkgnotify_from_sid(), ...).

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-11 16:17:07 +02:00
Dan McGee
9a79d2105e Segment the upload directory by package name prefix
This implements the following scheme:

* /packages/cower/ --> /packages/co/cower/
* /packages/j/     --> /packages/j/j/
* /packages/zqy/   --> /packages/zq/zqy/

We take up to the first two characters of each package name as a
intermediate subdirectory, and then the full package name lives
underneath that. Shorter named packages live in a single letter
directory.

Why, you ask? Well because earlier today the AUR hit 32,000 entries in
the unsupported/ directory, making new package uploads impossible. While
some might argue we shouldn't have so many damn packages in the repos,
we should be able to handle this case.

Why two characters instead of one? Our two biggest two-char groups, 'pe'
and 'py', both start with 'p', and have nearly 2000 packages each. Go
Python and Perl.

Still needed is a "move the existing data" script, as well as a set of
rewrite rules for those wishing to preserve backward compatible URLs for
any helper programs doing the wrong thing and relying on them.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-10 14:34:07 +02:00
Dan McGee
1f79258e81 Allow SQL_DEBUG to be undefined
We shouldn't require this as it is a new config parameter and it causes
PHP warnings to be spewed everywhere.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-08-10 14:34:07 +02:00
Lukas Fleischer
0f994df357 Simplify session ID generation
There was too much voodoo going on in new_sid(). Just use uniqid() with
a random seed and the optional entropy parameter to generate MD5 input.

Use the remote IP address as a salt to reduce the chance of two clients
getting the same ID if they login at exactly the same time.

Thanks-to: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-25 11:58:20 +02:00
Lukas Fleischer
e686b495a8 Stricter email address validation
Use PHP's filter extension to validate mail addresses. The filter
extension is enabled by default as of PHP 5.2.0.

Thanks-to: Florian Pritz <bluewind@xinu.at>
Thanks-to: Cédric Girard <girard.cedric@gmail.com>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-25 11:58:13 +02:00
Dan McGee
3c110b13b5 Use sane ORDER BY clauses in package list queries
We were doing some silly things here with an "ORDER BY Name, CategoryID"
clause, due to the fact that Name is unique, and thus any additional
ordering after Name will have no effect. Of course, the dumb as a box of
rocks MySQL query optimizer doesn't realize this, leading to full table
scans every time of ~30000 packages instead of using index scans for the
ordering and only retrieving the first 50 rows.

The biggest change is noted in the default sort order as it cuts down
the columns to one, but we can remove the redundant sort from other
orderings as well, even though those will still not be able to use an
index-driven query plan.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-25 11:29:55 +02:00
Lukas Fleischer
0e9f8c0ff5 Do not redirect on single search results
The use of header() to redirect after previous output was a design flaw.
Our only luck here was PHP's "output_buffering" config variable
defaulting to 4096, which kind of hid the bug for a long time.

Fixes FS#24580.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-22 15:42:38 +02:00
elij
5853097561 Make cache type selectable based on config value
Provie a mechanism to specify cache type from NONE, APC, or MEMCACHE
based on a config variable.

If MEMCACHE type is selected, a list of servers can be specified to
provide multiserver support. Note that php-memcaced is required for
MEMCACHE support.

Lukas: Minor commenting style fixes.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-22 15:21:21 +02:00
elij
888db089c5 rename *.inc files to *.inc.php and adjust imports and references
Lukas: Add note to "UPGRADING".

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-22 15:15:04 +02:00
elij
af5d05f4ad refactor apc code and move to aur.inc
- move apc cache code to aur.inc (centralize)
- refactor the apc usage in stats.inc to utilize new code in aur.inc

Lukas: Small commenting style and spelling changes.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-06-22 15:02:21 +02:00
kachelaqa
0df6d7b4e7 add first submitted and last modified to rpc output
after reading the discussion about the submitter field, i looked again
at my previous patch and realized that it was not very good at all.

hopefully this simplified version will be more acceptable.

the justification for this patch is that many third-party tools will
want to know how new a package is, or when it last changed. adding the
submitted and modified timestamps will allow users to get all the
relevant package data in a single query.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-17 10:46:56 +02:00
elij
77f3aa56b7 add support for etag and conditional get (if-none-match)
Add etag and if-none-match conditional get support. This will allow
'smart client' to save network bandwidth, as they can save the etag hash
value for queries and test it later. Still an http request because this
patch specifically sets a cache lifetime of zero, and must-revalidate.
The benefit here is bandwidth savings. Caching based on expires headers
would likely be counter productive, as the api data can change rather
quickly...but etag is a nice compromise, and could be quite beneficial
for bandwidth recution in some scenarios.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-17 10:43:49 +02:00
elij
3e81712fd6 fix case where user does not exist
the query was being performed when $id was not set, resulting in an
invalid sql query being performed.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-17 10:43:46 +02:00
elij
0898f1447a test return value from db_query before assuming it is valid
make the sql query form consistent in usage by cleaning up instances
where db_query's result was not inspected before attempting to fetch row
data from the handle

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-05-17 10:43:42 +02:00
elij
c43558ba60 add SQL_DEBUG variable and database logging
add a hook to db_query to log all sql queries when SQL_DEBUG is set
Additionally, provide better logging for sql error situations (provide
backtrace as well as error message).

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-27 14:50:44 +02:00
Dan McGee
33a765ef03 stats: capitalize FROM in all SQL queries
Matches our normal code conventions.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-27 14:50:28 +02:00
Dan McGee
3609cf140e SQL: treat all UID/ID values as numbers, not strings
Ensure we are not quoting these values in any of our SQL queries.

Thanks-to: elij <elij.mx@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-27 14:50:27 +02:00
Dan McGee
fcda6671f3 SQL: use standard LIMIT/OFFSET syntax
Increases compatibility with standard SQL dialect.

Thanks-to: elij <elij.mx@gmail.com>
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-27 14:50:27 +02:00
Lukas Fleischer
973e4f8558 Use HTTPs for links in out of date notification mails.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-20 00:02:09 +02:00
tuxce
c6d84b3a8d rpc: unify methods return (fixes FS#17597)
Include maintainer in info and search method.

Lukas: Adjustments for "multiinfo" queries.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-16 17:49:01 +02:00
Dan McGee
0488e8597c rpc: introduce multiinfo query (fixes FS#17583)
The majority of "real world" info requests [1] come in hefty batches. We
would be better served to handle these in one request rather than
multiple by allowing AUR clients to send multiple arguments.

This enables things like this to work:
    http://aur.test/rpc.php?type=multiinfo&arg[]=cups-xerox&arg[]=cups-mc2430dl&arg[]=10673

Note to RPC users: unfortunately due to the asinine design of PHP, you
unfortunately have to use the 'arg[]' syntax if you want more than one
query argument, or you will only get the package satisfying the last arg
you pass.

[1] Rough data from April 11, 2011, with a total hit count of 1,109,163:
     12 /login.php
     13 /rpc.php?type=sarch
     15 /rpc.php?type=msearch
     16 /pingserver.php
     16 /rpc.php
     22 /logout.php
    163 /passreset.php
    335 /account.php
    530 /pkgsubmit.php
    916 /rss2.php
   3838 /index.php
   6752 /rss.php
   9699 /
  42478 /rpc.php?type=search
 184737 /packages.php
 681725 /rpc.php?type=info

That means a whopping 61.5% of our requests were for info over the RPC
interface; package pages are a distant second at only 16.7%.

Lukas: Introduce "multiinfo" query instead of extending "info" (for the
sake of backward compatibility).

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-16 17:49:00 +02:00
Dan McGee
a3ad060158 rpc.php: be a bit more consistent in query building
Do the implode as the same but separate step each time, and remove
indentation where no other query has it.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-16 17:49:00 +02:00
Dan McGee
aa206b343a rpc.php overhaul
* Mark things static in the class rather than use a constructor every
  single invocation of the service.
* Don't call mysql_real_escape_string() before we even have a database
  connection, and don't do work in the database if we don't need to.
* Formatting consistency fixups in a few places.
* Add new process_query() helper function; use this instead of
  copy-pasted code in all of the RPC method calls.
* Remove the escaping code meant to fix FS#15526, introduced in commit
  4d1eb4dd7a. It broke more than it solved, only fixed the output in
  one of three RPC calls (and who knows what the web interface then also
  does), and proper encoding should be done at the database level rather
  than up here.

Lukas: Add special case for "info" queries to process_query() (return a
single result instead of an array of results here).

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-16 17:47:57 +02:00
Dan McGee
8fdb382d67 Don't allow dl() of json module
You need this enabled for the AUR, period. No need for this BS.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-13 12:36:07 +02:00
Lukas Fleischer
56de32c016 Pass array_map() callback function properly.
PHP requires callback functions to be passed as strings. Fix this to
supress PHP notices.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-13 12:24:01 +02:00
Lukas Fleischer
eef5353bde Remove "New Package Notify" option from user account settings.
Do this in preparation for the upcoming notification script removal.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-13 12:22:28 +02:00
Lukas Fleischer
3a5693166c Remove comment on translation helpers from "web/lib/translator.inc".
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-10 16:27:48 +02:00
Lukas Fleischer
22a718ea89 Replace translation engine with php-gettext.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-10 15:40:50 +02:00
Lukas Fleischer
779ecc8c3a Add php-gettext libraries to "web/lib/".
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-10 15:40:49 +02:00
Lukas Fleischer
668826226a Release 1.8.2.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-10 15:38:28 +02:00
Lukas Fleischer
1c55e6b40c Add option to search for exact name matches only (fixes FS#23556).
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-06 19:21:19 +02:00
Lukas Fleischer
8cd2aef50d Initialize "$name" in pkgname_from_id().
Ensures "$name" is always initialized, even if the package doesn't
exist. Fixes PHP warnings appearing when retrieving package details of a
package with an invalid ID.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-05 11:38:43 +02:00
Lukas Fleischer
8b4b2d207e Remove File_Find PEAR module from code base.
We removed the code depending on this a long time ago - drop it and add
some note to "UPGRADING".

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-05 11:31:12 +02:00
Dan McGee
2005a2889c Add more stats to the front page table
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-03 22:59:33 +02:00
Dan McGee
7c91c59245 Remove Dummy Package concept
Instead, we just store dependencies directly in the PackageDepends
table. Since we don't use this info anywhere besides the package details
page, there is little value in precalculating what is in the AUR vs.
what is not.

An upgrade path is provided via several SQL statements in the UPGRADING
document. There should be no user-visible change from this, but the DB
schema gets a bit more sane and we no longer have loads of junk packages
in our tables that are never shown to the end user. This should also
help the MySQL query planner in several cases as we no longer have to be
careful to exclude dummy packages on every query.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-03 22:59:32 +02:00
Dan McGee
1f252eba64 Always set ModifiedTS including new packages
Set it equal to the SubmittedTS field, which will be our indication the
package is new when we show the logo on the front page of the AUR.

This results in the ability to remove the use of the unindexable
GREATEST() function from the AUR code everywhere we had to use it before
to handle the 0 timestamp case.

Note that there is no race condition here in calling UNIX_TIMESTAMP()
twice- it always returns the time at the beginning of statment
execution:

    mysql> select unix_timestamp(), sleep(2), unix_timestamp();
    +------------------+----------+------------------+
    | unix_timestamp() | sleep(2) | unix_timestamp() |
    +------------------+----------+------------------+
    |       1300851746 |        0 |       1300851746 |
    +------------------+----------+------------------+
    1 row in set (2.00 sec)

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-04-03 22:59:31 +02:00
Dan McGee
b6a34d013b Bump up default per-page value to 50
25 is woefully small for the number of packages many searches can
return, and with 28000+ packages in AUR, it makes sense to show a lot
more per page by default.

The new choices of (50, 100, 250) happen to match those from the main
site.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-24 15:32:08 +01:00
Lukas Fleischer
bd36412321 Fix searching for orphans with "$LANG != 'en'" (fixes FS#23252).
Regression introduced in commit ef8fab0c. Removed the strict check again
instead of wrapping it in __() to ensure search URLs are language
independent.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-12 19:54:35 +01:00
Lukas Fleischer
7f9e498e48 Fix broken XHTML.
Fix a lot of invalid XHTML in the templates and actions. There might
still be some legacy code left, but this should cover most of it.

Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-11 19:00:50 +01:00
Dan McGee
c34bebf428 Perform a second query to find total search count
This removes the need for SQL_CALC_FOUND_ROWS which can really slow down
queries in a lot of cases. The COUNT(*) query we end up performing can
reuse a lot of the original clauses from our primary query, but we can
really slim it up by omitting some joins and the sorting/limiting
clauses.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-10 18:16:00 +01:00
Dan McGee
2cfcddf24e Split package search query into parts
No functional change here; we should be rebuilding the same query at the
end of the process.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-10 18:16:00 +01:00
Dan McGee
80401c6afc More PHP Notice undefined fixups
Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-10 18:15:59 +01:00
Lukas Fleischer
29b2f3b399 Release 1.8.1.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-09 17:44:21 +01:00
Dan McGee
d186bcfd89 Add a sanitize_ids function and use it in all pkg_* functions
And use implode() instead of some looping/first time logic.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-04 10:29:18 +01:00
Dan McGee
d4b1ca7cf1 Set a default timezone
With our use of strtotime() in stats.inc, we are "required" to do so or at
least the emitted warning tells us it is a good idea.

Signed-off-by: Dan McGee <dan@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
2011-03-04 10:29:17 +01:00