There's no need to use permanent storage for rate limiting information;
try to keep it in memory if caching is enabled.
From experiments with our live setup, this reduces the number of
INSERT/DELETE operations per second from 15 to almost 0. Disk writes on
the server hosting the AUR are reduced by 90% (from ~3MB/s to ~300kB/s).
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
The TTL for package details can be much longer than for generic values
since they never change. Note that when an update is pushed via Git, all
packages belonging to that package base are deleted and new packages are
created.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Cache the results of the extended fields computation if the global
caching mechanism is enabled.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
It is now possible to search for packages that depend on a given package,
for instance:
/rpc/?v=5&type=search&by=depends&arg=ocaml
It is similarly possible to match on "makedepends", "checkdepends" and
"optdepends".
Signed-off-by: Baptiste Jonglez <git@bitsofnetworks.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
This allows us to prevent users from hammering the API every few seconds
to check if any of their packages were updated. Real world users check
as often as every 5 or 10 seconds.
Signed-off-by: Florian Pritz <bluewind@xinu.at>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Make sure that the get_extended_fields() invocation succeeded before
merging regular and extended fields in process_query().
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
All error messages in aurjson except two end with a period. Add the
missing periods to make the messages consistent.
Signed-off-by: Michael Straube <straubem@gmx.de>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
The IDs of packages are unique, so there is no need to group search
results by package ID.
Note that the GROUP BY statement in question was introduced in commit
3447dfc (Support versioned RPC queries, 2014-04-28) for no apparent
reason and could even lead to errors in various DBMS.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
When performing info or multiinfo queries, one can currently either pass
package names or package IDs as parameters. As a consequence, it is
impossible to search for packages with a numeric package name because
numeric arguments are always treated as IDs. Since package IDs are not
public anymore these days, simply remove the possibility to search by ID
in revision 5 of the RPC interface.
Fixes FS#47324.
Suggested-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Fixes a regression introduced in 94aeead (aurjson: Pass http_data array
to all functions, 2015-06-28).
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Instead of introducing a new message "You do not have the right to edit
this comment." for the RPC interface, use "You are not allowed to edit
this comment." which we already show in the front-end.
Reported-by: Christoph Seitz <seitz.christoph@gmail.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
The callback parameter of the RPC interface currently allows for
specifying a prefix of arbitrary length of the returned result. This can
be exploited by certain attacks.
As a countermeasure, this patch restricts the allowed character set for
the callback name to letters, digits, underscores, parenthesis and dots.
It also limits the length of the name to 128 characters. Furthermore,
the reflected callback name is now always prepended with "/**/", which
is a common workaround to protect against attacks such as Rosetta Flash.
Fixes FS#46259.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
This method will be used by the JavaScript comment editing and produces
a form containing the comment.
Signed-off-by: Marcel Korpel <marcel.korpel@gmail.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
This is a preparatory patch that simplifies adding more arguments to
the parse functions
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Marcel Korpel <marcel.korpel@gmail.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
Remove package base categories. Instead, users can now specify up to
twenty custom keywords that are taken into consideration when searching.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
In commit 74edb6f (Use Git repositories to store packages, 2014-06-06),
the URLPath field was dropped from RPC package results. Reintroduce the
field for backwards compatibility, even though it is no longer
recommended to use snapshot tarballs.
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
* Remove package submission page from the web interface.
* Replace PKGBUILD and tarball links with links to cgit.
* Remove the "URLPath" field from RPC replies.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Replace web/lib/config.inc.php with an INI-style configuration file.
This allows us to get rid of several globals and makes it easier to use
the same configuration file in external scripts.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Add the typeahead implementation we already use for the package search
to the merge target text field. Instead of suggesting packages, suggest
package base names.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Add RPC v3 which always returns a list of objects in RPC responses,
regardless of the request type. An empty list is returned when an error
occurs. The error message is moved to a separate "error" field.
Fixes FS#40963.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
This is useful to tools that automatically generate requests to orphan,
delete or merge packages.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Most AUR helpers don't support the new format yet. Use version 1 of the
API by default. In order to use the new format, "v=2" can be appended to
the list of GET parameters.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
This adds a backwards compatibility mode for the old RPC API. The old
format can be requested by explicitly adding "v=1" to the list of GET
parameters. Note that due to internal changes, the old format only
returns the first license that belongs to a package.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
This is needed for clients to quickly differentiate between different
versions of the RPC API. The version number should be bumped whenever
there is a change that breaks backwards compatibility.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
This patch adds the following fields to info and multiinfo replies:
* Depends
* MakeDepends
* CheckDepends
* OptDepends
* Conflicts
* Provides
* Replaces
* Groups
* License
Each of these fields is an array.
Note that since collecting all these fields is CPU-intensive, they are
not included in replies to search queries.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
* Fix the SQL query to conform to the new database layout.
* Remove the license field from replies. The license field is now stored
in a separate table and no longer returned on search queries.
* Add a "PackageBase" field that contains the name of the package base
of every package in the result.
* Fix the source tarball URL. The URL is now built based on the package
base name instead of the package name.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Uses the Singleton pattern to ensure all queries use the same
database connection that is released upon script completion.
All database connections should now be called with DB::connect() and
not db_connect().
Signed-off-by: canyonknight <canyonknight@gmail.com>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Use Twitter Bootstrap JavaScript framework for typeahead support.
Add a new "suggest" JSON method, which returns the first 20
packages that match the beginning characters of a query.
canyonknight: Link format change, commit message
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
With no limit to the number of results, memory_limit set to 32M
can easily be exceeded for searches that have a large number of
results. This results in an HTTP error 500 for those queries.
Limit results to an amount set within config.inc.php to avoid
exceeding memory_limit. Introduce new JSON error code for when
the result limit is hit.
Fixes FS#31849
Signed-off-by: canyonknight <canyonknight@gmail.com>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
All DB code currently uses the quickly aging mysql_* functions. These
functions are strongly discouraged and may eventually be deprecated.
Transition all code to utilize the PDO data access abstraction layer. PDO
allows for consistent query code across multiple databases. This could
potentially allow for someone to use a database other than MySQL with
minimal code changes.
All functions and behaviors are reproduced as faithfully as possible with
PDO equivalents and some changes in code.
Signed-off-by: canyonknight <canyonknight@gmail.com>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
We already ask for the result count, but only use it as a basis for testing
query success or failure. Add the value to the JSON reply.
Signed-off-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Percent signs ("%") and underscores ("_") are not escaped by
mysql_real_escape_string() and are interpreted as wildcards if combined
with "LIKE". Write a wrapper function db_escape_like() and use it where
appropriate.
Note that we already fixed this for the RPC interface in commit
da2ebb667b but missed the other places.
This patch should fix all remaining flaws reported in FS#26527.
Signed-off-by: Lukas Fleischer <archlinux@cryptocrack.de>
Signed-off-by: Dan McGee <dan@archlinux.org>
