Commit graph

79 commits

Author SHA1 Message Date
Lukas Fleischer
5f43e2aaa9 t0002: Add tests for adopt/disown/set-comaintainers
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-09-18 14:01:08 +02:00
Lukas Fleischer
94ac084d9d git-serve: Add support for setting co-maintainers
Add support for changing co-maintainers from the SSH interface. The
syntax is `set-comaintainers <pkgbase> <user1> <user2>...`.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-09-18 14:01:08 +02:00
Lukas Fleischer
9b983ac03e git-serve: Add support for disowning package bases
Add support for disowning packages from the SSH interface. The syntax is
`disown <pkgbase>`.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-09-18 14:01:08 +02:00
Lukas Fleischer
ce12e3b82d git-serve: Add support for adopting package bases
Add support for adopting packages from the SSH interface. The syntax is
`adopt <pkgbase>`.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-09-17 20:17:39 +02:00
Lukas Fleischer
e045069421 git-serve: Format usage text automatically
Remove the formatting of the usage text and add code to columnize it
automatically instead.

Also, add more strict tests for the usage output. These new tests ensure
that the usage header is printed, commands are indented properly and no
overly long lines are produced.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-09-17 20:17:11 +02:00
Lukas Fleischer
ab228fd3c3 git-serve: Mark setup-repo as deprecated
Since 0c1187c (git-serve: Deprecate setup-repo, 2016-07-24), it is no
longer recommended to use setup-repo. Mark the command as deprecated in
the usage/help text.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-09-12 08:45:38 +02:00
Lukas Fleischer
e3bcf83feb git-update: Do not overwrite the repo variable
The repo variable is already used to store the pygit2.Repository.

Fixes a regression introduced in d273ee5 (Use the official provider list
to detect duplicates, 2016-05-17).

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-22 08:25:51 +02:00
Lukas Fleischer
29625e0744 git-update: Close cursor before closing database
When using SQLite as backend, we need to close the cursor before closing
the database to avoid the following error:

    sqlite3.OperationalError: unable to close due to unfinalized statements or unfinished backups

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-22 08:16:26 +02:00
Lukas Fleischer
c761631181 t0003: Make tests more strict
Instead of testing the exit code only, also check the error output.

This reveals two bugs. The corresponding tests are marked as known
breakages.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-22 08:10:35 +02:00
Lukas Fleischer
2c36c17a18 db.py: Support pyformat paramstyle
This is used by the MySQL database backend.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-14 21:56:45 +02:00
Lukas Fleischer
435b5fc902 t0003: Do not initialize package bases explicitly
Package bases are created by git-update automatically when the
repository receives a ref update for the first time.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-10 20:14:22 +02:00
Lukas Fleischer
936ee66f1e Lazy-add new package bases
Create new package bases just before saving package metadata. This
protects from stray package bases left behind when new packages are
rejected, e.g. when the user tries to push a package that is available
from the official repositories already.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-10 20:14:17 +02:00
Lukas Fleischer
83df9808b4 Add tests for the restore command
Test the restore mode of git-serve and git-update.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-06 01:50:17 +02:00
Lukas Fleischer
dd9c6f3ddc t0003: Add more git-update tests
Add tests for common scenarios that should be detected/handled by the
update hook.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 21:23:20 +02:00
Lukas Fleischer
9a03c7fbdd t0002: Add more git-serve tests
Add tests for common scenarios that should be detected/handled by the
git-serve script.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 21:23:20 +02:00
Lukas Fleischer
008eace8db t0001: Add more git-auth tests
Test the authentication script with an invalid key type and with a key
that does not exist in the database.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 21:02:21 +02:00
Lukas Fleischer
3a41f8d564 git-update: Remove package details before updating
Explicitly remove all package sources, dependencies, relations, licenses
and groups before inserting new ones.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 19:38:42 +02:00
Lukas Fleischer
6e38309c19 git-interface: Add test suite and basic tests
Add basic tests for the Git interface. The test suite is based on
sharness.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 12:12:09 +02:00
Lukas Fleischer
baf8a220ab git-interface: Support SQLite as database backend
In addition to MySQL, add support for SQLite to the database abstraction
layer. Also, add a new configuration option to select the DBMS.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 12:05:22 +02:00
Lukas Fleischer
f2a6bd207d git-interface: Do not use UNIX_TIMESTAMP
Avoid using UNIX_TIMESTAMP which is not part of the SQL standard.
Retrieve the current UNIX time in Python and substitute it into the SQL
queries instead.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 12:05:22 +02:00
Lukas Fleischer
27631f1157 git-interface: Do not use rowcount
Avoid using Cursor.rowcount to obtain the number of rows returned by a
SELECT statement as this is not guaranteed to be supported by every
database engine.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 12:05:22 +02:00
Lukas Fleischer
ecbf32f0cc git-interface: Add AUR_CONFIG environment variable
Introduce a new environment variable that can be used to specify the
path to an aurweb configuration file. If the environment variable is
unset, the default search path is used.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 12:05:22 +02:00
Lukas Fleischer
2f5f5583be git-interface: Factor out configuration file parsing
Add a new module that automatically locates the configuration file and
provides methods to obtain the values of configuration options.

Use the new module instead of ConfigParser everywhere.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 12:05:22 +02:00
Lukas Fleischer
2915abb9d3 git-interface: Add database abstraction layer
Add a new class that connects to the database specified in the
configuration file and provides an interface to execute SQL queries.
Prepared statements with qmark ("?") placeholders are supported.

Replace all direct database accesses with calls to the new abstraction
layer.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 12:05:22 +02:00
Lukas Fleischer
2cd69bf66d git-update: Make maximum blob size configurable
Support setting the maximum blob size in the configuration file.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 11:25:51 +02:00
Lukas Fleischer
87f5f1b407 git-update: Use AUR_PRIVILEGED for forced pushes
Instead of looking up the account type of the current user again, use
the AUR_PRIVILEGED environment variable to check whether the user is
allowed to perform non-fast-forward ref updates.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 01:22:52 +02:00
Lukas Fleischer
7a53ded5fe git-update: Fix some issues reported by pyflakes
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-05 01:22:51 +02:00
Lukas Fleischer
0c1187caa4 git-serve: Deprecate setup-repo
Since 02dd9c5 (git-serve.py: Automatically create repositories,
2015-01-06), one can create new package bases by running `git push`. It
is no longer necessary to run setup-repo manually.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-04 21:37:52 +02:00
Lukas Fleischer
415a2c836d git-update: Notify privileged users of forced uploads
Show a warning when a Trusted User or a developer creates a package that
is blacklisted or already provided by an official package.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-04 13:04:44 +02:00
Lukas Fleischer
573715afd9 git-serve: Refactor environment variable access
Read all environment variables at the beginning of the script and
immediately pre-process their values.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-08-04 13:04:44 +02:00
Lukas Fleischer
d273ee5eb2 Use the official provider list to detect duplicates
Instead of automatically adding packages from the official binary
repositories to the package blacklist, use the official provider list to
prevent users from uploading duplicates.

This does not only result in reduced disk usage but also has a nice
visible side effect. The error messages printed by the update hook now
look like

    error: package already provided by [community]: powerline-fonts

instead of

    error: package is blacklisted: powerline-fonts

which was confusing to most end users.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-05-18 09:22:44 +02:00
Lukas Fleischer
dc41a5afa5 git-serve: Change syntax for setting keywords
Instead of `config <pkgbase> keywords <keyword1> <keyword2>...`, the new
syntax is `set-keywords <pkgbase> <keyword1> <keyword2>...`.

The `config` keyword was rather general and it was not obvious that it
could be used to change package base settings. Instead of replacing it
with an even more verbose expression, remove that unnecessary level of
indirection. Since we do not (and probably never will) support keywords
anywhere else, the chance of name conflicts is small.

Suggested-by: Dave Reisner <dreisner@archlinux.org>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-09 21:03:47 +01:00
Lukas Fleischer
64072461df Add support for package update notifications
Introduce a new notification option to receive notifications when a new
commit is pushed to a package repository.

Implements FS#30109.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-07 10:54:20 +01:00
Lukas Fleischer
3412de21d3 Rename the CommentNotify table to PackageNotifications
As a preparatory step to adding support for package notifications on
events other than comments, rename the database table accordingly.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-07 10:39:22 +01:00
Johannes Löthberg
7a3a3876a8 git-update: Replace aurinfo.py with python-srcinfo
python-srcinfo is a more transparent and simpler library for parsing
SRCINFO files.

Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-07 09:48:32 +01:00
Lukas Fleischer
f8b3cb97e5 Fix issues reported by pyflakes
Fix several style issues and remove unneeded imports/assignments.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2016-02-06 16:23:01 +01:00
Mark Weiman
39280152ea git-serve: Fix pkgbase_exists()
Fixes a regression introduced in 8c87b1d (git-serve: Add support for
setting keywords, 2015-10-22).

Signed-off-by: Mark Weiman <mark.weiman@markzz.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-11-26 17:22:22 +01:00
Lukas Fleischer
8c87b1df0c git-serve: Add support for setting keywords
This allows for setting keywords using the SSH interface. The syntax is
`config <pkgbase> keywords <keyword1> <keyword2>...`.

Implements FS#45627.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-10-24 16:22:43 +02:00
Lukas Fleischer
62c13e2c0c Do not show ".SRCINFO unchanged" warning on restore
Since 612300b (Show a warning if .SRCINFO is unchanged, 2015-09-29), the
git-update script displays a warning when a ref update does not affect
the content of the package base meta data. We also invoke git-update to
rebuild the package base details in the aurweb database when a package
base is restored via the SSH interface. In that case, fake information
is passed to the update hook: Both the old and the new object IDs refer
to the current HEAD. Check for such "Everything up-to-date" updates and
not display the ".SRCINFO unchanged" in these cases.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-10-03 12:10:25 +02:00
Lukas Fleischer
612300b39c Show a warning if .SRCINFO is unchanged
Warn users when a remote ref update does not change the content of
.SRCINFO such that users are reminded of updating package meta data.

Implements FS#46130.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-09-29 20:54:55 +02:00
Lukas Fleischer
2c20403cc4 git-update: Use proper stop value for slice
Fixes a regression introduced in 4112e57 (Add a restore command to the
SSH interface, 2015-08-14).

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-09-16 22:10:06 +02:00
Lukas Fleischer
4112e572aa Add a restore command to the SSH interface
Implement a new command that can be used to restore deleted package
bases without having to push a new commit.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-16 06:49:12 +02:00
Lukas Fleischer
da875276d4 git-update: Add comments
Add some comments to explain the major steps performed in the update
hook.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-14 12:52:32 +02:00
Lukas Fleischer
80e06e5fc3 git-update: Remove superfluous assignment
The pkgbase variable already contains the package base name at this
point, no need to reassign it.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-14 12:50:35 +02:00
Lukas Fleischer
4bc6c55d98 git-update: Move blacklist reading further down
Since c4870a9 (git-update: Only check HEAD for blacklisted packages,
2015-06-04), only the HEAD commit package name is looked up in the
blacklist. This means that we no longer need to read the blacklist
before running the commit walker. Moving the blacklist reading code
further down makes the code easier to read.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-14 12:45:36 +02:00
Johannes Löthberg
950abb0189 aurinfo: Fix parsing with custom file when running script
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-04 05:57:02 +02:00
Johannes Löthberg
f19892f7c2 aurinfo: Allow lines starting with spaces
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-08-04 05:57:02 +02:00
Lukas Fleischer
38b1bbe78d git-update: Error out if PKGBUILD is missing
Fixes FS#45646.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-07-16 14:09:55 +02:00
Johannes Löthberg
f4f1921fc6 git-update: Fix error when printing SRCINFO errors
This commit fixes a bug introduced by ae2907a (git: Use .format
everywhere instead of %, 2015-06-27) where passing the error tuple to
.format wasn't prefixed with an asterisk.

Fixes FS#45545.

Reported-by: Marty <vadmium+al@gmail.com>
Signed-off-by: Johannes Löthberg <johannes@kyriasis.com>
Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-07-04 15:02:04 +02:00
Lukas Fleischer
d0be66015a git-update: Improve error message on invalid pkgbase
When the package base does not match the repository name, we currently
show an error message that only mentions the package base name:

    error: invalid pkgbase: yate-qt4

To the end user this sounds like the package base name is malformed
itself (and we show exactly the same error message when it contains
invalid characters, indeed). Clarify that there is a name mismatch by
also printing the expected value:

    error: invalid pkgbase: yate-qt4, expected: yate4-qt

Inspired by FS#45483.

Signed-off-by: Lukas Fleischer <lfleischer@archlinux.org>
2015-06-27 13:04:03 +02:00