mirror of
https://gitlab.archlinux.org/archlinux/aurweb.git
synced 2025-02-03 10:43:03 +01:00
99 lines
4 KiB
Text
99 lines
4 KiB
Text
TU Packaging Tools (tupkg)
|
|
--------------------------
|
|
- client side (python for proof of concept, later re-write to C?)
|
|
The main purpose of this tool is to upload the compiled
|
|
pkg.tar.gz to the server. It can (should?) do some verification
|
|
on the package prior to uploading to the server. It will have
|
|
a config file to store run-time information such as username
|
|
(email), password, and server name.
|
|
|
|
- server side (python for proof of concept, later re-write to C?)
|
|
The server side will handle incoming connections from its client
|
|
side counterpart. The server should bind to port 80 (maybe a
|
|
vhost such as tupkg.archlinux.org?) so that firewalls won't be
|
|
an issue. The server verifies the client authentication data,
|
|
and then accepts the package(s). If port 80 is not available,
|
|
perhaps 443, or are there other 'standard' ports that usually
|
|
do not get filtered?
|
|
|
|
I think the server should be multithreaded to handle simultaneous
|
|
uploads rather than queue up requests. The download should be
|
|
stored in a temp directory based on the username to prevent
|
|
directory, filename clashes.
|
|
|
|
Once the package(s) is uploaded, the server can either kick off
|
|
a gensync, or we can write a separate script to call gensync once
|
|
or twice a day. My preference would be a separate script to call
|
|
gensync (like the *NIX philosophy of one tool per task).
|
|
|
|
- protocol (c: => client, s: => server)
|
|
Whenever the client/server exchange a message, it is always
|
|
preceeded by two-bytes representing the following message's
|
|
length. For example, when the client connects, it will send:
|
|
|
|
0x0028username=bfinch@example.net&password=B0b
|
|
|
|
0x0028 is the 40 byte strlen of the message in two-bytes. The
|
|
client and server always read two-bytes from the socket, and
|
|
then know how much data is coming and can read that amount of
|
|
bytes from the socket.
|
|
|
|
==> authentication
|
|
c: username=emailaddy&password=mypassword
|
|
s: result=PASS|FAIL
|
|
|
|
NOTE: We can add encryption easily enough with the python
|
|
version using the socket.ssl method.
|
|
|
|
==> uploading package data
|
|
if PASS:
|
|
|
|
c: numpkgs=2&name1=p1.pkg.tar.gz&size1=123&md5sum1=abcd\
|
|
name2=p2.pkg.tar.gz&size2=3&md5sum2=def1
|
|
s: numpkgs=2&name1=p1.pkg.tar.gz&size1=119&\
|
|
name2=p2.pkg.tar.gz&size2=0 (*)
|
|
|
|
(*) NOTE: The server will reply back to the client how many
|
|
packages it has already received and its local file size.
|
|
This way, the client can resume an upload. In the example
|
|
above, the server still needs the last four (123-119) bytes
|
|
for the first package, and that it has no part of the
|
|
second package. The client would then begin sending the
|
|
last four bytes from the first package (p1.pkg.tar.gz) and
|
|
then follow it with the full second package (p2.pkg.tar.gz).
|
|
The data would be sent as a continuous chunk of data. The
|
|
server will then need to track which bytes belong to which
|
|
package.
|
|
|
|
else FAIL:
|
|
c: -spits out error message on stderr to user-
|
|
|
|
|
|
==> after upload completes
|
|
The server should verify the integrity of the uploaded packages
|
|
by doing an md5sum on each and sending the info back to the client
|
|
for comparison. After sending the message, the server waits for
|
|
the 'ack' message from the client and then closes the connection.
|
|
|
|
s: np=2&m1=PASS&m2=FAIL
|
|
c: ack
|
|
|
|
The client replies with the 'ack' and then closes its connection
|
|
to the server. It then reports the PASS/FAIL status of each
|
|
package's upload attempt.
|
|
|
|
NOTE: If the upload fails (client connection dies), the server
|
|
keeps any data it has received in order to support resuming an
|
|
upload. However, if the client uploads all data, and the server
|
|
successully reads all data and the final MD5 fails, the server
|
|
deletes the failed package.
|
|
|
|
|
|
Terms/definitions:
|
|
======================
|
|
TU - No change (trusted by the community, if anyone asks what trust
|
|
means)
|
|
TUR - renamed to Arch User-community Repo (AUR) (so we can use -u for
|
|
versions)
|
|
Incoming - renamed to "Unsupported"
|
|
|