[sbopkg-users] Using http, Curl as a repository tool

[Asaf]

On 08/04/2011 02:19 AM, slakmagik wrote:
> On 2011-08-03 (Wed) 23:18:21 [+0300], Asaf wrote:
>> Continued from:
>> http://sbopkg.org/pipermail/sbopkg-users/2010-July/000492.html
>>
>> I've forked sbopkg to implement Curl repository tool:
>> https://github.com/asafo/sbopkg
>>
>> there is also a sample repos.d/80-SBo-curl.repo
>>
>> I'll be happy for any suggestions to help it get merged
>>
> First, am I to understand that every use of 'sbopkg -r' using this would
> result in downloading the entire SBo repository? Because I don't think
> that's acceptable, both from a user's POV and from the SBo admin's POV.
Yes, the work is not complete yet, there are starting points for parsing
the ChangeLog.txt file (Actually, the diff with the old ChangleLog.txt, 
if there is one)
so that only new packages would get downloaded

> For less major points: your 80 repo file doesn't work because it
> attempts to sync to localhost. Even after correcting that to a full url,
> the protocol is duplicated in _curl_sync_package(). Even after
> correcting that, gunzip asks about whether it should overwrite
> SLACKBUILDS.TXT if that file already exists.
This is going to change, when ChangeLog.txt parsing implementation is 
complete,
also, there is currently no public repository available (I hope that 
they will open one at
slackbuilds.org), so I'm using localhost for testing
>
> For relatively minor points, we don't use the _func convention (not
> saying whether we should or not, but we don't) and your patches have
> trailing whitespace. Also we only use the '['-style test on pathnames
> and maybe not always then. We prefer '[[' for strings and '((' for
> numerics. We don't leave commented-out code in (if we do, it's a
> mistake). Error messages should announce that they're coming from
> $SCRIPT (and I don't mind even $FUNCNAME at times). While there's
> nothing in writing about and sbopkg is very bashistic, I'd personally
> still prefer to avoid stuff like BASH_REMATCH, though the use is really
> spiffy in this case and might well be the best thing to do. I'm not fond
> of _curl_fail() when it's only used once and its functionality could
> just be used directly in the _curl_sync_package() line where it's
> called. And I'm personally a<80 columns fanatic. Lines should fit
> within standard terminal widths wherever possible.
the use of BASH_REMATCH saves a lot of process forking (grep, cut, sed, 
etc..)
(or is there a third way ?)
>
> Lastly, a major point, I'd raise first except that I wouldn't document
> something until I knew it was going in, myself, but anything that would
> go in would have to be thoroughly documented (man pages, always, and any
> relevant text files).
>
> Well, lastly lastly, I've only briefly looked at the patches (or patch,
> as it could be profitably 'squashed' into one, IMO) and only briefly
> tested but, while I'm testing, I'm noticing a lot of hangs and 'Curl
> Failed: 6's so this doesn't seem very efficient and effective even
> without the non-differencing problem.
>
> But, other than these points, it's pretty neat. I definitely appreciate
> your interest and this contribution but, if I understand it correctly, I
> don't think I'm in favor of it, at least at this point. We really do
> intend to get an lftp sync method in soon and, while that's not http, it
> might help with some people who have rsync issues while still only
> updating the differences. Or possibly something could be worked out via
> parsing the ChangeLog and only http'ing those packages.
lftp wouldn't solve the problem  (restrictive firewalls)
>
> But maybe if Mauro's around, he'll have a different take.
>
> And, yeah, the http sync is still ongoing after having been started
> before I began this email (and I take forever with emails). ;)
> _______________________________________________
> sbopkg-users mailing list
> sbopkg-users at sbopkg.org
> http://sbopkg.org/mailman/listinfo/sbopkg-users
>


-- 
Asaf Ohaion
Lingnu Open Source Consulting Ltd
http://www.lingnu.com