New approach in failover libzypp development

Duncan (my “another mentor” in my Google Summer of Code project), proposed to me to try another approach to solve the failover concept within libzypp library. This time, I will try to create “another” MediaHandler class than will use an external program (aria2c) to download files from download.opensuse.org server. Thus, it will not be required to download metalink file, parse it and download selected package: aria2c will do all of this for us 🙂

Extracted from aria2c webpage:

aria2 is a utility for downloading files. The supported protocols are HTTP(S), FTP, BitTorrent (DHT, PEX, MSE/PE), and Metalink. It has a powerful segmented downloading ability, downloading a file from multiple sources and multiple protocols and utilizing your download bandwidth to the maximum. It even supports downloading a file from HTTP(S)/FTP and BitTorrent at the same time, while the data downloaded from HTTP(S)/FTP is uploaded to the BitTorrent swarm.

Using Metalink’s chunk checksums, aria2 automatically validates chunks of data while downloading a file like BitTorrent.

Installing gbrainy with zypper (compiled with libzypp + metalink support)

Right now, with libzypp hacked version than I have in my laptop, I can do this:

./zypper install gbrainy
Reading installed packages…

The following NEW package is going to be installed:
gbrainy

Overall download size: 127,0 K. After the operation, additional 472,0 K will be used.
Continue? [YES/no]: YES
Downloading package gbrainy-0.61-31.1.noarch (1/1), 127,0 K (472,0 K unpacked)

——————–
getFile ( /media.1/media )
doGetMetalinkFileCopy –> http://download.opensuse.org/distribution/11.0/repo/oss/media.1/media.metalink
\ Downloading: media.metalink [starting]Metalink file exists – We can parse it 😉
We have 69 mirrors for this file
Downloading from http://mirror.switch.ch/ftp/mirror/opensuse/distribution/11.0/repo/oss/media.1/media
HTTP response: 200
done: /var/adm/mount/AP_0x00000003/media.1/media{- 0644 0/0 size 42}

——————–
getFile ( ./suse/noarch/gbrainy-0.61-31.1.noarch.rpm )
doGetMetalinkFileCopy –> http://download.opensuse.org/distribution/11.0/repo/oss/suse/noarch/gbrainy-0.61-31.1.noarch.rpm.metalink
| Downloading: gbrainy-0.61-31.1.noarch.rpm.metalink [starting]Metalink file exists – We can parse it 😉
We have 69 mirrors for this file
Downloading from http://mirror.switch.ch/ftp/mirror/opensuse/distribution/11.0/repo/oss/suse/noarch/gbrainy-0.61-31.1.noarch.rpm
/ Downloading: gbrainy-0.61-31.1.noarch.rpm [starting]HTTP response: 200
done: /var/adm/mount/AP_0x00000003/suse/noarch/gbrainy-0.61-31.1.noarch.rpm{- 0644 0/0 size 129095}
Downloading:  [done]
Installing: gbrainy-0.61-31.1 [done]

All seems well and correct but… right now begins a new problem 🙁 If the server responds with HTTP code 304, I get this error message:

Can’t provide ./suse/noarch/gbrainy-0.61-31.1.noarch.rpm : Can’t copy /var/adm/mount/AP_0x00000003/suse/noarch/gbrainy-0.61-31.1.noarch.rpm to /var/cache/zypp/packages/repo-oss
Repository ‘repo-oss’ is out of date. Running ‘zypper refresh’ might help.

Really, this error is caused because this file (/var/adm/mount/AP_0x00000003/suse/noarch/gbrainy-0.61-31.1.noarch.rpm{does not exist}) NOT exists 🙁

Post updated 02/07: In effect, we receive 304 response code because we send “If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT” in the header.

GET /distribution/11.0/repo/oss/media.1/media HTTP/1.1
User-Agent: ZYpp 5.0.0 (MediaMetalink – with curl 7.18.1 & libmetalink )
Host: opensuse.mirror.solnet.ch
Accept: */*
If-Modified-Since: Thu, 01 Jan 1970 00:00:00 GMT

HTTP/1.1 304 Not Modified
Content-Type: text/plain
Accept-Ranges: bytes
ETag: “3592996126”
Last-Modified: Wed, 02 Jul 2008 02:42:38 GMT
Date: Wed, 02 Jul 2008 13:31:06 GMT
Server: httpd/1.4.x

I’m thinking in this actions:

  • Disable “If-Modified-Since” in our petition via libcurl, or,
  • Try the next mirror when libzypp receives 304 HTTP code response.

Updated: We can disable “If-Modified-Since” header in our petition, simply commenting these lines:

curl_easy_setopt(_curl, CURLOPT_TIMECONDITION, CURL_TIMECOND_IFMODSINCE);
curl_easy_setopt(_curl, CURLOPT_TIMEVALUE, PathInfo(target).mtime());

Good solution 🙂 ?