Ubuntu QA:
BlogBrainstormPackage status
Log in
Ubuntu QA
The Ubuntu community has contributed 22700 ideas, 138270 comments, 2629576 votes
Idea sandbox Idea sandbox
Popular ideas Popular ideas
Ideas in development Ideas in development
Implemented ideas Implemented ideas
Idea #29480: Reduce the size of apt-get update (not upgrade) downloads

Written by hit801 the 3 Apr 12 at 18:26. Category: Installation. Related project: Nothing/Others. Status: New
Rationale
While updating system using software update or apt-get update or by similar method, it takes quite along time and total data downloaded reches close to 16MB. On slower or I should say mobile broadband this is a waste of money or bandWidth.
Many time it happend that I had checked for software update and after downloading around 16 MB of Data or I should call "Software Catalauge/information" the update comes up is of 2MB or less.

7
votes
up equal down
Solution #1: System footprint file
Written by hit801 the 3 Apr 12 at 18:26.
Instead of checking and downloading data of more than 10 MB and most importantly wasting time, Software update should use a system footprint file! Or updater should prepare a file of few Kb which has signature/information of all the system installation and software versions currently installed. Whenever user checks for update system should compare this local system file with updated file which lies on update server. Any changes should be reflected in output on local computer as updates ready for download/installation. This will certainly saves time and bandwidth.
43
votes
up equal down
Solution #2: Use checksums and deltas and last changed meta information
Written by puxkggn the 6 Apr 12 at 12:46.
After checking a general date to see if a piece of software has a newer update available. The following things can be done to do this very effective and efficient:

Instead of complete files you could use checksums to check if files are the same.
Very effective and it's guaranteed to give good results when implemented good. We also don't need to compare the dates because the repository can check if the checksum coincidences with an older, newer or unknown version of the file. In the case of unknown the package manager should avoid messing with the file and reporting the situation to the User before doing anything.

Instead of replacing whole files could make something that only replaces the parts to update a file from a previous version to a next version. This could be implemented at the Ubuntu package manager and the Ubuntu repository software so that software developers don't have to make extra packages. Making deltas can even be fully automated. And very efficient: only make delta when a user is detected with a previous version who wants to update to a newer version. Generate delta between two version from packages in warehouse. Then cache the delta update files to speed up serving of them.

Very efficient without loosing effectiveness.
10
votes
up equal down
Solution #3: Use rsync
Written by aadityabhatia the 20 Apr 12 at 11:01.
We all love rsync because it transfers the deltas only. Why reinvent the wheel?
4
votes
up equal down
Solution #4: Third-Party Update Pushing
Written by ki4jgt the 28 Apr 12 at 00:59.
A service could be established for broadband users who are limited in bandwidth (it could be implemented into the software center). The service would manage the user's repositories (updating locally) and push updates to the client. The user would be allowed to add additional repositories. This would allow the user to only receive what updates were needed for their particular system.

EDIT: If the user wanted to update the available applications, they could do that as well.
2
votes
up equal down
Solution #5: delta + rsync + Ubuntu one packages list backup
Written by mitcoes the 17 Aug 12 at 10:21.

delta upgrades, using rsync and ...

With Ubuntu one keep the sources.list and a list of packages installed as system backup.

Almost every browser allows you to keep online your config allowing you to install at any computer your bookmarks, passwords,and web apps.

If ubuntu one can keep the list of packages installed, with a database format default + added packages and - uninstalled packages, each new distro upgrade reinstallation or installation at a new computer would be faster and it would be a great Ubuntu One service.

Propose your solution

Attachments
No attachments.


Duplicates


Comments
Darwin Survivor (Brainstorm moderator) wrote on the 4 Apr 12 at 03:55
Have you looked at what is currently being downloaded and found something that can be removed? There is most likely stuff in there that you are not thinking of. Off the top of my mind I can think of signatures, hashes and conflict resolutions that may be present.

I just downloaded http://packages.ubuntu.com/precise/allpackages?format=txt.gz (3.6MB) which is a simple list of each package available in the default repos (I don't think it includes multiverse/etc). Taking out the descriptions reduces it to about 1 or 2 MB. Remember that does not include signing, etc.

PaddyLandau wrote on the 4 Apr 12 at 17:07
This is a nice idea, but there is a problem. You would have to persuade every single manager of a PPA to do the same thing.

Perhaps it would be possible for the system to store the last-modified date of each file within a PPA, and to download only if that date has changed. However, this probably would make little difference for Canonical-held repositories, as the changes are quite frequent.

dolphinaura wrote on the 8 Apr 12 at 05:35
Im sure youve seen the md5sum lists in the ubuntu cd/dvd download pages.

Why not create one of those for the packages folder at the repo?

Then, apt can simply compare the md5sums of the packagelists, and determine if it has changed.

PaddyLandau wrote on the 9 Apr 12 at 12:46
@dolphinaura: Is that not solution #2?

Sepero wrote on the 11 Apr 12 at 12:58
Rsync is the first thing to come to mind when I hear "checksums and deltas".

arjo129 wrote on the 19 Apr 12 at 10:27
Maybe this is more apropriate if someone raised this with the debian guys?

PaddyLandau wrote on the 20 Apr 12 at 15:46
Solution #3: Sadly, rsync is not appropriate because we are not overwriting when we initially download; the package manager has to download, unpack and investigate the package before deciding how to proceed.

Ravi5kumar wrote on the 4 Jun 12 at 04:51
Lol, every time I do sudo apt-get on my kubuntu 12.04 it always downloads a huge 20 MB data?! Everyone don't own a high speed connection! Please implement this idea as soon as possible so that we can save our bandwidth and time.


Post your comment