Ubuntu QA:
BlogBrainstormPackage status
Log in
Ubuntu QA
The Ubuntu community has contributed 21986 ideas, 135057 comments, 2615221 votes
Idea sandbox Idea sandbox
Popular ideas Popular ideas
Ideas in development Ideas in development
Implemented ideas Implemented ideas
Idea #11088: Bittorrent P2P for update manager

Written by fz105 the 13 Jul 08 at 16:43. Related project: Update manager. Status: New
Rationale
The update manager should take advantage of P2P technology, im thinking the way Bittorrent does it. Where while you're downloading you're also sharing. This will greatly remove loads of many of the servers. Ubuntu, Linux, should utilize the huge community that it has.

Using P2P will ensure comfortable and reliable speeds when performing updates, especially when doing those Ubuntu distro updates. From my experience when a new distro is released getting an distro update is close to impossible, distro update SUCK!!!!! It's usually very slow or unaccessible. P2P would help!

61
votes
closed
Solution #1: Auto-generated solution of idea #11088
Written by fz105 the 13 Jul 08 at 16:43.
Ubuntu Brainstorm was updated in January 2009. Since the idea #11088 was submitted before this update, its rationale and solution are not separated. Please vote accordingly, and if you have the necessary rights, please separate the rationale from the solution. Thanks!

Propose your solution

Attachments
No attachments.


Duplicates


Comments
zooounds wrote on the 13 Jul 08 at 18:48
dup

zooounds wrote on the 13 Jul 08 at 18:49
... and btw, -1 from me, today with fast connections (here in sweden 10-100 MBit/s) downloading isn't an issue.

Eldmannen wrote on the 13 Jul 08 at 19:40
Update is usually many small files.

P2P are useful for very large files like 700+ megabyte. It is not so useful for small 1 megabyte file.

Eldmannen wrote on the 13 Jul 08 at 19:41
Also, last time I did an distro upgrade it worked great, I downloaded it in 9 megabyte/s.

Baggers wrote on the 13 Jul 08 at 22:05
Some users are in locations where p2p and bit-torrenting is blocked (workplaces & uni for me). It would certainly be an excellent option but probably should not be on as default.

Tree MendUs wrote on the 13 Jul 08 at 22:55
Situation ;
Some folks in other parts of the world still have dialup, so any speed increase for them would probably be considered helpful.

Others may even only have temporary and sporadic access to an internet connection.

Ideas ;
1) At the User's end, it would be useful to download the updates to one machine, and then share them locally from their.
Even though one PC may not be much of a problem re bandwidth, if you multiply that by a small community or business it could main the line gets tied up for quite a while (with a higher probability of the line being dropped during download).
2) So some form of synchronisation, or shred updating, possible use of P2P for locally updating off the PCs in the local community would be handy (the PCs may not be connected to each other all of the time).

3) Another way to reduce load on the Ubuntu servers, is to have more mirror servers at local countries. And for those local servers to have good speed, so as to encourage local use.
Some local mirrors may not be so fast for local users that are outside the network that the mirror is on. This tends to encourage the local users to still use the ubuntu main site server as their update source because it goes faster (even over an international connection).


Tabris wrote on the 13 Jul 08 at 23:25
This is really awesome

fcsonline wrote on the 14 Jul 08 at 08:04
dup dup dup dup

fordplay wrote on the 14 Jul 08 at 08:40
This system would be great but needs to be backed up with http downloading at the same time. This would reduce the load and download time from the ubuntu server with big files downloadin mainly through bittorrent. Smaller files would be downloaded mainly through http.

bianchins wrote on the 14 Jul 08 at 12:51
Umh... security problem!

Auzy wrote on the 14 Jul 08 at 12:54
Uhm, no bittorrent isn't. Each chunk is hashed so its secure. And if each file is digitally signed as well by Canonical, its even more secure.

But it is a dupe of http://brainstorm.ubuntu.com/idea/7792/

Mishtal wrote on the 14 Jul 08 at 15:26
Reasons against Using BitTorrent to download packages and upgrades.

1) In some parts of the world straight http or ftp connections are so fast that BitTorrent isnt necessary.
2) For some specific cases, such as corporate offices, and university campuses, BitTorrent and other P2P traffic is blocked.
3) Perceived security problems
4) BitTorrent is a protocol designed to download large files, as many debian packages are smaller than 100 KiB's it is unreasonable to use BitTorrent to download them
5) Distro upgrades worked amazingly blazingly fast for person X

Reasons why the above reasons are not issues.

1) If http or ftp is blindingly fast in your part of the world, that still doesnt mean that you dont have to pull down the same package for each computer you happen to own doing the upgrade. 5 10 megabyte packages, for 5 computers. is still 250 megabytes, any way you swing it. with BitTorrent, 5 10 megabyte packages may involve 100 megabytes of downloading from an external source, and the rest sharing among the internal lan computers.

2) if BitTorrent traffic is blocked on your network for any peer to peer application, there would be no problem with switching to HTTP or FTP. There might even be a simple option in software sources "disable P2P support for APT" or something else equally easy. Im not going to really go into much detail, but if your local network admin disables P2P across the board, your local network admin is missing out on very large cost savings involved with throughput and network capacity.

3)BitTorrent is absolutely secure. each chunk of a bittorrent file has its own hash. and the entire file also has its own hash, as far as im aware. if a malicious host attempted to send a peer a corrupted or infected chunk, that peer would reject the file when it failed its hash check against the torrentswarm wide standard for the chunk's hash. Therefore eliminating the possibility of, or even the need to care about, man in the middle attacks. The BitTorrent protocol supports encryption, and encryption forcing. seeing as how integrating BitTorrent into APT only involves package downloads to be done through bittorrent instead of HTTP, there really isnt a reason to use encryption for the sake of protecting others from knowing you downloaded the new version of pidgin, but you can encrypt the $*** out of your package downloads if you feel like it.

4) your right, the vanilla version of bit torrent is designed to download large files over several chunks. but theres really nothing stopping Devs from modifying BitTorrent to effectively support smaller files with extreme ease.

5) congratulations on being able to download your distribution upgrade so quickly. however, it took me several days for the download to complete. the day when hardy was released, i set my laptop up in my university's computer science lab and asked it to upgrade itself from the fastest mirror it could find. i went to my 6 hours of classes, and came back to find that ubuntu Gutsy Gibbon still needed 18 hours to finish downloading all of the information it needed. my university has fiber connection directly to the local internet backbone. when i went home for the night, the distro upgrade claimed to need another 10 hours on top of the 18 it previously claimed on my home cable connection. i left my system on the entire night and finally was able to finish some time around the next evening. it was not a problem with the local connection. i was on cable and fiber the entire time. it was the server strain. with bittorrent, i would have been able to download all of the needed files from my local peers, namely all of the other members of the linux users group at my university who were upgrading at the same time as me. with the ability to fall back to http, any file that i simply cant seem to get would be grabbed directly from the server. with bittorrent, the strain on the server would be shifted from transmitting large volumes of package files to transmitting .torrent files to allow for other peers to connect to the torrent swarm. torrent files are smaller than package files, on average. torrent files are only needed 1 time per all of the packages that they track. using bittorrent, the strain on the server could potentially be dropped by a significant amount, potentially by over half. many of the people who host repository mirrors do so out of the goodness of their hearts. implimenting bittorrent for package downloads will easy their burden. it will give them more free time, and will reduce their electricity and equipment costs.


additional thoughts)

many of the features that most of the comments ive seen on ideas such as this are ALREADY IMPLEMENTED with the package "debtorrent"

ive been running debtorrent on my local machines for the past week or so. it works magnificently. honestly, the only problem is that there is a lack of peers. http://debtorrent.alioth.debian.org/
look into it.

aelfwyne wrote on the 15 Jul 08 at 01:00
Agreed with the previous comments debunking the reasons against this - and want to add a couple of topics.

1) No need to use this for ALL updates, just the large ones. The server can be a peer as well (and should be) to avoid torrent-only slowdowns, to make sure users with fat pipes can still get full speed - just not all from ubuntu's servers: Cost Savings.

2) If you're one user at the end of a decent connection this won't help a lot, BUT imagine you're a sysadmin with 20 computers to upgrade. With p2p you can launch the updates on all the computers, and as they each get pieces from the outside network, they start sharing pieces over your LAN. Effectively you would only have to download the updates ONCE for 20 computers, rather than 20 separate times. Sure there are probably ways to locally mirror the updates, etc, but this would be totally transparent and need no setup - particularly if the torrent looks for LAN peers specifically.

fz105 wrote on the 17 Jul 08 at 16:44
Excellent points Mishtal. And thanks for the link to debtorrent, I'll will try it out.

It's great that Eldmannen was able to download at 9MBits/sec but my experience was about 3 days using the Update Manager; it really sucked updating to 8.04 (Hardy). I eventually gave up and downloaded the ISO image via bittorrent. I was able to download the ISO image within 10-15 min, there was a HUGE community seeding it.

I think a P2P technique, in combination with what's there, in performing updates would increase the overall experience and remove loads off these servers.

As someone pointed out that we need more mirrors. We take a lot of these mirror servers for granted and forget that they're being provided to the community for free. Meaning someone else pays the bills (to put it simply)!!

Also, sorry about creating a duplicate idea. I had checked (not thoroughly enough it seems) but didn't find similar ideas before sharing my own. So my apologies, and thanks to those that were kind enough to provide a link to the duplicate ideas.

TuxHHG wrote on the 4 Mar 10 at 20:51
Bad idea!!

Cause google for 'acta' anti counterfeiting agreement. Carriers and providers are ordered by law to break p2p traffic for anti piracy reasons. In germany most providers change their agb and establish heavy p2p blocking mechanisms.



Post your comment