Comments on: automatically ‘mirroring’ distro torrents? http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/ Ramblings on law school in New York, free software, and the spaces in between. Thu, 21 Aug 2008 23:45:04 +0000 http://wordpress.org/?v=2.6 By: Seth Vidal http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26047 Seth Vidal Fri, 21 Dec 2007 21:37:34 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26047 Torrents don't remove the need for mirrors b/c torrents don't work out well for single, small files. Also distributed mirrors deal well for places where a link internally is MUCH MUCH MUCH cheaper than an overseas link. You could, potentially, do distribution by a network of seeds and trackers in multiple locales but that let's people get ALL The bits, but not precisely "this bit" of the files hosted on the torrent. Torrents don’t remove the need for mirrors b/c torrents don’t work out well for single, small files. Also distributed mirrors deal well for places where a link internally is MUCH MUCH MUCH cheaper than an overseas link.

You could, potentially, do distribution by a network of seeds and trackers in multiple locales but that let’s people get ALL The bits, but not precisely “this bit” of the files hosted on the torrent.

]]> By: Luis http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26046 Luis Fri, 21 Dec 2007 21:34:27 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26046 Presumably Ubuntu manages in part by having piles of money. ;) But yes, I agree that (at least in theory) having torrents could make the whole mirror structure mostly redundant, except for those institutions that for some reason need to block torrents. Presumably Ubuntu manages in part by having piles of money. ;) But yes, I agree that (at least in theory) having torrents could make the whole mirror structure mostly redundant, except for those institutions that for some reason need to block torrents.

]]> By: Jon Cooper http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26045 Jon Cooper Fri, 21 Dec 2007 21:30:03 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26045 Surely torrents negate the need for historical "mirrors" as once a client has acquired 100% of the data, it can happily seed forever. I don't see the need for official mirrors, just fast seeds (that Ubuntu somehow seem to manage) Surely torrents negate the need for historical “mirrors” as once a client has acquired 100% of the data, it can happily seed forever.

I don’t see the need for official mirrors, just fast seeds (that Ubuntu somehow seem to manage)

]]> By: Luis http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26041 Luis Fri, 21 Dec 2007 12:54:12 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26041 No, I'm really not asking for a cloud file system :) I'll subscribe to the torrent RSS; it should be better documented on the torrent.fedoraproject page. No, I’m really not asking for a cloud file system :)

I’ll subscribe to the torrent RSS; it should be better documented on the torrent.fedoraproject page.

]]> By: Christian Lohmaier http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26040 Christian Lohmaier Fri, 21 Dec 2007 12:49:17 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26040 Many clients offer to watch a certain directory for torrent files and will then watch that directory for additions/deletions of torrents. So only the 'how to get hold of the torrents' part left. There you have all options: rsync, rss feeds, crawler like stuff, etc. There is not only one way to do it... Many clients offer to watch a certain directory for torrent files and will then watch that directory for additions/deletions of torrents.

So only the ‘how to get hold of the torrents’ part left. There you have all options: rsync, rss feeds, crawler like stuff, etc.

There is not only one way to do it…

]]> By: Jon Smirl http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26035 Jon Smirl Fri, 21 Dec 2007 01:04:50 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26035 You're asking for a cloud file system. It is a well know concept but it has never been implemented on a large scale. Cloud file systems always get filled with illegal content and that brings on the liability question. Mirrors are a controlled way of doing a primitive cloud file system. For an interesting intellectual exercise imagine a world without copyright on digital representations. In this world the cloud file system is the perfect solution. Microsoft loads a new Windows onto the cloud disk drive and everyone magically starts booting from it. All the books, music, and images of the world are on anther cloud disk. Everyone's disk becomes a cache and the network sorts out the optimal place to redundantly store everything. All information is stored in standardized form and perfectly indexed. But alas digital representations have copyright and we have this world of billions of private disk drives. Illegal copies are retransmitted trillions of times over the internet because of the liability of storing them. Everything is a confusing mess since all of the right owners don't cooperate with indexing - that's called marketing - make your product more visible and hide the others. Remove copyright and allow efficient caching. 90% of the world's internet traffic would disappear and we would all be able to consume 10x the content with out generating network load. You’re asking for a cloud file system. It is a well know concept but it has never been implemented on a large scale. Cloud file systems always get filled with illegal content and that brings on the liability question. Mirrors are a controlled way of doing a primitive cloud file system.

For an interesting intellectual exercise imagine a world without copyright on digital representations. In this world the cloud file system is the perfect solution. Microsoft loads a new Windows onto the cloud disk drive and everyone magically starts booting from it. All the books, music, and images of the world are on anther cloud disk. Everyone’s disk becomes a cache and the network sorts out the optimal place to redundantly store everything. All information is stored in standardized form and perfectly indexed.

But alas digital representations have copyright and we have this world of billions of private disk drives. Illegal copies are retransmitted trillions of times over the internet because of the liability of storing them. Everything is a confusing mess since all of the right owners don’t cooperate with indexing - that’s called marketing - make your product more visible and hide the others.

Remove copyright and allow efficient caching. 90% of the world’s internet traffic would disappear and we would all be able to consume 10x the content with out generating network load.

]]> By: Seth Vidal http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26020 Seth Vidal Thu, 20 Dec 2007 02:41:55 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26020 http://torrent.fedoraproject.org publishes an rss feed of all of its torrents. This was done to handle auto-mirroring of our torrent. A number of torrent clients obey the rss format. A contributor to fedora told me about the format and I wrote the script that adds them to it automatically. A lot of torrent servers have this already, I think. http://torrent.fedoraproject.org publishes an rss feed of all of its torrents. This was done to handle auto-mirroring of our torrent. A number of torrent clients obey the rss format. A contributor to fedora told me about the format and I wrote the script that adds them to it automatically.

A lot of torrent servers have this already, I think.

]]> By: Chris Leech http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26017 Chris Leech Thu, 20 Dec 2007 00:11:23 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26017 Many Bittorrent clients support discovering torrents via RSS, often through a plugin of some sort. So if the distro / projects you wanted to mirror published their list of torrents in an RSS feed it would be fairly easy to pick up new releases. Many Bittorrent clients support discovering torrents via RSS, often through a plugin of some sort. So if the distro / projects you wanted to mirror published their list of torrents in an RSS feed it would be fairly easy to pick up new releases.

]]> By: jef spaleta http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26016 jef spaleta Wed, 19 Dec 2007 23:46:44 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26016 the overhead for small files in torrent is pretty bad. Fedora's Mirror Manager works with the default mirrorlist url generator to help make use of local mirrors when available. Local admins for a netblock can define a local mirror location for the fedora repository and every client from that netblock will then be handed the address of the local mirror from the Fedora mirrorlist generator.. with no local configuration necessary. As long as the local miror admin keeps their mirror in sync, it will be the preferred mirror for the local netblock for which they are providing the local mirror. Check out mirror manager. -jef the overhead for small files in torrent is pretty bad.

Fedora’s Mirror Manager works with the default mirrorlist url generator to help make use of local mirrors when available. Local admins for a netblock can define a local mirror location for the fedora repository and every client from that netblock will then be handed the address of the local mirror from the Fedora mirrorlist generator.. with no local configuration necessary.

As long as the local miror admin keeps their mirror in sync, it will be the preferred mirror for the local netblock for which they are providing the local mirror.

Check out mirror manager.

-jef

]]> By: Jon Åslund http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26015 Jon Åslund Wed, 19 Dec 2007 23:40:05 +0000 http://tieguy.org/blog/2007/12/19/automatically-mirroring-distro-torrents/#comment-26015 There is something called debtorrent. I haven't tried it myself, but I think it works pretty well. http://debtorrent.alioth.debian.org/ There is something called debtorrent. I haven’t tried it myself, but I think it works pretty well.

http://debtorrent.alioth.debian.org/

]]>