BitTorrent

From Wikipedia, the free encyclopedia.

Jump to: navigation, search
Internet protocol suite
Layer Protocols
Application FTP, HTTP, IMAP, IRC, NNTP, POP3, SIP, SMTP, SNMP, SSH, Telnet, BitTorrent, …
Transport DCCP, SCTP, TCP, RTP, UDP, IL, RUDP, …
Network IPv4, IPv6, …
Data link Ethernet, Wi-Fi, Token ring, FDDI, PPP, …
Physical RS-232, EIA-422, RS-449, EIA-485, 10BASE2, 10BASE-T, …

BitTorrent logo
BitTorrent is both the protocol and the name of the peer-to-peer (P2P) file distribution application created by programmer Bram Cohen that makes it possible to massively distribute files without incurring the corresponding massive consumption in server and bandwidth resources.

The bittorrent protocol achieves this goal by breaking down a file into smaller fragments, typically a quarter of a megabyte (250 KB) in size. Peers download missing fragments from each other and upload those that they already have to peers that request them. The protocol is 'smart' enough to choose the peer with the best network connections for the fragments that its requesting. To increase the overall efficiency of the ad-hoc P2P network temporarily created to distribute a particular file, the bittorrent clients request from their peers the fragments that are most rare; in other words, the fragments that are available on the least number of peers, making most fragments available widely across many machines and avoiding bottlenecks. The file fragments are not usually downloaded in sequential order and need to be reassembled by the receiving machine. It is important to note that clients start uploading fragments to their peers before the entire file is downloaded. Sharing by each peer therefore begins when the first complete segment is downloaded and can begin to be uploaded if another peer requests it. This scheme is particularly useful for trading large files such as videos and operating systems. This is contrasted with conventional file serving where high demand can lead to saturation of the host's resources as the consumption of bandwidth to transfer the file to many requesting downloaders surges. With BitTorrent, high demand can actually increase throughput as more bandwidth and additional “seeds” of the file become available to the group. Cohen claims that for very popular files, BitTorrent can support about a thousand times as many downloads as HTTP.

The BitTorrent application is written in Python and its source code has been released under the BitTorrent Open Source License (a modified version of the Jabber Open Source License), as of version 4.0. The name "BitTorrent" refers to the distribution protocol, the original client application, and the .torrent file type.

Contents

Sharing files

BitTorrent greatly reduces the load on peers with complete copies, because peers with incomplete copies generally download the file from each other. As the colored bars below each client show, the file is downloaded in random order, instead of sequential order.
BitTorrent greatly reduces the load on peers with complete copies, because peers with incomplete copies generally download the file from each other. As the colored bars below each client show, the file is downloaded in random order, instead of sequential order.

To share a file using BitTorrent, a user creates a .torrent file, a small "pointer" file that contains:

  • the filename, size, and the hash of each block in the file (which allows users to make sure they are downloading the real thing)
  • the address of a "tracker" server (which is discussed below)
  • and some other data (like client instructions).

The torrent file is then distributed to users, often via email or placed on a website. The BitTorrent client is started as a "seed node", allowing other users to connect and begin downloading. When other users finish downloading the entire file, they can optionally "reseed" it--becoming an additional source for the file. One outcome of this approach is that if all seeds are taken offline, the file may no longer be available for download, even if a client has a copy of the torrent file. However, everyone can eventually get the complete file as long as there is at least one distributed copy of the file, even if there are no seeds.

Downloading with BitTorrent is straightforward. Each person who wants to download the file first downloads the torrent and opens it in the BitTorrent client software. The torrent file tells the client the address of the tracker, which, in turn, maintains a log of which users are downloading the file and where the file and its fragments reside. For each available source, the client considers which blocks of the file are available and then requests the rarest block it does not yet have. This makes it more likely that peers will have blocks to exchange. As soon as the client finishes importing a block, it hashes it to make sure that the block matches what the torrent file said it should be. Then it begins looking for someone to upload the block to.

BitTorrent gives the best download performance to the people who upload the most, a property known as "leech resistance", since it discourages "leechers" from trying to download the file without uploading it to anyone. (Although, confusingly, when used in opposition to "seeds" or "seeders" as in "S/L ratio" (meaning "seed/leech ratio"), "leecher" only means someone who hasn't downloaded the full file yet.)

Though BitTorrent is a good protocol for a broadband user, it is less effective for dial up connections, where disconnections are common. On the other hand, many HTTP servers drop connections over several hours, while many torrents exist long enough to complete a multi-day download.

Terminology

torrent
A torrent can mean either a .torrent meta data file or all files described by it, depending on context. The .torrent file contains meta data about shared files. It contains the names of the files, their sizes and the checksums of all blocks in the torrent. It also holds the address of a tracker that hosts the torrent.
peer
A peer is one instance of a BitTorrent client running on a computer on the Internet that you connect to and transfer data. Usually a peer does not have the complete file, but only parts of it.
seed
A seed is a peer that has a complete copy of the torrent and still offers it for upload. The more seeds there are, the better the chances are for completion of the file.
leech
A leech is a peer that does not have a complete copy of the torrent yet. When downloading is complete, it may stay around and seed the file as a seed so that others can complete their download. The term leech is also used for peers that have very poor upload/download ratios or leave the swarm immediately after their downloads are complete. (See Etiquette.) The leeches usually contribute a majority of the bandwidth in a swarm.
swarm
Together, all peers sharing a torrent are called a swarm. Six leeches and two seeds makes a swarm of eight.
tracker
A tracker is a broker service that mediates contacts between peers. The tracker is not directly involved in the data transfer and does not have a copy of the file.

Comparison to other file sharing systems

Version 4.0.4 running in Windows XP
Enlarge
Version 4.0.4 running in Windows XP

The method used by BitTorrent to distribute files parallels to a large extent the one used by the eDonkey2000 network, but nodes in eDonkey's file sharing network usually share and download a much larger number of files, making the bandwidth available to each transfer much smaller. BitTorrent transfers are typically very fast, because all nodes in a group concentrate on transferring a single file or collection of files. While the original eDonkey2000 client provided little "leech resistance", most new clients have some sort of system to encourage uploaders. eMule, for example, has a credits system whereby a client rewards other clients that upload to it by increasing their priority in its queue. However, the nature of the eDonkey2000 concept means download speeds tend to be much more variable, although the number of available files is far greater.

A similar method to BitTorrent was the Participation Level introduced in KaZaA in 2002. The Participation Level would increase when you upload and decrease when you download. Then when you upload a file to someone else the person with the highest Participation Level gets it first, then they upload it on to the person with the next highest Participation Level, and so on. This can be visualised as a pyramid, with the people who have the most upload bandwidth available at the top and people with less bandwidth on progressively lower levels. This is the most efficient way to distribute a file to a large number of users: it is probable that even the people at the bottom of the pyramid will get the file faster than if the file was served by a non-P2P method. Unfortunately, the system adopted by KaZaA is considered by some to be flawed as it relies on the client accurately reporting their Participation Level and therefore it is easy to cheat with the many "unofficial" clients.

Legal issues

BitTorrent, like any other file transfer protocol, can be used to distribute files without the permission of the copyright holder. BitTorrent has become famous for its ability to also share copyrighted files. Those who intend to distribute illegitimate files often have no or limited funds available, and BitTorrent's efficiency provides a viable distribution method.

Copyright enforcement

In December 2004, the Finnish police raided a major BitTorrent site. [1]. The court hearing will take place later this year.

Suprnova.org, one of the most popular BitTorrent sites, closed in December 2004, supposedly due to the pressure felt by the founder and administrator of the site. LokiTorrent, arguably the biggest torrent source after the demise of Suprnova, closed down soon after. Allegedly, after threats from the MPAA, Edward Webber (known as 'lowkee'), webmaster of the site, was ordered by the court to pay a fine and supply the MPAA with logs (the IP addresses of visitors).

Other sites that offer files such as anime fansubs are careful to shut down torrents that have been licensed in the United States, and only provide files that cannot be bought in the U.S., presumably in order to stay on the good side of the "effect on the market" factor of the fair use definition.

Webber, in the weeks following his reception of the subpoena, had begun a fundraising campaign to pay lawyers fees in a legal battle against the MPAA. In news reports, Webber said he would stand up to protect the rights of file sharers, which he did not. Webber raised approximately $45,000 (USD) through a PayPal-based donation system. It is unclear how much of that money went to the MPAA, but taking into account the amount of damages he most likely had to pay, probably much of it. Following the agreement, the MPAA changed the LokiTorrent website to display a message intended to intimidate filesharers. Webber did not comment on this change.

On 24 October 2005, a 38-year-old Hong Kong BitTorrent user Chan Nai-Ming (陳乃明, using the handle 古惑天皇 Lit. The master of cunning) allegedly distributed the three movies Daredevil, Red Planet and Miss Congeniality, subsequently uploading the torrent file to a newsgroup. He was convicted of breaching the Copyright ordinance, Chapter 528 of Hong Kong law. The magistrate remarked that Chan's act of distributing the seed caused significant damage to the interest of copyright holders, up to thirty users downloading the torrent simultaneously.

He was released on bail for HK$5,000, awaiting a sentencing hearing, though the magistrate himself admitted the difficulty of determining how he should be sentenced due to the lack of precedent for such a case. On 7 November 2005, he was sentenced to jail for three months for film piracy.

Legal defenses

There are two major differences between BitTorrent and many other peer-to-peer file-trading systems, which advocates suggest make it less useful to copyright violators. First, BitTorrent does not offer a search facility to find files by name. A user must find the initial torrent file by other means, such as a Web search. Second, BitTorrent makes no attempt to conceal the host ultimately responsible for a given file's availability: a person who wishes to make a file available must run a tracker on a specific host or hosts and distribute the tracker address(es) in the .torrent file. While it is possible to simply operate a tracker on a server that is located where the copyright holder cannot take legal action, this feature of the protocol does imply some degree of accountability that other protocols lack. It is far easier to request that the server's internet service provider shut the site down than it is to find and identify every user sharing a file on a traditional peer-to-peer network.

Legal uses for BitTorrent

BitTorrent can be used by software developers who want to ease the bandwidth strain on their servers. If a developer offers a large file for download, the bandwidth limit of their server may be exceeded if a large number of people download the file. By offering the file via BitTorrent, they transfer much of the bandwidth burden to downloaders of the file. For example, the demo of the flight sim X-Plane is offered via BitTorrent, as well as the World of Warcraft ingame patches. Another such example is PlaneShift, a free open-source MMORPG, which uses BitTorrent for its primary method of distribution. The fan-film Star Wars: Revelations is distributing two DVD images as well as the film by itself via BitTorrent, and Star Wreck: In the Pirkinning, a feature-length film, was provided for download via the network besides a centralized server. In 2005, the rock group Harvey Danger began distributing their third full-length album, Little by Little..., using BitTorrent. Also, the NetBSD operating system version 1.6.2 and later as well as most major Linux distributions use BitTorrent as an alternative way of distributing ISO images of their releases.

Following the success of the BitTorrent protocol, Bram Cohen, its creator, was hired in 2004 by Valve Software to develop a means of distributing patches and other content for online video games, proving that there are some less controversial reasons for the development of this technology. While many legal files, including Linux distributions, are available on other networks such as eDonkey2000 and Gnutella these are placed there by users and not generally part of the official distribution mechanism. So far, BitTorrent seems to be the most popular P2P protocol adopted officially for legal uses.

Etiquette

Because BitTorrent relies on the upstream bandwidth of its users — and the more users, the more aggregate bandwidth is available for sharing the files — it is considered good etiquette to leave one's BitTorrent client open after downloading has completed so that others may continue to gain from the file that has been distributed.

It is not clear, however, how long one should leave their client open after downloading has finished. Many clients report the byte traffic upstream as well as down, so the user can see how much they have contributed back to the network. Some clients also report the "share ratio", a number relating the amount of data uploaded to the amount downloaded. It is generally considered good form to at least share back the equivalent amount of traffic as the original file size.

It is worth noting that the requirement of a "1.00" share ratio (uploading as much data as you have downloaded) is rather hotly contested given its relative impossibility to achieve for every person. On any given torrent, the best possible outcome is the original seeder with an infinite ratio (having only uploaded data and never downloaded any data), a number of peers with 1.00 ratios (having downloaded the file, uploaded just as much data, and then promptly logged off), and two users with a .50 ratio (the last two having each downloaded a separate half of the file and then shared their half with the other). This is highly unlikely to be achieved due to the very small chance of the last two peers downloading completely opposite halves and finishing just as the last seeder logged off and the fact that not all people will upload the same amount of data they downloaded as some will upload less and others will upload more. Ultimately, a perfect torrent would leave two end users with only a .50 ratio for the torrent, which means every user would have to provide new content at least equal to the portion of data they did not get to upload in the last torrent to maintain an overall ratio of 1.00.

While a 1.00 ratio per torrent is not feasible as a rule for all users who download that file, it is more of a guideline to encourage the average upstream of a given network. Some networks, for example, prevents access to torrents for the first 24-48 hours to people with overall ratios of less than 1.0.

The amount of time the client is left open may be more important than the amount of traffic contributed, since new users attempting to download a file will first need to find peers hosting the file.

Many advanced trackers now track statistics such as how many seeders and downloaders were on a torrent at the time of a user's disconnect as many consider this information more important than just the user's ratio of downloaded/uploaded.

New developments

The BitTorrent protocol is still under development and therefore may still acquire new features and other enhancements such as improved efficiency.

In May 2005, on the same day the tracker website elitetorrents.org was shut down, Bram Cohen released a new beta version of BitTorrent that eliminated the need for Web site hosting of centralized servers known as "trackers". It is now possible to have a torrent up in minutes, with a file, a website, and no understanding of how it works. In addition, Cohen launched a new service on BitTorrent's website, which helps you find files - both legal and illegal.

Cohen explained that the tracker removal feature is part of his ongoing effort to make publishing files online "painless and disruptively cheap." The move is only one of several designed to remove BitTorrent's dependence on centralized trackers.

This change is said to cause some trouble in the legal efforts to shut down illegal file sharing. However, Tarun Sawney, BSA Asia antipiracy director, said BitTorrent files could still be identified, since with or without the tracker sites, someone still hosts the infringing files (see [2] [3]).

Alternative approaches

The BitTorrent protocol provides no way to index torrent files. As a result, a comparatively small number of websites have hosted the large majority of copyright infringing torrents, rendering those sites especially vulnerable to lawsuits. In response, some developers have sought ways to make publishing of files more anonymous while still retaining BitTorrent's speed advantage. The Shareaza client, for example, provides three alternatives to BitTorrent: eDonkey2000, Gnutella, and Shareaza's native network, Gnutella2. If the tracker is down, it can finish the file over the other protocols, and/or find new (Shareaza) peers over G2. BitTorrent also inspired the privately held company InmateMediaGroup: eXeem. It is backed by Andrej Preston, the administrator of the now defunct Suprnova BitTorrent website. eXeem is supposed to decentralize BitTorrent and eliminate the need for web-based trackers (easy targets for the RIAA or the MPAA). Unlike BitTorrent (the protocol and the official client), eXeem is closed-source and owned by a corporation. Distributed trackers is also one of the goals for Azureus 2.3.0.2 and BitTorrent 4.1.2. Another interesting idea that has surfaced recently in Azureus is virtual torrent. This idea is based on the distributed tracker approach and is used to describe some web resource. Right now, it is used for instant messaging. It is implemented using a special messaging protocol and requires an appropriate plugin. Anatomic P2P is another approach, which uses a decentralised networks of nodes that route traffic to dynamic trackers.

BitTorrent search / Trackerless

A new BitTorrent online search was recently implemented (Betanews article). One can search content provided by BT. A new beta "trackerless" client was also released. This eliminates the need for a tracker.

Web seeding (unofficial feature)

One recently implemented feature of BitTorrent is web seeding. The advantage of this feature is that a site may distribute a torrent for a particular file or batch of files and make those files available for download from that same web server application; this can simplify seeding and load balancing greatly once support for this feature is implemented in the various BitTorrent clients. In theory, this would make using BitTorrent almost as easy for a web publisher as simply creating a direct download while allowing some of the upload bandwidth demands to be placed upon the downloaders (who normally use only a very small portion of their upload bandwidth capacity). This feature is an unofficial one, created by the author of a specific third-party client.

Broadcatching

Another proposed feature combines RSS and BitTorrent to create a content delivery system dubbed broadcatching. Since a Steve Gillmor column for Ziff-Davis in December 2003, the discussion has spread quickly among many bloggers (Techdirt, Ernest Miller, and former Tech TV host Chris Pirillo, for example). In an interview, Scott Raymond explained:

"I want RSS feeds of BitTorrent files. A script would periodically check the feed for new items, and use them to start the download. Then, I could find a trusted publisher of an Alias RSS feed, and 'subscribe' to all new episodes of the show, which would then start downloading automatically — like the 'season pass' feature of the TiVo."

While potential illegal uses abound as is the case with any new distribution method, this idea lends itself to a great number of ideas that could turn traditional distribution models on their heads, giving smaller operations a new opportunity for content distribution. Similar to Linspire, the system leans on the cost-saving benefit of BitTorrent, where expenses are virtually non-existent; each downloader of a file participates in a portion of the distribution.

RSS feeds layered on top keep track of the content, and because BitTorrent does cryptographic hashing of all data, subscribers to the feed can be sure they're getting what they think they're getting, whether that winds up being the latest Sopranos episode, or the latest Sveasoft firmware upgrade. (Naturally, however, ensuring that the same data reaches all nodes neglects the possibility that the original, source file may be corrupted or incorrectly labeled.)

API

The BitTorrent web-service Prodigem has made available a BitTorrent API for the creation and seeding of BitTorrents. Combining the ideas of both web-seeding and broadcatching, listed above, this API extends BitTorrent creation ability to any web application capable of parsing XML through its standard Representational State Transfer (REST) based interface. Alongside this release is a first application built using the API called PEP which, in 400 lines of PHP, will parse any Really Simple Syndication (RSS 2.0) feed and automatically create and seed a torrent for each enclosure found in that feed.

BitTorrent clients and applications, which support the protocol

Because of the open nature of the protocol, many clients have been developed that support numerous platforms and written using various programming languages.

Clients

  • Arctic Torrent is a minimalist, fast, C++ BitTorrent client.
  • Azureus is a Java-based client with many features, including an RSS feedreader plug-in that can be used in conjunction with the sites mentioned above.
  • BitAnarch is a BitTorrent client packed with a lot of useful features, such as a newsgroup .torrent scanner, unicode filename support, automation, download and upload bandwidth limit, etc. (Project suspended from Sept. 2003)
  • BitComet is a client written in C++, as opposed to many of the above that are written in python, and thus is much faster, with lower CPU overhead. It is available for Windows only and allows you to create torrents and use a method of DHT compatible with the official BitTorrent client by Bram Cohen, but not compatible with the DHT in Azureus.
  • BitLord is a nice and easy to use torrent client, written in C++, it is fast and stable client with browsing facility and is very similar to BitComet. There are two versions, the unpaid for one features annoyance ads similar to Kaaza.
  • Bits on Wheels is a native BitTorrent client for Mac OS X. It provides several interesting features, including a fun 3D swarm view.
  • BitSpirit is a powerful and easy-to-use BitTorrent client which provides not only full BitTorrent protocol implementation but also many personalization features.
  • BitTornado is based off the original BitTorrent code with a few enhancements.
  • burst! is port of BitTorrent's console btdownloadheadless.py to Win32, with a GUI front-end. MakeTorrent is a Win32 solution for easily creating .torrent files with features such as multi-tracker .torrents and a custom tracker list.
  • CTorrent is a client written in C++. Thus like BitComet it has low CPU overhead. It is written for Unix/Linux platforms only.
  • Flash! Torrent is a BitTorrent client that includes a small web browser for navigate in the webs about BitTorrent, an own tracker, etc. You can configure upload speed, know the number of seeds and peers of a torrent, multilanguage and much more.
  • Rufus is feature rich and graphically empowered bittorrent client for Win32 written in Python.
  • Localhost is a new client that creates a decentralised hierarchy-index of torrents, which can be thought of as a world-wide file system of files. Localhost is based off Azureus.
  • TvM is another BT/RSS feedreader for Windows designed to be as simple as possible.
  • Torment is a C++ bittorent client that has many features including remote administration, a client list, and a block map.
  • Transmission is a free BitTorrent client written from scratch in C. It aims to be as efficient as possible. It runs on Mac OS X, BeOS, Linux, NetBSD and FreeBSD. Only the Mac OS X version has a graphical interface yet. Smallest and fastest Mac client available.
  • µTorrent written in C++, the program is only 98k and the client is efficient with a surprising number of extra features.
  • WinMobile Torrent is the first BitTorrent Client for Windows Mobile PocketPC.
  • WritTorrent offers a blog posting plugin for Azureus, as well as RSS feeds for the built-in Azureus tracker.
  • Yet ABC is an improved client for the Bittorrent which supports queueing system with priority, global and local preference setting for downloading torrent, 3 upload options to do with completed file, with ABC_TWEAK you can adjust status display by yourself.

More BitTorrent Clients

Applications

  • Programmer Andrew Grumet has announced the release of a beta version of an RSS and BitTorrent integration tool for Radio Userland's news aggregator [4].
  • Buttress is a light weight simple Java solution.
  • TV RSS is a Linux solution.
  • SwarmTv project working on a c++ rss bittorrent library.
  • Undercurrents, a UK based video activist group is working on an offline TV service using RSS and BitTorrent. It can be seen at beyondTV.
  • Thinkingest is providing commercial broadcatching and pro-bono consulting for community projects, such as OurMedia [5].
  • Anime fansub communities often use BitTorrent for their releases, and the most popular announce sites like animesuki, Tokyo Toshokan, and downloadanime.org [6] have RSS feeds.
  • X Hollywood aggregates RSS feeds from BitTorrent websites.
  • Podcasting is starting to integrate BitTorrent to help podcasters deal with the download demands of their MP3 "radio" programs. Specifically, iPodder supports BitTorrent for the RSS 2.0 enclosures that power podcasting.
  • Blog Torrent offers a simplified BitTorrent tracker to enable bloggers and non-technical users to run a tracker off their site with the added functionality of letting visitors download a file even if they do not have a BitTorrent client installed by automatically installing a client and the desired file. (GrepLaw interviews makers of Blog Torrent.)
  • TBSource and TBDev provide information and help on the TorrentBits sourcecode, a PHP based BitTorrent Tracker.
  • XBNBT is a modifed BNBT BitTorrent tracker written in C++. XBNBT offers many additional features beyond the original BNBT tracker and has been designed with the Linux user in mind in an attempt to make it as easy as it is for Windows.
  • TorrentSniff is a tool written in perl, that reports status of the torrent

See also

External links

Torrent sites and trackers

Some of these sites temporarily go down due to bandwidth limitations, legal issues or technical difficulties. Most are run by volunteer staff and most return within a day.

General collections

Wikibooks
Wikibooks Internet Server Directory has more about this subject:

Trackers

Indexers

Meta-search

Unsorted

Forums

Usenet newsgroups

News

Technical analyses

Personal tools