Admin: Created page with "'''Metalink''' is an extensible metadata file format that describes one or more computer files available for download. It specifies files appropriate for the u..."

2019-06-14T04:10:31Z

Created page with "'''Metalink''' is an extensible metadata file format that describes one or more computer files available for download. It specifies files appropriate for the u..."

New page

'''Metalink''' is an extensible [[metadata]] [[file format]] that describes one or more [[computer file]]s available for [[download]]. It specifies files appropriate for the user's language and [[operating system]]; facilitates [[file verification]] and recovery from [[data corruption]]; and lists alternate download sources ([[Web mirror|mirror]] [[Uniform resource identifier|URIs]]).

The metadata is encoded in [[List of HTTP header fields|HTTP header fields]] and/or in an [[Extensible Markup Language|XML]] file with [[File extension|extension]] <tt>.meta4</tt> or <tt>.metalink</tt>. The duplicate download locations provide reliability in case one method fails. Some clients also achieve faster download speeds by allowing different chunks/segments of each file to be downloaded from multiple resources at the same time ([[segmented downloading]]).

Metalink supports listing multiple partial and full file [[Cryptographic hash function|hashes]] along with [[Pretty Good Privacy|PGP]] signatures. Most clients only support verifying [[MD5]], [[SHA-1]], and [[SHA-256]], however. Besides FTP and HTTP [[Web mirror|mirror]] locations and [[rsync]], it also supports listing the P2P methods [[BitTorrent (protocol)|BitTorrent]], [[Ed2k: URI scheme|ed2k]], [[Magnet URI scheme|magnet link]] or any other that uses a [[Uniform Resource Identifier|URI]].

== Development History ==
Metalink 3.0 was publicly released in 2005. It was designed to aid in downloading Linux [[ISO image]]s and other large files on release day, when servers would be overloaded (each server would have to be tried manually) and to repair large downloads by replacing only the parts with errors instead of fully re-downloading them. It was initially adopted by [[download manager]]s, and was used by [[open source]] projects such as [[OpenOffice.org]] and [[Linux distribution]]s. A community developed around it, more download programs supported it (including proprietary ones) and it saw commercial adoption. In 2008, the community took their work to the [[Internet Engineering Task Force]] which resulted in Metalink 4.0 in 2010, described in a Standards Track [[Request for Comments|RFC]]. Metalink 3.0 (with the extension <tt>.metalink</tt>) and Metalink 4.0 (with the extension <tt>.meta4</tt>) are incompatible because they have a slightly different format. In 2011, another Standards Track [[Request for Comments|RFC]] described Metalink in [[List of HTTP header fields|HTTP header fields]].

== Client programs ==

*[https://aria2.github.io/ aria2] is a utility for downloading files. The supported protocols are [[Hypertext Transfer Protocol|HTTP]][[HTTPS|(S)]], [[File Transfer Protocol|FTP]], [[SSH File Transfer Protocol|SFTP]], [[BitTorrent]], and [[Metalink]].
* [[cURL]] is a computer software project providing a library and command-line tool for transferring data using various protocols.
* [[DownThemAll]] ([[GNU General Public License|GPL]]), a Firefox extension and download manager that has basic Metalink support.
* [[FlashGot]] ([[GNU General Public License|GPL]]), a Firefox extension for integrating with many Download Managers, supports Metalink.
* [[Free Download Manager]] ([[GNU General Public License|GPL]]), a Windows Download Manager, supports Metalink (HTTP/FTP) in version 2.5.
* [[GetRight]] 6, a Windows Download Manager, supports Metalink (HTTP/FTP/BitTorrent).
* [[jDownloader]] is an open source download manager written in [[Java (software platform)|Java]].
* [[KGet]] ([[GNU General Public License|GPL]]), [[KDE]]'s native download manager, supports Metalink in [[KDE 4]].
* [[Orbit Downloader]], a Windows Download Manager, supports Metalink (HTTP/FTP).
* [[Phex]] ([[GNU General Public License|GPL]]), a Mac/Unix/Windows [[gnutella]] client, supports Metalink downloads along with exporting a library to Metalink for sharing between users.
* [[SmartFTP]], a Windows GUI FTP client, supports Metalink for adding files to an FTP transfer queue and hash verification.
* [[VeryCD]]'s easyMule (version 1 is [[GNU General Public License|GPLed]]) is a version of [[eMule]] with Metalink support.
* [[Yellow dog Updater, Modified|yum]], an [[open source software|open source]] [[command line interface|command line]] [[package management system|package management]] utility for [[RPM Package Manager|RPM]]-compatible [[Linux]] [[operating system]]s, uses Metalink internally, invisible to the user.
* [[ZYpp]], the [[package management]] engine that powers [[YaST2]] and [[Zypper]], supports Metalink for downloading. In the beginning it used aria2 internally, meanwhile (since Jul. 2010) its own libcurl based implementation

== Client Libraries ==
* libmetalink ([[MIT License]]) is a Metalink library written in [[C (programming language)|C]]. It provides the parsing of Metalink XML files to programs written in C and uses [[Expat (library)]] or [[libxml2]]. It does not handle the actual downloading of files. It is used by [[cURL]] and other client programs.
* metalink-checker ([[GNU General Public License|GPL]]) is a command line downloader written in [[Python (programming language)|Python]] that supports multi-source downloads and chunk checksums, as well as checking mirrors. It can also be used as a [[Python (programming language)|Python]] library.

== In use ==
[[Mandriva Linux]] has integrated Metalink into package management with [[urpmi]] and aria2.
[[Fedora (operating system)|Fedora]] has integrated Metalink into package management with [[Yellow dog Updater, Modified|yum]]. [[openSUSE]] has integrated Metalink into package management with [[ZYpp]] and aria2.

[[Wubi (installer)|Wubi]], the Windows-based [[Ubuntu (operating system)|Ubuntu]] installer, uses [http://hampus.vox.nu/metadl/ Metadl] ([[LGPL]]) to download Ubuntu [[ISO image]]s and takes advantage of Metalink's features of higher availability and increased reliability. If there are errors in the download, they are repaired, instead of restarting the large download.

Appupdater ([[GNU General Public License|GPL]]) for Windows "is similar to apt-get or yum on Linux. It automates the process of installing and maintaining up to date versions of programs."

Currently, [[OpenOffice.org]] uses Metalinks to distribute their free office suite. [[cURL]] offers Metalinks of their downloads. [[UniProt]] Consortium also provides a Metalink for their large protein database. [[Dofus]], a Flash [[MMORPG]] uses Metalinks for downloads and so does [[SageMath]], the open-source mathematical software.

The software hosting and collaboration platform Origo generates Metalinks for all hosted releases.

The following [[Linux]] distributions use Metalink for [[ISO image]] distribution: [[Adios Live CD|Adios]], [[Bayanihan Linux]], [[BeleniX]], [[Berry Linux]], [[BLAG Linux and GNU]], [[Bluewhite64 Linux]], [[Damn Small Linux]], [[Fedora (operating system)|Fedora]], [[GoboLinux]], [[Granular Linux]], [[KateOS]], [[Linux Mint]], [[openSUSE]], [[Pardus (operating system)|Pardus Linux]], [[PCLinuxOS]], [[PuppyLinux]], [[Sabayon Linux]], StartCom Enterprise Linux, [[SUSE Linux]], [[Ubuntu (operating system)|Ubuntu]].

The following [[FreeBSD]] based distributions use Metalink for [[ISO image]] distribution: [[DesktopBSD]], [[MidnightBSD]], and [[TrueOS]].

File based search engine [http://www.findthatfile.com/ findthatfile] provides search for Metalinks.

== Metalink client feature comparison ==
A guide to which clients support which Metalink features:
{| class="wikitable mw-collapsible" style="font-size: 85%; text-align: center;"
!Client
! [[cURL]]
! [[DownThemAll]]
! [[Free Download Manager]]
! [[GetRight]]
! [[KGet]]
! [[Orbit Downloader]]
! [[Phex]]
! [[SmartFTP]]
|-
! Licensing
| [[Free Software]] ([http://curl.haxx.se/docs/copyright.html MIT/X derivate license])
| [[Free Software]] ([[GNU General Public License]])
| [[Free Software]] ([[GNU General Public License]])
| Proprietary / $29.95 for Standard, $49.95 for Pro
| [[Free Software]] ([[GNU General Public License]])
| Proprietary / [[Freeware]]
| [[Free Software]] ([[GNU General Public License]])
| Proprietary / [[Freeware]] for personal, $36.95 commercial
|-

! Interface
| CLI
| GUI
| GUI
| GUI
| GUI
| GUI
| GUI
| GUI
|-

! Metalink 4.0
|
|
|
|
|
|
|
|
|-

! Runs on [[Linux]]
|
|
|
|
|
|
|
|
|-

! Runs on [[Microsoft Windows|Windows]]
|
|
|
|
|
|
|
|
|-

! Runs on [[Mac OS X]]
|
|
|
|
|
|
|
|
|-

! Resume Downloads
|
|
|
|
|
|
|
|
|-

! Segmented Downloads
| ?
|
|
|
|
|
|
|
|-

! Multiple File support
| ?
|
|
|
|
|
|
|
|-

! "location"
| ?
|
|
|
|
|
|
|
|-

! OS
| ?
|
|
|
|
|
|
|
|-

! Language
| ?
|
|
|
|
|
|
|
|-

! HTTP with Transparent Negotiation
| ?
|
|
|
|
|
|
|
|-

! Metalink/HTTP
| ?
|
|
|
|
|
|
|
|-

! HTTP
|
|
|
|
|
|
|
|
|-

! FTP
|
|
|
|
|
|
|
|
|-

! BitTorrent
|
|
|
|
|
|
|
|
|-

! magnet
| ?
|
|
|
|
|
|
|
|-

! MD5 Hashes
|
|
|
|
|
|
| ?
|
|-

! SHA-1 Hashes
|
|
|
|
|
|
|
|
|-

! SHA-256 Hashes
|
|
|
|
|
|
|
|
|-

! Piece Hashes
| ?
|
|
|
|
|
|
|
|-

! PGP Signatures
| ?
|
|
|
|
|
|
|
|-
|}

No clients support [[rsync]] at this time.

== Metalink generation ==
Metalink is formatted as XML so it can be either hand-crafted or created with an XML library, useful for automation. These tools are specifically for Metalink.
* Bouncer is "database driven mirror management app that has three main components - a bounce script, a sentry script to check mirrors and files for integrity and a set of web-based administration tools." Bouncer supports Metalink via a patch that has not yet been integrated upstream.
* Geo McFly ([[GNU General Public License|GPL]]) is an on the fly generator of metalinks based on the downloader's geographical location, written in [[Python (programming language)|Python]].
* Metalink Editor (GPL) is a Cross Platform GUI Metalink editor written in Python which allows you to create and edit Metalinks.
* Metalink Library (GPL) is a Cross Platform Metalink library for Python and PHP based on Metalink Editor with many automatic features and an easy command-line interface to generate Metalink and Torrent files.
* Metalink Generator can also be used to create Metalinks by filling out a Web form.
* Metalink tools (GPL) Unix/Windows command line tools for generating Metalinks with hashes and mirror/p2p lists.
* Dynamic online metalinks based on metadata. Automatically created metalinks based on metadata published by each mirror.
* MirrorManager ([[MIT X11 license]]) is used by the [[Fedora Project]] for dynamically listing mirrors.
* MirrorBrain ([[GNU General Public License|GPL]], [[Apache License]]) is a real-time Metalink generator and download redirector. It can either return Metalinks, or automatically redirect non-Metalink clients (per HTTP redirection) to a mirror server near them. Transparent negotiation of Metalinks, embedding PGP signatures, Metalink/HTTP and other advanced features are supported. It is used for instance by [[openSUSE]] and [[OpenOffice.org]].

== Example Metalink 4.0 .meta4 file ==
Metalink 4.0 files have the extension <tt>.meta4</tt> and are [[Extensible Markup Language|XML]] text files. They are served with the application/metalink4+xml [[Internet media type]].

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<metalink xmlns="urn:ietf:params:xml:ns:metalink">
<published>2009-05-15T12:23:23Z</published>
<file name="example.ext">
<size>14471447</size>
<identity>Example</identity>
<version>1.0</version>
<language>en</language>
<description>
A description of the example file for download.
</description>
<hash type="sha-256">3d6fece8033d146d8611eab4f032df738c8c1283620fd02a1f2bfec6e27d590d</hash>
<url location="de" priority="1">ftp://ftp.example.com/example.ext</url>
<url location="fr" priority="1">http://example.com/example.ext</url>
<metaurl mediatype="torrent" priority="2">http://example.com/example.ext.torrent</metaurl>
</file>
</metalink>
</source>

== Example Metalink/HTTP header fields ==
Metalink in HTTP header fields makes use of existing standard [[List of HTTP header fields|HTTP header fields]] such as [[HTTP ETag|ETags]], Link header fields (for mirrors and P2P), and Instance Digests (for hashes).

<pre>
Etag: "thvDyvhfIqlvFe+A9MYgxAfm1q5="
Link: <http://www2.example.com/example.ext>; rel=duplicate
Link: <ftp://ftp.example.com/example.ext>; rel=duplicate
Link: <http://example.com/example.ext.torrent>; rel=describedby; type="application/x-bittorrent"
Link: <http://example.com/example.ext.meta4>; rel=describedby; type="application/metalink4+xml"
Link: <http://example.com/example.ext.asc>; rel=describedby; type="application/pgp-signature"
Digest: SHA-256=MWVkMWQxYTRiMzk5MDQ0MzI3NGU5NDEyZTk5OWY1ZGFmNzgyZTJlODYzYjRjYzFhOTlmNTQwYzI2M2QwM2U2MQ==
</pre>

== Example Metalink 3.0 .metalink file ==
Metalink 3.0 files have the extension <tt>.metalink</tt> and are [[Extensible Markup Language|XML]] text files.

<source lang="xml">
<?xml version="1.0" encoding="UTF-8"?>
<metalink version="3.0" xmlns="http://www.metalinker.org/">
<files>
<file name="example.ext">
<verification>
<hash type="md5">example-md5-hash</hash>
<hash type="sha1">example-sha1-hash</hash>
<signature type="pgp"/>
</verification>
<resources>
<url type="ftp" location="us" preference="90">ftp://ftp.example.com/example.ext</url>
<url type="ftp" location="uk" preference="90">ftp://ftp.example.net/example.ext</url>
<url type="http" location="us" preference="90">http://example.com/example.ext</url>
<url type="http" location="de" preference="90">http://example.net/example.ext</url>
<url type="bittorrent" preference="100">http://example.org/example.ext.torrent</url>
<url type="rsync"/>
<url type="magnet"/>
<url type="ed2k"/>
</resources>
</file>
</files>
</metalink>
</source>

== See also ==
* [[ed2k URI scheme]]
* [[Magnet URI scheme]]
* [[Named data networking]]

== External links ==
*

==Source==

[http://wikipedia.org/ http://wikipedia.org/]
[[Category:Data storage]]

Metalink - Revision history

Admin: Created page with "'''Metalink''' is an extensible metadata file format that describes one or more computer files available for download. It specifies files appropriate for the u..."