Contact: Micah Beck, mbeck@cs.utk.edu,
+1-865-974-0455
LINUX USERS ON INTERNET2 NETWORKS ENJOY THE
BENEFITS OF LOGISTICAL
NETWORKING
enjoying the benefits of a new approach to high
performance content
distribution, called Logistical Networking, which will be
on display
this week in the Internet2 booth at SC2003 in
a research team from the Logistical Computing
and Internetworking (LoCI)
Laboratory at the
combines state-of-the-art data transfer technology
with storage
resources provisioned throughout the network to create
a convenient and
powerful new paradigm for distributed data
management.
To test this technology, the LoCI team has used the 22 terabyte (TB)
testbed of LN "depots" deployed across
Internet2 networks to create an
ad hoc content distribution network for
distributing 650 megabyte (MB)
CD-images (called
"ISO's") of Linux and FreeBSD software. Users are now
employing the Logistical Runtime System (LoRS) tools to download these
ISO's at speeds of 30 to 80 megabits per
second (Mbps)—roughly tens
times faster than from traditional HTTP or FTP
mirror sites. Downloads
for gigabit Ethernet connected users can exceed
150 Mbps.
"We think that this kind of network
storage infrastructure paves the way
for a new era in content distribution,"
said Dr. Micah Beck, Co-Director
of LoCI Laboratory
and the chair of Internet2's Network Storage Special
Interest Group. "For example, although using multiple
copies and
multiple TCP streams to increase transfer speed is
similar to what some
peer-to-peer systems do, with our fixed but shared
infrastructure of
well connected nodes, you can scale up the size
of the content without
sacrificing performance."
What makes this unique combination of
flexibility and performance
possible is an XML encoded metadata file called an exNode. A content
publisher who uploads a file to the testbed of LN depots, which is
called the Logistical Backbone (L-Bone), receives
an exNode containing
metadata that maps the segments of the file's content
to L-Bone storage
allocations, which are time-limited to make them more
shareable. A
single exNode can
represent content that has been fragmented across
multiple depots to accommodate large sizes,
replicated to ensure fault
tolerance, or both replicated and geographically
dispersed to improve
accessibility and performance. A single exNode used to distribute a
Linux ISO represents eight copies of the
ISO's content, which has been
broken up into 8-20MB chunks and spread across
L-Bone depots nationwide.
Publishing content that has been stored in
the L-Bone is as simple as
sharing the exNode that
represents it. Since exNodes are text files,
they can be published via HTTP, sent as e-mail
attachments, or passed on
a floppy disk. When they are posted on a Web
site, as with the exNodes
for Linux ISOs, the
result is called an exNode Distribution Network, or
XDN. To access the content in an XDN, users
simply retrieve the relevant
exNode from the site, and then use them with the LoRS tools to download
the content. The LoRS
tools are freely available and easy to set up,
have a convenient GUI, and run on Microsoft
Windows, Apple OS X, and all
common variants of UNIX and Linux. The LoRS tools make fast,
mulitsource/multistream downloads routine for Internet2 users when
the
content is suitably replicated, as in the Linux XDN,
"As compared to some of the other things
we're doing with Logistical
Networking, like managing the data sets from
supercomputer simulations
or accelerating remote browsing of massive
image databases, putting up
an XDN is a pretty simple application that
anyone on Internet2 can do,"
explains Dr. Beck. "And prototype applications
like IBPvo show that
there are some easy variations on XDN that can
automate different parts
of the process."
IBPvo is an Internet2-enabled personal video
recorder (PVR) service
created to show the flexibility of LN technology.
Like TiVo or ReplayTV,
IBPvo can be scheduled in advance (via a Web
interface) to record
standard television programs. But unlike any other PVRs, the recorded
video is automatically uploaded and replicated in
the L-Bone, and the
user receives an exNode
for the video content in e-mail. Once
the user
has the exNode, the LoRS tools can be used for a high performance,
multisource/multistream download.
Members of the research and education
community who want to set up their
own XDN now have a substantial infrastructure to
work with. Worldwide
there are over 23 TB of storage space available on
more than 300 depots
in 20 countries. A portion of the L-Bone is the
National Logistical
Networking Testbed,
funded by a grant from the National Science
Foundation (NSF) and a donation from Yotta Yotta, a leading Canadian
storage company.
Other depots use the resources of PlanetLab, a
collaborative research infrastructure located at
universities throughout
the
from Intel, has now received additional funding
from the NSF and a
contribution of 30 more nodes from research and
technology collaborator
Hewlett Packard. As with the Internet itself, some L-Bone
resources are
provisioned by the nation's leading colleges and
universities in order
to support research and education
applications. The Department of
Energy (DOE) also provisions some private
Logistical Networking
resources in support of science projects based at the
National Laboratories.
For more information:
Logistical Networking -- http://loci.cs.utk.edu
Linux/FreeBSD XDN -- http://loci.cs.utk.edu/lors
IBPvo -- http://loci.cs.utk.edu/ibp/files/IBPvo_final1.pdf
The Logistical Computing and Internetworking
(LoCI) Laboratory of the
Computer Science Department of the
research on information logistics for distributed
computer systems and
networks. Information logistics studies architectures
and strategies for
the flexible coscheduling
of the physical resources that underpin
computer systems — storage, computation, and data
transmission. Formed
in 2001 with support from UT's Center for
Information Technology
Research, and led by Assoc. Profs. Micah Beck and James
S. Plank of
UT's Computer Science Dept., LoCI Lab has pioneered in the application
of the Internet model of scalable resource
sharing to physical storage,
creating a unified communication infrastructure that
can support
advanced applications not adequately served by the
conventional model of
Internetworking. Its work is funded by grants from the NSF
and the DOE.
---------------------------------------------------------------i2-news-+
For list utilities, archives, subscribe,
unsubscribe, etc. please visit the ListProc web
interface at
http://archives.internet2.edu/
---------------------------------------------------------------i2-news--