|
|
MDis - Multicast Distribution ServiceWeather forecasting is a data-intensive process. It results in models that can be a significant amount of data. These models need to be recomputed periodically, e.g. every hour. Many weather related applications need the detailed models or significant amounts of other information generated from them. Most national weather services have a centralized site for computing weather models. Service operations are, however, generally distributed to multiple sites all over the country. These sites need many of the data generated at the central site, calling for a distribution service. Currently, most weather services use a home-grown distribution service to make the weather data available to their service sites. This typically means that multiple file transfer connections are made from the central site to each of the service sites. The load both on the central site and the network can be tremendous. Sending results separately to each site also incurs delays that can reduce the timeliness of the data. Multicasting, the simultaneous transfer of data to multiple recipients, is becoming one of the key technologies for content distribution. Deployed today in local area and many well-managed corporate networks, it is becoming available in larger parts of the Internet -- enabling efficient distribution of information to millions of recipients. So far, multicast has mainly been used to distribute audio and video streams, where small amounts of packet loss do not present a problem. It is significantly more difficult to reliably transfer data based on the best-effort IP multicast service. This makes it hard to use exploit the benefits of multicasting -- significantly reducing network bandwidth requirements, minimizing transfer delay and reducing the load on the transmitting servers -- for data distribution applications. The goal of the MDiS project is to provide a multicast-enabled platform for distribution services. The core of the platform is MTP/SO, a proven multicast transport protocol developed by Tellique, Berlin, based on the Internet RFC 1301. In MDiS, MTP/SO plays the role for multicast content distribution that TCP has as the foundation of the unicast FTP file transfer service. MTP/SO provides a socket-style programming interface, which facilitates its use by many applications. In the MDiS project, the MTP/SO protocol is complemented by an adaptation layer for the most frequent applications, data transfer. This allows building applications that use the adaptation layer for multicast data transfer just as they would use FTP for unicast transfer. As a demonstrator for this adaptation layer, an application will be built that can replicate entire file systems to as many target systems as desired. The pilot user for the MDiS project will be the German Weather Service (Deutscher Wetterdienst, DWD). DWD needs to distribute 12 GBytes per hour of model data to its main branch offices. In addition, smaller data sets need to be distributed to around 50 more branch offices. To this end, the Multicast Distribution Service developed by MDiS will be integrated into the central DWD IT infrastructure. To verify that the MDiS infrastructure can also be used in more complex networks than the internal network of the DWD, a pilot installation will also be made in the German Research Network (Wissenschaftsnetz, WiN). After the project is completed, the DFN will retain rights to use the results of the project within the DFN network. MDiS is supported by the German Federal Ministry of Education and Research (BMBF) through the Development Program of the German Research Network (DFN). The MDiS project partners are:
In summary, this layer model allows using reliable multicast at a number of different levels (in addition to the specific requirements of the DWD):
|
|
|
|
|
|
|
