Highly efficient data transmission facility through very long distance high-speed networks

This document is made of the slides of the presentation. For global collaboration as a large scale scientific project, utilization of very high-speed, long distance internet is essential part of the research infrastructure. By the rapid progress of network technology, such as optical fiber and network switches, very high-speed networks are connecting research institutes all over the world and network bandwidth grows and grows. Currently, 10 Gbps network is commonly used and 40 and 100 Gbps network is coming. As the network bandwidth increase, it becomes more difficult to use network efficiently, especially when we use TCP/IP. Furthermore, parallel TCP/IP streams are known to be much more difficult to utilize than single TCP/IP stream. To tackle with these problems, we developed a middle-hardware approach to take care of network streams with wire-rate speed; a programmable hardware framework on which we can realize a network function by developing a firmware for FPGA. We design a middle-hardware-box consists of 5-port network interface and FPGA with large buffer memories, whose firmware can be changed dynamically for the flexibility. Middle-Hardware-Box can be used as a proxy to be settled at the entrance between LAN and WAN, as well as used as a terminal-node to sends and receives packets from and to the network. One of the important features is, it can handle each stream of TCP separately. As the result, all streams between cluster systems can be handled at the gateway, and are controlled by fine-grained scheduling and placing. Since 2001, we have developed 'Data Reservoir System' to transfer huge amount of data between clusters settled in long distant places using parallel TCP streams. For fast data transfer, the important issues are (1) single stream performance is high and (2) performance of each stream is balanced. At the same time, to improve the performance, network environment is very important both as a test environment, that is, artificial network inside laboratory, and as a real environment to be used. We performed long distance network experiments to evaluate the effect of our approach. We used network path from Tokyo to Seattle, Usa with a 10 Gbps WAN PHY link. The results of experiments show that four TCP/IP streams on a 10 Gbps link are ideally balanced, and the utilization factor of the bandwidth is more than 80 percent. (authors)