Massive data transfer between supercomputers and grid clusters using SCP controlled by FTS

Data transfers between Supercomputing centers and Grid sites may be complicated since, in general, Supercomputing centers are not completely incorporated in the Grid environment. In particular, the Port d'Informació Científica (PIC) had to transfer data of cosmological simulations from the Barcelona Supercomputing Center (BSC) to Grid storage resources in order to post-process those data at PIC. The data connection is limited to the Secure Shell (SSH) protocol using the scp command. Besides a limit of bandwidth, we had to deal with a lack of automation and reliability of the transfers. PIC managed to control those scp transfers through gLite's File Transfer Service (FTS). Therefore, the discs containing the data at the BSC are mounted via SSH to a local Storage Resource Management (SRM) server at PIC. Once established the access to the data through the SRM server, the data can be transferred to any other SRM server, including the storage system at PIC, using grid protocols. Although we cannot improve the bandwidth, we reach a level of automation and reliability comparable to transfers from the Tier-0 to Tier-1 sites of the Large Hadron Collider (LHC) experiments.