Deploying perfSONAR-base End-2-End monitoring for production US CMS networking

Fermilab is the US Tier-1 Center for CMS data storage and analysis. End-2-End (E2E) circuits are utilized to support high impact data movement into and out of the Tier-1 Center. E2E circuits have been implemented to facilitate the movement of raw experiment data from the Tier-0 Center at CERN, as well as processed data to a number of the US Tier-2 sites. Troubleshooting and monitoring of those circuits presents a significant challenge, since the circuits typically cross multiple research and education networks, each with its own management domain and customized monitoring capabilities. The perfSONAR Monitoring Project was established to facilitate development and deployment of a common monitoring infrastructure across multiple network management domains. Fermilab has deployed perfSONAR across its E2E circuit infrastructure and enhanced the product with several tools that ease the monitoring and management of those circuits. This paper will present the current state of perfSONAR monitoring at Fermilab and detail our experiences using perfSONAR to manage our current E2E circuit infrastructure. We will describe how production network circuits are monitored by perfSONAR E2E Monitoring Points (MPs), and the benefits it has brought to production US CMS networking support