Virtual round robin – A new opportunity to study NDT reliability

Highlights: • Virtual flaws enable new kind of round robin focused on data evaluation. • Virtual flaws enable POD evaluation for ultrasonic inspection for nuclear industry. • The applied inspection strategy has significant effect on the attained POD. • Inspectors sometimes miss large cracks, even when they find smaller cracks. Round robin exercises have traditionally been difficult to arrange in the field of non-destructive testing (NDT). To create a representative round robin exercise, representative mock-ups with representative flaws are needed. The mock-ups are costly and transporting them around the world to facilitate testing by numerous laboratories is difficult. The few round robins that have been completed have often contributed significantly to our understanding on the capability of the used NDT methods and procedures. Recently, the increased use of automated inspections together with the development of virtual flaws (independently by Trueflaw and EPRI) has enabled a new type of round robin, where instead of moving samples around the world, the round robin is focused on the data analysis and only pre-acquired data files are distributed. This makes conducting a round robin much more cost-effective both in terms of arrangement and in terms of inspection effort from the participating companies. In addition, the virtual flaw technology allows unprecedented number and variety of flaws to be included. With high number of flaws included, the results present statistically meaningful sample and can be further analyzed to estimate the probability of detection (POD) with standard statistical tools. In connection with the international project "PIONIC", such a virtual round robin was arranged for the first time. The exercise showed, that virtual flaws and virtual round robins can be used to extract important information about NDT reliability and performance. Also, some points of development were identified for further studies: the sizing and detection files should be better optimized for their respective uses and the data could be further obfuscated to avoid any possibility of inspectors learning to recognize repeating signal patterns. 12 inspectors submitted results to the virtual round robin. The results showed a_90/95 ranging from 1.2 to 7.0 mm – a significant variation in performance. The difference was mainly attributed to different inspection strategies. In addition, an unexplained tendency to miss big cracks was noted on some result sets. One of the data files did not contain any flaws. None of the inspectors correctly identified the file as flawless.