Quantitative validation of nuclear safeguards monitoring systems and their simulations

Nuclear safeguards monitoring systems like those used by the International Atomic Energy Agency (IAEA) are comprised of instruments to detect the misuse of nuclear materials. Attempting to prove that these systems meet their objectives in their intended facilities under operational conditions is too difficult, expensive, and risky. Alternatively, the system may be proved in a laboratory under like conditions, or in a simulation if its quality is known and is acceptable. The main result of this research is a statistical procedure for determining the quality of a safeguards monitoring systems and its simulation. Quality is manifest in two metrics, validity and fidelity, which can be determined quantitatively by administering a validation test. Validity is the demonstrated ability of the system or its simulation to make correct decisions whereas fidelity is the realism of the data used in making the aforementioned decisions. The metrics fit into a Bayesian hypothesis test wherefrom optimal and automated decisions are possible. The technique was successfully tested using real and simulated versions of a laboratory nuclear safeguards monitoring system comprised of an IAEA video camera and Swedish-made boron-trifluoride neutron detector. Validation test data sets were prepared by acquiring data in the real safeguards system about movements of a drum containing a beryllium-plutonium radiation source. The real and simulated automatic evaluation systems were then trained to recognize named movements by providing them recorded data. The validation test was administered by letting the system in question, either real or simulated, attempt to name the events in the validation test data set. Recognition by either real or simulated systems was done by automatically isolating events in the data and comparing them to their respective library data, and then choosing the best match. For images, matching was based on the coordinates of the centroid of pixels resulting from subtraction between successive images. For radiation data, matching was based on the coordinate of the centroid of an event in radiation data. Validity of the system is the ratio of matching successes to the number attempted. Fidelity is the ratio of successful matches where also a particular result was observed from comparing the validation test data to the matched library data. For images, the result sought was a threshold fraction of overlapping pixels that remained after image subtraction. In a radiation event, it was a threshold of the root-mean-square of the differences in frequency of counts-per-second data. In a validation test, validity and fidelity are solved for iteratively because they appear on opposite sides of the Bayesian hypothesis test inequality. Once known, however, they may be used as unchanging parameters in a system that automatically evaluates operational data. Experiments focused on the determination of fidelity changes for improvements in the simulation's radiation model. (author)