Freedman's Paradox: an Info-Metrics Perspective

In linear regression models where there are no relationships between the dependent variable and each of the potential explanatory variables – a usual scenario in real-world problems – some of them can be identified as relevant by standard statistical procedures. This incorrect identification is usually known as Freedman's paradox. To avoid this disturbing effect in regression analysis, an info-metrics approach based on normalized entropy is discussed and illustrated in this work. The results suggest that normalized entropy is a powerful alternative to traditional statistical methodologies currently used by practitioners.