Transport properties at quantum critical points are strongly constrained by dimensional analysis. They are typically governed by two critical exponents: the dynamical critical exponent z determining the relative scaling of spatial and temporal coordinates and the hyperscaling violating exponent theta. We will show that in general response to electromagnetic fields requires a third exponent, an anomalous dimension for the coupling to background fields. We show that this exponent is generically non-zero in critical points constructed via holography and discuss its potential relevance to the physics of cuprates.