Bailey hits back at non-referral finger-pointing