‘Difficult’ Brown was ‘maddening’: Blair