Louis Walsh has insisted he cannot be axed from 'The X Factor' because his contract is "watertight", despite recent claims.