RBI Governor wants to withdraw nominee from bank boards