Only reschedule refresh if we successfully removed the scheduled one.