fix(poller): only unmark stale if there's activity after marking as stale