Let $A$ be a closed semialgebraic subset of euclidean space of codimension at least one, and containing the origin $O$ as a non-isolated point. We prove that, for every real $s\ge 1$, there exists an algebraic set $V$ which approximates $A$ to order $s$ at $O$. The special case $s=1$ generalizes the result of the authors that every semialgebraic cone of codimension at least one is the tangent cone of an algebraic set.