The pin that holds the outer section on is tapered, meaning it will only tap out in one direction. If you look closely one end is slightly larger than the other. They can sometimes be difficult to knock out.