What's the Deal With Staring?

Gazing too long upon another person is almost universally viewed as anywhere from impolite to hostile, which is odd considering science isn't fully certain why we stare - and why we're so good at knowing when we're being stared at.