Small Water is entirely natural and lies at an altitude of 457 m (1,500 ft) in one of the more isolated parts of the Lake District. It is south of the neighbouring and larger tarn Blea Water and separated from it by Piot Crag. It nestles at the base of a circle of crags formed by Harter Fell and Mardale Ill Bell, so that only the eastern end offers anything in the way of a view, back down the Haweswater valley.
A level area of ground to the north of the tarn provides space for wild camping, although it can be quite exposed to winds from the east. The only road access is from the road that runs alongside Haweswater. Walkers can reach the tarn via the footpath from Mardale Head into the Kentmere valley, running over Nan Bield Pass.