The village is densely populated and originates from medieval times. It is built amphitheatrically above the port, at an altitude of 260 metres.

With 270 inhabitants today, Chora is distinguished from the capital of other Cycladic islands due to its numerous arched single-room homes, which occupy a large area of the settlement. Narrow cobbled streets criss-cross the village, leading to the Venetian Kastro (fortress), the initial nucleus and heart of the village. Several churches are scattered throughout Chora, including those of Agios Nikolaos, the Kimissi (Assumption), Agios Charalambos, Christos tou Stavrou (Christ of the Cross), Aghioi Anargyroi and Agios Georgios at the top of the Kastro, all with Byzantine or post-Byzantine icons. Findings from ancient Anafi can be viewed in the Archaeological Collection.