The Númenóreans now make permanent dwellings on the shores of Middle-earth, seeking wealth and dominion; they build many havens and fortresses. The Elf-friends go chiefly to the North-west, but their strongest place is at Pelargir.... The King's Folk establish lordships in Umbar and Harad and in many other places on the coasts of the Great Lands.
-
HoME 12; The Tale of Years of the Second Age
The great cape and land-locked firth of Umbar had been Númenórean land since days of old; but it was a stronghold of the King's Men, who were afterwards called the Black Númenóreans, corrupted by Sauron, and who hated above all the followers of Elendil.
-
RotK; Appendix A
And Sauron gathered to him great strength of his servants out of the east and the south; and among them were not a few of the high race of Númenor. For in the days of the sojourn of Sauron in that land the hearts of well nigh all its people had been turned towards darkness. Therefore many of those who sailed east in that time and made fortresses and dwellings upon the coasts were already bent to his will, and they served him still gladly in Middle-earth. But because of the power of Gil-galad these renegades, lords both mighty and evil, for the most part took up their abodes in the southlands far away
-
Silmarillion, Of the Rings of Power and the Third Age