Do you mean proto-Aksumite or pre-Aksumite? If you mean the latter i guess you are refering to the Punt reliefs, because (oficially called) pre-Aksumite Ethiopia (ca. 800-400 BC, also known as the period of "Dm't"), was massively influenced by ancient Sabaens, which transfered much of their culture to the indigenous Kushite people:
Consider though that this picture here shows no indigenous pre/Aksumite Ethiopian but rather a Sabaen.
After this pre-Aksumite period, the bad known proto-Aksumite period begins, which stretches from after the fall of Dm't until the first mention of Aksum in the first century AD. Indeed, during this time Ethiopia was also under Kushite influence, which can be seen by the fact that proto-Aksumites imported pottery from there.
Considering your assumption about the look of ancient Aksumites: We not only have the coins which depict the Aksumite kings, but also anthropomorphic terracottas, very likely depicting ancient Aksumites in a naturalistic way:
The haircuts of these terracottas also fit with a depiction of a warrior on an Egyptian curtain (See the left guy), which shows a battle between Aksumites and Sassanians during the reign of Chosrau I:
Based on that sources we can say that atleast the core population of the Aksumite kingdom already had the look modern habitants of Eritrea and northeastern Ethiopia has.