Every Viking raider was a warrior. Most of the saxon fighting force was composed of the fyrd, which is all men capable of bearing arms. Not owning arms or proficient of their usage, but capable of bearing arms.
P.S. You cannot compare the average person of the Dark age with a modern person. You eat enough, are disease- and parasite-free. This alone gives you an advantage over them. The Norsemen enjoyed a better diet, out of necessity(grain does not grow on rocks) and they were better warriors due to their culture.