Skenfrith Castle may have been first established in c.1070 by William Fitz Osbern.

However, the earliest documented reference to the Castle was in the 12th century, at which point it was part of the territory of the three castles acquired by King Stephen. During this time the first stone fortifications were built, possibly by Ralph Grosmont, who was granted the Lordship of the Three Castles in 1187. This was shortly followed by the construction of a palisade around the partially built stone castle and ditch in 1193, and the establishment of a wharf and slipway.

There is no known evidence for any settlement prior to the Roman period. Excavations at the castle have found evidence of Roman ironworking and a Roman coin was uncovered at Malthouse Farm. It is uncertain whether occupation continued in the area following the Roman period, some accounts suggest that the church was founded prior to the Norman conquest, however, no archaeological evidence to corroborate this has yet been found.