The first castle at Skipton was a Norman stone fortress built on a naturally defensive position by Robert de Romille, one of William the Conqueror's barons. One twelfth century arch is all that remains of this original castle as it was rebuilt in the early fourteenth century by Robert de Clifford in an Edwardian concentric style.

The castle is entered, from the south, through a gatehouse flanked by two drum towers. Above the entrance is the motto of the Clifford family Desormais - 'Henceforth'.

Beyond the gatehouse is the outer bailey which was surrounded by a light curtain wall and a moat on the east, south and west approaches. Within this enclosure stands the now residential castle. This was raised having a very small inner bailey surrounded by a semi-circular curtain wall inset with six large round towers. The northern walls, complete with a surviving garderobe, back onto a 100ft cliff dropping down into a gorge formed by the Haw beck.

During The English Civil War (1642 - 48 ) Skipton Castle was held by the Royalist, Sir John Mallory. With a garrison of three hundred men he held out for three years against a Parliamentary siege finally surrendering in 1645. Oliver Cromwell regarded Skipton Castle as one of the most serious threats to his campaigns in the North and as a result it was slighted, rendering it indefensible. Between 1657 - 58 Lady Anne Clifford was granted permission to repair the castle with the provison that it would no longer be capable of a serious defence. The upper parts of the round towers were rebuilt but with much thinner walls than those below, windows were added to the castle and the outer curtain wall and gatehouse were also repaired.