While castles may have protected those living within from the elements, by no means were they a place of warmth. Castles were built using stone as a purpose to keep enemies out, not necessarily for comfort. 

With the stone providing no insulation, castles kept the cold in, which was especially problematic in a region that is known to be cold and wet. In addition, the windows were so small that they let in minimal sunlight, and those were only located in certain parts of the castle. Many of the rooms in the castle had no windows at all and were more or less an icebox.

https://www.healthygem.com/health/what-it-was-like-to-live-in-a-medieval-castle/12/?utm_campaign=svn-d-ww-c-0-0-230223-hg-ob-d1&utm_source=ob&utm_medium=00b7509d7288e631e0dcc91d879c4f94c0-ob&utm_content=00b3da2fa8c0d756fdd696c3a26f7d231e&utm_cpc=0.010&dicbo=v4-A3pcWBe-1077451370&safari=1

https://edition.cnn.com/style/article/beethoven-hair-study-scn/index.html