||In 1837 Lewis writes that the flour mill at Castle Matrix ''has been fitted up by the proprietor J. Southwell Brown esq in the most complete manner'' and that the Elizabethan square castle was being repaired. The Ordnance Survey Field Name Book records Castlematrix as a large 2 storey house ''with a new castle 6 stories high adjoining''. John S. Brown held Castle Matrix from Lord Southwell. In the mid 19th century the buildings including the flour mills were valued at £90. In the 1853 sale rental which includes a lithograph, the castle is described as having been repaired and added to "regardless of expense. There are 9 bedrooms in the Castle, besides dressing closets, bathrooms, water closets, a large dining room, drawing room and library with extensive suites of servants' apartments, and the entire fitted up in elegant and substanial style". Castle Matrix was the headquarters of the International Institute of Military History and of the Heraldy Society of Ireland until 1991.