Records say that both the towers, the gopurams were constructed by Muluthukanda Rama Pandiya, whereas King Nindraseer Nedumaran, who governed the area during the seventh century, erected the remaining significant elements of the shrine. He built the mani mandap, when he was presiding over the area in the seventh century itself. Earlier, the shrines of Kanthimathi as well as that of Nellaiappar were actually two separate structures, and there was some distance between them. However, two shrines were linked or connected to each other, by the help of a chain mandap, in the year 1647, by Vadamalaiappa Pillaiyan, who was an ardent follower of Lord Shiva. One can see a flower garden, which was developed by Thiruvengadakrishna Mudaliar that is located in the western part of the chain mandap, and was constructed in the year 1756. There is a square shaped vasantha mandap, held together by hundred beams, which can be found in the middle of the flower garden. The Nandi Mandap is said to have erecte by Sivanthiappa Nayak, and the flagpole that is located at a close distance from it was built in the year 1155. One is able to view in all their beauty, many inscriptions that are made of stone, inside the shrine. The most well known out of the many are those of Rajendran I and Kulothonha Chola I. Kulasekkara Pandiyan's inscriptions tell us that he won the battles that he fought against the kings that belonged to the Chola, Hoysala and the Chera dynasties. He was the one who erected the outer walls of the shrine, using the money that he acquired when he won the wars.