According to the belief, breasts of Mata Sati fell at Kumari Hills at the bank of Rushikulya River in Ganjam district of Odisha. Hence, the temple is also known as Sthana Peeths or the breast shrine. The pilgrimage is ancient (believed to be in existence since Satya Yuga) and there is no estimation of its actual age.

Mythology of the temple is related to Daksha Yagya and self-immolation of Devi Sati. When Lord Siva knew about it, he became incredibly angry. He carried the corpse of Devi on his shoulders and started Tandava. There was a complete havoc in the universe, and everyone got terrified with the destruction. All gods went to Lord Vishnu and prayed for rescue. Lord Vishnu ordered his Chakra to destroy the body of Devi Sati because Lord Siva was getting furious by looking at her dead body. Pieces of body spread across the country.The idol is very peculiar in this temple. There are two ancient stone statues representing the breasts of Mata Sati. Two brass heads also placed in between these stone deities, known as Chalanti Pratima or a living image.

Many festivals and special occasions get celebrated in the temple. Not only from India, but people from various countries come and get blessings of Adi Mata (The ancient Mother). Chaitra Mela gets organized in the first month of Hindu calender. It is the most important and prestigious event in the shrine. Every Tuesday of Chaitra month, thousands of people gather in the premises. Temple remains open from Monday mid-night to Tuesday afternoon. During normal days, temple remains close in the afternoon, but during Chaitra month it remains closed during night time only. A lot of people take their newborn children and perform their Mundana Samskar in the temple premises. They offer a bunch of hair to the goddess and pray for good health and safety from evil spirits. Tara Tarini temple is one of the important temples among all Shakti Peethas with immense powers.