The history of Rajpuri Caves is intertwined with local legends and religious beliefs. While the exact origins of the caves are unclear, they are believed to have been used as a place of worship for centuries. The presiding deity is Lord Kartikeya, also known as Murugan in South India. Legend has it that Lord Kartikeya used these caves during his exile. The water kunds within the caves are believed to have been created by Lord Kartikeya's divine powers. Over time, the caves have become a significant pilgrimage site for Hindus, particularly those from Maharashtra and nearby regions. The local community has played a vital role in maintaining the temple and preserving the sanctity of the caves. The caves represent a blend of natural formations and human faith, creating a unique cultural and historical landmark.