Historians of analytic philosophy have increasingly questioned the historical adequacy of standard interpretations of canonical texts in analytic philosophy. This has led to a reassessment of Frege’s foundational status in analytic philosophy of language. This project contributes to this growing trend by examining Frege’s views on language and formalization through his polemic with mathematical formalism, revealing a distinctly Fregean ‘philosophy of the sign’ that clarifies key interpretative puzzles in his work and sheds new light on contemporary debates in the philosophy of language. Though Frege is widely known for his formalization of logic, his lifelong polemic with mathematical formalism—targeting figures like Hankel, Heine, Thomae, and Hilbert—is largely overlooked. This neglect has left the nature and philosophical status of that formalization unclear. As a result, existing scholarship on Frege's polemic with formalism is fragmented, poorly understood, and insufficiently integrated into Frege's broader philosophy. First, this project examines how Frege’s critique of formalism is unified by an overlooked philosophy of the sign, encapsulated in his distinction between sign and figure. Second, it investigates how this critique and its underlying sign-conception bring to light the unique ontological status of signs both within Frege’s logical framework and in contemporary debates on quotation.