Due to their unique physical properties, physically unclonable functions (PUF) have been proposed widely as versatile cryptographic primitives. It is desirable that silicon PUF circuits should be lightweight, i.e., have low-hardware resource requirements. However, it is also of primary importance that such demands of low hardware overhead should not compromise the security aspects of PUF circuits. In this paper, we develop two different mathematical attacks on previously proposed lightweight PUF circuits, namely composite PUF and the multibit output lightweight secure PUF (LSPUF). We show that independence of various components of composite PUF can be used to develop divide and conquer attacks which can be used to determine the responses to unknown challenges. We reduce the complexity of the attack using a machine learning-based modeling analysis. In addition, we elucidate a special property of the output network of LSPUF to show how such feature can be leveraged by an adversary to perform an intelligent model building attack. The theoretical inferences are validated through experimental results. More specifically, proposed attacks on composite PUF are validated using the challenge-response pairs (CRPs) from its field programmable gate array (FPGA) implementation, and attack on LSPUF is validated using the CRPs of both simulated and FPGA implemented LSPUF.