The user interface (UI) is a primary source of interaction with a device. Since the introduction of graphical user interface (GUI), software engineers and designers have been trying to make user-friendly UIs for various computing devices, including smartphones, tablets, and computers. The modern smart TV also comes with built-in operating systems. However, little attention has been given to this prominent entertainment device, i.e., smart TV. The technological advancement and proliferation of smart TV enabled the manufacturer to provide rich functionalities and features; however, this richness resulted in more clutter and attention-demanding interfaces. Besides, smart TV is a lean-back supporting device having a diverse range of users. Therefore, smart TV’s usability and user experience (UX) are questionable due to diverse user interests and limited features of traditional remote controls. This study aimed to discuss and critically analyze the features and functionalities of the existing well-known smart TV UIs of various operating systems in the context of usability, cognition, and UX. Moreover, this study highlights the issues and challenges in the current smart TV UIs and recommends some research opportunities to cope with the smart TV UIs. This study further reports and validates some overlooked factors affecting smart TV UIs and UX. A subjective study and usability tests from diverse users are presented to validate these factors. The study concludes that a one-size-fits-all UI design is unsuitable for shared devices, i.e., smart TV. This study further recommends a personalized adaptive UI, which may enhance the learnability and UXs of the smart TV viewers.