A significant number of individuals in the United States report a disability that limits their ability to travel, including many people who are blind or visually impaired (BVI). The implications of restricted transportation result in negative impacts related to economic security, physical and mental health, and overall quality of life. Fully autonomous vehicles (FAVs) present a means to mitigate travel barriers for this population by providing new, safe, and independent travel opportunities. However, current policies governing interactions with the artificial intelligence (AI) ‘at the wheel’ of FAVs do not reflect the accessibility needs articulated by BVI people in the extant literature, failing to encourage use cases that would result in life changing mobility. By reviewing the legislative and policy efforts surrounding FAVs, we argue that the heart of this problem is due to a disjointed, laissez-faire approach to FAV accessibility that has yet to actualize the full benefits of this new transportation mode, not only for BVI people, but also for all users. We outline the necessity for a policy framework that guides the design of FAVs to include the concerns of BVI people and then propose legislative and design recommendations aimed to promote enhanced accessibility, transparency, and fairness during FAV travel.