Abstract We present an unsupervised search for outliers in the Bright Galaxy Survey (BGS) data set from the DESI Early Data Release. This analysis utilizes an autoencoder to compress galaxy spectra into a compact, redshift-invariant latent space, and a normalizing flow to identify low-probability objects. The most prominent outliers show distinctive spectral features, such as irregular or double-peaked emission lines or originate from galaxy mergers, blended sources, and rare quasar types, including one previously unknown broad absorption line system. A significant portion of the BGS outliers are stars spectroscopically misclassified as galaxies. By building our own star model trained on spectra from the DESI Milky Way Survey, we have determined that the misclassification likely stems from the principle component analysis of stars in the DESI pipeline. To aid follow-up studies, we make the full probability catalog of all BGS objects and our pretrained models publicly available.