Antibodies-recognising peptides bound to the major histocompatibility complex (pMHC) represent potentially valuable and promising targets for chimeric antigen receptor (CAR) T cells to treat patients with cancer. Here, a human phage-Fab library has been selected using HLA-A2 complexed with a heteroclitic peptide variant from an epitope shared among multiple melanoma-associated antigens (MAGEs). DNA restriction analyses and phage ELISAs confirmed selection of unique antibody clones that specifically bind to HLA-A2 complexes or HLA-A2-positive target cells loaded with native or heteroclitic peptide. Antibodies selected against heteroclitic peptide, in contrast to native peptide, demonstrated significantly lower to even negligible binding towards native peptide or tumour cells that naturally expressed peptides. The binding to native peptide was not rescued by phage panning with antigen-positive tumour cells. Importantly, when antibodies directed against heteroclitic peptides were engineered into CARs and expressed by T cells, binding to native peptides and tumour cells was minimal to absent. In short, TCR-like antibodies, when isolated from a human Fab phage library using heteroclitic peptide, fail to recognise its native peptide. We therefore argue that peptide modifications to improve antibody selections should be performed with caution as resulting antibodies, either used directly or as CARs, may lose activity towards endogenously presented tumour epitopes.