The present study aimed to evaluate the effect of the inclusion of different olive cakes in the diet of Bísaro pigs on the carcass, meat and fat. The carcasses of 40 animals fed a diet with five treatments (T1—Basic diet and commercial feed; T2—Basic diet + 10% crude olive cake; T3—Basic diet + 10% olive cake, two phases; T4—Basic diet + 10% exhausted olive cake; T5—Basic diet + 10% exhausted olive cake + 1% olive oil) were used to study the effect on carcass traits, physicochemical meat quality and lipid composition of meat and backfat. There were no significant differences between treatments for the conformation measurements performed, except for the length at the seventh and last rib (p < 0.05). The percentage of prime cuts of the carcass in Bísaro pig is within the values indicated by the Portuguese Standard 2931. No significant differences between treatments for body weight, pH and carcass weight were found. The values of ultimate pH (5.7), L* (51–52), b* (11–12) and SF (3.4–4.2) observed confirm a non-exudative and firm meat without quality deviations, such as DFD or PSE. Thus, as a general conclusion, the inclusion of different olive cakes in the diet of Bísaro pigs did not cause any negative consequences on the carcass characteristics and conformation as well as in the meat and lipidic quality. In addition, the inclusion of this olive industry by-product in the animal diet would be an important contribution to solving the problem of the great environmental impact from olive-mill wastewaters from the extractive industries.