This study evaluated the effect of alum sludge as an alternative to fly ash in fabricating geopolymer paste and mortar. The blending of this industrial waste (alum sludge and fly ash) is not only for the benefit of sustainable construction and disposal of industrial waste but also for the reduction of CO2 emissions due to the increasing production of Portland cement from the cement production industry. A laboratory investigation was carried out on the workability and mechanical properties of geopolymer paste and mortar produced with alum sludge replacement in different proportions (0, 10, 20, 30, 40, 50, 60, 70, 80, 90, and 100). A combination of an alkaline solution of sodium silicate and sodium hydroxide of 14 molarity was employed as an activator for the manufacturing of both paste and mortar geopolymer specimens. It was observed from the findings that geopolymer paste and mortar was flowable and workable when alum sludge is replaced for fly ash at higher replacement content. The addition of alum sludge to the mix improved some properties such as density, strength, water absorption, and the elevated temperature behavior. It was observed that the addition of alum sludge was optimum at the 50% replacement level. The addition of alum sludge up to 50% significantly increased the compressive strength of mortar (up to 80% increase in 28 days strength). The compressive strength of the paste and mortar increased with an increase in curing age. Thus, alum sludge and fly ash can be employed together in the production of eco-friendly cementing material for environmental sustainability.