We theoretically study for the first time the waveguiding properties of an all-solid photonic bandgap fiber with elliptical cladding rods. High birefringence in the order of 10^-3 is easily achievable in such fibers. The contributions of the cladding rod's ellipticity, the lattice ratio of the cladding microstructure as well as the index contrast of two soft glasses to the birefringence are systematically evaluated. The evolutions of birefringence with the structure and material variations show that our highly birefringent fiber design can be tailored