Rubber-based conductive composites are widely used in sensors, wearable electronic devices and electromagnetic fields. In this work, by using the two-roll milling and hot-pressing process, chopped glass fiber (CGF) and graphene (Gr) as additives, and acrtlinitrile-brtadiene rubber (NBR) as the matrix, a series of anisotropic flexible rubber-based composites were prepared. Using this preparation method, both CGF and Gr additives were directly arranged in the material. When the content of CGF was 1 wt.%, the tensile strength in both the T and W directions of the material reached 27 MPa and 28 MPa, respectively. When the content of CGF was fixed at 1 wt.% and Gr was 1.5 wt.% and the elongation at break in both directions reached 328% and 347%. By focusing on the comparison of the dielectric differences in the T and W directions in the X band, it was found that the directional arrangement of the additives led to differences in the dielectric properties.