We have investigated the non-centrosymmetric tetragonal heavy-fermion compound CeAuAl3 using muon spin rotation (muSR), neutron diffraction (ND) and inelastic neutron scattering (INS) measurements. We have also revisited the magnetic, transport and thermal properties. The magnetic susceptibility reveals an antiferromagnetic transition at 1.1 K with a possibility of another magnetic transition near 0.18 K. The heat capacity shows a sharp lambda-type anomaly at 1.1 K in zero-filed, which broadens and moves to higher temperature in applied magnetic field. Our zero-field muSR and ND measurements confirm the existence of a long-range magnetic ground state below 1.2 K. Further the ND study reveals an incommensurate magnetic ordering with a magnetic propagation vector k = (0, 0, 0.52) and a spiral structure of Ce moments coupled ferromagnetically within the ab-plane. Our INS study reveals the presence of two well-defined crystal electric field (CEF) excitations at 5.1 meV and 24.6 meV in the paramagnetic phase of CeAuAl3 which can be explained on the basis of the CEF theory. Furthermore, low energy quasi-elastic excitations show a Gaussian line shape below 30 K compared to a Lorentzian line shape above 30 K, indicating a slowdown of spin fluctuation below 30 K. We have estimated a Kondo temperature of TK=3.5 K from the quasi-elastic linewidth, which is in good agreement with that estimated from the heat capacity. This study also indicates the absence of any CEF-phonon coupling unlike that observed in isostructural CeCuAl3. The CEF parameters, energy level scheme and their wave functions obtained from the analysis of INS data explain satisfactorily the single crystal susceptibility in the presence of two-ion anisotropic exchange interaction in CeAuAl3. ; Comment: 28 pages and 17 figures