Context. RX J1604.3-2130A is a young, dipper-type, variable star in the Upper Scorpius association, suspected to have an inclined inner disk, with respect to its face-on outer disk. Aims. We aim to study the eclipses to constrain the inner disk properties. Methods. We used time-resolved photometry from the Rapid Eye Mount telescope and Kepler 2 data to study the multi-wavelength variability, and archival optical and infrared data to track accretion, rotation, and changes in disk structure. Results. The observations reveal details of the structure and matter transport through the inner disk. The eclipses show 5 d quasi-periodicity, with the phase drifting in time and some periods showing increased/decreased eclipse depth and frequency. Dips are consistent with extinction by slightly processed dust grains in an inclined, irregularly-shaped inner disk locked to the star through two relatively stable accretion structures. The grains are located near the dust sublimation radius (~0.06 au) at the corotation radius, and can explain the shadows observed in the outer disk. The total mass (gas and dust) required to produce the eclipses and shadows is a few % of a Ceres mass. Such an amount of mass is accreted/replenished by accretion in days to weeks, which explains the variability from period to period. Spitzer and WISE infrared variability reveal variations in the dust content in the innermost disk on a timescale of a few years, which is consistent with small imbalances (compared to the stellar accretion rate) in the matter transport from the outer to the inner disk. A decrease in the accretion rate is observed at the times of less eclipsing variability and low mid-IR fluxes, confirming this picture. The v sin i = 16 km s⁻¹ confirms that the star cannot be aligned with the outer disk, but is likely close to equator-on and to be aligned with the inner disk. This anomalous orientation is a challenge for standard theories of protoplanetary disk formation.