Motor-cargo recruitment to microtubules is often the rate-limiting step of intracellular transport, and defects in this recruitment can cause neurodegenerative disease. Here, we use in vitro reconstitution assays with single-molecule resolution, live-cell transport assays in primary neurons, computational image analysis, and computer simulations to investigate the factors regulating retrograde transport initiation in the distal axon. We find that phosphorylation of the cytoskeletal-organelle linker protein CLIP-170 and post-translational modifications of the microtubule track combine to precisely control the initiation of retrograde transport. Computer simulations of organelle dynamics in the distal axon indicate that while CLIP-170 primarily regulates the time to microtubule encounter, the tyrosination state of the microtubule lattice regulates the likelihood of binding. These mechanisms interact to control transport initiation in the axon in a manner sensitive to the specialized cytoskeletal architecture of the neuron.