Diffusion and mobility are essential for cellular functions, as molecules are usually distributed throughout the cell and have to meet to interact and perform their function. This also involves the cytosolic migration of cellular organelles. However, observing such diffusion and interaction dynamics is challenging due to the high spatial and temporal resolution required and the accurate analysis of the diffusional tracks. The latter is especially important when identifying anomalous diffusion events, such as directed motions, which are often rare. Here, we investigate the migration modes of peroxisome organelles in the cytosol of living cells. Peroxisomes predominantly migrate randomly, but occasionally they bind to the cell’s microtubular network and perform directed migration, which is difficult to quantify, and so far, accurate analysis of switching between these migration modes is missing. We set out to solve this limitation by experiments and analysis with high statistical accuracy. Specifically, we collect temporal diffusion tracks of thousands of individual peroxisomes in the HEK 293 cell line using two-dimensional spinning disc fluorescence microscopy at a high acquisition rate of 10 frames/s. We use a Hidden Markov Model with two hidden states to (1) automatically identify directed migration segments of the tracks and (2) quantify the migration properties for comparison between states and between different experimental conditions. Comparing different cellular conditions, we show that the knockout of the peroxisomal membrane protein PEX14 leads to a decrease in the directed movement due to a lowered binding probability to the microtubule. However, it does not eradicate binding, highlighting further microtubule-binding mechanisms of peroxisomes than via PEX14. In contrast, structural changes of the microtubular network explain perceived eradication of directed movement by disassembly of microtubules by Nocodazole-treatment.