Processes of electron scattering on fine ionic structure leading to the decay of dipole plasmon oscillations in alkali metal clusters are investigated. The study is based on separation of the collective plasmon oscillations and single particle excitations. Coupling between these two types of electronic motions caused by electron-ion scattering leads to the plasmon damping. The relative contributions of elastic and inelastic electron-ion scattering to plasmon linewidth are considered. The results are compared with experimental data on cluster photoabsorption for sodium clusters of different size.