The potential energy surfaces (PESs) and S(1)-T(1) spin-orbit coupling matrix element (SOCME) surfaces are investigated for the toluene-X weakly bound clusters (X=Ne, Ar, and Kr). Calculations of the vibrational wave functions using a one-dimensional stretch model are presented and used to determine vibrationally averaged values of the SOCMEs. Our ab initio theoretical results compare well with intersystem crossing rates derived from recent experimental fluorescence lifetime data [Doyle et al., J. Chem. Phys. 122, 194315 (2005)]. Vibrational averaging is shown to change the absolute magnitude of the calculated SOCMEs, but the ratio between them remains very similar to that of the single-point values calculated at the minima of the PESs.