The preventive effect of late-life physical exercise on cognitive deterioration has been reported in many cohort studies. However, the effect of exercise, independent of other cognitively demanding and social activities, is equivocal and little is known about the relative contributions of frequency, intensity, and duration of exercise. This study aimed to examine the relationships of exercise and its underlying components with cognitive function and rate of cognitive change over an 8-year period in a nationally representative sample of older Taiwanese. Data from the 1999, 2003, and 2007 phases of the nationwide longitudinal survey were used. Data from a fixed cohort of 1,268 participants aged 70 years or older in 1999 with 8 years of follow-up were analyzed. Cognitive function was assessed using the Short Portable Mental Status Questionnaire. Self-reported frequency, intensity, and duration of exercise were collected. A generalized estimating equation with multivariate adjustment for sociodemographic variables, cognitive and social leisure activities, lifestyle behaviors, and health status was calculated. Participants who were physically active during leisure time had better subsequent cognitive function (incident rate ratios [IRR] = 0.63; 95 % CI, 0.54-0.75) and a slower rate of cognitive decline (p = 0.01). Among the components of exercise, only duration emerged as a predictor of cognitive function (p = 0.01). Older adults engaging in exercise for at least 30 min or more per session are likely to reduce the risk of subsequent cognitive decline. This research supports the case for physical exercise programs for older adults in order to help prevent loss of cognitive function.