This study aimed to examine the prospective associations between total physical activity, leisure-time physical activity (LTPA), and domestic and work-related physical activity (DWPA) involving heavy physical labor, with all-cause mortality in patients with cardiovascular disease (CVD). A 7-year follow-up cohort design was used based on the data from the Taiwan 2005 National Health Interview Survey, which was linked to the 2005 to 2012 Taiwan National Health Insurance claims data. Multivariable Cox proportional hazard models were utilized to assess the associations between physical activity and all-cause mortality in 2,370 patients with CVD. Participants who achieved a volume of all physical activities of ≥1,000 kcal/week experienced lower risks of all-cause mortality than those who achieved less. Additionally, an inverse relation between LTPA and all-cause mortality was observed. Furthermore, participants who reported exertions (1 to 999 kcal/week) in DWPA had the lowest risk of all-cause mortality. This study provides evidence that patients with CVD who achieve at least a moderate volume of physical activity reduce their mortality risk. It also reveals that both LTPA and DWPA may be important contributors to the reduced risk of premature death among this clinical population.