The development of thermal convection is studied for a viscoplastic fluid. If the viscosity is finite at zero shear rate, the critical Rayleigh number for linear instability takes the value given by a Newtonian fluid with that viscosity. The subsequent weakly nonlinear behaviour depends on the degree of shear thinning: with moderate shear thinning, convective overturning for a given temperature difference is amplified relative to the Newtonian case. If the reduction in viscosity is sufficiently sharp the transition becomes subcritical (the relevant situation for many regularized constitutive laws). For an infinite viscosity at zero shear rate, or a yield-stress, the critical Rayleigh number for linear instability is infinite. Nonlinear convective overturning, however, is still possible; we trace out how the finite-amplitude solution branches develop from their Newtonian counterparts as the yield stress is increased from zero for the Bingham fluid. Laboratory experiments with a layer of Carbopol fluid heated from below confirm that yield strength inhibits convection but a sufficiently strong perturbation can initiate overturning.