Associations of height and head circumference with IQ are well documented, but much less is known about the association of IQ with other anthropometric measures or the mechanisms behind these associations. We therefore analyzed the associations between IQ and several anthropometric measures using a twin-study design. Twins born in Minnesota were assessed at either age 11 (756 complete pairs) or 17 (626 complete pairs) and analyzed using genetic modeling. Head circumference and height showed the most consistent positive associations with IQ, whereas more detailed anthropometric measures were not significantly better predictors of IQ. These associations were mainly due to common genetic factors. Our results suggest that the same genetic factors have an effect on physical and cognitive development. Head circumference and height capture information on children’s physical development, which is partly associated also with cognitive development.