The biological basis of male-female brain differences has been difficult to elucidate in humans. The most notable morphological difference is size, with male individuals having on average a larger brain than female individuals1,2, but a mechanistic understanding of how this difference arises remains unknown. Here we use brain organoids3 to show that although sex chromosomal complement has no observable effect on neurogenesis, sex steroids-namely androgens-lead to increased proliferation of cortical progenitors and an increased neurogenic pool. Transcriptomic analysis and functional studies demonstrate downstream effects on histone deacetylase activity and the mTOR pathway. Finally, we show that androgens specifically increase the neurogenic output of excitatory neuronal progenitors, whereas inhibitory neuronal progenitors are not increased. These findings reveal a role for androgens in regulating the number of excitatory neurons and represent a step towards understanding the origin of sex-related brain differences in humans.