The circuit mechanisms that give rise to direction selectivity in the retina have been studied extensively but how direction selectivity is established in retinorecipient areas of the brain is less well understood. Using functional imaging in larval zebrafish we examine how the direction of motion is encoded by populations of neurons at three layers of the optic tectum; retinal ganglion cell axons (RGCs), a layer of superficial inhibitory interneurons (SINs), and periventricular neurons (PVNs), which constitute the majority of neurons in the tectum. We show that the representation of motion direction is transformed at each layer. At the level of RGCs and SINs the direction of motion is encoded by three direction-selective (DS) subtypes tuned to upward, downward, and caudal-to-rostral motion. However, the tuning of SINs is significantly narrower and this leads to a conspicuous gap in the representation of motion in the rostral-to-caudal direction at the level of SINs. Consistent with previous findings we demonstrate that, at the level of PVNs the direction of motion is encoded by four DS cell types which include an additional DS PVN cell type tuned to rostral-to-caudal motion. Strikingly, the tuning profile of this emergent cell type overlaps with the gap in the representation of rostral-to-caudal motion at the level of SINs. Using our functional imaging data we constructed a simple computational model that demonstrates how the emergent population of PVNs is generated by the interactions of cells at each layer of the tectal network. The model predicts that PVNs tuned to rostral-to-caudal motion can be generated via convergence of DS RGCs tuned to upward and downward motion and feedforward tuned inhibition via SINs which suppresses responses to non-preferred directions. Thus, by reshaping directional tuning that is inherited from the retina inhibitory inputs from SINs can generate a novel subtype of DS PVN and in so doing enhance the encoding of directional stimuli.