We present a deep generative model that is able to synthesized the movement of a given basketball player conditioning on the trajectory of the ball. For that, we used the freely available NBA SportVU tracking data and a Conditional Variational Autoencoder (CVAE). Similar to previous approaches, we capture the movement of the player and the ball using pictorial representations. The temporal aspects of the trajectories are encoded using ‘fading’. We show that our architecture is able to correctly model the movement of the players, the movement of a given player, and the movement of a given player conditioning on the trajectory of the ball. To the best of our knowledge, this work constitutes one of the first attempts to synthesize the movement of basketball players using a deep generative approach.