We determine experimentally and using first-principles theory the evolution of phonons and the relaxation of photo-excited carriers on picosecond timescales across the Brillouin zone of GaAs by electron–phonon scattering. We simulate the time-evolution of phonon populations, based on first-principles band structure and electron–phonon matrix elements, and compare them to data from time-resolved x-ray diffuse scattering experiments, performed at the Pohang Accelerator Laboratory X-ray Free Electron Laser (PAL-XFEL) facility, following photo-excitation by a 50 fs near-infrared optical pulse. We show that the intensity of the non-thermal x-ray diffuse scattering signal, which is observed to grow substantially at the L-point of the Brillouin zone over several ps, is due to phonons generated by scattering of carriers between the X and L valleys.  Besides phonon generation, we observe regions of strong phonon absorption due to the cooling of the electron plasma and strong electron-polar-phonon interaction.