Thin crystalline layers of TaAs Weyl semimetal are grown by molecular beam epitaxy on GaAs(001) substrates. The (001) planes of the tetragonal TaAs lattice are parallel to the GaAs(001) substrate, but the corresponding in-plane crystallographic directions of the substrate and the layer are rotated by 45 degrees. In spite of a substantial lattice mismatch (about 19%) between the GaAs(001) substrate and TaAs epilayer, no misfit dislocations are observed at the GaAs(001)/TaAs(001) interface. Only stacking fault defects in TaAs are detected by transmission electron microscopy. Thorough X-ray diffraction measurements and analysis of the in situ reflection high-energy electron diffraction images indicate that TaAs layers are fully relaxed already at the initial deposition stage. Atomic force microscopy imaging reveals the columnar structure of the layers, with lateral (parallel to the layer's surface) columns about 20 nm wide and 200 nm long. Both X-ray diffraction and transmission electron microscopy measurements indicate that the columns share the same orientation and crystalline structure.