Extended X-ray absorption fine structure (EXAFS) experiments at the Zr K-edge were carried out on perovskite relaxor ferroelectrics BaTi(1-x)Zr(x)O3 (BTZ) (x = 0.25, 0.30, 0.35), and on BaZrO3 for comparison. Structural information up to 4.5 A around the Zr atoms is obtained, revealing that the local structure differs notably from the average Pm-3m cubic structure deduced from X-ray diffraction. In particular, our results show that the distance between Zr atoms and their first oxygen neighbors is independent of the Zr substitution rate x and equal to that measured in BaZrO3, while the X-ray cubic cell parameter increases linearly with x. Furthermore, we show that the Zr atoms tend to segregate in Zr-rich regions. We propose that the relaxor behavior in BTZ is linked to random elastic fields generated by this particular chemical arrangement, rather than to random electric fields as is the case in most relaxors.