Libraries like this enable differentiable programming, which lets you backprop through more than just neural networks. For instance, people have built a differentiable raytracer and plugged a physics engine into reinforcement learning to accelerate training.

https://en.wikipedia.org/wiki/Differentiable_programming