A general framework for calculating shape derivatives for optimization problems with partial differential equations as constraints is presented. The proposed technique allows to obtain the shape derivative of the cost without the necessity to involve the shape derivative of the state variable. In fact, the state variable is only required to be Lipschitz continuous with respect to the geometry perturbations. Applications to inverse interface problems, and shape optimization for elliptic systems...