Testing product changes with network effects

Experimentation is ubiquitous in online services such as Facebook, where the effects of product changes are explicitly tested and analyzed in randomized trials. Interference, sometimes referred to as network effects in the context of online social networks, is a threat to the validity of these randomized trials as the presence of interference violates the stable unit treatment value assumption (SUTVA) important to the analysis of these experiments.

Colloquially, interference means that an experimental unit’s response to an intervention depends not just on its own treatment, but also on other units’ treatments. For example, consider a food delivery marketplace that tests a treatment that causes users to order deliveries faster. This could reduce the supply of delivery drivers to users in the control group, leading the experimenter to overstate the effects of the treatment.

Read more: https://research.facebook.com/blog/2021/8/testing-product-changes-with-network-effects/