Abstract

Many modern tech companies, such as Google, Uber, and Didi, utilize online experiments (also known as A/B testing) to evaluate new policies against existing ones. Analyzing the causal relationship between platform policies and outcomes of interest is of great importance to improve key platform metrics. This study focuses on capturing dynamic treatment effects in complex temporal/spatial experiments and designing informative experiments. We propose a temporal/spatio-temporal varying coefficient decision process (VCDP) model to characterize dynamic treatment effects. Average treatment effects are decomposed into direct and indirect effects (DE and IE) with estimation and inference procedures developed for both. Meanwhile, we establish a framework for calculating conditional quantile treatment effects (CQTE) based on independent characteristics. Notably, we demonstrate that dynamic CQTE equals the sum of individual CQTEs across time under specific model assumptions. Additionally, we propose three optimal allocation strategies for sequential treatments in dynamic settings to minimize variance in treatment effect estimation. Estimation procedures based on off-policy evaluation (OPE) methods are developed. Theoretical properties of the proposed methods are established, including weak convergence, asymptotic power, and optimality of the proposed treatment allocation design. Extensive simulations and real data analyses support the usefulness of the proposed methods.

Speaker Intro

Hongtu Zhu is a tenured professor of biostatistics, statistics, computer science, and genetics at University of North Carolina at Chapel Hill. He was DiDi Fellow and Chief Scientist of Statistics at DiDi Chuxing between 2018 and 2020 and was Endowed Bao-Shan Jing Professorship in Diagnostic Imaging at MD Anderson Cancer Center between 2016 and 2018. He is an internationally recognized expert in statistical learning, medical image analysis, precision medicine, biostatistics, artificial intelligence, and big data analytics. He has been an elected Fellow of American Statistical Association and Institute of Mathematical Statistics since 2011. He received an established investigator award from Cancer Prevention Research Institute of Texas in 2016 and received the INFORMS Daniel H. Wagner Prize for Excellence in Operations Research Practice in 2019. He has published more than 320+ papers in top journals including Nature, Science, Cell, Nature Genetics, PNAS, AOS, JASA, and JRSSB, as well as 55+ conference papers in top conferences including NeurIPS, AAAI, KDD, ICDM, MICCAI, and IPMI.