As one of the hypothetical principles in the Standard Model (SM), lepton flavor universality (LFU) should be tested with a precision as high as possible such that the physics violating this principle can be fully examined. The run of $Z$ factory at a future $e^+e^-$ collider such as CEPC or FCC-$ee$ provides a great opportunity to perform this task because of the large statistics and high reconstruction efficiencies for $b$-hadrons at $Z$ pole. In this paper, we present a systematic study on the LFU test in the future $Z$ factories. The goal is three-fold. Firstly, we study the sensitivities of measuring the LFU-violating observables of $b\to c \tau \nu$, {\it i.e.}, $R_{J/\psi}$, $R_{D_s}$, $R_{D_s^\ast}$ and $R_{\Lambda_c}$, where $\tau$ decays muonically. For this purpose, we develop the strategies for event reconstruction, based on the track information significantly. Secondly, we explore the sensitivity robustness against detector performance and its potential improvement with the message of event shape or beyond the $b$-hadron decays. A picture is drawn on the variation of analysis sensitivities with the detector tracking resolution and soft photon detectability, and the impact of Fox-Wolfram moments is studied on the measurement of relevant flavor events. Finally, we interpret the projected sensitivities in the SM effective field theory, by combining the LFU tests of $b\to c \tau \nu$ and the measurements of $b\to s \tau^+\tau^-$ and $b\to s \bar{\nu} \nu$. We show that the limits on the LFU-violating energy scale can be pushed up to $\sim \mathcal{O} (10)$~TeV for $\lesssim \mathcal O(1)$ Wilson coefficients at Tera-$Z$.