In this talk, I discuss the Hamiltonian formulation of lattice gauge theory using the uniform matrix product state and its application to the single flavor Schwinger model. We perform simulations based on the variational uniform matrix product state (VUMPS) algorithm with a gauge-invariant matrix product ansatz that locally enforces the Gauss law constraint. Both the continuum and lattice versions of the Schwinger model with θ = π are known to exhibit first-order phase transitions for fermion masses above a critical value, at which a second-order phase transition occurs. Our algorithm enables a precise determination of the critical point in the continuum theory.