We present the results from cross-correlation analysis of weak gravitational lensing and the extra-galactic gamma-ray radiation. The former provides the large scale distribution of dark matter, and the latter possibly includes high energy photons produced by dark matter self-annihilation or decay. We use data from CFHT, SDSS, RCSLens galaxy surveys and the Fermi all-sky survey. We generate a large set of full-sky mock catalogs from cosmological N-body simulations and use them to estimate statistical errors accurately. The measured cross-correlation is consistent with null detection, which is then used to place strong cosmological constraints on annihilating and decaying DM. For leptophilic DM, the constraints are improved by a factor of 100 in the mass range of ~TeV when including contributions from secondary gamma-rays due to the inverse -Compton upscattering of background photons. Annihilation cross sections of 10^{-23} cm3/s are excluded for TeV-scale DM. Lifetimes of 10^25 sec are also excluded for the decaying TeV-scale DM. These are real cosmological constraints, rather than being obtained from local structures.