Channel state information (CSI) plays a vital role in wireless communication systems. However, the CSI acquisition overhead is an enormous obstacle to realize the system performance improvements promised by massive connectivity and massive multiple-input multiple-output (MIMO). To alleviate this overhead, this paper proposes a remote channel inference framework by probing the channels occupied by a source base station (BS) and inferring the channels of target BSs at geographically separated sites. The work generalizes existing literature which mainly focuses on utilizing CSI linear correlations of adjacent antennas, by adopting a model-free deep learning framework to investigate non-linear remote CSI correlations. The existence of such cross-BS CSI correlations is first shown by calculating the mutual information between remote channels, and the Cram��r-Rao lower bound of remote CSI inference performance based on a one-ring channel model. Inspired by this finding, modern deep learning approaches are leveraged to perform remote channel inference in heterogeneous networks for both single user and multi-user scenarios. Simulation results based on ray tracing data show evident performance advantages over conventional methods, under both homogeneous and heterogeneous frequency coverage. The proposed framework achieves beamformer inference accuracy within 4.6% of the genie-aided optimum at the cost of sweeping only two beams.