摘要：It has been shown that the deployment of device-to-device(D2D) communication in cellular systems can provide better support for local services. However, improper design of the hybrid system may cause severe interference between cellular and D2D links. In this paper, we consider transceiver design for the system employing multiple antennas to mitigate the interference. The precoder and decoder matrices are optimized in terms of sum mean squared error(MSE) and capacity, respectively. For the MSE minimization problem, we present an alternative transceiver optimization algorithm. While for the non-convex capacity maximization problem, we decompose the primal problem into a sequence of standard convex quadratic programs for efficient optimization. The evaluation of our proposed algorithms for performance enhancement of the entire D2D integrated cellular system is carried out through simulations.