Spectral Efficiency Maximization for mmWave MIMO-Aided Integrated Sensing and Communication Under Practical Constraints

A hybrid transmit precoder (TPC) and receive combiner (RC) pair is conceived for millimeter wave (mmWave) multiple input multiple output (MIMO) integrated sensing and communication (ISAC) systems. The proposed design considers a practical mean squared error (MSE) constraint between the desired and the achieved beampatterns constructed for identifying radar targets (RTs). To achieve optimal performance, we formulate an optimization problem relying on sum spectral efficiency (SE) maximization of the communication users (CUs), while satisfying certain radar beampattern similarity (RBPS), total transmit power, and constant modulus constraints, where the latter are attributed to the hybrid mmWave MIMO architecture. Since the aforementioned problem is non-convex and intractable, a sequential approach is proposed wherein the TPCs are designed first, followed by the RCs. To deal with the non-convex MSE and constant modulus constraints in the TPC design problem, we propose a majorization and minimization (MM) based Riemannian conjugate gradient (RCG) method, which restricts the tolerable MSE of the beampattern to within a predefined limit. Moreover, the least squares and the zero-forcing methods are adopted for maximizing the sum-SE and for mitigating the multiuser interference (MUI), respectively. Furthermore, to design the RC at each CU, we propose a linear MM-based blind combiner (LMBC) scheme that does not rely on the knowledge of the TPC at the CUs and has a low complexity. To achieve user fairness, we further extend the proposed sequential approach for maximizing the geometric mean (GM) of the CU's rate. Simulation results are presented, which show the superior performance of the proposed hybrid TPC and RC in comparison to the state-of-the-art designs in the mmWave MIMO ISAC systems under consideration.