On the extended G′G2$\left(\frac{{G}^{\prime }}{{G}^{2}}\right)$-expansion method for an optical solution of the Schrödinger equation in quantum mechanics

Abstract This paper intends to investigate the optical solution of the generalized nonlinear Schrodinger equation (NLSE) of higher order that includes cubic-quintic nonlinear terms and third-fourth order dispersion, focusing on the dynamics of ultra-short pulse transmission in optical fibers and quantum mechanics within the framework of nonlinear optics. The new analytical and solitary wave solutions for higher order generalized NLSE is determined with the extended <m:math xmlns:m="http://www.w3.org/1998/Math/MathML"> <m:mrow> <m:mfenced open="(" close=")" separators="|"> <m:mrow> <m:mfrac> <m:msup> <m:mi>G</m:mi> <m:mo>′</m:mo> </m:msup> <m:msup> <m:mi>G</m:mi> <m:mn>2</m:mn> </m:msup> </m:mfrac> </m:mrow> </m:mfenced> </m:mrow> </m:math> $\left(\frac{{G}^{\prime }}{{G}^{2}}\right)$ -expansion method. This method is a significant mathematical instrument for identifying the exact traveling wave solution of the complex partial differential equations with greater nonlinearity and also applications in various fields of physical sciences. A variety of soliton solutions, such as periodic, plane wave and singular solutions are derived along with suitable values of the parameters. Furthermore, the examination of wave propagation through the trigonometric, hyperbolic and rational functions is represented via physical interpretation with the help of 2D and 3D surf plots. This study has a great practical significance and has the potential for broad application in several scientific domains, including nonlinear optics and quantum mechanics.