How Loss and Maximum Error Relate to Taylor-Like Regression Coefficients When Approximating Cosine

A Taylor polynomial uses derivatives of the target function at a point, whereas polynomial regression uses training samples to fit across the entire interval. We ask whether having a better regression fit or a smaller domain lowers its largest error and brings its coefficients closer to Taylor coefficients. To test this, we compare polynomial regression trained using gradient descent (GD) to Taylor polynomials centered at zero for approximating the cosine function on intervals [0,4] and [0,10]. We vary training settings (viz., polynomial degree, learning rate, and number of training updates) and measure training loss (half of the mean squared error over 1,000 training samples), maximum absolute error, root mean squared error on [0,1], and absolute differences between the constant, linear, and quadratic polynomial regression coefficients and their corresponding Taylor coefficients. For degree 10 GD on [0,10] with learning rate 0.07, increasing the training budget from 5,000 to 50,000 updates more than halves the loss value but increases the maximum error by about 13%. After 50,000 updates using the same settings, the first three GD coefficients are closer to Taylor coefficients on [0,4] than on [0,10]. Further, increasing the degree from 18 to 30 using 10,000 updates and learning rate 0.07 reduces GD loss on [0,10] but slightly increases it on [0,4]. In a separate degree 10 least‑squares check, we find losses on both intervals are lower than GD’s using learning rate 0.07 and 50,000 updates. The constant, linear, and quadratic least‑squares coefficients are also closer to Taylor coefficients than GD coefficients from the degree 10 comparison described above. So overall, which method performs better depends on the chosen polynomial degree, interval, error measure, and training settings. Also, a lower loss can coincide with polynomial coefficients being closer to Taylor coefficients, but it does not guarantee such agreement. This upload contains the main paper and accompanying supporting material. Code is not included in this release.