Why not use the “normal equations” to find simple least squares coefficients?

I saw this list here and couldn’t believe there were so many ways to solve least squares. The “normal equations” on Wikipedia seemed to be a fairly straight forward way:
$$
{displaystyle {begin{aligned}{hat {alpha }}&={bar {y}}-{hat {beta }},{bar {x}},\{hat {beta }}&={frac {sum _{i=1}^{n}(x_{i}-{bar {x}})(y_{i}-{bar {y}})}{sum _{i=1}^{n}(x_{i}-{bar {x}})^{2}}}end{aligned}}}
$$

So why not just use them? I assumed there must be a computational or precision issue given that in the first link above Mark L. Stone mentions that SVD or QR are popular methods in statistical software and that the normal equations are “TERRIBLE from reliability and numerical accuracy standpoint”. However, in the following code, the normal equations are giving me accuracy to ~12 decimal places when compared to three popular python functions: numpy’s polyfit; scipy’s linregress; and scikit-learn’s LinearRegression.

What’s more interesting is that the normal equation method is the fastest when n = 100000000. Computational times for me are: 2.5s for linregress; 12.9s for polyfit; 4.2s for LinearRegression; and 1.8s for the normal equation.

Code:

import numpy as np
from sklearn.linear_model import LinearRegression
from scipy.stats import linregress
import timeit

b0 = 0
b1 = 1
n = 100000000
x = np.linspace(-5, 5, n)
np.random.seed(42)
e = np.random.randn(n)
y = b0 + b1*x + e

# scipy                                                                                                                                     
start = timeit.default_timer()
print(str.format('{0:.30f}', linregress(x, y)[0]))
stop = timeit.default_timer()
print(stop - start)

# numpy                                                                                                                                      
start = timeit.default_timer()
print(str.format('{0:.30f}', np.polyfit(x, y, 1)[0]))
stop = timeit.default_timer()
print(stop - start)

# sklearn                                                                                                                                    
clf = LinearRegression()
start = timeit.default_timer()
clf.fit(x.reshape(-1, 1), y.reshape(-1, 1))
stop = timeit.default_timer()
print(str.format('{0:.30f}', clf.coef_[0, 0]))
print(stop - start)

# normal equation                                                                                                                            
start = timeit.default_timer()
slope = np.sum((x-x.mean())*(y-y.mean()))/np.sum((x-x.mean())**2)
stop = timeit.default_timer()
print(str.format('{0:.30f}', slope))
print(stop - start) 

How would I differentiate with respect to x equations with Euler’s number in them?

I understand how to differentiate but am not sure as to how to do it with Euler’s number. (Please keep in mind that I have not had any experience whatsoever with Euler’s number, sorry if it’s a stupid question)

question i need to ask

Differential equations for a transformer

I would like to set up differential equations (please, no phasors in answers) for the circuit below:

circuit

So far I have two equations:

eq1

eq2

where I assume perfect magnetic coupling (so I treat M as a known constant).

Initial conditions are that at time=0, both i1 and i2 are zero.

The system is underdetermined and I have to put one more equation. What equation is it?

My intention is to solve for i2. Again, please no suggestions with phasors. I am looking into a more general case where v1 isn’t necessarily a perfect sinusoidal function.

Is it possible to train the neural network to solve math equations?

I’m aware that neural networks are probably not designed to do that, however asking hypothetically, is it possible to train the deep neural network (or similar) to solve math equations?

So given the 3 inputs: 1st number, operator sign represented by the number (1 – +, 2 – -, 3 – /, 4 – *, and so on), and the 2nd number, then after training the network should give me the valid results.

Example 1 (2+2):

  • Input 1: 2; Input 2: 1 (+); Input 3: 2; Expected output: 4
  • Input 1: 10; Input 2: 2 (-); Input 3: 10; Expected output: 0
  • Input 1: 5; Input 2: 4 (*); Input 3: 5; Expected output: 25
  • and so

The above can be extended to more sophisticated examples.

Is that possible? If so, what kind of network can learn/achieve that?

Question about fundamental matrix in differential equations.

Suppose I have a fundamental matrix $Phi(t)$ of the homogeneous system $y’=A(t)y$. I am trying to show the derivation for the variation of constants method of solving non-homogeneous systems $y’=A(t)y+g(t)$.

Suppose $psi(t)=Phi(t)v(t)$ is a solution to the non-homogeneous equation for some non-constant vector $v(t)$.

Then $Psi'(t)=Phi(t)v(t)+Psi(t)v'(t)=A(t)Phi(t)v(t)+g(t)$

My textbook continues to say “Since $Phi$is a fundamental matrix of the homogeneous system, $Phi'(t)=A(t)Phi(t)$…”

My question is, is this claim a fundamental property of a fundamental matrix, as the text seems to suggest, or is it an argument made by matching the terms attached to $v(t)$? i.e. $Phi(t)v(t)=A(t)Phi(t)v(t)$?

Ratio of solutions to two heat equations

Let $u(x,t)$ and $v(x,t)$ respectively solve the two one-dimensional heat equations with different (real) diffusion coefficients on the same domain $D$ and the same initial & boundary conditions, i.e.,
$$u_t(x,t) = alpha u_{xx}(x,t), quad v_t(x,t) = beta v_{xx}(x,t), quad (x,t) in D $$

For simplicity, say $D$ is the domain $t in [0,1]$, $x leq c(t)$ where $c in C^infty[0,1]$,

Initial condition: $u(x,0) = v(x,0) = f(x) in C^2$ with $f(x) leq exp{-x^2}$,

and boundary condition $u(t,c(t)) = v(t,c(t)) equiv 0$.

Question: What can be said about the asymptotic behaviour of the ratio $u/v$ when $alpha / beta to 0$ or $alpha / beta to infty$?

If $c = 0$, (i.e $D$ is a rectangle), we have closed form solutions $u$ and $v$ but I’m looking for more general results.

Ratio of solutions to heat equations

Let $u(x,t)$ and $v(x,t)$ respectively solve the two one-dimensional heat equations with different (real) diffusion coefficients on the same domain $D$ and the same initial & boundary conditions, i.e.,
$$u_t(x,t) = alpha u_{xx}(x,t), quad v_t(x,t) = beta v_{xx}(x,t), quad (x,t) in D $$
For simplicity, say $D$ is the quarter plane $x geq 0$, $t geq 0$,

initial condition: $u(x,0) = v(x,0) = f(x) in C^2$ with $f(x) leq exp{-x^2}$,

and boundary condition $u(t,0) = v(t,0) equiv 0$.

Question: What can be said about the asymptotic behaviour of the ratio $u/v$ when $alpha / beta to 0$ or $alpha / beta to infty$?

Diophantine equations for septic $(7$th$)$ power reciprocity

Let $p$ be a prime, and $p-1=fd$. There is exactly one unique subfield of $mathbb{Q}(zeta_p)$ which is of degree $d$. The defining polynomial $P_d(x)$ for this subfield has the root:

$$sum_{r=1}^{f} (zeta_p^{g^{rd}}) = zeta_p^{g^{d}} + zeta_p^{g^{2d}} + zeta_p^{g^{3d}}… + zeta_p^{g^{(f-2)d}} + zeta_p^{g^{(f-1)d}} + zeta_p^{g^{fd}}$$

where $g$ is a primitive root $pmod p$.

When $d=3$, it is well known that $P_3(x) = x^3+x^2+cx+c_2x$ where

$c=-(p-1)/3$ and $c_2=(p(L+3)-1)/27$

The integer $L=1pmod 3$ is given by the solution of $4p = L^2+27M^2$.

The discriminant of $P_3(x)$ is $p^2M^2$.

Are there any known results when $d=7$?

If $p=1pmod 7$, then the subfield of $mathbb{Q}(zeta_p)$ of degree $7$ is given by:

Let $P_7(x): x^7 + x^6 + cx^5 + c_2x^4 + c_3x^3 + c_4x^2 + c_5x + c_6$

One finds that $c=-3(p-1)/7$, but the question of interest is how do $c_2, c_3, c_4, c_5,$ and $c_6$ depend on $p$? Any links to articles or Diophantine equations necessary to find how the remaining coefficients depend on $p$ would be helpful. Thanks in advance!