Post History

Why can't we use linear regression on logits for solving logistic regression problems?

Consider a machine learning problem with inputs $\boldsymbol{X} \in \mathbb{R}^{N \times D}$ and corresponding labels $\boldsymbol{y} \in \mathcal{Y}^N$.

When the problem is to solve a regression task, $\mathcal{Y}^N = \mathbb{R}^N$ and we can use linear regression model,
$$\hat{\boldsymbol{y}} = \boldsymbol{X} \boldsymbol{w},$$ 
which has a closed-form solution for the parameters of our model:
$$\boldsymbol{w} = \big(\boldsymbol{X}^\mathsf{T} \boldsymbol{X}\big)^{-1} \boldsymbol{X}^\mathsf{T} \boldsymbol{y}.$$

When the problem is a classification task, such that $\mathcal{Y}^N = \{0, 1\}^N,$ we would typically use a logistic regression model,
$$\hat{\boldsymbol{y}} = \mathop{\sigma}(\boldsymbol{X} \boldsymbol{w}),$$
where $\sigma(s) = \big(1 + \exp(-s)\big)^{-1}$ is the logistic sigmoid.
Unfortunately, there is no closed-form solution for logistic regression.
Typically, we have to use methods like gradient descent to find a solution.
However, since the logistic sigmoid is invertible, I would assume that we can also write the logistic regression model as
$$\ln\Bigl(\frac{\hat{\boldsymbol{y}}}{1 - \hat{\boldsymbol{y}}}\Bigr) = \boldsymbol{X} \boldsymbol{w}$$
to directly model the logits, $\ln\Bigl(\frac{y}{1 - y}\Bigr)$.
After all, this would allow us to use the analytical solution from linear regression to solve logistic regression problems.

Why is this approach not used anywhere? Is there a mistake in my reasoning or are there problems with this approach that I am not aware of?