Deep learning theory: why are hidden layers necessary?

For this question, I’ll refer to the popular YouTube video by 3Blue1Brown on deep learning applied to recognition of written numbers.

The video describes a neural network with the following layers:

1. Input (individual grayscale pixels)

2. Small-scale feature determination (e.g., the top quarter of the loop in the number 9)

3. Larger-scale features (e.g., the entire loop in the number 9)

4. The output layer, which shows the probability of an input image being each of the numbers 0-9

I’m also going to try to read through the entire wiki section here, and I’m currently on the neural networks page.

I particularly like the explanation of coefficients… “A node combines input from the data with a set of coefficients, or weights, that either amplify or dampen that input, thereby assigning significance to inputs with regard to the task the algorithm is trying to learn; e.g. which input is most helpful is classifying data without error?“

Essentially, it’s saying that each input has some level of importance to each output, which to me begs the question… are component features/hidden layers necessary at all? In the handwriting example, couldn’t every input node be connected to every output node without the use of hidden layers? The idea is that all of the high-weight input pixels or a given output would still have high weights for that output, but the network would skip the feature/aggregation stages. Is this just a matter of training efficiency (i.e., to prevent duplication by effectively extracting the same features more than once)?

Also, do the connections between various nodes need to specifically chosen so that the number of nodes and number and selection of connections are intelligently chosen?

Is it accurate to say that a sufficiently deep neural network looks to find the significance of all relevant combinations of input values and that’s basically all it’s doing?