How to classify data which is spiral in shape?

May be you need reset all settings, and select x squared and y squared, only 1 hidden layer with 5 neurons.

This is the architecture proposed and tested on the playground tensor flow for the Spiral Dataset. Two Hidden Layers with 8 neurons each is proposed with Tanh activation function.

You can increase no of hidden layers. Following is an example (But not very efficient)

The solution I reached after an hour of trial usually converges in just 100 epochs.

Yeah, I know it does not have the smoothest decision boundary out there, but it converges pretty fast.

I learned a few things from this spiral experiment:-

• The output layer should be greater than or equal to the input layer. At least that's what I noticed in the case of this spiral problem.
• Keep the initial learning rate high, like 0.1 in this case, then as you approach a low test error like 3-5% or less, decrease the learning rate by a notch(0.03) or two. This helps in converging faster and avoids jumping around the global minima.
• You can see the effects of keeping the learning rate high by checking the error graph at the top right.
• For smaller batch sizes like 1, 0.1 is too high a learning rate as the model fails to converge as it jumps around the global minima.
• So, if you would like to keep a high learning rate(0.1), keep the batch size high(10) as well. This usually gives a slow yet smoother convergence.

Coincidentally the solution I came up with is very similar to the one provided by Salvador Dali.

Kindly add a comment, if you find any more intuitions or reasonings.

This is an example of vanilla Tensorflow playground with no added features and no modifications. The run for Spiral was between 187 to ~300 Epoch, depending. I used Lasso Regularization L1 so I could eliminate coefficients. I decreased the batch size by 1 to keep the output from over fitting. In my second example I added some noise to the data set then upped the L1 to compensate.

By cheating... theta is $arctan(y,x)$, $r$ is $sqrt{(x^2 + y^2)}$.

In theory, $x^2$ and $y^2$ should work, but, in practice, they somehow failed, even though, occasionally, it works.

Ideally neural networks should be able to find out the function out on it's own without us providing the spherical features. After some experimentation I was able to reach a configuration where we do not need anything except $X_1$ and $X_2$. This net converged after about 1500 epochs which is quite long. So the best way might still be to add additional features but I am just trying to say that it is still possible to converge without them.