Can RNN be replaced with non-recurrent classifier for Sequence Classification problem?

Conceptually, the sound of dropping a metal chain on the floor is different from the sound of dropping the separated links of that chain.

Feedforward NN

In a feed forward neural network all of sequential features would be consumed independently:

$f(x)=WX+b=w_1x_1+..+w_nx_n+b$

This is all good, as far as you’re ready to sacrifice the step-wise dependency between x1, x2 etc.

Recurrent NN

To be able to utilise the temporal or sequential signal in your dataset you need a method that “chains” each feature with its past/future state, right?

We "connect" these sequential events with the alleged "hidden state":

$a_n = f(W_n, a_{n-1}, x_n)$

By exploding the above equation you can see how past information is accumulated in the subspace of the hidden state:

$a_n = f(W_n, a_{n-1}, x_n) = f(W_n, f(W_{n-1}, a_{n-2}, x_{n-1}), x_n)$, since $ a_{n-1}=f(W_n, a_{n-2}, x_n)$.

You may be able to see some issues with the above especially when it comes to sequences of larger length, usually tackled with LSTM and attention pooling architectures, but this is a different discussion!

Hope it helps.