How to perform feature selection with Categorical Variables and Continuous Target, provided that data is not normally distributed?

If I understand your question correctly, You are asking how to reduce categorical features from a dataset. If yes then few of the approach I can think of -

1> Iterative Process - Build a model with all numerical features and one categorical feature then evaluate your improvement of the model by whatever metrics you are using and then add other categorical features and so on. So if you have N cat features you will be building N+1 models.

2> chi square test of predictor and target variables.

3> (what I use) Build a model with all the available features and measure it's performance and then use the feature importance functionality of that model to determine which feature is important. In case of linear regression, Higher the value of coefficient better the feature. Alternatively, you can use L1 regularization to check for non zero features. Do check for multi-collinearity before considering feature importance in linear regression.