ML regression poor performance

Question

I am experimenting with 3 years time series electrical demand data (kW) for a building and attempting to create regression supervised ML models from sci kit learn regressor algorithms but I have very poor performance (very high mean squared error). I have a GitHub Gist of the entire IPython notebook here.

There isn't a lot of wisdom here (and I don't have anyone to consult with) for what I am doing other than I know there is well developed analytic software (demand forecasting) that the power consulting industry uses and I am just attempting to recreate from scratch on own experimentation methods in Python.

The data that I am processing look like this below all recorded in 15 minute intervals.

Date_Time    kW
0 2011-03-01 00:15:00 171.36
1 2011-03-01 00:30:00 181.44
2 2011-03-01 00:45:00 175.68
3 2011-03-01 01:00:00 180.00

The distribution of the kW data looks like this pic below which doesn't appear to have a bell shaped curve: (Could this be a poor performance reason?)

EDIT rolling average plot

Also in my experimentation I am adding in additional Python Pandas dataframes to represent the integer value of the time stamp 'day of the week', hour, minute, and month; where logically I am know electrical demand fluctuates greatly depending on these variables. These are some scatter plots below of the data compared to kW. (which maybe screwing everything up) For example the first scatter below is the hour of the day which is typical for buildings that the electrical demand increases during a typical work day. The outliers are most likely extreme weather conditions causing high demand where I do not have any weather data incorporated here...

In python if I do a df.describe:

Ultimately I am hoping someone can give me some tips on why the model is horrible but maybe its just due to not enough data and/or strategy... Another person I have been questioning uses a clustering unsupervised learning approach but that doesn't make any sense to me...

Machine learning mastery also has a mini course and a large book I could purchase on time series forecasting. Is this more of a statistics approach? Does it require more 'normal' bell shaped distribution of the data?

Any tips to try or avenues to march down is greatly appreciated :)

EDIT 
GitHub gist was updated for a rolling average of the data as well as distribution column plot of kW data

Amit Bhardwaj · Accepted Answer

I understand you must have researched before opting the regression model.But I want to highlight one thing here that since that data is time series one,we have to be more careful here and use the models which takes the account of moving average and other time series factors.
Also,cross validation is different for timeseries analysis.
Having said that please use other models like ARIMA or SARIMA for your time series if your objective is to have higher accuracy.

I_Play_With_Data · Answer

It seems to me that your premise for doing this is potentially flawed. It sounds like you're trying to replicate some information that power companies can generate but they are working with more broad datasets than you have presented here. In turn, this can be a reason why your accuracy scores as so poor.

For example, consider that weather has an effect on power usage. So unless you're wrangling in weather data at some point, you will (1) most likely never achieve the most accurate model possible (might just be "adequate") and (2) most likely never approach similar results to what your power company can generate.

So, I would take a step back and consider your current data points; it's highly likely that you just don't have the right factors there in place to create the accurate model you seek.

ML regression poor performance

2 Answers

Add your own answers!

Ask a Question