Data Science Asked by Chris Cabral on June 25, 2021
I am attempting to adapt the frame prediction model from the keras examples to work with a set of 1-d sensors. I have android wearable sensor data and am designing an algorithm that can hopefully predict what the future sensor readings will be based on the past sensor readings.
model = Sequential()
model.add(ConvLSTM2D(filters=filters, kernel_size=(1, 1), input_shape=(None, 9, 1, 1), padding='same', return_sequences=True))
model.add(BatchNormalization())
model.add(ConvLSTM2D(filters=filters, kernel_size=(1, 1), padding='same', return_sequences=True))
model.add(BatchNormalization())
model.add(ConvLSTM2D(filters=filters, kernel_size=(1, 1), padding='same', return_sequences=True))
model.add(BatchNormalization())
model.add(Conv3D(filters=1, kernel_size=(3, 3, 3), activation='sigmoid', padding='same', data_format='channels_last'))
model.compile(loss='binary_crossentropy', optimizer='adadelta')
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
conv_lst_m2d_1 (ConvLSTM2D) (None, None, 9, 1, 10) 480
_________________________________________________________________
batch_normalization_1 (Batch (None, None, 9, 1, 10) 40
_________________________________________________________________
conv_lst_m2d_2 (ConvLSTM2D) (None, None, 9, 1, 10) 840
_________________________________________________________________
batch_normalization_2 (Batch (None, None, 9, 1, 10) 40
_________________________________________________________________
conv_lst_m2d_3 (ConvLSTM2D) (None, None, 9, 1, 10) 840
_________________________________________________________________
batch_normalization_3 (Batch (None, None, 9, 1, 10) 40
_________________________________________________________________
conv3d_1 (Conv3D) (None, None, 9, 1, 1) 271
=================================================================
Total params: 2,551
Trainable params: 2,491
Non-trainable params: 60
_________________________________________________________________
I have a series of csv files with sensor data (9 sensors, acceleration on 3 axis, rotation on 3 axis, and yaw, pitch and roll). The data was sampled at 10 hertz. The input of the model is a “frame” or a snapshot in time of the sensor data and the output is the next “moment” or snapshot in time of the sensor data.
I have looked at the frame prediction algorithm and wanted to understand how to adapt my loaded data to be used to train the model.
My data consists of 15 csvs with about 10,000 rows each representing a set of sensors at a snapshot. Each csvs is from a different person and I would like to enhance my model to understand the differences between each person if possible.
import pandas as pd
csvs = get_csvs()
for file in csvs:
data = transform_df_to_deep_frame(pd.read_csv(file))
model.fit(data[:-1], data[1:], batch_size=10, epochs=300, validation_split=0.05)
This code produces the following output/error for relatively obvious reasons that I am still unable to fix.
(10778, 9, 1, 1)
Traceback (most recent call last):
File "compare_models.py", line 220, in leave_one_out_comparisons
model.fit(data[:-1], data[1:], batch_size=10, epochs=300, validation_split=0.05)
File "/home/victor/anaconda3/lib/python3.7/site-packages/keras/engine/training.py", line 952, in fit
batch_size=batch_size)
File "/home/victor/anaconda3/lib/python3.7/site-packages/keras/engine/training.py", line 751, in _standardize_user_data
exception_prefix='input')
File "/home/victor/anaconda3/lib/python3.7/site-packages/keras/engine/training_utils.py", line 128, in standardize_input_data
'with shape ' + str(data_shape))
ValueError: Error when checking input: expected conv_lst_m2d_1_input to have 5 dimensions, but got array with shape (10778, 9, 1, 1)
The problem seems be that I am not batching my data up properly. The None confused me. I am relatively new to ML, so forgive me. How do I adapt my input and output of shape (10778, 9, 1, 1) to fit as inputs and outputs to this model.
I needed to reshape my data into frames. The expected shape is
(samples, no_frames, row, col, grayscale)
I was putting in
(samples, row, col, grayscale)
In the end, I reshaped the data.
size = data.shape[0]//frames
data = data[:size*frames, :, :, :]
print(data.shape)
data = data.reshape(size, frames, 9, 1, 1)
print(data.shape)
Answered by Chris Cabral on June 25, 2021
Get help from others!
Recent Questions
Recent Answers
© 2024 TransWikia.com. All rights reserved. Sites we Love: PCI Database, UKBizDB, Menu Kuliner, Sharing RPP