Deep Learning for Real Estate Price Prediction

Task 1. Exploratory Data Analysis

In [1]:
import warnings
warnings.filterwarnings("ignore")
import pandas as pd
import numpy as np
import seaborn as sns
import matplotlib.pyplot as plt
In [2]:
house_df=pd.read_csv('/Users/mekki/Python_Projects_Datasets/Deep Learning for Real Estate Price Prediction/00-realestate_prices.csv')
house_df
Out[2]:
id date price bedrooms bathrooms sqft_living sqft_lot floors waterfront view ... grade sqft_above sqft_basement yr_built yr_renovated zipcode lat long sqft_living15 sqft_lot15
0 7129300520 20141013T000000 221900.0 3 1.00 1180 5650 1.0 0 0 ... 7 1180 0 1955 0 98178 47.5112 -122.257 1340 5650
1 6414100192 20141209T000000 538000.0 3 2.25 2570 7242 2.0 0 0 ... 7 2170 400 1951 1991 98125 47.7210 -122.319 1690 7639
2 5631500400 20150225T000000 180000.0 2 1.00 770 10000 1.0 0 0 ... 6 770 0 1933 0 98028 47.7379 -122.233 2720 8062
3 2487200875 20141209T000000 604000.0 4 3.00 1960 5000 1.0 0 0 ... 7 1050 910 1965 0 98136 47.5208 -122.393 1360 5000
4 1954400510 20150218T000000 510000.0 3 2.00 1680 8080 1.0 0 0 ... 8 1680 0 1987 0 98074 47.6168 -122.045 1800 7503
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
21608 263000018 20140521T000000 360000.0 3 2.50 1530 1131 3.0 0 0 ... 8 1530 0 2009 0 98103 47.6993 -122.346 1530 1509
21609 6600060120 20150223T000000 400000.0 4 2.50 2310 5813 2.0 0 0 ... 8 2310 0 2014 0 98146 47.5107 -122.362 1830 7200
21610 1523300141 20140623T000000 402101.0 2 0.75 1020 1350 2.0 0 0 ... 7 1020 0 2009 0 98144 47.5944 -122.299 1020 2007
21611 291310100 20150116T000000 400000.0 3 2.50 1600 2388 2.0 0 0 ... 8 1600 0 2004 0 98027 47.5345 -122.069 1410 1287
21612 1523300157 20141015T000000 325000.0 2 0.75 1020 1076 2.0 0 0 ... 7 1020 0 2008 0 98144 47.5941 -122.299 1020 1357

21613 rows × 21 columns

Retrieve the :

  • Average House Price
  • Price of the Cheapest House
  • Average number of Bedrooms and Bathrooms
  • Maximum number of Bedrooms

    • In [3]:
    pd.set_option('display.float_format', '{:.0f}'.format)
    In [4]:
    house_df[['price','bedrooms', 'bathrooms']].describe()
    Out[4]:
    price bedrooms bathrooms
    count 21613 21613 21613
    mean 540088 3 2
    std 367127 1 1
    min 75000 0 0
    25% 321950 3 2
    50% 450000 3 2
    75% 645000 4 2
    max 7700000 33 8

    Task 2. Perform Data Visualization

    See if there is any relationship between the sqft_living and the price of the houses

    In [5]:
    sns.scatterplot(data=house_df, x='sqft_living', y='price', color='#EF5A41')
    Out[5]:
    <Axes: xlabel='sqft_living', ylabel='price'>
    No description has been provided for this image

    Plot a Histogram for each value

    In [6]:
    house_df.hist(bins=20, figsize=(20,20), color='#EF5A41')
    Out[6]:
    array([[<Axes: title={'center': 'id'}>,
        <Axes: title={'center': 'price'}>,
        <Axes: title={'center': 'bedrooms'}>,
        <Axes: title={'center': 'bathrooms'}>],
       [<Axes: title={'center': 'sqft_living'}>,
        <Axes: title={'center': 'sqft_lot'}>,
        <Axes: title={'center': 'floors'}>,
        <Axes: title={'center': 'waterfront'}>],
       [<Axes: title={'center': 'view'}>,
        <Axes: title={'center': 'condition'}>,
        <Axes: title={'center': 'grade'}>,
        <Axes: title={'center': 'sqft_above'}>],
       [<Axes: title={'center': 'sqft_basement'}>,
        <Axes: title={'center': 'yr_built'}>,
        <Axes: title={'center': 'yr_renovated'}>,
        <Axes: title={'center': 'zipcode'}>],
       [<Axes: title={'center': 'lat'}>,
        <Axes: title={'center': 'long'}>,
        <Axes: title={'center': 'sqft_living15'}>,
        <Axes: title={'center': 'sqft_lot15'}>]], dtype=object)
    No description has been provided for this image

    Visualize the Correlation between each feature

    In [7]:
    numeric_columns = house_df.select_dtypes(include=['float64', 'int64'])
f, ax = plt.subplots(figsize=(20,20))
sns.heatmap(numeric_columns.corr(), annot=True)
    Out[7]:
    <Axes: >
    No description has been provided for this image

    Select a sample of the features

    In [8]:
    house_df_sample=house_df[['price','bedrooms','bathrooms','sqft_living','sqft_lot','sqft_above','sqft_basement','yr_built']]
house_df_sample
    Out[8]:
    price bedrooms bathrooms sqft_living sqft_lot sqft_above sqft_basement yr_built
    0 221900 3 1 1180 5650 1180 0 1955
    1 538000 3 2 2570 7242 2170 400 1951
    2 180000 2 1 770 10000 770 0 1933
    3 604000 4 3 1960 5000 1050 910 1965
    4 510000 3 2 1680 8080 1680 0 1987
    ... ... ... ... ... ... ... ... ...
    21608 360000 3 2 1530 1131 1530 0 2009
    21609 400000 4 2 2310 5813 2310 0 2014
    21610 402101 2 1 1020 1350 1020 0 2009
    21611 400000 3 2 1600 2388 1600 0 2004
    21612 325000 2 1 1020 1076 1020 0 2008

    21613 rows × 8 columns

    Using Seaborn, plot the pairplot for the features in house_df_sample, explore the data and perform a sanity check

    In [9]:
    g = sns.PairGrid(data=house_df_sample, diag_sharey=False)
g.map_upper(sns.scatterplot, color='#681F55')
g.map_lower(sns.scatterplot, color='#681F55')
g.map_diag(sns.histplot, color='#EF5A41')
    Out[9]:
    <seaborn.axisgrid.PairGrid at 0x177973490>
    No description has been provided for this image

    Task 3. Perform Data Cleaning and Feature Engineering

    Select a sample of the features for the Deep Learning Phase

    In [10]:
    selected_features = ['bedrooms','bathrooms','sqft_living','sqft_lot','floors','sqft_above','sqft_basement']
x=house_df[selected_features]
x
    Out[10]:
    bedrooms bathrooms sqft_living sqft_lot floors sqft_above sqft_basement
    0 3 1 1180 5650 1 1180 0
    1 3 2 2570 7242 2 2170 400
    2 2 1 770 10000 1 770 0
    3 4 3 1960 5000 1 1050 910
    4 3 2 1680 8080 1 1680 0
    ... ... ... ... ... ... ... ...
    21608 3 2 1530 1131 3 1530 0
    21609 4 2 2310 5813 2 2310 0
    21610 2 1 1020 1350 2 1020 0
    21611 3 2 1600 2388 2 1600 0
    21612 2 1 1020 1076 2 1020 0

    21613 rows × 7 columns

    In [11]:
    y=house_df['price']
y
    Out[11]:
    0       221900
1       538000
2       180000
3       604000
4       510000
         ...  
21608   360000
21609   400000
21610   402101
21611   400000
21612   325000
Name: price, Length: 21613, dtype: float64
    In [12]:
    x.shape
    Out[12]:
    (21613, 7)
    In [13]:
    y.shape
    Out[13]:
    (21613,)

    Scaling the Data to Train it

    In [14]:
    from sklearn.preprocessing import MinMaxScaler
    In [15]:
    scaler=MinMaxScaler()
x_scaled=scaler.fit_transform(x)
    In [16]:
    x_scaled
    Out[16]:
    array([[0.09090909, 0.125     , 0.06716981, ..., 0.        , 0.09758772,
        0.        ],
       [0.09090909, 0.28125   , 0.17207547, ..., 0.4       , 0.20614035,
        0.08298755],
       [0.06060606, 0.125     , 0.03622642, ..., 0.        , 0.05263158,
        0.        ],
       ...,
       [0.06060606, 0.09375   , 0.05509434, ..., 0.4       , 0.08004386,
        0.        ],
       [0.09090909, 0.3125    , 0.09886792, ..., 0.4       , 0.14364035,
        0.        ],
       [0.06060606, 0.09375   , 0.05509434, ..., 0.4       , 0.08004386,
        0.        ]])
    In [17]:
    x_scaled.shape
    Out[17]:
    (21613, 7)
    In [18]:
    scaler.data_max_
    Out[18]:
    array([3.300000e+01, 8.000000e+00, 1.354000e+04, 1.651359e+06,
       3.500000e+00, 9.410000e+03, 4.820000e+03])
    In [19]:
    scaler.data_min_
    Out[19]:
    array([  0.,   0., 290., 520.,   1., 290.,   0.])
    In [20]:
    y=y.values.reshape(-1,1)
    In [21]:
    y_scaled=scaler.fit_transform(y)
y_scaled
    Out[21]:
    array([[0.01926557],
       [0.06072131],
       [0.01377049],
       ...,
       [0.04289849],
       [0.04262295],
       [0.03278689]])

    Task 4. Train a Deep Learning Model with Limited Number of Features

    Train Test Split

    In [22]:
    from sklearn.model_selection import train_test_split
    In [23]:
    x_train, x_test, y_train, y_test = train_test_split(x_scaled, y_scaled, test_size=0.25)
    In [24]:
    x_train.shape
    Out[24]:
    (16209, 7)
    In [25]:
    x_test.shape
    Out[25]:
    (5404, 7)

    Instantiating a Model Object

    In [26]:
    import tensorflow.keras
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense

model=Sequential()

    Adding Layers

    In [27]:
    model.add(Dense(100, input_dim=7, activation='relu'))
model.add(Dense(100, activation='relu'))
model.add(Dense(100, activation='relu'))
model.add(Dense(200, activation='relu'))
model.add(Dense(1, activation='linear'))
    In [28]:
    model.summary()

    Model: "sequential"
_________________________________________________________________
 Layer (type)                Output Shape              Param #   
=================================================================
 dense (Dense)               (None, 100)               800       
                                                                 
 dense_1 (Dense)             (None, 100)               10100     
                                                                 
 dense_2 (Dense)             (None, 100)               10100     
                                                                 
 dense_3 (Dense)             (None, 200)               20200     
                                                                 
 dense_4 (Dense)             (None, 1)                 201       
                                                                 
=================================================================
Total params: 41401 (161.72 KB)
Trainable params: 41401 (161.72 KB)
Non-trainable params: 0 (0.00 Byte)
_________________________________________________________________

    Compiling and Training the Model

    In [29]:
    model.compile(optimizer='Adam', loss='mean_squared_error')
    In [30]:
    epochs_hist=model.fit(x_train, y_train, epochs=40, batch_size=50, validation_split=0.2)

    Epoch 1/40
260/260 [==============================] - 1s 1ms/step - loss: 0.0012 - val_loss: 0.0011
Epoch 2/40
260/260 [==============================] - 0s 940us/step - loss: 0.0010 - val_loss: 0.0011
Epoch 3/40
260/260 [==============================] - 0s 883us/step - loss: 0.0010 - val_loss: 0.0011
Epoch 4/40
260/260 [==============================] - 0s 861us/step - loss: 0.0010 - val_loss: 0.0011
Epoch 5/40
260/260 [==============================] - 0s 846us/step - loss: 9.9282e-04 - val_loss: 0.0011
Epoch 6/40
260/260 [==============================] - 0s 871us/step - loss: 9.8137e-04 - val_loss: 0.0010
Epoch 7/40
260/260 [==============================] - 0s 848us/step - loss: 9.6953e-04 - val_loss: 0.0010
Epoch 8/40
260/260 [==============================] - 0s 863us/step - loss: 9.6622e-04 - val_loss: 0.0011
Epoch 9/40
260/260 [==============================] - 0s 854us/step - loss: 9.7589e-04 - val_loss: 0.0010
Epoch 10/40
260/260 [==============================] - 0s 838us/step - loss: 9.6671e-04 - val_loss: 9.9508e-04
Epoch 11/40
260/260 [==============================] - 0s 843us/step - loss: 9.5596e-04 - val_loss: 0.0011
Epoch 12/40
260/260 [==============================] - 0s 846us/step - loss: 9.5228e-04 - val_loss: 0.0011
Epoch 13/40
260/260 [==============================] - 0s 856us/step - loss: 9.7486e-04 - val_loss: 0.0010
Epoch 14/40
260/260 [==============================] - 0s 850us/step - loss: 9.4639e-04 - val_loss: 0.0010
Epoch 15/40
260/260 [==============================] - 0s 918us/step - loss: 9.4163e-04 - val_loss: 0.0010
Epoch 16/40
260/260 [==============================] - 0s 914us/step - loss: 9.7185e-04 - val_loss: 0.0010
Epoch 17/40
260/260 [==============================] - 0s 872us/step - loss: 9.4328e-04 - val_loss: 0.0010
Epoch 18/40
260/260 [==============================] - 0s 885us/step - loss: 9.4496e-04 - val_loss: 0.0010
Epoch 19/40
260/260 [==============================] - 0s 953us/step - loss: 9.3023e-04 - val_loss: 0.0012
Epoch 20/40
260/260 [==============================] - 0s 888us/step - loss: 9.4264e-04 - val_loss: 0.0010
Epoch 21/40
260/260 [==============================] - 0s 865us/step - loss: 9.2959e-04 - val_loss: 0.0010
Epoch 22/40
260/260 [==============================] - 0s 889us/step - loss: 9.3602e-04 - val_loss: 0.0011
Epoch 23/40
260/260 [==============================] - 0s 882us/step - loss: 9.3827e-04 - val_loss: 9.9735e-04
Epoch 24/40
260/260 [==============================] - 0s 870us/step - loss: 9.2816e-04 - val_loss: 0.0010
Epoch 25/40
260/260 [==============================] - 0s 863us/step - loss: 9.2620e-04 - val_loss: 9.8791e-04
Epoch 26/40
260/260 [==============================] - 0s 876us/step - loss: 9.2761e-04 - val_loss: 0.0010
Epoch 27/40
260/260 [==============================] - 0s 902us/step - loss: 9.3383e-04 - val_loss: 9.9569e-04
Epoch 28/40
260/260 [==============================] - 0s 875us/step - loss: 9.2334e-04 - val_loss: 9.8781e-04
Epoch 29/40
260/260 [==============================] - 0s 878us/step - loss: 9.2401e-04 - val_loss: 9.9307e-04
Epoch 30/40
260/260 [==============================] - 0s 888us/step - loss: 9.1545e-04 - val_loss: 9.8973e-04
Epoch 31/40
260/260 [==============================] - 0s 939us/step - loss: 9.2033e-04 - val_loss: 9.8311e-04
Epoch 32/40
260/260 [==============================] - 0s 895us/step - loss: 9.2280e-04 - val_loss: 9.8225e-04
Epoch 33/40
260/260 [==============================] - 0s 911us/step - loss: 9.2556e-04 - val_loss: 0.0010
Epoch 34/40
260/260 [==============================] - 0s 913us/step - loss: 9.1427e-04 - val_loss: 9.9323e-04
Epoch 35/40
260/260 [==============================] - 0s 868us/step - loss: 9.2140e-04 - val_loss: 0.0010
Epoch 36/40
260/260 [==============================] - 0s 883us/step - loss: 9.2044e-04 - val_loss: 9.9073e-04
Epoch 37/40
260/260 [==============================] - 0s 895us/step - loss: 9.0728e-04 - val_loss: 0.0010
Epoch 38/40
260/260 [==============================] - 0s 920us/step - loss: 9.0718e-04 - val_loss: 0.0010
Epoch 39/40
260/260 [==============================] - 0s 881us/step - loss: 9.1437e-04 - val_loss: 9.8026e-04
Epoch 40/40
260/260 [==============================] - 0s 883us/step - loss: 9.1776e-04 - val_loss: 9.8231e-04

    Task 5. Evaluate Trained Deep Learning Model Performance

    Plot the Training Loss vs. Validation Loss

    In [31]:
    epochs_hist.history.keys()
    Out[31]:
    dict_keys(['loss', 'val_loss'])
    In [32]:
    plt.plot(epochs_hist.history['loss'], color='#EF5A41')
plt.plot(epochs_hist.history['val_loss'], color='#9A1B5B')
plt.title('Model Loss Progress During Training')
plt.xlabel('Epoch')
plt.ylabel('Training and Validation Loss')
plt.legend(['Training Loss','Validation Loss'])
    Out[32]:
    <matplotlib.legend.Legend at 0x2935592d0>
    No description has been provided for this image

    Predict the price of a house using the feature values : 4, 3, 1960, 5000, 1, 2000, 3000

    In [33]:
    x_test_1 = np.array([[4, 3, 1960, 5000, 1, 2000, 3000]])
scaler_1=MinMaxScaler()
x_test_scaled_1=scaler_1.fit_transform(x_test_1)

y_predict_1=model.predict(x_test_scaled_1)

y_predict_1=scaler.inverse_transform(y_predict_1)
y_predict_1

    1/1 [==============================] - 0s 36ms/step
    Out[33]:
    array([[235966.42]], dtype=float32)

    Plot the Model Prediction vs. True Values

    In [34]:
    y_predict = model.predict(x_test)
plt.plot(y_test, y_predict, '+', color='#EF5A41', markersize=6)
plt.xlabel('Model Prediction')
plt.ylabel('True Value')

    169/169 [==============================] - 0s 400us/step
    Out[34]:
    Text(0, 0.5, 'True Value')
    No description has been provided for this image

    Plot using the actual values using inverse_transform

    In [35]:
    y_predict_original=scaler.inverse_transform(y_predict)
y_test_original=scaler.inverse_transform(y_test)
    In [36]:
    y_predict = model.predict(x_test)
plt.plot(y_test_original, y_predict_original, '+', color='#EF5A41', markersize=6)
plt.xlabel('Model Prediction')
plt.ylabel('True Value')
plt.xlim(0, 5000000)
plt.ylim(0,3000000)

    169/169 [==============================] - 0s 391us/step
    Out[36]:
    (0.0, 3000000.0)
    No description has been provided for this image

    Generate the different metrics

    In [37]:
    k=x_test.shape[1]
n=len(x_test)
n
    Out[37]:
    5404
    In [38]:
    k
    Out[38]:
    7
    In [39]:
    from sklearn.metrics import r2_score, mean_squared_error, mean_absolute_error
from math import sqrt

    List the :

  • Root Mean Squared Error
  • Mean Squared Error
  • Mean Absolute Error
  • Coefficient of Determination
  • Adjusted Coefficient of Determination

    • In [40]:
    RMSE=float(format(np.sqrt(mean_squared_error(y_test_original, y_predict_original)),'.3f'))
MSE=mean_squared_error(y_test_original, y_predict_original)
MAE=mean_absolute_error(y_test_original, y_predict_original)
r2=r2_score(y_test_original, y_predict_original)
adj_r2=1-(1-r2)*(n-1)/(n-k-1)

print('RMSE =', RMSE)
print('MSE =', MSE)
print('MAE =', MAE)
print('R2 =', r2)
print('Adjusted R2 =', adj_r2)

    RMSE = 254933.803
MSE = 64991243773.00386
MAE = 159263.76404919967
R2 = 0.6007554733734726
Adjusted R2 = 0.6002375505257361

    Task 5. Train a Deep Learning Model with a Higher Number of Features

    In [41]:
    selected_features = ['bedrooms','bathrooms','sqft_living','sqft_lot','floors','sqft_above','sqft_basement',
                     'yr_renovated','zipcode','lat','long','sqft_living15','sqft_lot15']

x = house_df[selected_features]
x
    Out[41]:
    bedrooms bathrooms sqft_living sqft_lot floors sqft_above sqft_basement yr_renovated zipcode lat long sqft_living15 sqft_lot15
    0 3 1 1180 5650 1 1180 0 0 98178 48 -122 1340 5650
    1 3 2 2570 7242 2 2170 400 1991 98125 48 -122 1690 7639
    2 2 1 770 10000 1 770 0 0 98028 48 -122 2720 8062
    3 4 3 1960 5000 1 1050 910 0 98136 48 -122 1360 5000
    4 3 2 1680 8080 1 1680 0 0 98074 48 -122 1800 7503
    ... ... ... ... ... ... ... ... ... ... ... ... ... ...
    21608 3 2 1530 1131 3 1530 0 0 98103 48 -122 1530 1509
    21609 4 2 2310 5813 2 2310 0 0 98146 48 -122 1830 7200
    21610 2 1 1020 1350 2 1020 0 0 98144 48 -122 1020 2007
    21611 3 2 1600 2388 2 1600 0 0 98027 48 -122 1410 1287
    21612 2 1 1020 1076 2 1020 0 0 98144 48 -122 1020 1357

    21613 rows × 13 columns

    In [42]:
    from sklearn.preprocessing import MinMaxScaler
scaler=MinMaxScaler()
x_scaled=scaler.fit_transform(x)
    In [43]:
    x_scaled.shape
    Out[43]:
    (21613, 13)
    In [44]:
    y=house_df['price']
    In [45]:
    y=y.values.reshape(-1,1)
y_scaled=scaler.fit_transform(y)
    In [46]:
    from sklearn.model_selection import train_test_split
x_train, x_test, y_train, y_test = train_test_split(x_scaled, y_scaled, test_size=0.25)
    In [47]:
    model=Sequential()

model.add(Dense(10, input_dim=13, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(1, activation='linear'))
    In [48]:
    model.compile(optimizer='Adam', loss='mean_squared_error')
    In [49]:
    epochs_hist=model.fit(x_train, y_train, epochs=100, batch_size=50, verbose=1, validation_split=0.2)

    Epoch 1/100
260/260 [==============================] - 0s 685us/step - loss: 0.0014 - val_loss: 0.0010
Epoch 2/100
260/260 [==============================] - 0s 512us/step - loss: 9.8655e-04 - val_loss: 9.1588e-04
Epoch 3/100
260/260 [==============================] - 0s 522us/step - loss: 9.1356e-04 - val_loss: 8.2766e-04
Epoch 4/100
260/260 [==============================] - 0s 484us/step - loss: 8.1252e-04 - val_loss: 6.9093e-04
Epoch 5/100
260/260 [==============================] - 0s 504us/step - loss: 7.4539e-04 - val_loss: 6.3415e-04
Epoch 6/100
260/260 [==============================] - 0s 509us/step - loss: 7.0054e-04 - val_loss: 6.1010e-04
Epoch 7/100
260/260 [==============================] - 0s 500us/step - loss: 6.7116e-04 - val_loss: 5.7200e-04
Epoch 8/100
260/260 [==============================] - 0s 466us/step - loss: 6.3796e-04 - val_loss: 6.0879e-04
Epoch 9/100
260/260 [==============================] - 0s 493us/step - loss: 6.1259e-04 - val_loss: 5.4255e-04
Epoch 10/100
260/260 [==============================] - 0s 487us/step - loss: 6.0233e-04 - val_loss: 5.2240e-04
Epoch 11/100
260/260 [==============================] - 0s 482us/step - loss: 5.8094e-04 - val_loss: 5.2628e-04
Epoch 12/100
260/260 [==============================] - 0s 488us/step - loss: 5.6496e-04 - val_loss: 4.9161e-04
Epoch 13/100
260/260 [==============================] - 0s 472us/step - loss: 5.6531e-04 - val_loss: 5.7880e-04
Epoch 14/100
260/260 [==============================] - 0s 498us/step - loss: 5.6829e-04 - val_loss: 4.9359e-04
Epoch 15/100
260/260 [==============================] - 0s 484us/step - loss: 5.4840e-04 - val_loss: 4.9673e-04
Epoch 16/100
260/260 [==============================] - 0s 498us/step - loss: 5.4731e-04 - val_loss: 4.7401e-04
Epoch 17/100
260/260 [==============================] - 0s 480us/step - loss: 5.4758e-04 - val_loss: 4.8439e-04
Epoch 18/100
260/260 [==============================] - 0s 474us/step - loss: 5.3812e-04 - val_loss: 4.8150e-04
Epoch 19/100
260/260 [==============================] - 0s 487us/step - loss: 5.3209e-04 - val_loss: 5.2022e-04
Epoch 20/100
260/260 [==============================] - 0s 491us/step - loss: 5.3957e-04 - val_loss: 4.7427e-04
Epoch 21/100
260/260 [==============================] - 0s 484us/step - loss: 5.3188e-04 - val_loss: 5.0369e-04
Epoch 22/100
260/260 [==============================] - 0s 486us/step - loss: 5.2812e-04 - val_loss: 4.6702e-04
Epoch 23/100
260/260 [==============================] - 0s 491us/step - loss: 5.2687e-04 - val_loss: 4.6215e-04
Epoch 24/100
260/260 [==============================] - 0s 564us/step - loss: 5.2611e-04 - val_loss: 4.7124e-04
Epoch 25/100
260/260 [==============================] - 0s 523us/step - loss: 5.2297e-04 - val_loss: 4.7145e-04
Epoch 26/100
260/260 [==============================] - 0s 510us/step - loss: 5.2560e-04 - val_loss: 4.6142e-04
Epoch 27/100
260/260 [==============================] - 0s 496us/step - loss: 5.1666e-04 - val_loss: 4.6126e-04
Epoch 28/100
260/260 [==============================] - 0s 483us/step - loss: 5.1242e-04 - val_loss: 4.7089e-04
Epoch 29/100
260/260 [==============================] - 0s 485us/step - loss: 5.1919e-04 - val_loss: 4.5322e-04
Epoch 30/100
260/260 [==============================] - 0s 481us/step - loss: 5.1712e-04 - val_loss: 4.5280e-04
Epoch 31/100
260/260 [==============================] - 0s 486us/step - loss: 5.1124e-04 - val_loss: 4.4612e-04
Epoch 32/100
260/260 [==============================] - 0s 498us/step - loss: 5.0667e-04 - val_loss: 4.5507e-04
Epoch 33/100
260/260 [==============================] - 0s 508us/step - loss: 5.0634e-04 - val_loss: 4.4977e-04
Epoch 34/100
260/260 [==============================] - 0s 477us/step - loss: 5.0611e-04 - val_loss: 4.7769e-04
Epoch 35/100
260/260 [==============================] - 0s 481us/step - loss: 5.0728e-04 - val_loss: 4.6175e-04
Epoch 36/100
260/260 [==============================] - 0s 481us/step - loss: 5.0250e-04 - val_loss: 4.4267e-04
Epoch 37/100
260/260 [==============================] - 0s 488us/step - loss: 4.9903e-04 - val_loss: 4.5836e-04
Epoch 38/100
260/260 [==============================] - 0s 477us/step - loss: 5.0251e-04 - val_loss: 4.7478e-04
Epoch 39/100
260/260 [==============================] - 0s 479us/step - loss: 4.9616e-04 - val_loss: 4.5989e-04
Epoch 40/100
260/260 [==============================] - 0s 476us/step - loss: 4.9775e-04 - val_loss: 4.8445e-04
Epoch 41/100
260/260 [==============================] - 0s 486us/step - loss: 4.9031e-04 - val_loss: 4.5187e-04
Epoch 42/100
260/260 [==============================] - 0s 483us/step - loss: 4.9426e-04 - val_loss: 5.0214e-04
Epoch 43/100
260/260 [==============================] - 0s 482us/step - loss: 4.8952e-04 - val_loss: 4.7890e-04
Epoch 44/100
260/260 [==============================] - 0s 487us/step - loss: 4.9193e-04 - val_loss: 4.4524e-04
Epoch 45/100
260/260 [==============================] - 0s 520us/step - loss: 4.8517e-04 - val_loss: 4.4299e-04
Epoch 46/100
260/260 [==============================] - 0s 664us/step - loss: 4.8192e-04 - val_loss: 4.3565e-04
Epoch 47/100
260/260 [==============================] - 0s 507us/step - loss: 4.7916e-04 - val_loss: 4.5358e-04
Epoch 48/100
260/260 [==============================] - 0s 578us/step - loss: 4.7537e-04 - val_loss: 4.3119e-04
Epoch 49/100
260/260 [==============================] - 0s 484us/step - loss: 4.7735e-04 - val_loss: 4.3379e-04
Epoch 50/100
260/260 [==============================] - 0s 482us/step - loss: 4.8026e-04 - val_loss: 4.2112e-04
Epoch 51/100
260/260 [==============================] - 0s 488us/step - loss: 4.7509e-04 - val_loss: 4.4513e-04
Epoch 52/100
260/260 [==============================] - 0s 521us/step - loss: 4.7452e-04 - val_loss: 4.4619e-04
Epoch 53/100
260/260 [==============================] - 0s 492us/step - loss: 4.7449e-04 - val_loss: 4.1621e-04
Epoch 54/100
260/260 [==============================] - 0s 490us/step - loss: 4.7124e-04 - val_loss: 4.2618e-04
Epoch 55/100
260/260 [==============================] - 0s 503us/step - loss: 4.7473e-04 - val_loss: 4.3231e-04
Epoch 56/100
260/260 [==============================] - 0s 475us/step - loss: 4.6783e-04 - val_loss: 4.2514e-04
Epoch 57/100
260/260 [==============================] - 0s 486us/step - loss: 4.6216e-04 - val_loss: 4.2986e-04
Epoch 58/100
260/260 [==============================] - 0s 474us/step - loss: 4.6207e-04 - val_loss: 4.3327e-04
Epoch 59/100
260/260 [==============================] - 0s 488us/step - loss: 4.6343e-04 - val_loss: 4.7058e-04
Epoch 60/100
260/260 [==============================] - 0s 484us/step - loss: 4.5562e-04 - val_loss: 4.4049e-04
Epoch 61/100
260/260 [==============================] - 0s 470us/step - loss: 4.5528e-04 - val_loss: 4.4535e-04
Epoch 62/100
260/260 [==============================] - 0s 481us/step - loss: 4.6022e-04 - val_loss: 4.2820e-04
Epoch 63/100
260/260 [==============================] - 0s 509us/step - loss: 4.5444e-04 - val_loss: 4.1424e-04
Epoch 64/100
260/260 [==============================] - 0s 513us/step - loss: 4.4972e-04 - val_loss: 4.3377e-04
Epoch 65/100
260/260 [==============================] - 0s 585us/step - loss: 4.5189e-04 - val_loss: 4.3070e-04
Epoch 66/100
260/260 [==============================] - 0s 512us/step - loss: 4.5068e-04 - val_loss: 4.1717e-04
Epoch 67/100
260/260 [==============================] - 0s 506us/step - loss: 4.4975e-04 - val_loss: 4.3070e-04
Epoch 68/100
260/260 [==============================] - 0s 489us/step - loss: 4.4721e-04 - val_loss: 4.2432e-04
Epoch 69/100
260/260 [==============================] - 0s 486us/step - loss: 4.4663e-04 - val_loss: 4.1461e-04
Epoch 70/100
260/260 [==============================] - 0s 485us/step - loss: 4.4664e-04 - val_loss: 4.4409e-04
Epoch 71/100
260/260 [==============================] - 0s 532us/step - loss: 4.4509e-04 - val_loss: 4.3348e-04
Epoch 72/100
260/260 [==============================] - 0s 504us/step - loss: 4.4435e-04 - val_loss: 4.0393e-04
Epoch 73/100
260/260 [==============================] - 0s 491us/step - loss: 4.4184e-04 - val_loss: 4.1078e-04
Epoch 74/100
260/260 [==============================] - 0s 489us/step - loss: 4.3905e-04 - val_loss: 4.0586e-04
Epoch 75/100
260/260 [==============================] - 0s 491us/step - loss: 4.4429e-04 - val_loss: 4.9049e-04
Epoch 76/100
260/260 [==============================] - 0s 586us/step - loss: 4.4245e-04 - val_loss: 4.2431e-04
Epoch 77/100
260/260 [==============================] - 0s 600us/step - loss: 4.3445e-04 - val_loss: 4.2144e-04
Epoch 78/100
260/260 [==============================] - 0s 513us/step - loss: 4.3516e-04 - val_loss: 3.9872e-04
Epoch 79/100
260/260 [==============================] - 0s 506us/step - loss: 4.3872e-04 - val_loss: 3.9920e-04
Epoch 80/100
260/260 [==============================] - 0s 479us/step - loss: 4.3398e-04 - val_loss: 4.0783e-04
Epoch 81/100
260/260 [==============================] - 0s 489us/step - loss: 4.3280e-04 - val_loss: 4.0244e-04
Epoch 82/100
260/260 [==============================] - 0s 482us/step - loss: 4.3203e-04 - val_loss: 3.9208e-04
Epoch 83/100
260/260 [==============================] - 0s 484us/step - loss: 4.3452e-04 - val_loss: 4.3321e-04
Epoch 84/100
260/260 [==============================] - 0s 485us/step - loss: 4.3918e-04 - val_loss: 4.0215e-04
Epoch 85/100
260/260 [==============================] - 0s 480us/step - loss: 4.2415e-04 - val_loss: 3.9426e-04
Epoch 86/100
260/260 [==============================] - 0s 481us/step - loss: 4.3164e-04 - val_loss: 4.1263e-04
Epoch 87/100
260/260 [==============================] - 0s 485us/step - loss: 4.2936e-04 - val_loss: 4.0783e-04
Epoch 88/100
260/260 [==============================] - 0s 508us/step - loss: 4.2321e-04 - val_loss: 3.9601e-04
Epoch 89/100
260/260 [==============================] - 0s 485us/step - loss: 4.2139e-04 - val_loss: 4.1398e-04
Epoch 90/100
260/260 [==============================] - 0s 492us/step - loss: 4.2829e-04 - val_loss: 3.9073e-04
Epoch 91/100
260/260 [==============================] - 0s 512us/step - loss: 4.2635e-04 - val_loss: 4.0511e-04
Epoch 92/100
260/260 [==============================] - 0s 481us/step - loss: 4.3305e-04 - val_loss: 4.0167e-04
Epoch 93/100
260/260 [==============================] - 0s 572us/step - loss: 4.2303e-04 - val_loss: 3.9452e-04
Epoch 94/100
260/260 [==============================] - 0s 502us/step - loss: 4.2977e-04 - val_loss: 4.0568e-04
Epoch 95/100
260/260 [==============================] - 0s 490us/step - loss: 4.2278e-04 - val_loss: 4.0852e-04
Epoch 96/100
260/260 [==============================] - 0s 486us/step - loss: 4.3675e-04 - val_loss: 4.1940e-04
Epoch 97/100
260/260 [==============================] - 0s 494us/step - loss: 4.2929e-04 - val_loss: 3.9623e-04
Epoch 98/100
260/260 [==============================] - 0s 490us/step - loss: 4.2588e-04 - val_loss: 4.0703e-04
Epoch 99/100
260/260 [==============================] - 0s 475us/step - loss: 4.2414e-04 - val_loss: 3.9912e-04
Epoch 100/100
260/260 [==============================] - 0s 502us/step - loss: 4.2318e-04 - val_loss: 4.2747e-04
    In [50]:
    plt.plot(epochs_hist.history['loss'], color='#EF5A41')
plt.plot(epochs_hist.history['val_loss'], color='#9A1B5B')
plt.title('Model Loss Progress During Training')
plt.xlabel('Epoch')
plt.ylabel('Training and Validation Loss')
plt.legend(['Training Loss','Validation Loss'])
    Out[50]:
    <matplotlib.legend.Legend at 0x2946092d0>
    No description has been provided for this image
    In [51]:
    y_predict = model.predict(x_test)
plt.plot(y_test, y_predict, '+', color='#EF5A41', markersize=6)
plt.title('Linear Regression Predictions')
plt.xlabel('Model Prediction')
plt.ylabel('True Value')
plt.show()

    169/169 [==============================] - 0s 353us/step
    No description has been provided for this image
    In [52]:
    y_predict_original=scaler.inverse_transform(y_predict)
y_test_original=scaler.inverse_transform(y_test)
    In [53]:
    y_predict = model.predict(x_test)
plt.plot(y_test_original, y_predict_original, '+', color='#EF5A41', markersize=6)
plt.xlabel('Model Prediction')
plt.ylabel('True Value')
plt.xlim(0, 5000000)
plt.ylim(0,3000000)

    169/169 [==============================] - 0s 296us/step
    Out[53]:
    (0.0, 3000000.0)
    No description has been provided for this image
    In [54]:
    k=x_test.shape[1]
n=len(x_test)
n
    Out[54]:
    5404
    In [55]:
    k
    Out[55]:
    13
    In [56]:
    RMSE=float(format(np.sqrt(mean_squared_error(y_test_original, y_predict_original)),'.3f'))
MSE=mean_squared_error(y_test_original, y_predict_original)
MAE=mean_absolute_error(y_test_original, y_predict_original)
r2=r2_score(y_test_original, y_predict_original)
adj_r2=1-(1-r2)*(n-1)/(n-k-1)

print('RMSE =', RMSE)
print('MSE =', MSE)
print('MAE =', MAE)
print('R2 =', r2)
print('Adjusted R2 =', adj_r2)

    RMSE = 159494.805
MSE = 25438592732.23761
MAE = 95997.44692155116
R2 = 0.8128842575647005
Adjusted R2 = 0.8124329580003853

    Task 6. Train a Deep Learning Model with a Higher Number of Neurons

    In [57]:
    model=Sequential()

model.add(Dense(10, input_dim=13, activation='relu'))
model.add(Dense(10, activation='relu'))
model.add(Dense(200, activation='relu'))
model.add(Dense(200, activation='relu'))
model.add(Dense(1, activation='linear'))
    In [58]:
    model.compile(optimizer='Adam', loss='mean_squared_error')
    In [59]:
    epochs_hist=model.fit(x_train, y_train, epochs=100, batch_size=50, verbose=1, validation_split=0.2)

    Epoch 1/100
260/260 [==============================] - 1s 1ms/step - loss: 0.0016 - val_loss: 9.8649e-04
Epoch 2/100
260/260 [==============================] - 0s 921us/step - loss: 9.4928e-04 - val_loss: 7.7548e-04
Epoch 3/100
260/260 [==============================] - 0s 900us/step - loss: 7.5622e-04 - val_loss: 7.7875e-04
Epoch 4/100
260/260 [==============================] - 0s 911us/step - loss: 6.9572e-04 - val_loss: 5.5175e-04
Epoch 5/100
260/260 [==============================] - 0s 908us/step - loss: 6.3614e-04 - val_loss: 5.6789e-04
Epoch 6/100
260/260 [==============================] - 0s 920us/step - loss: 6.0920e-04 - val_loss: 5.5736e-04
Epoch 7/100
260/260 [==============================] - 0s 924us/step - loss: 5.8342e-04 - val_loss: 5.1721e-04
Epoch 8/100
260/260 [==============================] - 0s 916us/step - loss: 5.9573e-04 - val_loss: 5.4101e-04
Epoch 9/100
260/260 [==============================] - 0s 895us/step - loss: 5.4592e-04 - val_loss: 4.4515e-04
Epoch 10/100
260/260 [==============================] - 0s 884us/step - loss: 5.1552e-04 - val_loss: 4.7656e-04
Epoch 11/100
260/260 [==============================] - 0s 972us/step - loss: 5.2550e-04 - val_loss: 4.1534e-04
Epoch 12/100
260/260 [==============================] - 0s 906us/step - loss: 5.2490e-04 - val_loss: 4.5475e-04
Epoch 13/100
260/260 [==============================] - 0s 945us/step - loss: 5.0719e-04 - val_loss: 4.1551e-04
Epoch 14/100
260/260 [==============================] - 0s 919us/step - loss: 4.8089e-04 - val_loss: 3.9222e-04
Epoch 15/100
260/260 [==============================] - 0s 940us/step - loss: 4.8053e-04 - val_loss: 3.9263e-04
Epoch 16/100
260/260 [==============================] - 0s 1ms/step - loss: 4.7236e-04 - val_loss: 4.5121e-04
Epoch 17/100
260/260 [==============================] - 0s 945us/step - loss: 4.7714e-04 - val_loss: 3.7444e-04
Epoch 18/100
260/260 [==============================] - 0s 911us/step - loss: 4.5230e-04 - val_loss: 4.0788e-04
Epoch 19/100
260/260 [==============================] - 0s 924us/step - loss: 4.5152e-04 - val_loss: 4.7962e-04
Epoch 20/100
260/260 [==============================] - 0s 911us/step - loss: 4.4627e-04 - val_loss: 4.2470e-04
Epoch 21/100
260/260 [==============================] - 0s 916us/step - loss: 4.3764e-04 - val_loss: 3.8686e-04
Epoch 22/100
260/260 [==============================] - 0s 916us/step - loss: 4.4636e-04 - val_loss: 6.3127e-04
Epoch 23/100
260/260 [==============================] - 0s 893us/step - loss: 4.6282e-04 - val_loss: 3.5916e-04
Epoch 24/100
260/260 [==============================] - 0s 913us/step - loss: 4.3849e-04 - val_loss: 3.7643e-04
Epoch 25/100
260/260 [==============================] - 0s 910us/step - loss: 4.4550e-04 - val_loss: 4.6009e-04
Epoch 26/100
260/260 [==============================] - 0s 912us/step - loss: 4.4478e-04 - val_loss: 3.5830e-04
Epoch 27/100
260/260 [==============================] - 0s 899us/step - loss: 4.1965e-04 - val_loss: 3.5006e-04
Epoch 28/100
260/260 [==============================] - 0s 990us/step - loss: 4.2643e-04 - val_loss: 3.5560e-04
Epoch 29/100
260/260 [==============================] - 0s 920us/step - loss: 4.2529e-04 - val_loss: 4.6456e-04
Epoch 30/100
260/260 [==============================] - 0s 958us/step - loss: 4.1524e-04 - val_loss: 4.3442e-04
Epoch 31/100
260/260 [==============================] - 0s 894us/step - loss: 4.1918e-04 - val_loss: 3.4946e-04
Epoch 32/100
260/260 [==============================] - 0s 907us/step - loss: 4.1664e-04 - val_loss: 3.3847e-04
Epoch 33/100
260/260 [==============================] - 0s 903us/step - loss: 4.0626e-04 - val_loss: 4.3640e-04
Epoch 34/100
260/260 [==============================] - 0s 897us/step - loss: 4.2061e-04 - val_loss: 3.5299e-04
Epoch 35/100
260/260 [==============================] - 0s 1ms/step - loss: 4.2408e-04 - val_loss: 3.5629e-04
Epoch 36/100
260/260 [==============================] - 0s 906us/step - loss: 4.2520e-04 - val_loss: 4.1602e-04
Epoch 37/100
260/260 [==============================] - 0s 918us/step - loss: 3.9879e-04 - val_loss: 3.5318e-04
Epoch 38/100
260/260 [==============================] - 0s 896us/step - loss: 4.0380e-04 - val_loss: 3.7103e-04
Epoch 39/100
260/260 [==============================] - 0s 910us/step - loss: 4.2351e-04 - val_loss: 4.0435e-04
Epoch 40/100
260/260 [==============================] - 0s 898us/step - loss: 4.0546e-04 - val_loss: 5.2386e-04
Epoch 41/100
260/260 [==============================] - 0s 925us/step - loss: 4.2760e-04 - val_loss: 3.5751e-04
Epoch 42/100
260/260 [==============================] - 0s 1ms/step - loss: 3.9094e-04 - val_loss: 3.8858e-04
Epoch 43/100
260/260 [==============================] - 0s 926us/step - loss: 3.9693e-04 - val_loss: 3.5649e-04
Epoch 44/100
260/260 [==============================] - 0s 914us/step - loss: 3.9600e-04 - val_loss: 4.3183e-04
Epoch 45/100
260/260 [==============================] - 0s 931us/step - loss: 3.9876e-04 - val_loss: 3.5637e-04
Epoch 46/100
260/260 [==============================] - 0s 937us/step - loss: 3.8709e-04 - val_loss: 3.5753e-04
Epoch 47/100
260/260 [==============================] - 0s 954us/step - loss: 3.9070e-04 - val_loss: 4.4879e-04
Epoch 48/100
260/260 [==============================] - 0s 913us/step - loss: 4.0001e-04 - val_loss: 3.8129e-04
Epoch 49/100
260/260 [==============================] - 0s 929us/step - loss: 3.9784e-04 - val_loss: 3.4729e-04
Epoch 50/100
260/260 [==============================] - 0s 893us/step - loss: 3.8879e-04 - val_loss: 3.4094e-04
Epoch 51/100
260/260 [==============================] - 0s 904us/step - loss: 3.7971e-04 - val_loss: 3.5302e-04
Epoch 52/100
260/260 [==============================] - 0s 986us/step - loss: 3.8531e-04 - val_loss: 3.6147e-04
Epoch 53/100
260/260 [==============================] - 0s 972us/step - loss: 3.8796e-04 - val_loss: 3.3611e-04
Epoch 54/100
260/260 [==============================] - 0s 964us/step - loss: 3.8931e-04 - val_loss: 3.6652e-04
Epoch 55/100
260/260 [==============================] - 0s 929us/step - loss: 3.7971e-04 - val_loss: 3.4375e-04
Epoch 56/100
260/260 [==============================] - 0s 892us/step - loss: 3.8271e-04 - val_loss: 3.3765e-04
Epoch 57/100
260/260 [==============================] - 0s 899us/step - loss: 3.8250e-04 - val_loss: 3.4022e-04
Epoch 58/100
260/260 [==============================] - 0s 924us/step - loss: 3.7790e-04 - val_loss: 3.4815e-04
Epoch 59/100
260/260 [==============================] - 0s 952us/step - loss: 3.7882e-04 - val_loss: 3.6458e-04
Epoch 60/100
260/260 [==============================] - 0s 928us/step - loss: 3.7421e-04 - val_loss: 3.5461e-04
Epoch 61/100
260/260 [==============================] - 0s 906us/step - loss: 3.7147e-04 - val_loss: 3.4390e-04
Epoch 62/100
260/260 [==============================] - 0s 926us/step - loss: 3.8664e-04 - val_loss: 4.1396e-04
Epoch 63/100
260/260 [==============================] - 0s 929us/step - loss: 3.7643e-04 - val_loss: 3.5885e-04
Epoch 64/100
260/260 [==============================] - 0s 985us/step - loss: 3.7307e-04 - val_loss: 3.1915e-04
Epoch 65/100
260/260 [==============================] - 0s 916us/step - loss: 3.5527e-04 - val_loss: 3.6746e-04
Epoch 66/100
260/260 [==============================] - 0s 915us/step - loss: 3.7806e-04 - val_loss: 4.1192e-04
Epoch 67/100
260/260 [==============================] - 0s 910us/step - loss: 3.7332e-04 - val_loss: 3.2803e-04
Epoch 68/100
260/260 [==============================] - 0s 896us/step - loss: 3.6035e-04 - val_loss: 3.5944e-04
Epoch 69/100
260/260 [==============================] - 0s 905us/step - loss: 3.9454e-04 - val_loss: 3.4213e-04
Epoch 70/100
260/260 [==============================] - 0s 890us/step - loss: 3.6252e-04 - val_loss: 3.9332e-04
Epoch 71/100
260/260 [==============================] - 0s 929us/step - loss: 3.7558e-04 - val_loss: 4.3639e-04
Epoch 72/100
260/260 [==============================] - 0s 925us/step - loss: 3.8003e-04 - val_loss: 3.6735e-04
Epoch 73/100
260/260 [==============================] - 0s 892us/step - loss: 3.6065e-04 - val_loss: 3.8908e-04
Epoch 74/100
260/260 [==============================] - 0s 919us/step - loss: 3.5992e-04 - val_loss: 3.4571e-04
Epoch 75/100
260/260 [==============================] - 0s 916us/step - loss: 3.5439e-04 - val_loss: 3.7431e-04
Epoch 76/100
260/260 [==============================] - 0s 928us/step - loss: 3.7222e-04 - val_loss: 3.8962e-04
Epoch 77/100
260/260 [==============================] - 0s 895us/step - loss: 3.6041e-04 - val_loss: 3.2952e-04
Epoch 78/100
260/260 [==============================] - 0s 996us/step - loss: 3.5551e-04 - val_loss: 3.5227e-04
Epoch 79/100
260/260 [==============================] - 0s 919us/step - loss: 3.5773e-04 - val_loss: 3.6855e-04
Epoch 80/100
260/260 [==============================] - 0s 933us/step - loss: 3.6016e-04 - val_loss: 3.2864e-04
Epoch 81/100
260/260 [==============================] - 0s 943us/step - loss: 3.6669e-04 - val_loss: 3.9169e-04
Epoch 82/100
260/260 [==============================] - 0s 918us/step - loss: 3.5063e-04 - val_loss: 3.4247e-04
Epoch 83/100
260/260 [==============================] - 0s 914us/step - loss: 3.5130e-04 - val_loss: 3.5517e-04
Epoch 84/100
260/260 [==============================] - 0s 929us/step - loss: 3.5735e-04 - val_loss: 3.7150e-04
Epoch 85/100
260/260 [==============================] - 0s 897us/step - loss: 3.5838e-04 - val_loss: 3.3777e-04
Epoch 86/100
260/260 [==============================] - 0s 915us/step - loss: 3.3658e-04 - val_loss: 3.8450e-04
Epoch 87/100
260/260 [==============================] - 0s 897us/step - loss: 3.5305e-04 - val_loss: 3.4894e-04
Epoch 88/100
260/260 [==============================] - 0s 975us/step - loss: 3.4101e-04 - val_loss: 3.5481e-04
Epoch 89/100
260/260 [==============================] - 0s 911us/step - loss: 3.5236e-04 - val_loss: 3.3046e-04
Epoch 90/100
260/260 [==============================] - 0s 929us/step - loss: 3.4905e-04 - val_loss: 3.3836e-04
Epoch 91/100
260/260 [==============================] - 0s 914us/step - loss: 3.4622e-04 - val_loss: 3.2883e-04
Epoch 92/100
260/260 [==============================] - 0s 926us/step - loss: 3.4353e-04 - val_loss: 3.3694e-04
Epoch 93/100
260/260 [==============================] - 0s 916us/step - loss: 3.3719e-04 - val_loss: 3.6588e-04
Epoch 94/100
260/260 [==============================] - 0s 896us/step - loss: 3.5121e-04 - val_loss: 3.5338e-04
Epoch 95/100
260/260 [==============================] - 0s 927us/step - loss: 3.4302e-04 - val_loss: 3.9683e-04
Epoch 96/100
260/260 [==============================] - 0s 928us/step - loss: 3.3276e-04 - val_loss: 3.3400e-04
Epoch 97/100
260/260 [==============================] - 0s 1ms/step - loss: 3.3025e-04 - val_loss: 3.4384e-04
Epoch 98/100
260/260 [==============================] - 0s 922us/step - loss: 3.3522e-04 - val_loss: 4.6095e-04
Epoch 99/100
260/260 [==============================] - 0s 937us/step - loss: 3.5135e-04 - val_loss: 3.6126e-04
Epoch 100/100
260/260 [==============================] - 0s 918us/step - loss: 3.2818e-04 - val_loss: 3.4800e-04
    In [60]:
    plt.plot(epochs_hist.history['loss'], color='#EF5A41')
plt.plot(epochs_hist.history['val_loss'], color='#9A1B5B')
plt.title('Model Loss Progress During Training')
plt.xlabel('Epoch')
plt.ylabel('Training and Validation Loss')
plt.legend(['Training Loss','Validation Loss'])
    Out[60]:
    <matplotlib.legend.Legend at 0x291b7d590>
    No description has been provided for this image
    In [61]:
    y_predict = model.predict(x_test)
plt.plot(y_test, y_predict, '+', color='#EF5A41', markersize=6)
plt.title('Linear Regression Predictions')
plt.xlabel('Model Prediction')
plt.ylabel('True Value')
plt.show()

    169/169 [==============================] - 0s 420us/step
    No description has been provided for this image
    In [62]:
    y_predict_original=scaler.inverse_transform(y_predict)
y_test_original=scaler.inverse_transform(y_test)
    In [63]:
    y_predict = model.predict(x_test)
plt.plot(y_test_original, y_predict_original, '+', color='#EF5A41', markersize=6)
plt.xlabel('Model Prediction')
plt.ylabel('True Value')
plt.xlim(0, 5000000)
plt.ylim(0,3000000)

    169/169 [==============================] - 0s 396us/step
    Out[63]:
    (0.0, 3000000.0)
    No description has been provided for this image
    In [64]:
    RMSE=float(format(np.sqrt(mean_squared_error(y_test_original, y_predict_original)),'.3f'))
MSE=mean_squared_error(y_test_original, y_predict_original)
MAE=mean_absolute_error(y_test_original, y_predict_original)
r2=r2_score(y_test_original, y_predict_original)
adj_r2=1-(1-r2)*(n-1)/(n-k-1)

print('RMSE =', RMSE)
print('MSE =', MSE)
print('MAE =', MAE)
print('R2 =', r2)
print('Adjusted R2 =', adj_r2)

    RMSE = 150048.282
MSE = 22514486929.695152
MAE = 82312.30946983716
R2 = 0.8343927676446907
Adjusted R2 = 0.8339933438931844