Hyperparameter tuning is done to increase the efficiency of a model by tuning the parameters of the neural network. Some scikit-learn APIs like GridSearchCV and RandomizedSearchCV are used to perform hyper parameter tuning.
In this article, you’ll learn how to use GridSearchCV to tune Keras Neural Networks hyper parameters.
Approach:
- We will wrap Keras models for use in scikit-learn using KerasClassifier which is a wrapper.
- We will use cross validation using KerasClassifier and GridSearchCV
- Tune hyperparameters like number of epochs, number of neurons and batch size.
Implementation of the scikit-learn classifier API for Keras:
tf.keras.wrappers.scikit_learn.KerasClassifier(
build_fn=None, **sk_params
)
# import the libraries import tensorflow as tf import pandas as pd from sklearn.compose import ColumnTransformer from sklearn.preprocessing import OneHotEncoder from keras.wrappers.scikit_learn import KerasClassifier from sklearn.model_selection import GridSearchCV from sklearn.preprocessing import LabelEncoder from sklearn.preprocessing import StandardScaler |
The dataset can be downloaded from here
Import the dataset using which we’ll predict if a customer stays or leave.
Code:
# The last column is a binary value dataset = pd.read_csv('Churn_Modelling.csv') X = dataset.iloc[:, 3:-1].values y = dataset.iloc[:, -1].values |
Code: Preprocess the data
le = LabelEncoder() X[:, 2] = le.fit_transform(X[:, 2]) #perform one hot encoding ct = ColumnTransformer(transformers=[('encoder', OneHotEncoder(), [1])], remainder='passthrough') X = np.array(ct.fit_transform(X)) # perform standardization of the data. sc = StandardScaler() X = sc.fit_transform(X) |
To use the KerasClassifier wrapper, we will need to build our model in a function which needs to be passed to the build_fn argument in the KerasClassifier constructor.
Code:
def build_clf(unit): # creating the layers of the NN ann = tf.keras.models.Sequential() ann.add(tf.keras.layers.Dense(units=unit, activation='relu')) ann.add(tf.keras.layers.Dense(units=unit, activation='relu')) ann.add(tf.keras.layers.Dense(units=1, activation='sigmoid')) ann.compile(optimizer = 'adam', loss = 'binary_crossentropy', metrics = ['accuracy']) return ann |
Code: create the object of KerasClassifier class
model=KerasClassifier(build_fn=build_clf) |
Now we will create the dictionary of the parameters we want to tune and pass as an argument in GridSearchCV.
Code:
params={'batch_size':[100, 20, 50, 25, 32], 'nb_epoch':[200, 100, 300, 400], 'unit':[5,6, 10, 11, 12, 15], } gs=GridSearchCV(estimator=model, param_grid=params, cv=10) # now fit the dataset to the GridSearchCV object. gs = gs.fit(X, y) |
The best_score_ member gives the best score observed during the optimization procedure and the best_params_ describes the combination of parameters that achieved the best results.
Code:
best_params=gs.best_params_ accuracy=gs.best_score_ |
Output:
Accuracy: 0.80325
Best Params: {‘batch_size’: 20, ‘nb_epoch’: 200, ‘unit’: 15}
<!–
–>
Please Login to comment…