Cfrgtkky

I recently wrote a Logistic regression model using Scikit Module. However, I'm having a REALLY HARD time plotting the decision boundary line. I'm explicitly multiplying the Coefficients and the Intercepts and plotting them (which in turn throws a wrong figure).

Could someone point me in the right direction on how to plot the decision boundary?

Is there an easier way to plot the line without having to manually multiply the coefficients and the intercepts?

Thanks a Million!

import pandas as pd

import matplotlib.pyplot as plt

import numpy as np

from sklearn.linear_model import LogisticRegression

from sklearn.model_selection import train_test_split

from sklearn.metrics import accuracy_score



#Import Dataset

dataset = pd.read_csv("Students Exam Dataset.txt", names=["Exam 1", "Exam 2", "Admitted"])

print(dataset.head())



#Visualizing Dataset

positive = dataset[dataset["Admitted"] == 1]

negative = dataset[dataset["Admitted"] == 0]



plt.scatter(positive["Exam 1"], positive["Exam 2"], color="blue", marker="o", label="Admitted")

plt.scatter(negative["Exam 1"], negative["Exam 2"], color="red", marker="x", label="Not Admitted")

plt.title("Student Admission Plot")

plt.xlabel("Exam 1")

plt.ylabel("Exam 2")

plt.legend()

plt.plot()

plt.show()



#Preprocessing Data

col = len(dataset.columns)

x = dataset.iloc[:,0:col].values

y = dataset.iloc[:,col-1:col].values

print(f"X Shape: {x.shape}   Y Shape: {y.shape}")



x_train, x_test, y_train, y_test = train_test_split(x, y, test_size=0.2, random_state=1306)



#Initialize Model

reg = LogisticRegression()

reg.fit(x_train, y_train)



#Output

predictions = reg.predict(x_test)

accuracy = accuracy_score(y_test, predictions) * 100

coeff = reg.coef_

intercept = reg.intercept_

print(f"Accuracy Score : {accuracy} %")

print(f"Coefficients = {coeff}")

print(f"Intercept Coefficient = {intercept}")



#Visualizing Output

xx = np.linspace(30,100,100)

decision_boundary = (coeff[0,0] * xx + intercept.item()) / coeff[0,1]

plt.scatter(positive["Exam 1"], positive["Exam 2"], color="blue", marker="o", label="Admitted")

plt.scatter(negative["Exam 1"], negative["Exam 2"], color="red", marker="x", label="Not Admitted")

plt.plot(xx, decision_boundary, color="green", label="Decision Boundary")

plt.title("Student Admission Plot")

plt.xlabel("Exam 1")

plt.ylabel("Exam 2")

plt.legend()

plt.show()

Dataset: Student Dataset.txt

Is there an easier way to plot the line without having to manually multiply the coefficients and the intercepts?

Yes, if you don't need to build this from scratch, there is an excellent implementation of plotting decision boundaries from scikit-learn classifiers in the mlxtend package. The documentation is extensive in the link provided and it's easy to install with pip install mlxtend.

First, a couple points about the Preprocessing block of the code you posted:

1. x should not include the class labels.

2. y should be a 1d array.

#Preprocessing Data

col = len(dataset.columns)

x = dataset.iloc[:,0:col-1].values # assumes your labels are always in the final column.

y = dataset.iloc[:,col-1:col].values

y = y.reshape(-1) # convert to 1d

Now the plotting is as easy as:

from mlxtend.plotting import plot_decision_regions

plot_decision_regions(x, y,

                      X_highlight=x_test,

                      clf=reg,

                      legend=2)

This particular plot highlights x_test data points by encircling them.