LogisticRegression
Enables you to fit a logistic regression model to your data and then predict new class labels and probabilities.
LogisticRegression from SAS Viya on Heart Disease
Source
This example is adapted from Heart Disease UCI by Harshal Gadhe.
Data Preparation
About the data set
The original data contains 76 different attributes of patients from four different hospital databases. The goal is to determine if the attributes can be used to predict whether patients are diagnosed with heart disease. However, this data has been subset to contain only 14 factors from only the Cleveland database. Here are the variables included and their interpretations:
- age
- trestbps: resting blood pressure
- chol: serum cholesterol
- thalch: maximum heart reate achieved
- ca: number of major vessels (0-3) colored by fluoroscopy
- sex
- cp: chest pain type
- exang: exercise-induced angina
- slope: slope of the peak exercise ST segment
- thal: thalassemia (3 = normal; 6=fixed defect; 7 = reversable defect)
- restecg: resting electrocardiographic results
- fbs: fasting blood sugar
- target: diagnosis of heart disease
- oldpeak: ST depression induced by exercise relative to rest
import os
import pandas as pd
import matplotlib.pyplot as plt
import seaborn as sn
import warnings
warnings.filterwarnings("ignore", category=RuntimeWarning)
warnings.filterwarnings("ignore", category=UserWarning)
warnings.simplefilter(action='ignore', category=FutureWarning)
pd.options.mode.chained_assignment = NoneImporting the data set
heart_df = pd.read_csv("https://raw.githubusercontent.com/sassoftware/sas-viya-workbench-examples/refs/heads/main/data/heart_disease.csv")
print(heart_df.head())age trestbps chol thalch oldpeak ... slope thal restecg fbs target 0 63 145 233 150 2.3 ... 0 1 0 1 0 1 67 160 286 108 1.5 ... 1 2 0 0 1 2 67 120 229 129 2.6 ... 1 0 0 0 1 3 37 130 250 187 3.5 ... 0 2 1 0 0 4 41 130 204 172 1.4 ... 2 2 0 0 0
Data Preprocessing
Print information about the data
Start by getting some general characteristics about the data set.
print(heart_df.info())<class 'pandas.core.frame.DataFrame'> RangeIndex: 304 entries, 0 to 303 Data columns (total 14 columns): # Column Non-Null Count Dtype --- ------ -------------- ----- 0 age 304 non-null int64 1 trestbps 304 non-null int64 2 chol 304 non-null int64 3 thalch 304 non-null int64 4 oldpeak 304 non-null float64 5 ca 299 non-null float64 6 sex 304 non-null int64 7 cp 304 non-null int64 8 exang 304 non-null int64 9 slope 304 non-null int64 10 thal 304 non-null int64 11 restecg 304 non-null int64 12 fbs 304 non-null int64 13 target 304 non-null int64 dtypes: float64(2), int64(12) memory usage: 33.3 KB
Replacing NaN values with mean
There are some missing values in the data set. Replace the missing values with the mean for the column.
cols_sum_null = heart_df.isnull().sum()
print(cols_sum_null)
age 0 trestbps 0 chol 0 thalch 0 oldpeak 0 ca 5 sex 0 cp 0 exang 0 slope 0 thal 0 restecg 0 fbs 0 target 0 dtype: int64
hasnull_cols = cols_sum_null[cols_sum_null != 0]
for col in hasnull_cols.index:
mean = heart_df[col].mean()
heart_df[col].fillna(mean, inplace=True)
print(heart_df.isnull().sum())age 0 trestbps 0 chol 0 thalch 0 oldpeak 0 ca 0 sex 0 cp 0 exang 0 slope 0 thal 0 restecg 0 fbs 0 target 0 dtype: int64
Correlations
print(heart_df.corr())age trestbps chol ... restecg fbs target age 1.000000 0.281916 0.225333 ... -0.102142 0.120826 0.228467 trestbps 0.281916 1.000000 0.129773 ... -0.119188 0.175419 0.150904 chol 0.225333 0.129773 1.000000 ... -0.142105 0.012782 0.091021 thalch -0.401508 -0.045660 -0.014568 ... 0.034189 -0.009941 -0.419619 oldpeak 0.209235 0.189202 0.052615 ... -0.050019 0.006968 0.426069 ca 0.354519 0.098706 0.117576 ... -0.107101 0.143925 0.456962 sex -0.102636 -0.064622 -0.203126 ... -0.057334 0.046870 0.274144 cp -0.064347 0.041611 -0.075577 ... 0.044318 0.097061 -0.433479 exang 0.096937 0.064930 0.065811 ... -0.063887 0.026593 0.433054 slope -0.168549 -0.117505 -0.003035 ... 0.083403 -0.061122 -0.341159 thal -0.105685 -0.123009 -0.011651 ... 0.025765 -0.065833 -0.510246 restecg -0.102142 -0.119188 -0.142105 ... 1.000000 -0.080011 -0.134547 fbs 0.120826 0.175419 0.012782 ... -0.080011 1.000000 0.026484 target 0.228467 0.150904 0.091021 ... -0.134547 0.026484 1.000000 [14 rows x 14 columns]
Visualizing the Data
Look at a variety of plots to get a sense of the data set.
- a pair plot of each pair of variables
- a heat map of the correlations
- scatterplots of the target versus each factor
- a histogram of the target's values
cat_threshold = 8
quantitative = [c for c in heart_df.columns if len(heart_df[c].unique()) > cat_threshold]
sn.pairplot(heart_df[quantitative])
SAS.pyplot(plt)
plt.figure(figsize=(12,10))
sn.heatmap(heart_df.corr(),annot=True,cmap=plt.cm.plasma)
SAS.pyplot(plt)
plt.figure(figsize=(15,15))
for i in range(len(heart_df.columns)-1):
plt.subplot(5, 3, i+1)
sn.scatterplot(data=heart_df, x='target', y=heart_df.columns[i], hue='target')
plt.xticks([0, 1])
plt.tight_layout(pad=4.0)
plt.show()
SAS.pyplot(plt)
sn.countplot(x='target', hue='target', legend=False, data=heart_df)
plt.grid()
plt.ylabel("count")
SAS.pyplot(plt)
Building and Training the Model
For details about using the classes in
sasviya.ml, see the Python API documentation.
Data preprocessing
from sklearn.preprocessing import StandardScaler
sc=StandardScaler()
X=heart_df.drop('target',axis=1)
Y=heart_df['target']
heart_df=sc.fit(X).transform(X)Creating training and test data
Split the original data by putting 75% of the data in the training set and 25% of the data into the test set.
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test=train_test_split(X,Y,test_size=0.25,random_state=3)Training the model for six different algorithms
You will train six different models against the training set: LogisticRegression, DecisionTreeClassifier, SVC, ForestClassifier, KNeighborsClassifier, and GaussianNB. The latter two are sklearn classifiers.
from sasviya.ml.linear_model import LogisticRegression
from sasviya.ml.tree import DecisionTreeClassifier
from sasviya.ml.svm import SVC
from sasviya.ml.tree import ForestClassifier
from sklearn.neighbors import KNeighborsClassifier
from sklearn.naive_bayes import GaussianNBdef model(X_train,y_train):
models=[]
lr = LogisticRegression(
solver='lbfgs',
tol=1e-4,
max_iter=1000)
lr.fit(X_train,y_train)
models.append(lr)
tree=DecisionTreeClassifier()
tree.fit(X_train,y_train)
models.append(tree)
svm=SVC(kernel='rbf', coef0=0.1, C=1.0)
svm.fit(X_train,y_train)
models.append(svm)
knn=KNeighborsClassifier(n_neighbors=5)
knn.fit(X_train,y_train)
models.append(knn)
rfc=ForestClassifier()
rfc.fit(X_train,y_train)
models.append(rfc)
nb=GaussianNB()
nb.fit(X_train,y_train)
models.append(nb)
return modelsmodels=model(X_train,y_train)Gathering the accuracy scores
After running the six models, you should examine how well they did relative to each other. For each, gather the accuracy scores for the training and test data and add them into a summary dataframe.
from sklearn.metrics import accuracy_score
train_accuracy=[]
test_accuracy=[]
for m in models:
train_accuracy.append(round(m.score(X_train, y_train),2))
test_accuracy.append(round(m.score(X_test, y_test),2))
Accuracy_score=pd.DataFrame({
'Model': [str(m.__repr__()).split('(')[0]for m in models],
'Train_Accuracy':train_accuracy,
'Test_Accuracy':test_accuracy
})Finding the Best Model
After displaying the table of all accuracy scores for the training and test data, graph the test accuracy for each algorithm to see how they all fared on the data.
print(Accuracy_score)[14 rows x 14 columns]
Model Train_Accuracy Test_Accuracy
0 LogisticRegression 0.88 0.82
1 DecisionTreeClassifier 0.90 0.78
2 SVC 0.85 0.87
3 KNeighborsClassifier 0.75 0.64
4 ForestClassifier 0.92 0.83
5 GaussianNB 0.86 0.87
plt.figure(figsize=(12,6))
plt.plot(Accuracy_score['Model'],Accuracy_score['Test_Accuracy'],marker='x',color='red')
plt.xlabel('Model')
plt.ylabel('Test Accuracy')
plt.title('Test Accuracy by Model')
plt.grid()
SAS.pyplot(plt)