Skip to main content

Logistic Regression with TensorFlow

In this section, we will implement Logistic Regression using the TensorFlow library. Like in the scikit-learn example, we will use the Pima Indians Diabetes dataset to predict whether or not a person has diabetes.

Steps Covered:​

  1. Loading and preparing the dataset.
  2. Building the logistic regression model.
  3. Training the model using gradient descent.
  4. Evaluating the model’s performance.
  5. Making predictions on new data.

1. Load and Prepare the Dataset​

We'll start by loading the Pima Indians Diabetes dataset using pandas and preparing it for use in TensorFlow.

# Import required libraries
import tensorflow as tf
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler

# Load the dataset
url = "https://raw.githubusercontent.com/jbrownlee/Datasets/master/pima-indians-diabetes.data.csv"
column_names = ['Pregnancies', 'Glucose', 'BloodPressure', 'SkinThickness', 'Insulin', 
                'BMI', 'DiabetesPedigreeFunction', 'Age', 'Outcome']

data = pd.read_csv(url, names=column_names)

# Split into features (X) and target (y)
X = data.drop('Outcome', axis=1)
y = data['Outcome']

# Split the data into 80% training and 20% testing sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)

# Normalize the features for better performance with gradient-based methods
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# Convert the data into TensorFlow tensors
X_train_tensor = tf.convert_to_tensor(X_train_scaled, dtype=tf.float32)
X_test_tensor = tf.convert_to_tensor(X_test_scaled, dtype=tf.float32)
y_train_tensor = tf.convert_to_tensor(y_train.values, dtype=tf.float32)
y_test_tensor = tf.convert_to_tensor(y_test.values, dtype=tf.float32)

Explanation:​

  • We load the dataset and split it into features (X) and target (y).
  • Standardization: We standardize the features using StandardScaler to improve the performance of gradient descent.
  • The data is then converted into TensorFlow tensors for further processing.

2. Build the Logistic Regression Model​

In TensorFlow, we will define the logistic regression model using a simple linear layer followed by a sigmoid activation function.

# Set seed for reproducibility
tf.random.set_seed(42)

# Define the logistic regression model using a single linear layer
model = tf.keras.Sequential([
    tf.keras.layers.Dense(1, input_shape=(X_train.shape[1],), activation='sigmoid')
])

# Compile the model with binary cross-entropy loss and stochastic gradient descent optimizer
model.compile(optimizer='sgd', loss='binary_crossentropy', metrics=['accuracy'])

# Print the model summary
model.summary()

Explanation:​

  • Sequential Model: We define a simple sequential model with one dense layer and a sigmoid activation function. The sigmoid ensures the output is between 0 and 1, suitable for binary classification.
  • Binary Cross-Entropy Loss: We use binary_crossentropy as the loss function, which is ideal for binary classification problems.
  • SGD Optimizer: We use stochastic gradient descent (SGD) to optimize the model.

3. Train the Logistic Regression Model​

We now train the logistic regression model on the training data using the fit() method.

# Train the model for 100 epochs
history = model.fit(X_train_tensor, y_train_tensor, epochs=100, batch_size=32, validation_split=0.2)

Explanation:​

  • Epochs: The model will train for 100 complete passes over the training data.
  • Batch Size: We use a batch size of 32 for gradient descent updates.
  • Validation Split: 20% of the training data is used for validation during training to monitor the model’s performance on unseen data.

4. Evaluate the Model​

Once the model is trained, we will evaluate its performance on the test set by calculating the accuracy.

# Evaluate the model on the test data
test_loss, test_accuracy = model.evaluate(X_test_tensor, y_test_tensor)

print(f"Test Accuracy: {test_accuracy * 100:.2f}%")

Explanation:​

  • The evaluate() function calculates the loss and accuracy of the model on the test set.
  • The result will show how well the model generalizes to unseen data.

5. Make Predictions on New Data​

Finally, we use the trained model to make predictions on new data.

# Example of new data for prediction (standardized input)
new_data = [[6, 148, 72, 35, 0, 33.6, 0.627, 50]]  # Example input
new_data_scaled = scaler.transform(new_data)  # Scale the new data

# Convert to tensor and make prediction
new_data_tensor = tf.convert_to_tensor(new_data_scaled, dtype=tf.float32)
predicted_proba = model.predict(new_data_tensor)

print(f"Probability of being Diabetic: {predicted_proba[0][0] * 100:.2f}%")

Explanation:​

  • New Data: We use a single new data point and predict the probability that the individual is diabetic.
  • The output is the predicted probability, which can be interpreted as a percentage.

Summary​

In this section, we successfully implemented Logistic Regression using TensorFlow by following these steps:

  • Loading and preparing the dataset: We normalized the data and converted it into TensorFlow tensors.
  • Building the model: We created a logistic regression model using a single dense layer with a sigmoid activation function.
  • Training the model: We trained the model using stochastic gradient descent (SGD) and monitored its performance.
  • Evaluating the model: We calculated the accuracy on the test set.
  • Making predictions: We used the trained model to predict the probability of diabetes for new data.

In the next section, we will explore how to implement logistic regression using PyTorch to provide an additional implementation option.