Moron Manipulator: The Impact of Generative AI on Critical Thinking

notebook.ipynb

{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "install-libraries",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Install necessary libraries\n",
    "!pip install ipywidgets seaborn statsmodels\n",
    "import ipywidgets as widgets\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "introduction",
   "metadata": {},
   "source": [
    "# Moron Manipulator Introduction: Algorithmic Manipulation in Generative AI\n",
    "\n",
    "As Generative AI becomes increasingly integrated into knowledge work, there is a growing need to ensure that these systems enhance rather than diminish human critical thinking. One of the primary risks is **cognitive offloading**, where users overly rely on AI outputs, bypassing essential evaluative processes.\n",
    "\n",
    "This tutorial introduces an algorithmic approach to counteract such risks by implementing a feature known as the **\"Moron Manipulator\"**. Despite its provocative name, the Moron Manipulator is designed to algorithmically detect when users may be at risk of uncritical acceptance and to trigger interventions that prompt deeper scrutiny of AI-generated outputs.\n",
    "\n",
    "In the following sections, we will detail the features of this approach, discuss the inherent risks, outline various usage scenarios, and provide step-by-step instructions for implementing the algorithm."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "features",
   "metadata": {},
   "source": [
    "# Features of the Moron Manipulator\n",
    "\n",
    "1. **Dynamic Thresholding:** The algorithm dynamically adjusts intervention thresholds based on user confidence metrics.\n",
    "2. **Risk Detection:** It employs a rule-based mechanism to flag when high AI confidence coincides with low self-confidence.\n",
    "3. **Automated Intervention:** When risk conditions are met, the system automatically prompts the user to re-evaluate the AI output.\n",
    "4. **Interactive Customization:** Users can interactively modify thresholds to fine-tune the intervention sensitivity to their specific context."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "risks",
   "metadata": {},
   "source": [
    "# Risks of Algorithmic Manipulation\n",
    "\n",
    "While algorithmic manipulation can safeguard against cognitive offloading, it also carries inherent risks:\n",
    "\n",
    "1. **Overreliance on AI:** Excessive interventions may inadvertently lead to dependence on automated prompts.\n",
    "2. **Algorithmic Bias:** Poorly calibrated thresholds might flag or miss cases, introducing bias in the intervention process.\n",
    "3. **False Positives/Negatives:** The rule-based system may incorrectly flag benign behavior or fail to detect genuine risk scenarios.\n",
    "4. **User Frustration:** Frequent or poorly timed prompts may disrupt workflow and reduce user satisfaction."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "usages",
   "metadata": {},
   "source": [
    "# Usages of the Moron Manipulator\n",
    "\n",
    "The algorithmic intervention strategy can be applied across various domains:\n",
    "\n",
    "1. **Quality Assurance:** In knowledge work, ensuring that AI outputs are critically evaluated before adoption.\n",
    "2. **Educational Settings:** Helping learners develop critical thinking skills when using AI assistance.\n",
    "3. **Decision Support Systems:** Assisting professionals in high-stakes environments to verify AI recommendations.\n",
    "4. **Behavioral Research:** Studying the impact of interventions on user behavior and cognitive engagement."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "step-by-step",
   "metadata": {},
   "source": [
    "# Step-by-Step Instructions\n",
    "\n",
    "This tutorial is organized into the following algorithmic steps:\n",
    "\n",
    "1. **Data Simulation & Cleaning:** Generate a synthetic dataset that simulates survey data on AI confidence, self-confidence, and critical thinking effort. Clean the data to handle missing or out-of-range values.\n",
    "2. **Exploratory Data Analysis (EDA):** Analyze the dataset using summary statistics and visualizations to understand key relationships.\n",
    "3. **Regression Modeling:** Build a linear regression model to quantify how AI confidence and self-confidence impact critical thinking effort.\n",
    "4. **Intervention Simulation (Moron Manipulator):** Implement a rule-based algorithm to detect risk conditions and trigger interventions.\n",
    "5. **Interactive UI Elements:** Use interactive widgets to adjust intervention thresholds in real time.\n",
    "6. **Evaluation:** Assess model performance and intervention effectiveness using error metrics and residual analysis.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "simulate-data",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "np.random.seed(42)  # For reproducibility\n",
    "\n",
    "# Number of survey responses\n",
    "n_samples = 936\n",
    "\n",
    "# Simulate data: AI confidence, self-confidence, and critical thinking effort\n",
    "confidence_AI = np.random.uniform(0, 100, n_samples)  # AI confidence (0-100)\n",
    "self_confidence = np.random.uniform(0, 100, n_samples)    # Self-confidence (0-100)\n",
    "\n",
    "# Model: higher AI confidence reduces critical thinking, higher self-confidence increases it\n",
    "critical_thinking_effort = 50 + 0.5 * self_confidence - 0.5 * confidence_AI + np.random.normal(0, 10, n_samples)\n",
    "\n",
    "# Create DataFrame\n",
    "df = pd.DataFrame({\n",
    "    'confidence_AI': confidence_AI,\n",
    "    'self_confidence': self_confidence,\n",
    "    'critical_thinking_effort': critical_thinking_effort\n",
    "})\n",
    "\n",
    "# Introduce intentional errors for demonstration\n",
    "df.loc[0, 'critical_thinking_effort'] = -5   # Negative value\n",
    "df.loc[1, 'critical_thinking_effort'] = 120   # Value above 100\n",
    "df.loc[10, 'self_confidence'] = np.nan\n",
    "df.loc[11, 'confidence_AI'] = np.nan\n",
    "df.loc[12, 'critical_thinking_effort'] = np.nan\n",
    "\n",
    "print(\"Raw data sample (first 5 rows):\\n\", df.head())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "clean-data",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Data Cleaning\n",
    "print(\"Missing values in each column:\\n\", df.isna().sum())\n",
    "\n",
    "# Remove rows with missing values\n",
    "df_clean = df.dropna().copy()\n",
    "print(f\"\\nAfter dropping missing values, dataset has {df_clean.shape[0]} entries (out of {n_samples}).\")\n",
    "\n",
    "# Clamp critical thinking effort to [0, 100]\n",
    "df_clean['critical_thinking_effort'] = df_clean['critical_thinking_effort'].clip(lower=0, upper=100)\n",
    "\n",
    "# Reset index\n",
    "df_clean.reset_index(drop=True, inplace=True)\n",
    "\n",
    "print(\"\\nCleaned data sample (first 5 rows):\\n\", df_clean.head())\n",
    "print(\"\\nData summary after cleaning:\\n\", df_clean.describe().round(2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eda",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Exploratory Data Analysis (EDA)\n",
    "print(\"Summary statistics:\\n\", df_clean[['confidence_AI','self_confidence','critical_thinking_effort']].describe().round(2))\n",
    "\n",
    "# Compute and display correlation matrix\n",
    "corr_matrix = df_clean[['confidence_AI','self_confidence','critical_thinking_effort']].corr().round(2)\n",
    "print(\"\\nCorrelation matrix:\\n\", corr_matrix)\n",
    "\n",
    "import matplotlib.pyplot as plt\n",
    "plt.figure(figsize=(8,5))\n",
    "plt.hist(df_clean['confidence_AI'], bins=20, alpha=0.5, label='AI Confidence')\n",
    "plt.hist(df_clean['self_confidence'], bins=20, alpha=0.5, label='Self-Confidence')\n",
    "plt.hist(df_clean['critical_thinking_effort'], bins=20, alpha=0.5, label='Critical Thinking Effort')\n",
    "plt.title('Distribution of Confidence Levels and Critical Thinking Effort')\n",
    "plt.xlabel('Value')\n",
    "plt.ylabel('Frequency')\n",
    "plt.legend()\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "regression-model",
   "metadata": {},
   "outputs": [],
   "source": [
    "import statsmodels.formula.api as smf\n",
    "\n",
    "# Regression Modeling: Quantify the impact on critical thinking effort\n",
    "model = smf.ols('critical_thinking_effort ~ confidence_AI + self_confidence', data=df_clean).fit()\n",
    "coeffs = model.params\n",
    "pvals = model.pvalues\n",
    "\n",
    "print(\"Regression coefficients:\")\n",
    "for var, coef in coeffs.items():\n",
    "    print(f\"  {var}: {coef:.3f}\")\n",
    "\n",
    "print(\"\\nP-values:\")\n",
    "for var, p in pvals.items():\n",
    "    print(f\"  {var}: {p:.3f}\")\n",
    "\n",
    "print(\"\\nModel Summary:\\n\", model.summary())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "moron-manipulator",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Intervention Simulation: Implement the Moron Manipulator\n",
    "\n",
    "# Define intervention thresholds\n",
    "high_ai_threshold = 75.0\n",
    "low_self_threshold = 50.0\n",
    "\n",
    "# Rule-based flagging: If AI confidence is high and self-confidence is low, trigger intervention\n",
    "df_clean['intervention_flag'] = (\n",
    "    (df_clean['confidence_AI'] > high_ai_threshold) & \n",
    "    (df_clean['self_confidence'] < low_self_threshold)\n",
    ")\n",
    "\n",
    "num_flagged = df_clean['intervention_flag'].sum()\n",
    "perc_flagged = 100 * num_flagged / df_clean.shape[0]\n",
    "print(f\"Intervention suggested in {num_flagged} out of {df_clean.shape[0]} cases ({perc_flagged:.1f}%).\")\n",
    "\n",
    "# Compare average critical thinking effort in flagged vs. unflagged cases\n",
    "flagged_ct_mean = df_clean[df_clean['intervention_flag'] == True]['critical_thinking_effort'].mean()\n",
    "unflagged_ct_mean = df_clean[df_clean['intervention_flag'] == False]['critical_thinking_effort'].mean()\n",
    "print(f\"\\nAverage critical thinking effort in flagged cases: {flagged_ct_mean:.1f}\")\n",
    "print(f\"Average critical thinking effort in unflagged cases: {unflagged_ct_mean:.1f}\")\n",
    "\n",
    "def check_intervention(conf_ai, self_conf):\n",
    "    return (conf_ai > high_ai_threshold) and (self_conf < low_self_threshold)\n",
    "\n",
    "example_high_AI = {'confidence_AI': 90, 'self_confidence': 30}\n",
    "example_low_AI = {'confidence_AI': 40, 'self_confidence': 80}\n",
    "\n",
    "print(\"\\nExample 1:\", example_high_AI, \"-> Intervention?\", check_intervention(**example_high_AI))\n",
    "print(\"Example 2:\", example_low_AI, \"-> Intervention?\", check_intervention(**example_low_AI))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ui-interactive",
   "metadata": {},
   "outputs": [],
   "source": [
    "from ipywidgets import interact\n",
    "\n",
    "def update_intervention(high_ai, low_self):\n",
    "    # Update intervention flags using user-specified thresholds\n",
    "    df_clean['intervention_flag'] = (df_clean['confidence_AI'] > high_ai) & (df_clean['self_confidence'] < low_self)\n",
    "    num_flagged = df_clean['intervention_flag'].sum()\n",
    "    perc_flagged = 100 * num_flagged / df_clean.shape[0]\n",
    "    print(f\"Intervention suggested in {num_flagged} out of {df_clean.shape[0]} cases ({perc_flagged:.1f}%).\")\n",
    "    \n",
    "    flagged_ct_mean = df_clean[df_clean['intervention_flag']]['critical_thinking_effort'].mean()\n",
    "    unflagged_ct_mean = df_clean[~df_clean['intervention_flag']]['critical_thinking_effort'].mean()\n",
    "    print(f\"Average critical thinking effort in flagged cases: {flagged_ct_mean:.1f}\")\n",
    "    print(f\"Average critical thinking effort in unflagged cases: {unflagged_ct_mean:.1f}\")\n",
    "\n",
    "interact(update_intervention, \n",
    "         high_ai=widgets.IntSlider(min=0, max=100, step=1, value=75, description=\"High AI Threshold\"),\n",
    "         low_self=widgets.IntSlider(min=0, max=100, step=1, value=50, description=\"Low Self-Threshold\"));\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "evaluation",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "# Evaluate the regression model\n",
    "predictions = model.predict(df_clean[['confidence_AI', 'self_confidence']])\n",
    "rmse = np.sqrt(mean_squared_error(df_clean['critical_thinking_effort'], predictions))\n",
    "print(f\"RMSE of the regression model: {rmse:.2f}\")\n",
    "\n",
    "print(f\"Total interventions flagged: {num_flagged}\")\n",
    "\n",
    "# Residual analysis: Histogram of residuals\n",
    "residuals = df_clean['critical_thinking_effort'] - predictions\n",
    "plt.figure(figsize=(6,4))\n",
    "sns.histplot(residuals, bins=20, kde=True)\n",
    "plt.title('Residual Distribution')\n",
    "plt.xlabel('Residual')\n",
    "plt.ylabel('Frequency')\n",
    "plt.show()\n",
    "\n",
    "# Scatter plot of residuals vs. predicted values\n",
    "plt.figure(figsize=(6,4))\n",
    "plt.scatter(predictions, residuals, alpha=0.5)\n",
    "plt.axhline(0, color='red', linestyle='--')\n",
    "plt.title('Residuals vs Predicted Values')\n",
    "plt.xlabel('Predicted Critical Thinking Effort')\n",
    "plt.ylabel('Residuals')\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "conclusions",
   "metadata": {},
   "source": [
    "# Conclusions & Summary\n",
    "\n",
    "This tutorial presented an algorithmically focused approach to enhancing critical thinking in AI-assisted workflows. By simulating a dataset, performing regression modeling, and implementing the **Moron Manipulator** intervention, we demonstrated how algorithmic manipulation can help detect and mitigate cognitive offloading.\n",
    "\n",
    "The step-by-step instructions provided a framework to:\n",
    "\n",
    "- Simulate and clean data\n",
    "- Explore and analyze key metrics\n",
    "- Build and evaluate a predictive model\n",
    "- Implement and adjust rule-based interventions in real time\n",
    "\n",
    "By leveraging these techniques, practitioners can develop AI systems that balance automation with critical engagement, ensuring that users remain thoughtful and resilient in their decision-making processes."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "name": "python",
   "version": "3.8.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}