# Data Handling and Manipulation
import pandas as pd
import numpy as np
# Data Preprocessing
from sklearn.preprocessing import StandardScaler, OneHotEncoder
from mlxtend.preprocessing import TransactionEncoder
# Model Selection and Evaluation
from sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV
from sklearn.metrics import silhouette_score, davies_bouldin_score, calinski_harabasz_score
from sklearn.mixture import GaussianMixture
# Machine Learning Models
from mlxtend.frequent_patterns import apriori, association_rules, fpgrowth
from mlxtend.preprocessing import TransactionEncoder
# Stats
from scipy.stats import pearsonr
import statsmodels.api as sm
# Data Visualization
import matplotlib.pyplot as plt
import seaborn as sns
# Set the default style for visualization
sns.set_theme(style = "white", palette = "colorblind")
# Increase font size of all Seaborn plot elements
sns.set(font_scale = 1.25)Association
Data Mining and Discovery
Frequent itemset generation
Given \(d\) items, there are \(2^d\) itemsets:
Apriori method
All subsets of a frequent itemset must also be frequent.
Rule generation
- Calculate support for each itemset to determine which meet the threshold.
- Generate association rules.
\(\text{Support}(A \rightarrow B) = \frac{Freq(A,B)}{N}\)
Evaluate rule strength
- Assess the likelihood that consequent item is purchased with antecedent.
- Filtering based on a confidence threshold.
\(\text{Confidence}(A \rightarrow B) = \frac{Support(A \rightarrow B)}{Support(A)}\)
Contribute to decision making
- Identify strong associations
- Improving predictive accuracy
Pros:
Simplicity: Straightforward and easy to understand.
Generality: Applicable to any transaction dataset without needing domain-specific knowledge.
Widely Used: A foundational technique that informs many other algorithms.
Cons:
Performance Issues: Computationally expensive for large datasets with many items.
Memory Intensive: Requires significant memory for storing candidates at lower support thresholds.
Redundant Computations: Generates many candidates, leading to unnecessary computations.
Frequent itemset generation: ECLAT
ECLAT uses vertical data layout to store a list of transaction ids for each item:
Advantage: very fast support counting
Disadvantage: intermediate tid-lists may become too large for memory
Frequent itemset generation: FP-Growth
Add Items from Transaction T1 to the FP Tree
Initialize Null root node for the FP-tree.
Select first transaction T1: sorted items [I3, I1, I4].
At root, no child for I3 exists; create child node for I3 with count 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T1 to the FP Tree
Move to node I3 and proceed to next item I1 in transaction.
No child node for I1 under I3; create child node for I1 with count 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T1 to the FP Tree
Move to node I1 and next item I4 in transaction.
No child node for I4 under I1; create child node for I4 with count 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T2 to the FP Tree
Complete traversal of T1; move to T2 with items [I3, I5, I2, I6].
At root, find child node with I3; increment its count by 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T2 to the FP Tree
Move to modified I3 node and next item I5 in T2.
No child node for I5 under I3; create child node for I5 with count 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T2 to the FP Tree
Move to newly created I5 node and next item I2 in transaction.
No child node for I2 under I5; create child node for I2 with count 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T2 to the FP Tree
Move to newly created I2 node and next item I6 in transaction.
No child node for I6 under I2; create child node for I6 with count 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T3 to the FP Tree
Finished traversing T2; start T3 with items [I3, I5, I1, I2].
At root, find I3; increment its count by 1 since it exists.
Frequent itemset generation: FP-Growth
Add Items from Transaction T3 to the FP Tree
Move to modified I3 node and next item I5 in transaction.
Find existing child node for I5 under I3; increment I5’s count by 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T3 to the FP Tree
Move to modified I5 node and next item I1 in transaction.
No child node for I1 under I5; create child node for I1 with count 1.
Frequent itemset generation: FP-Growth
Add Items from Transaction T3 to the FP Tree
Move to newly created I1 node and next item I2 in transaction.
No child node for I2 under I1; create child node for I2 with count 1.
Frequent itemset generation: FP-Growth
Add Items From Transaction T4 to the FP Tree
Starting with I5 at root, check for child node with I5.
No child node for I5; create new child node for I5 with count 1 at root.
Frequent itemset generation: FP-Growth
Add Items From Transaction T4 to the FP Tree
Starting with I5 at root, check for child node with I5.
No child node for I5; create new child node for I5 with count 1 at root.
Maximal frequent itemsets
An itemset is maximal frequent if none of its immediate supersets is frequent:
Closed itemsets
None of an itemset’s its immediate supersets has the same support as the itemset
- Can only have smaller support \(\rightarrow\) see APRIORI principle above
Closed vs. maximal itemsets
In-class Exercise
Go to ex-13 and perform the tasks
