UMAP 作为分类的特征提取技术

以下脚本展示了 UMAP 如何用作特征提取技术，以提高分类任务的准确性。它还展示了如何将 UMAP 集成到标准的 scikit-learn 管线中。

第一步是创建一个用于分类任务的数据集，这通过函数 sklearn.datasets.make_classification 完成。然后使用 sklearn.model_selection.train_test_split 函数将数据集分为训练集和测试集。

其次，在线性 SVM 上对训练集进行拟合。为了自动选择最佳超参数，在训练集上执行网格搜索。然后使用准确度指标在测试集上评估模型的性能。

第三，重复上一步，但稍作修改：UMAP 被用作特征提取技术。这一微小改动相比使用原始数据的模型带来了显著的改进。

from sklearn.datasets import make_classification
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.pipeline import Pipeline
from sklearn.svm import LinearSVC

from umap import UMAP

# Make a toy dataset
X, y = make_classification(
    n_samples=1000,
    n_features=300,
    n_informative=250,
    n_redundant=0,
    n_repeated=0,
    n_classes=2,
    random_state=1212,
)

# Split the dataset into a training set and a test set
X_train, X_test, y_train, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)

# Classification with a linear SVM
svc = LinearSVC(dual=False, random_state=123)
params_grid = {"C": [10**k for k in range(-3, 4)]}
clf = GridSearchCV(svc, params_grid)
clf.fit(X_train, y_train)
print(
    "Accuracy on the test set with raw data: {:.3f}".format(clf.score(X_test, y_test))
)

# Transformation with UMAP followed by classification with a linear SVM
umap = UMAP(random_state=456)
pipeline = Pipeline([("umap", umap), ("svc", svc)])
params_grid_pipeline = {
    "umap__n_neighbors": [5, 20],
    "umap__n_components": [15, 25, 50],
    "svc__C": [10**k for k in range(-3, 4)],
}


clf_pipeline = GridSearchCV(pipeline, params_grid_pipeline)
clf_pipeline.fit(X_train, y_train)
print(
    "Accuracy on the test set with UMAP transformation: {:.3f}".format(
        clf_pipeline.score(X_test, y_test)
    )
)