蜜芽av一区二区三区四区,日韩亚洲福利社区,五月婷婷色五月色综合

特征重要性分析用于了解每個(gè)特征(變量或輸入)對(duì)于做出預(yù)測(cè)的有用性或價(jià)值。目標(biāo)是確定對(duì)模型輸出影響最大的最重要的特征，它是機(jī)器學(xué)習(xí)中經(jīng)常使用的一種方法。

為什么特征重要性分析很重要?

如果有一個(gè)包含數(shù)十個(gè)甚至數(shù)百個(gè)特征的數(shù)據(jù)集，每個(gè)特征都可能對(duì)你的機(jī)器學(xué)習(xí)模型的性能有所貢獻(xiàn)。但是并不是所有的特征都是一樣的。有些可能是冗余的或不相關(guān)的，這會(huì)增加建模的復(fù)雜性并可能導(dǎo)致過(guò)擬合。

特征重要性分析可以識(shí)別并關(guān)注最具信息量的特征，從而帶來(lái)以下幾個(gè)優(yōu)勢(shì):

改進(jìn)的模型性能
減少過(guò)度擬合
更快的訓(xùn)練和推理
增強(qiáng)的可解釋性

下面我們深入了解在Python/ target=_blank class=infotextkey>Python中的一些特性重要性分析的方法。

特征重要性分析方法

1、排列重要性 PermutationImportance

該方法會(huì)隨機(jī)排列每個(gè)特征的值，然后監(jiān)控模型性能下降的程度。如果獲得了更大的下降意味著特征更重要

 from sklearn.datasets import load_breast_cancer
 from sklearn.ensemble import RandomForestClassifier
 from sklearn.inspection import permutation_importance
 from sklearn.model_selection import trAIn_test_split
 import matplotlib.pyplot as plt
 
 cancer = load_breast_cancer()
 
 X_train, X_test, y_train, y_test = train_test_split(cancer.data, cancer.target, random_state=1)
 
 rf = RandomForestClassifier(n_estimators=100, random_state=1)
 rf.fit(X_train, y_train)
 
 baseline = rf.score(X_test, y_test)
 result = permutation_importance(rf, X_test, y_test, n_repeats=10, random_state=1, scoring='accuracy')
 
 importances = result.importances_mean
 
 # Visualize permutation importances
 plt.bar(range(len(importances)), importances)
 plt.xlabel('Feature Index')
 plt.ylabel('Permutation Importance')
 plt.show()

2、內(nèi)置特征重要性(coef_或feature_importances_)

一些模型，如線性回歸和隨機(jī)森林，可以直接輸出特征重要性分?jǐn)?shù)。這些顯示了每個(gè)特征對(duì)最終預(yù)測(cè)的貢獻(xiàn)。

 from sklearn.datasets import load_breast_cancer
 from sklearn.ensemble import RandomForestClassifier
 
 X, y = load_breast_cancer(return_X_y=True)
 
 rf = RandomForestClassifier(n_estimators=100, random_state=1)
 rf.fit(X, y)
 
 importances = rf.feature_importances_
 
 # Plot importances
 plt.bar(range(X.shape[1]), importances)
 plt.xlabel('Feature Index')
 plt.ylabel('Feature Importance')
 plt.show()

3、Leave-one-out

迭代地每次刪除一個(gè)特征并評(píng)估準(zhǔn)確性。

 from sklearn.datasets import load_breast_cancer
 from sklearn.model_selection import train_test_split
 from sklearn.ensemble import RandomForestClassifier
 from sklearn.metrics import accuracy_score
 import matplotlib.pyplot as plt
 import numpy as np
 
 # Load sample data
 X, y = load_breast_cancer(return_X_y=True)
 
 # Split data into train and test sets
 X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=1)
 
 # Train a random forest model
 rf = RandomForestClassifier(n_estimators=100, random_state=1)
 rf.fit(X_train, y_train)
 
 # Get baseline accuracy on test data
 base_acc = accuracy_score(y_test, rf.predict(X_test))
 
 # Initialize empty list to store importances
 importances = []
 
 # Iterate over all columns and remove one at a time
 for i in range(X_train.shape[1]):
    X_temp = np.delete(X_train, i, axis=1)
    rf.fit(X_temp, y_train)
    acc = accuracy_score(y_test, rf.predict(np.delete(X_test, i, axis=1)))
    importances.Append(base_acc - acc)
     
 # Plot importance scores    
 plt.bar(range(len(importances)), importances)
 plt.show()

4、相關(guān)性分析

計(jì)算各特征與目標(biāo)變量之間的相關(guān)性。相關(guān)性越高的特征越重要。

 import pandas as pd
 from sklearn.datasets import load_breast_cancer
 
 X, y = load_breast_cancer(return_X_y=True)
 df = pd.DataFrame(X, columns=range(30))
 df['y'] = y
 
 correlations = df.corrwith(df.y).abs()
 correlations.sort_values(ascending=False, inplace=True)
 
 correlations.plot.bar()

5、遞歸特征消除 Recursive Feature Elimination

遞歸地刪除特征并查看它如何影響模型性能。刪除時(shí)會(huì)導(dǎo)致更大下降的特征更重要。

 from sklearn.ensemble import RandomForestClassifier
 from sklearn.feature_selection import RFE
 import pandas as pd
 from sklearn.datasets import load_breast_cancer
 import matplotlib.pyplot as plt
 
 X, y = load_breast_cancer(return_X_y=True)
 df = pd.DataFrame(X, columns=range(30))
 df['y'] = y
 
 rf = RandomForestClassifier()
 
 rfe = RFE(rf, n_features_to_select=10)
 rfe.fit(X, y)
 
 print(rfe.ranking_)

輸出為[6 4 11 12 7 11 18 21 8 16 10 3 15 14 19 17 20 13 11 11 12 9 11 5 11]

6、XGBoost特性重要性

計(jì)算一個(gè)特性用于跨所有樹(shù)拆分?jǐn)?shù)據(jù)的次數(shù)。更多的分裂意味著更重要。

 import xgboost as xgb
 import pandas as pd
 from sklearn.datasets import load_breast_cancer
 import matplotlib.pyplot as plt
 
 X, y = load_breast_cancer(return_X_y=True)
 df = pd.DataFrame(X, columns=range(30))
 df['y'] = y
 
 model = xgb.XGBClassifier()
 model.fit(X, y)
 
 importances = model.feature_importances_
 importances = pd.Series(importances, index=range(X.shape[1]))
 importances.plot.bar()

7、主成分分析 PCA

對(duì)特征進(jìn)行主成分分析，并查看每個(gè)主成分的解釋方差比。在前幾個(gè)組件上具有較高負(fù)載的特性更為重要。

 from sklearn.decomposition import PCA
 import pandas as pd
 from sklearn.datasets import load_breast_cancer
 import matplotlib.pyplot as plt
 
 X, y = load_breast_cancer(return_X_y=True)
 df = pd.DataFrame(X, columns=range(30))
 df['y'] = y
 
 pca = PCA()
 pca.fit(X)
 
 plt.bar(range(pca.n_components_), pca.explained_variance_ratio_)
 plt.xlabel('PCA components')
 plt.ylabel('Explained Variance')

8、方差分析 ANOVA

使用f_classif()獲得每個(gè)特征的方差分析f值。f值越高，表明特征與目標(biāo)的相關(guān)性越強(qiáng)。

 from sklearn.feature_selection import f_classif
 import pandas as pd
 from sklearn.datasets import load_breast_cancer
 import matplotlib.pyplot as plt
 
 X, y = load_breast_cancer(return_X_y=True)
 df = pd.DataFrame(X, columns=range(30))
 df['y'] = y
 
 fval = f_classif(X, y)
 fval = pd.Series(fval[0], index=range(X.shape[1]))
 fval.plot.bar()

9、卡方檢驗(yàn)

使用chi2()獲得每個(gè)特征的卡方統(tǒng)計(jì)信息。得分越高的特征越有可能獨(dú)立于目標(biāo)。

 from sklearn.feature_selection import chi2
 import pandas as pd
 from sklearn.datasets import load_breast_cancer
 import matplotlib.pyplot as plt
 
 X, y = load_breast_cancer(return_X_y=True)
 df = pd.DataFrame(X, columns=range(30))
 df['y'] = y
 
 chi_scores = chi2(X, y)
 chi_scores = pd.Series(chi_scores[0], index=range(X.shape[1]))
 chi_scores.plot.bar()