Python Lesson 8: Machine Learning

Machine learning is about extracting knowledge from data. It is a research field at the intersection of statistics, artificial intelligence, and computer science and is also known as predictive analytics or statistical learning. The application of machine learning methods has in recent years become ubiquitous in everyday life. From auto‐ matic recommendations of which movies to watch, to what food to order or which products to buy, to personalized online radio and recognizing your friends in your photos, many modern websites and devices have machine learning algorithms at their core. When you look at a complex website like Facebook, Amazon, or Netflix, it is very likely that every part of the site contains multiple machine learning models.

The most successful kinds of machine learning algorithms are those that automate decision-making processes by generalizing from known examples. In this setting, which is known as supervised learning, the user provides the algorithm with pairs of inputs and desired outputs, and the algorithm finds a way to produce the desired out‐ put given an input. In particular, the algorithm is able to create an output for an input it has never seen before without any help from a human.

Machine learning algorithms that learn from input/output pairs are called supervised learning algorithms because a “teacher” provides supervision to the algorithms in the form of the desired outputs for each example that they learn from. Examples: identifying the zip code from handwritten digits on an envelope, determining whether a tumor is benign based on a medical image, or detecting fraudulent activity in credit card transactions

In unsupervised learning, only the input data is known, and no known output data is given to the algorithm. While there are many successful applications of these methods, they are usually harder to understand and evaluate. Examples: identifying topics in a set of blog posts, segmenting customers into groups with similar preferences, or detecting abnormal access patterns to a website.

Concerning data, often it is helpful to think of your data as a table. Each data point that you want to reason about (each email, each customer, each transaction) is a row, and each property that describes that data point (say, the age of a customer or the amount or location of a transaction) is a column. You might describe users by their age, their gender, when they created an account, and how often they have bought from your online shop. You might describe the image of a tumor by the grayscale values of each pixel, or maybe by using the size, shape, and color of the tumor. Each entity or row here is known as a sample (or data point) in machine learning, while the columns~—the properties that describe these entities—are called ~features

Scikit-learn is an open source machine learning library that supports supervised and unsupervised learning. It also provides various tools for model fitting, data preprocessing, model selection, model evaluation, and many other utilities. To install scikit-learn simply type:

conda install scikit-learn

The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    joblib-1.1.0               |     pyhd8ed1ab_0         210 KB  conda-forge
    scikit-learn-1.1.2         |   py39he5e8d7e_0         8.2 MB  conda-forge
    threadpoolctl-3.1.0        |     pyh8a188c0_0          18 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         8.5 MB

The following NEW packages will be INSTALLED:

  joblib             conda-forge/noarch::joblib-1.1.0-pyhd8ed1ab_0
  scikit-learn       conda-forge/linux-64::scikit-learn-1.1.2-py39he5e8d7e_0
  threadpoolctl      conda-forge/noarch::threadpoolctl-3.1.0-pyh8a188c0_0


Proceed ([y]/n)? 


Downloading and Extracting Packages
scikit-learn-1.1.2   | 8.2 MB    | ##################################### | 100% 
joblib-1.1.0         | 210 KB    | ##################################### | 100% 
threadpoolctl-3.1.0  | 18 KB     | ##################################### | 100% 
Preparing transaction: done
Verifying transaction: done

An alternative way is to use pip install scikit-learn

Moreover, we will use also by default the libraries: numpy, pandas, scipy, and matplotlib.

First we read a set of standard databases include in skit-learn, correspondig to a copy of the test set of the UCI ML hand-written digits datasets

https://archive.ics.uci.edu/ml/datasets/Optical+Recognition+of+Handwritten+Digits

from sklearn import datasets
digits = datasets.load_digits()

See https://scikit-learn.org/stable/datasets.html for the complete list of datasets or go to https://archive-beta.ics.uci.edu/

To see the keys of the dataset or to show the content of one the keys:

print("Keys of digit dataset: \n{}".format(digits.keys()))
print("Description of digit dataset: \n{}".format(digits['DESCR']))

To get access to the data

print(digits.data)
print(digits.data.shape)
print(digits.target)
print(digits.target.shape)

The attribute data provides the instances while target provides the class of the different instances. Note that data correspond to 2D arrays (8×8). In particular, for this dataset one ca use the attribute image

print(digits.images[0])
digits.images.shape

In this case, we will consider a digit dataset, the task is to predict, given an image, which digit it represents. We are given samples of each of the 10 possible classes (the digits zero through nine) on which we fit an estimator to be able to predict the classes to which unseen samples belong.

In scikit-learn, an estimator for classification is a Python object that implements the methods fit(X, y) and predict(T).

An example of an estimator is the class sklearn.svm.SVC, which implements the support vector classification. The estimator’s constructor takes as arguments the model’s parameters.

from sklearn import svm
clf = svm.SVC(gamma=0.001, C=100.)
clf.fit(digits.data[:-10], digits.target[:-10])
print(clf.predict(digits.data[-10:]))
print(digits.target[-10:])

The clf (for classifier) estimator instance is first fitted to the model; that is, it must learn from the model. This is done by passing our training set to the fit method. For the training set, we’ll use all the images from our dataset, except for the last ten images, which we’ll reserve for our predicting. We select the training set with the [:-10] Python syntax, which produces a new array that contains all but the last ten items from digits.data. You can predict new values. In this case, you’ll predict using the last image from digits.data. By predicting, you’ll determine the image from the training set that best matches the last image. In this example we are using the Support Vector Machines which are a set of supervised learning methods used for classification, regression and outliers detection. You can see the different possible supervised methods in https://scikit-learn.org/stable/supervised_learning.html or the unsupervised ones in https://scikit-learn.org/stable/unsupervised_learning.html

Also you can plot the images and check visually the performance of the calculation.

fig, ax = plt.subplots(2,5,figsize=(10,4))
i=0
for row in range(2):
    for col in range(5):
        ax[row,col].imshow(digits.images[-10+i], cmap=plt.cm.gray_r, interpolation="nearest")
        i+=1

However it is much more convenient to separate our original in two sets, training and test sets, selected both in a random way.

from sklearn.model_selection import train_test_split

X_train, X_test, y_train, y_test = train_test_split(
    digits.data, digits.target, test_size=0.5, shuffle=False
)

clf.fit(X_train, y_train)
predicted = clf.predict(X_test)

Now can analyze the performance of our learning procedure:

print(
    f"Classification report for classifier {clf}:\n"
    f"{metrics.classification_report(y_test, predicted)}\n"
)
#
disp = metrics.ConfusionMatrixDisplay.from_predictions(y_test, predicted)
disp.figure_.suptitle("Confusion Matrix")
print(f"Confusion matrix:\n{disp.confusion_matrix}")

plt.show()

Note that the performace of the prediction can be measured in different ways: precission= (true positives)/(trues positives + true negatives), recall= (true positives)/(trues positives + false potitives), F1 = 2 * (precision * recall) / (precision + recall), among other options. On the other hand, the confusion matrix provides a detailed view of the agreement between the predition and the tests. Note that for other election and training and test sets, the results could be different.

Linear regression, or ordinary least squares (OLS), is the simplest and most classic linear method for regression. Linear regression finds the parameters \(w\) and \(b\) that minimize the mean squared error between predictions and the true regression targets, \(y\), on the training set.

We define our working dataset

from sklearn import metrics 
from sklearn.model_selection import train_test_split
from sklearn import datasets
from sklearn.linear_model import LinearRegression
data_reg=datasets.make_regression(n_features=1,n_samples=100, noise=10, bias=2)
X=data_reg[0]
y=data_reg[1]

We separate into training and test sets, train the model

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42)
lr = LinearRegression().fit(X_train, y_train)

We show the coefficients

print("lr.coef_: {}".format(lr.coef_))
print("lr.intercept_: {}".format(lr.intercept_))

We present graphically the results

plt.plot(X_test,y_test,'.r', label='test')
plt.plot(X_train,y_train,'.b', label='train')
plt.plot(X,lr.predict(X),'-k', label='fit')
plt.legend()

This example can be extended easily to the multidimensional case.

As one can see there is no way to modify anything in the fitting process, for example in the case of overfitting. To this end there exists other alternatives, e.g., Ridge and Lasso.

Ridge regression is also a linear model for regression, so the formula it uses to make predictions is the same one used for ordinary least squares. In ridge regression, though, the coefficients (\(w\)) are chosen not only so that they predict well on the training data, but also to fit an additional constraint. We also want the magnitude of coefficients to be as small as possible; in other words, all entries of \(w\) should be close to zero. This constraint is an example of what is called regularization. In the case of Lasso as many as possible coefficients are taken as zero.

In general with linear regression we are overfitting the data. Ridge is a more restricted model, so we are less likely to overfit. A less complex model means worse performance on the training set, but better generalization. As we are only interested in generalization performance, we should choose the Ridge model over the LinearRegression model. How much importance the model places on simplicity versus training set performance can be specified by the user, using the alpha parameter. The optimum setting of alpha depends on the particular dataset we are using. Increasing alpha forces coefficients to move more toward zero, which decreases training set performance but might help generalization.

from sklearn.linear_model import Ridge
data_reg=datasets.make_regression(n_features=100, n_samples=1000, bias=3.5, noise=10)
X=data_reg[0]
y=data_reg[1]*np.random.rand(len(data_reg[1]))

X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=42, train_size=0.7)

First, we see the linear regression case

lr = LinearRegression().fit(X_train, y_train)
print("Training set score: {:.9f}".format(lr.score(X_train, y_train)))
print("Test set score: {:.9f}".format(lr.score(X_test, y_test)))

Now we explore the effect of alpha

ridge = Ridge().fit(X_train, y_train)
print("Training set score: {:.9f}".format(ridge.score(X_train, y_train)))
print("Test set score: {:.9f}".format(ridge.score(X_test, y_test)))

We can see the effect of the alpha parameter

ridge = Ridge(alpha=1000).fit(X_train, y_train)
print("Training set score: {:.9f}".format(ridge.score(X_train, y_train)))
print("Test set score: {:.9f}".format(ridge.score(X_test, y_test)))

Finally, we see we alpha affects the value of the coefficients.

coeffs= []
#alphas = np.linspace(0,10,50)
alphas = np.logspace(-6, 6, 200)
for i in alphas:
    ridge = Ridge(alpha=i).fit(X_train, y_train)
    coeffs.append(ridge.coef_)

plt.plot(alphas,coeffs)
plt.xscale("log")
plt.xlabel("alpha")
plt.ylabel("weights")
plt.title("Ridge coefficients as a function of the regularization")

Linear models are not only used for regression, but they are also extensively used for classification. In this case the prediction is done using the formula

The formula looks very similar to the one for linear regression, but instead of just returning the weighted sum of the features, we threshold the predicted value at zero. If the function is smaller than zero, we predict the class –1; if it is larger than zero, we predict the class +1. This prediction rule is common to all linear models for classification. Again, there are many different ways to find the coefficients (w) and the intercept (b).

The two most common linear classification algorithms are logistic regression, implemented in linear_model.LogisticRegression, and linear support vector machines (linear SVMs), implemented in svm.LinearSVC (SVC stands for support vector classifier). Despite its name, LogisticRegression is a classification algorithm and not a regression algorithm, and it should not be confused with LinearRegression.

For LogisticRegression and LinearSVC there is a trade-off parameter that determines the strength of the regularization is called C , and higher values of C correspond to less regularization. In other words, when you use a high value for the parameter C , LogisticRegression and LinearSVC try to fit the training set as best as possible, while with low values of the parameter C , the models put more emphasis on finding a coefficient vector (w) that is close to zero.

We consider the database of breast cancer where only two classes exit,

from sklearn.datasets import load_breast_cancer
from sklearn.linear_model import LogisticRegression
from sklearn import datasets, svm, metrics
from sklearn.model_selection import train_test_split

We split into two sets and we train the model,

X_train, X_test, y_train, y_test = train_test_split(
cancer.data, cancer.target, stratify=cancer.target, random_state=42)
logreg = LogisticRegression(C=1).fit(X_train, y_train)
print("Training set score: {:.3f}".format(logreg.score(X_train, y_train)))
print("Test set score: {:.3f}".format(logreg.score(X_test, y_test)))

We can check how the two clases separate plotting the classes as a function of two features (we can choose other two to see how the behave),

plt.scatter(cancer.data[:,0],cancer.data[:,11],c=cancer.target)
plt.xlabel('feature 0')
plt.ylabel('feature 11')

Now we move into the case of linear models for multiclass classification. In this case we will use a dataset for wines

wine=datasets.load_wine()
print(wine.DESCR)

One can visualize the classes (for only part of the attributes)

fig, ax = plt.subplots(8,8,figsize=(15,15))
for row in range(8):
    for col in range(8):
	ax[row,col].scatter(wine.data[:,row], wine.data[:,col], c=wine.target)
	ax[row,col].set_xlabel("{}".format(row))
	ax[row,col].set_ylabel("{}".format(col))

In this case we will use LinearSVC

from sklearn.svm import LinearSVC

X_train, X_test, y_train, y_test = train_test_split(
    wine.data, wine.target, stratify=wine.target, random_state=42)
SVC = LinearSVC(C=10.01,max_iter=10000).fit(X_train, y_train)
print("Training set score: {:.3f}".format(SVC.score(X_train, y_train)))
print("Test set score: {:.3f}".format(SVC.score(X_test, y_test)))

One can see that there are convergence problems.

Preprocessing methods like the scalers are usually applied before applying a supervised machine learning algorithm. As an example, say we want to apply the kernel SVM (SVC) to the cancer dataset, and use MinMaxScaler for preprocessing the data. We will use the cancer dataset

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(cancer.data, cancer.target,
random_state=1)
print(X_train.shape)
print(X_test.shape)

We will start with MinMaxScaler

from sklearn.preprocessing import MinMaxScaler
scaler = MinMaxScaler()

We then fit the scaler using the fit method, applied to the training data.

scaler.fit(X_train)

#transform data
X_train_scaled = scaler.transform(X_train)
# print dataset properties before and after scaling
print("transformed shape: {}".format(X_train_scaled.shape))
print("per-feature minimum before scaling:\n {}".format(X_train.min(axis=0)))
print("per-feature maximum before scaling:\n {}".format(X_train.max(axis=0)))
print("per-feature minimum after scaling:\n {}".format(
X_train_scaled.min(axis=0)))
print("per-feature maximum after scaling:\n {}".format(
X_train_scaled.max(axis=0)))

The transformed data has the same shape as the original data—the features are simply shifted and scaled. You can see that all of the features are now between 0 and 1, as desired.

To apply the SVM to the scaled data, we also need to transform the test set. This is again done by calling the transform method, this time on X_test:

# transform test data
X_test_scaled = scaler.transform(X_test)
# print test data properties after scaling
print("per-feature minimum after scaling:\n{}".format(X_test_scaled.min(axis=0)))
print("per-feature maximum after scaling:\n{}".format(X_test_scaled.max(axis=0)))

Maybe somewhat surprisingly, you can see that for the test set, after scaling, the minimum and maximum are not 0 and 1. Some of the features are even outside the 0–1 range! The problem is that we used the training data to fix the filter, but then we applied it to a different set of data.

It is important to apply exactly the same transformation to the training set and the test set for the supervised model to work on the test set. Here, we show what happens if the scaling is not done properly

from sklearn.datasets import make_blobs
# make synthetic data
X, _ = make_blobs(n_samples=50, centers=5, random_state=4, cluster_std=2)
# split it into training and test sets
X_train, X_test = train_test_split(X, random_state=5, test_size=.1)
# plot the training and test sets
fig, axes = plt.subplots(1, 3, figsize=(13, 4))
axes[0].scatter(X_train[:, 0], X_train[:, 1],c='blue', label="Training set", s=60)
axes[0].scatter(X_test[:, 0], X_test[:, 1], marker='^', c='red', label="Test set", s=60)
axes[0].legend(loc='upper left')
axes[0].set_title("Original Data")
# scale the data using MinMaxScaler
scaler = MinMaxScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
# visualize the properly scaled data
axes[1].scatter(X_train_scaled[:, 0], X_train_scaled[:, 1],c='blue', label="Training set", s=60)
axes[1].scatter(X_test_scaled[:, 0], X_test_scaled[:, 1], marker='^',c='red', label="Test set", s=60)
axes[1].set_title("Scaled Data")
# rescale the test set separately
# so test set min is 0 and test set max is 1
# DO NOT DO THIS! For illustration purposes only.
test_scaler = MinMaxScaler()
test_scaler.fit(X_test)
X_test_scaled_badly = test_scaler.transform(X_test)
# visualize wrongly scaled data
axes[2].scatter(X_train_scaled[:, 0], X_train_scaled[:, 1],c='blue', label="training set", s=60)
axes[2].scatter(X_test_scaled_badly[:, 0], X_test_scaled_badly[:, 1],marker='^', c='red', label="test set", s=60)
axes[2].set_title("Improperly Scaled Data")
for ax in axes:
    ax.set_xlabel("Feature 0")
    ax.set_ylabel("Feature 1")

The third panel shows what would happen if we scaled the training set and test set separately. In this case, the minimum and maximum feature values for both the training and the test set are 0 and 1. But now the dataset looks different. The test points moved incongruously to the training set, as they were scaled differently. We changed the arrangement of the data in an arbitrary way. Clearly this is not what we want to do.

Note that, in genera, the fit and the transform can be done more efficiently in a single step using the fit_transform method:

from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
# calling fit and transform in sequence (using method chaining)
X_scaled = scaler.fit(X).transform(X)
# same result, but more efficient computation
X_scaled_d = scaler.fit_transform(X)

Other important scale methods are StandardScaler and RobustScaler. The StandardScaler in scikit-learn ensures that for each feature the mean is 0 and the variance is 1, bringing all features to the same magnitude. However, this scaling does not ensure any particular minimum and maximum values for the features. The RobustScaler works similarly to the StandardScaler in that it ensures statistical properties for each feature that guarantee that they are on the same scale. However, the RobustScaler uses the median and quartiles, instead of mean and variance. This makes the RobustScaler ignore data points that are very different from the rest (like measurement errors). The Normalizer does a very different kind of rescaling. It scales each data point such that the feature vector has a Euclidean length of 1. In other words, it projects a data point on the circle (or sphere, in the case of higher dimensions) with a radius of 1.

The effect of working with scaled data can be really important. First we show the performance just using raw data:

from sklearn.svm import SVC
X_train, X_test, y_train, y_test = train_test_split(cancer.data, cancer.target,
random_state=0)
svm = SVC(C=100)
svm.fit(X_train, y_train)
print("Test set accuracy: {:.2f}".format(svm.score(X_test, y_test)))

and now the scaled ones

scaler = MinMaxScaler()
scaler.fit(X_train)
X_train_scaled = scaler.transform(X_train)
X_test_scaled = scaler.transform(X_test)
# learning an SVM on the scaled training data
svm.fit(X_train_scaled, y_train)
# scoring on the scaled test set
print("Scaled test set accuracy: {:.2f}".format(
    svm.score(X_test_scaled, y_test)))

As we saw before, the effect of scaling the data is quite significant. Even though scaling the data doesn’t involve any complicated math, it is good practice to use the scaling mechanisms.

Principal component analysis is a method that rotates the dataset in a way such that the rotated features are statistically uncorrelated. This rotation is often followed by selecting only a subset of the new features, according to how important they are for explaining the data.

We will work once more with the cancer dataset. First we scale the data,

from sklearn.datasets import load_breast_cancer
cancer = load_breast_cancer()
scaler = StandardScaler()
scaler.fit(cancer.data)
X_scaled = scaler.transform(cancer.data)

and then we obtain the two first principal components,

from sklearn.decomposition import PCA
# keep the first two principal components of the data
pca = PCA(n_components=2)
# fit PCA model to breast cancer data
pca.fit(X_scaled)
# transform data onto the first two principal components
X_pca = pca.transform(X_scaled)
print("Original shape: {}".format(str(X_scaled.shape)))
print("Reduced shape: {}".format(str(X_pca.shape)))

We can see how look like the new X data

# plot first vs. second principal component, colored by class
plt.figure(figsize=(8, 8))
plt.scatter(X_pca[:, 0], X_pca[:, 1], cancer.target)
plt.legend(cancer.target_names, loc="best")
plt.gca().set_aspect("equal")
plt.xlabel("First principal component")
plt.ylabel("Second principal component")

It is important to note that PCA is an unsupervised method, and does not use any class information when finding the rotation. It simply looks at the correlations in the data. You can see that the two classes separate quite well in this two-dimensional space. This leads us to believe that even a linear classifier (that would learn a line in this space) could do a reasonably good job at distinguishing the two classes.

A downside of PCA is that the two axes in the plot are often not very easy to interpret. The principal components correspond to directions in the original data, so they are combinations of the original features. However, these combinations are usually very complex, as we’ll see shortly.

print("PCA components:\n{}".format(pca.components_))

plt.matshow(pca.components_, cmap='viridis')
plt.yticks([0, 1], ["First component", "Second component"])
plt.colorbar()
plt.xticks(range(len(cancer.feature_names)),
cancer.feature_names, rotation=60, ha='left')
plt.xlabel("Feature")
plt.ylabel("Principal components")

Once the PCA has been performed, the fitting procedure can be conducted with the PCA components:

from sklearn.svm import SVC
X_train_pca, X_test_pca, y_train, y_test = train_test_split(X_pca, cancer.target,
random_state=10)
#
svm = SVC(C=100)
svm.fit(X_train_pca, y_train)
print("Test set accuracy: {:.2f}".format(svm.score(X_test_pca, y_test)))

Most machine learning applications require not only the application of a single algorithm, but the chaining together of many different processing steps and machine learning models. In this chapter, we will cover how to use the Pipeline class to simplify the process of building chains of transformations and models. In particular, we will see how we to use Pipeline for all processing steps at once.

Let’s look at how we can use the Pipeline class to express the workflow for training an SVM after scaling the data with MinMaxScaler (for now without the grid search). First, we build a pipeline object by providing it with a list of steps. Each step is a tuple containing a name (any string of your choosing) and an instance of an estimator:

from sklearn.datasets import load_breast_cancer
from sklearn.model_selection import train_test_split
cancer = load_breast_cancer()
X_train, X_test, y_train, y_test = train_test_split(cancer.data, cancer.target,
random_state=0)

from sklearn.pipeline import Pipeline
from sklearn.preprocessing import StandardScaler, MinMaxScaler
from sklearn.svm import SVC
pipe = Pipeline([("scaler", MinMaxScaler()), ("svm", SVC())])

Here, we created two steps: the first, called “scaler”, is an instance of MinMaxScaler, and the second, called “svm”, is an instance of SVC . One can access different steps with pipe[0] or pipe[1]. Now, we can fit the pipeline, like any other scikit-learn estimator:

pipe.fit(X_train, y_train)

print("Test score: {:.2f}".format(pipe.score(X_test, y_test)))

Calling the score method on the pipeline first transforms the test data using the scaler, and then calls the score method on the SVM using the scaled test data.

The Pipeline class is not restricted to preprocessing and classification, but can in fact join any number of estimators together. For example, you could build a pipeline containing feature extraction, feature selection, scaling, and classification, for a total of four steps. Similarly, the last step could be regression or clustering instead of classification. The only requirement for estimators in a pipeline is that all but the last step need to have a transform method, so they can produce a new representation of the data that can be used in the next step.

Creating a pipeline using the syntax described earlier is sometimes a bit cumbersome, and we often don’t need user-specified names for each step. There is a convenience function, make_pipeline , that will create a pipeline for us and automatically name each step based on its class. The syntax for make_pipeline is as follows:

from sklearn.pipeline import make_pipeline
# standard syntax
pipe_long = Pipeline([("scaler", MinMaxScaler()), ("svm", SVC(C=100))])
# abbreviated syntax
pipe_short = make_pipeline(MinMaxScaler(), SVC(C=100))

The pipeline objects pipe_long and pipe_short do exactly the same thing, but pipe_short has steps that were automatically named. We can see the names of the steps by looking at the steps attribute:

print("Pipeline steps:\n{}".format(pipe_short.steps))
print("Pipeline steps:\n{}".format(pipe_long.steps))

One can access the attributes at any step,

from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
pipe = make_pipeline(StandardScaler(), PCA(n_components=2), StandardScaler())
pipe.fit(cancer.data)
print("Pipeline steps:\n{}".format(pipe.steps))
print(pipe.named_steps["pca"].components_)
pipe.transform(cancer.data)

It is possible to change the value of any control parameters of the different steps

pipe.set_params(pca__n_components=3)
pipe.fit(cancer.data)
pipe.fit(cancer.data)
pipe.transform(cancer.data).shape

Note the double _.