Feature Engineering Tutorial: Encoding, Scaling, Crosses, Selection (2026)
Feature engineering transforms raw data into features that better represent the problem. It has a larger impact on performance than algorithm selection. A simple linear model with great features beats a complex neural network with raw features.
By 2026, automated feature tools exist, but fundamental techniques remain essential. Understanding encoding, scaling, crosses, and selection directly translates to better models.
Encoding Categorical Variables
One-Hot Encoding for low cardinality, Target Encoding with cross-validation for high cardinality, Count Encoding for extreme cardinality like ZIP codes.
No single method works for all cases. Choice depends on category count, data size, and relationship with the target.
from sklearn.preprocessing import OneHotEncoder\nfrom category_encoders import TargetEncoder\nohe = OneHotEncoder(sparse_output=False, handle_unknown="infrequent_if_exist")\nencoder = TargetEncoder(cols=["high_card_col"], smoothing=10)Numerical Feature Scaling
StandardScaler, RobustScaler, and PowerTransformer handle different distributions. Scaling is essential for SVM, KNN, and neural networks but not tree-based models.
For long-tailed features, apply log transform before scaling. PowerTransformer with Yeo-Johnson automatically finds the optimal transformation.
from sklearn.preprocessing import StandardScaler, PowerTransformer\nscaler = StandardScaler()\nqt = PowerTransformer(method="yeo-johnson")Feature Crosses and Interactions
PolynomialFeatures creates interaction terms and powers. Feature crosses capture non-linear relationships in linear models. The combination of user_age and ad_category was more predictive than either alone.
Applied feature selection after crossing since features grew from 50 to 1,275. AUC improved from 0.78 to 0.83.
from sklearn.preprocessing import PolynomialFeatures\npoly = PolynomialFeatures(degree=2, interaction_only=True)\nselector = SelectKBest(mutual_info_classif, k=100)Date and Time Features
Extract year, month, day of week, is_weekend, quarter. Cyclical encoding with sin/cos preserves circular nature. Lag features and rolling windows capture history.
For retail demand forecasting, date features were the most important predictor group. Cyclical encoding captured smooth December-to-January transitions.
def create_date_features(df, date_col):\n dates = pd.to_datetime(df[date_col])\n df["month_sin"] = np.sin(2 * np.pi * dates.dt.month / 12)\n df["month_cos"] = np.cos(2 * np.pi * dates.dt.month / 12)\n return dfAutomated Feature Engineering
Featuretools uses Deep Feature Synthesis to generate features from relational data. It creates groupby transformations across table relationships.
Generated 2,000 features from 15 tables in 10 minutes. Reduced to top 200 using LightGBM importance, outperforming manual engineering by 4 percent AUC.
import featuretools as ft\nes = ft.EntitySet("dataset")\nes = es.add_dataframe(dataframe_name="customers", dataframe=customers, index="customer_id")\nfeature_matrix, defs = ft.dfs(entityset=es, target_dataframe_name="customers", max_depth=2)Feature Selection Methods
Filter methods (mutual information) are cheap, wrapper methods (RFE) are accurate but expensive, embedded methods (LASSO) combine best of both.
My pipeline: filter to top 500, then RFE with Random Forest for optimal subset. For production, LASSO regularization selects features during training.
from sklearn.feature_selection import SelectFromModel, RFE\nfrom sklearn.linear_model import LogisticRegression\nselector = SelectFromModel(LogisticRegression(penalty="l1", C=0.1, solver="saga"), max_features=50)Frequently Asked Questions
Most important technique?
Aggregate features from relational data — groupby mean, sum, count, trend over time — provide the largest gains across domains.
Feature engineering before or after split?
Always fit transforms on training set only and apply to test set to prevent data leakage.
How many features to create?
Create as many as useful, then select. Rule: 10x more samples than features. Tree models handle more features than linear models.
Feature selection vs dimensionality reduction?
Selection chooses original features (interpretable). Reduction creates new combinations (PCA, t-SNE). Selection preserves interpretability.
Originally published on Ayodhyyya. Last updated June 1, 2026.