ISYE 6501 Final Exam Questions

ISYE 6501 Final Exam – Questions With Correct Solutions Factor Based Models ✔️Ans - classification, clustering, regression.Implicitly assumed that we have a lot of factors in the final model Why limit number of factors in a model? 2 reasons ✔️Ans - overfitting: when # of factors is close to or larger than # of data points. Model may fit too closely to random effects

simplicity: simple models are usually better

Classical variable selection approaches ✔️Ans - 1. Forward selection

• Backwards elimination
• Stepwise regression

greedy algorithms Backward elimination ✔️Ans - variable selection; classical Opposite of forward selection. Start with model with all factors, at each step find worst factor and remove from model. Continue until no more to add, # of factor threshold is satisfied. Remove factors at the end that were not good enough Forward selection ✔️Ans - variable selection; classical Start with model with no factors, at each step find best new factor to add.Continue until none bad enough to remove, # of factor threshold is satisfied.Remove factors at the end that were not good enough Stepwise regression ✔️Ans - variable selection; classical Combination of forward selection and backwards elimination. Start with all or no factors. Each step remove/add a factor. As it continues, after adding in new factor we eliminate right away any factors that may be good. Helps model adjust when new factors are added, goodness values change Ways of determining if factors are good enough in variable selection ✔️Ans - p-value, Rsquared, AIC, BIC 1 / 2

Greedy algorithm ✔️Ans - At each step, it does the one thing that looks best without taking future options into consideration. Good for initial analysis

• Forward selection
• Backwards elimination
• Stepwise regression

Global variable selection approaches ✔️Ans - 1. LASSO

• Elastic Net

Slower, but tend to give better predictive models LASSO ✔️Ans - variable selection; global

• SCALE the date (as with any constrained sum of coefficients)
• add a constraint to the standard regression equation
• minimize sum of squared errors
• T = limit or "budget" on how large the sum of squared errors can get. Budget

will be used on most important coefficients

• Method for limiting the number of variables in a model by limiting the sum of

all coefficients' absolute values. Can be very helpful when number of data points is less than number of factors.Elastic Net ✔️Ans - variable selection; global

• SCALE the date (as with any constrained sum of coefficients)
• T = limit or "budget" on how large the sum of squared errors can get. Budget

will be used on most important coefficients

• Combination of lasso and ridge regression.
• Variable selection benefits of LASSO
• Predictive benefits of ridge regression

Ridge Regression ✔️Ans - - Method of regularization by limiting the sum of the squares of the coefficients. Will reduce the magnitude of coefficients, not the number of variables chosen.

• The quadratic term in ridge regression

tends to shrink the coefficient values i.e Whatever the basic regression model coefficients would be, the quadratic constraint pushes them toward zero or regularizes them.

• / 2