Individual Conditional Expectation for black box model

Similarly to continuous variables the first thing we do is to get black box model response on single variable, called effect.

For continuous variable we used PD or ALE plots, which was average model response on predictor value. In a discrete case we use Individual Conditional Expectation (ICE). The construction is simple:

In training data we replace all values for selected predictor with one of the factor levels and then we perform predictions on resulting dataset. The action is repeated for all factor levels. As a result we get \(n * m\) predicted values, where \(n\) is number of observations in original training dataset and \(m\) is number of selected factor levels.

For above example (and salary variable), we will get nrow(HR_data) * 3 predicted values, as salary has 3 levels.

To generate model ICE xspliner uses pdp::partial(ice = TRUE) function. To specify additional options for the response, we may customize the effect parameter for xf, just like it was using xs:

average_monthly_hours ~ last_evaluation + xf(salary, effect = list(...)) + satisfaction_level

Possible parameters are inherited from pdp::partial function (except ice that is always TRUE).

How can we use info gathered from above data?

Merging similar groups with factorMerger

In continuous case, we simplified the effect with spline approximation. The main idea for discrete case is to find out which groups give similar black box model response and merge them into common groups. The final model is built on simpler variables that store some information sourced from black box. As a result the GLM is much easier to interpret (for example we reduce 10-level factor into 3-level one).

How is that implemented in xspliner?

The transformation is based on factorMerger package that “Support Adaptive Post-Hoc Fusing of Groups”.

Merging the groups uses just two functions from the package:

• mergeFactors
• getOptimalPartitionDf

In order to customize variable transition, just specified (inherited from above functions) parameters inside transition parameter of xf formula symbol. For example to use “fast-adaptive” method for groups merging with optimal partition at GIC statistics value of 4, we set:

xf(salary, transition = list(method = "fast-adaptive", value = 4))

In below example, we will transform salary predictor with cutting of GIC statistics at value = 2. As in continuous case we need to use the formula within xspline function:

library(xspliner)
model_xs <- xspline(
  average_montly_hours ~ last_evaluation + xf(salary, transition = list(value = 2)) + satisfaction_level,
  model = model_rf
)

summary(model_xs)

## 
## Call:
## stats::glm(formula = average_montly_hours ~ last_evaluation + 
##     xf(salary) + satisfaction_level, family = family, data = data)
## 
## Deviance Residuals: 
##      Min        1Q    Median        3Q       Max  
## -137.197   -37.956    -1.425    37.728   129.925  
## 
## Coefficients:
##                     Estimate Std. Error t value Pr(>|t|)    
## (Intercept)         136.0467     2.2213  61.248  < 2e-16 ***
## last_evaluation     100.8673     2.2501  44.828  < 2e-16 ***
## xf(salary)lowmedium  -0.3096     1.3929  -0.222    0.824    
## satisfaction_level  -11.3302     1.5494  -7.312 2.76e-13 ***
## ---
## Signif. codes:  0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
## 
## (Dispersion parameter for gaussian family taken to be 2199.008)
## 
##     Null deviance: 37409709  on 14998  degrees of freedom
## Residual deviance: 32974127  on 14995  degrees of freedom
## AIC: 158000
## 
## Number of Fisher Scoring iterations: 2

Checking out the model summary, we can realize that “low” and “medium” values were merged into single level (generating “lowmedium” level).

It can be also found by:

summary(model_xs, "salary")

##     orig      pred
## 1   high      high
## 2    low lowmedium
## 3 medium lowmedium

The graphical result if fully sourced from factorMerger. It is enough to run:

plot_variable_transition(model_xs, "salary")