Fitting User-Specified Models with Group Lasso Penalty

Description

Fits user-specified (GLM-) models with group lasso penalty.

Details

The DESCRIPTION file:

Index of help topics:

grpl.control            Options for the Group Lasso Algorithm
grpl.control-class      Class "grpl.control": Options for the Group
                        Lasso Algorithm
grpl.model              Group Lasso Models
grpl.model-class        Class "grpl.model": Group Lasso Models
grplasso                Function to Fit a Solution of a Group Lasso
                        Problem
grplasso-package        Fitting User-Specified Models with Group Lasso
                        Penalty
lambdamax               Function to Find the Maximal Value of the
                        Penalty Parameter Lambda
plot.grplasso           Plots the Solution Path of a grplasso Object
predict.grplasso        Predict Method for grplasso Objects
splice                  Dataset of Human Donor Splice Sites

The best entry point for the package are the examples in the help file of the function grplasso.

Author(s)

Lukas Meier

Maintainer: Lukas Meier <meier@stat.math.ethz.ch>

References

Lukas Meier, Sara van de Geer and Peter B\"uhlmann (2008), The Group Lasso for Logistic Regression, Journal of the Royal Statistical Society, 70 (1), 53 - 71

Options for the Group Lasso Algorithm

Description

Definition of options such as bounds on the Hessian, convergence criteria and output management for the group lasso algorithm.

Usage

grpl.control(save.x = FALSE, save.y = TRUE,
             update.hess = c("lambda", "always"), update.every = 3,
             inner.loops = 10, line.search = TRUE, max.iter = 500,
             tol = 5 * 10^-8, lower = 10^-2, upper = Inf, beta = 0.5,
             sigma = 0.1, trace = 1)

Arguments

Details

For the convergence criteria see chapter 8.2.3.2 of Gill et al. (1981).

Value

An object of class grpl.control.

References

Philip E. Gill, Walter Murray and Margaret H. Wright (1981) Practical Optimization, Academic Press.

Dimitri P. Bertsekas (2003) Nonlinear Programming, Athena Scientific.

Class "grpl.control": Options for the Group Lasso Algorithm

Description

Objects of class "grpl.control" define options such as bounds on the Hessian, convergence criteria and output management for the Group Lasso algorithm.

Details

For the convergence criteria see chapter 8.2.3.2 of Gill et al. (1981).

Objects from the Class

Objects can be created by calls of the form grpl.control(...)

Slots

save.x

a logical indicating whether the design matrix should be saved.

save.y

a logical indicating whether the response should be saved.

update.hess

should the hessian be updated in each iteration ("always")? update.hess = "lambda" will update the Hessian once for each component of the penalty parameter "lambda" based on the parameter estimates corresponding to the previous value of the penalty parameter.

update.every

Only used if update.hess = "lambda". E.g. set to 3 if you want to update the Hessian only every third grid point.

inner.loops

How many loops should be done (at maximum) when solving only the active set (without considering the remaining predictors). Useful if the number of predictors is large. Set to 0 if no inner loops should be performed.

line.search

Should line searches be performed?

max.iter

Maximal number of loops through all groups

tol

convergence tolerance; the smaller the more precise.

lower

lower bound for the diagonal approximation of the corresponding block submatrix of the Hessian of the negative log-likelihood function.

upper

upper bound for the diagonal approximation of the corresponding block submatrix of the Hessian of the negative log-likelihood function.

beta

scaling factor \beta < 1 of the Armijo line search.

sigma

0 < \sigma < 1 used in the Armijo line search.

trace

integer. 1 prints the current lambda value, 2 prints the improvement in the objective function after each sweep through all the parameter groups and additional information.

References

Philip E. Gill, Walter Murray and Margaret H. Wright (1981) Practical Optimization, Academic Press.

Dimitri P. Bertsekas (2003) Nonlinear Programming, Athena Scientific.

Group Lasso Models

Description

Generates models to be used for the group lasso algorithm.

Usage

grpl.model(invlink, link, nloglik, ngradient, nhessian, check,
           name = "user-specified", comment = "user-specified")
LogReg()
LinReg()
PoissReg()

Arguments

Value

An object of class grpl.model.

Examples

    LogReg()

Class "grpl.model": Group Lasso Models

Description

Objects of class "grpl.model" define link function, negative log-likelihood and corresponding gradient and Hessian for the model to be used in a group lasso problem.

Objects from the Class

Objects can be created by calls of the form grpl.model(...)

Slots

invlink

a function with arguments eta implementing the inverse link function.

link

a function with arguments mu implementing the link function.

nloglik

a function with arguments y, mu and weights implementing the negative log-likelihood function.

ngradient

a function with arguments x, y, mu and weights implementing the negative gradient of the log-likelihood function.

nhessian

a function with arguments x, mu and weights implementing the negative hessian of the log-likelihood function.

check

a function with argument y to check whether the response has the correct format.

name

a character name

comment

a character comment

Methods

show

object

Examples

  LogReg()

Function to Fit a Solution of a Group Lasso Problem

Description

Fits the solution of a group lasso problem for a model of type grpl.model.

Usage

grplasso(x, ...)

## S3 method for class 'formula'
grplasso(formula, nonpen = ~ 1, data, weights,
         subset, na.action, lambda, coef.init, penscale = sqrt,
         model = LogReg(), center = TRUE, standardize = TRUE,
         control = grpl.control(), contrasts = NULL, ...)

## Default S3 method:
grplasso(x, y, index, weights = rep(1, length(y)), offset = rep(0,
         length(y)), lambda, coef.init = rep(0, ncol(x)),
         penscale = sqrt, model = LogReg(), center = TRUE,
         standardize = TRUE, control = grpl.control(), ...)

Arguments

Details

When using grplasso.formula, the grouping of the variables is derived from the type of the variables: The dummy variables of a factor will be automatically treated as a group.

The optimization process starts using the first component of lambda as penalty parameter \lambda and with starting values defined in coef.init for the parameter vector. Once fitted, the next component of lambda is considered as penalty parameter with starting values defined as the (fitted) coefficient vector based on the previous component of lambda.

Value

A grplasso object is returned, for which coef, print, plot and predict methods exist.

Author(s)

Lukas Meier, meier@stat.math.ethz.ch

References

Lukas Meier, Sara van de Geer and Peter B\"uhlmann (2008), The Group Lasso for Logistic Regression, Journal of the Royal Statistical Society, 70 (1), 53 - 71

Examples

## Use the Logistic Group Lasso on the splice data set
data(splice)

## Define a list with the contrasts of the factors
contr <- rep(list("contr.sum"), ncol(splice) - 1)
names(contr) <- names(splice)[-1]

## Fit a logistic model 
fit.splice <- grplasso(y ~ ., data = splice, model = LogReg(), lambda = 20,
                       contrasts = contr, center = TRUE, standardize = TRUE)

## Perform the Logistic Group Lasso on a random dataset
set.seed(79)

n <- 50  ## observations
p <- 4   ## variables

## First variable (intercept) not penalized, two groups having 2 degrees
## of freedom each

index <- c(NA, 2, 2, 3, 3)

## Create a random design matrix, including the intercept (first column)
x <- cbind(1, matrix(rnorm(p * n), nrow = n))
colnames(x) <- c("Intercept", paste("X", 1:4, sep = ""))

par <- c(0, 2.1, -1.8, 0, 0)
prob <- 1 / (1 + exp(-x %*% par))
mean(pmin(prob, 1 - prob)) ## Bayes risk
y <- rbinom(n, size = 1, prob = prob) ## binary response vector

## Use a multiplicative grid for the penalty parameter lambda, starting
## at the maximal lambda value
lambda <- lambdamax(x, y = y, index = index, penscale = sqrt,
                    model = LogReg()) * 0.5^(0:5)

## Fit the solution path on the lambda grid
fit <- grplasso(x, y = y, index = index, lambda = lambda, model = LogReg(),
                penscale = sqrt,
                control = grpl.control(update.hess = "lambda", trace = 0))

## Plot coefficient paths
plot(fit)

Internal grplasso Functions

Description

Internal grplasso functions.

Usage

blockstand(x, ipen.which, inotpen.which)
create.design(m, formula, nonpen  = ~ 1, data, weights, subset,
              na.action, contrasts, env)

Details

These are not to be called by the user.

Function to Find the Maximal Value of the Penalty Parameter Lambda

Description

Determines the value of the penalty parameter lambda when the first penalized parameter group enters the model.

Usage

lambdamax(x, ...)

## S3 method for class 'formula'
lambdamax(formula, nonpen = ~1, data, weights, subset,
          na.action, coef.init, penscale = sqrt, model = LogReg(),
          center = TRUE, standardize = TRUE, contrasts = NULL,
          nlminb.opt = list(), ...)

## Default S3 method:
lambdamax(x, y, index, weights = rep(1, length(y)),
          offset = rep(0, length(y)), coef.init = rep(0, ncol(x)),
          penscale = sqrt, model = LogReg(), center = TRUE,
          standardize = TRUE, nlminb.opt = list(), ...)

Arguments

Details

Uses nlminb to optimize the non-penalized parameters.

Value

An object of type numeric is returned.

References

Lukas Meier, Sara van de Geer and Peter B\"uhlmann (2008), The Group Lasso for Logistic Regression, Journal of the Royal Statistical Society, 70 (1), 53 - 71

Examples

data(splice)
lambdamax(y ~ ., data = splice, model = LogReg(), center = TRUE,
          standardize = TRUE)

Plots the Solution Path of a grplasso Object

Description

Plots the solution path of a grplasso object.

Usage

## S3 method for class 'grplasso'
plot(x, type = "coefficients", col = NULL, ...)

Arguments

Examples

data(splice)

contr <- list(Pos.1 = "contr.sum", Pos.2 = "contr.sum")
lambda <- lambdamax(y ~ Pos.1 * Pos.2, data = splice, model = LogReg(),
                    contrasts = contr, standardize = TRUE) * 0.8^(0:8)

fit <- grplasso(y ~ Pos.1 * Pos.2, data = splice, model = LogReg(),
                lambda = lambda, contrasts = contr, standardize = TRUE,
                control = grpl.control(trace = 0, inner.loops = 0,
                update.every = 1, update.hess = "lambda"))
plot(fit, log = "x")

Predict Method for grplasso Objects

Description

Obtains predictions from a grplasso object.

Usage

## S3 method for class 'grplasso'
predict(object, newdata, type = c("link", "response"),
        na.action = na.pass, ...)

Arguments

Value

A matrix whose columns correspond to the different values of the penalty parameter lambda of the grplasso object.

Note

If newdata is given, offsets specified by offset in the fit by grplasso.default will not be included in predictions, whereas those specified by an offset term in the formula will be considered.

See Also

grplasso

Examples

data(splice)

contr <- rep(list("contr.sum"), ncol(splice) - 1)
names(contr) <- names(splice)[-1]

fit <- grplasso(y ~ ., data = splice, model = LogReg(), lambda = 10,
                contrasts = contr, standardize = TRUE)

pred <- predict(fit)
pred.resp <- predict(fit, type = "response")

## The following points should lie on the sigmoid curve
plot(pred, pred.resp)

Dataset of Human Donor Splice Sites

Description

Dataset of 400 human donor splice sites with a sequence length of 7 base pairs.

Usage

data(splice)

Format

y

binary response. True (1) or false (0) splice site.

Pos.x

DNA letter (A, C, G, T) at position x, where x ranges from 1 to 7.

Details

The dataset is a random subset of the MEMset Donor dataset used in Gene et al. (2004).

References

Gene, Y. and Burge, C. (2004) Maximum Entropy Modeling of Short Sequence Motifs with Applications to RNA Splicing Signals, Journal of Computational Biology, 11, 475 - 494.

Examples

  data(splice)