%\VignetteEngine{knitr::knitr}
%\VignetteIndexEntry{Regression Models Supported by the effects Package}

\documentclass{article}


\usepackage[utf8]{inputenc}
\usepackage{graphicx}
\usepackage[american]{babel}
\newcommand{\R}{{\sf R}}
\usepackage{url}
\usepackage{hyperref}
\usepackage{alltt}
\usepackage{fancyvrb}
\usepackage{natbib}
\usepackage{amsmath}

\usepackage[margin=1in]{geometry}
\usepackage{ragged2e}

\VerbatimFootnotes
\bibliographystyle{chicago}

\newcommand{\x}{\mathbf{x}}
\newcommand{\code}[1]{\normalfont\texttt{\hyphenchar\font45\relax #1}}
\newcommand{\E}{\mathrm{E}}
\newcommand{\tild}{\symbol{126}}
\newcommand{\Rtilde}{\,\raisebox{-.5ex}{\code{\tild{}}}\,}
\newcommand{\captilde}{\mbox{\protect\Rtilde}} % use in figure captions.
\newcommand{\Rmod}[2]{\code{#1 \raisebox{-.5ex}{\tild{}} #2}}
\newcommand{\Rmoda}[2]{\code{#1} &\code{\raisebox{-.5ex}{\tild{}} #2}}
\newcommand{\Rmodb}[2]{\code{#1 &\raisebox{-.5ex}{\tild{}}& #2}}
\newcommand{\C}{\mathbf{C}}
\newcommand{\betahat}{\widehat{\beta}}
\newcommand{\bbetahat}{\widehat{\boldsymbol{\beta}}}
\newcommand{\bbeta}{\boldsymbol{\beta}}
\newcommand{\xbf}{\x_{\backslash{}f}}
\newcommand{\hbf}{h_{\backslash{}f}}
\newcommand{\xtb}{\x_{2\backslash{}f}}
\newcommand{\xbfi}{\x_{\backslash{}f,i}}
\newcommand{\inter}[2]{\mbox{$#1$:$#2$}}
\newcommand{\cross}[2]{\mbox{$#1$\code{*}$#2$}}
\newcommand{\N}{\mathrm{N}}

\newcommand{\yx}{\widehat{y}(\x)}
\newcommand{\lvn}[1]{\mbox{$\log(\mbox{\texttt{#1}})$}}

\newcommand{\fn}[1]{\code{#1()}}
\newcommand{\pkg}[1]{\textbf{#1}}
\newcommand{\proglang}[1]{\textsf{#1}}
\newcommand{\class}[1]{\texttt{"#1"}}

\begin{document}

\title{Regression Methods Supported by the effects Package}

\author{John Fox and Sanford Weisberg}

\date{2020-07-21}

\maketitle

<<setopts,echo=FALSE>>=
library("knitr")
opts_chunk$set(fig.width=5,fig.height=5,tidy=TRUE,
               out.width="0.8\\textwidth",echo=TRUE)
options(prompt=" ")
@ 

<<echo=FALSE, results='hide', include=FALSE>>=
#options(continue="+    ", prompt="R> ", width=76)
options(show.signif.stars=FALSE)
options(scipen=3)
library(effects)
@

<<include=FALSE>>=
library(knitr)
opts_chunk$set(
tidy=FALSE,fig.width=5,fig.height=5,cache=FALSE,comment=NA, prompt=TRUE
)
render_sweave()
@


<<echo=FALSE, results='hide', include=FALSE>>=
options(continue="    ", prompt=" ", width=76)
options(show.signif.stars=FALSE)
options(scipen=3)
@

\emph{Effect plots} allow visualizing the effect of a predictor on a response in models in which the dependence of the response depends on a linear combination of main-effects and interactions \citep[Sec.~4.6.3]{FoxWeisberg19}.  Table~\ref{tab1} provides a list of \emph{some} of the regression modeling methods in \R{} that can be used with effect plots.

\begin{table}
\caption{\R{} regression models known to be compatible with the \pkg{effects} package.  The name before the double-colon is the name of the package that includes the function; for example \fn{stats::lm} means that \fn{lm} is in the \pkg{stats} package.\label{tab1}}
\begin{center}
\begin{tabular}{|l|p{4.0in}|}\hline
Function & Comments \\ \hline
\textbf{glm-type models}&\\ \hline
\fn{stats::lm} & Standard linear regression model.  A multivariate response, thus fitting a multivariatel linear model, are permitted, and effect plots are drawn for each response separately.\\
\fn{stats::glm} & Generalized linear models\\
\fn{nlme::lme} & Linear mixed-effects models.  Effects plots for predictors in the fixed-effects part of the model.\\
\fn{nlme::gls} & Linear model fit with generalized least squares\\
\fn{lmer::lmer} & Linear mixed-effect models. Effects plots are for predictors in the fixed-effects part of the model  \\
\fn{lmer::glmer} & Generalized linear mixed-effect models. Effects plots for predictors in the fixed-effects part of the model\\
\fn{survey::svyglm} & Survey-weighted generalized linear models\\
\fn{MASS:glmmPQL} & Generalized linear mixed models via partial quadratic likelihood\\
\fn{robustlmm::rlmer} & Robust linear mixed-models\\ 
\fn{betareg::betareg} & Beta regression for rates and proportions\\
\fn{AER::ivreg} & Instrumental-variable regression \\
\hline
\textbf{multinomial type models}&\\
\hline
\fn{nnet::multinom} & Multinomial log-linear models \\
\fn{poLCA::poLCA} & Latent class analysis of polytomous outcomes, even though this is not strictly a regression model\\
\hline
\textbf{ordinal type models}&\\ \hline
\fn{MASS:polr} & Ordinal logistic and probit models\\
\fn{ordinal::clm} & Cumulative link proportional odds models, similar to \fn{polr}\\
\fn{ordinal::clm2}& Updated version of \fn{ordinal::clm}\\
\fn{ordinal::clmm} & Cumulative link proportional odds models with random effects\\
\hline
\end{tabular}
\end{center}
\end{table}

The most basic type of model for which effects are appropriate is a standard linear model, for example

<<fig.height=4,fig.width=8>>=
library(effects)
g1 <- lm(prestige ~ education + type + education:type, data = Prestige)
plot(predictorEffects(g1), lines=list(multiline=TRUE))
@

\noindent
In this example the response \code{prestige} is modeled as a linear function of years of \code{education}, a factor \code{type} either blue collar, professional or white collar. Because of the interaction the estimated change in \code{prestige} as a function of \code{education} is different for each level of \code{type}, as is plainly evident in the graphs shown.  The graph shown at the left varies \code{education} and fixes \code{type}, while the right graph varies \code{type} and fixes \code{education}. 
A more complicated model is a linear mixed model, for example
<<>>=
data(Orthodont, package="nlme")
g2 <- lme4::lmer(distance ~ age + Sex + (1 |Subject), data = Orthodont)
g2
@
This model has a fixed effect part with response \code{distance} and predictors \code{age} and \code{Sex}.  The random effect varies by \code{Subject}.  Effect plots are based only on the fixed-effects in the model,
<<fig.height=4,fig.width=8>>=
plot(predictorEffects(g2))
@

\section{Types of Effects Plots}

There are three basic types of effects plots.  Both of the models just fit are of the  \code{glm-type} which visualizes the dependence of a response on a set of main effects and interactions among fixed effect predictors.  As shown in Table~\ref{tab1} most of the models used with effects are of this type.  

The \code{mutlinomial-type} arises when the response is a multinomial random variable, also modeled as a linear function of fixed-effect main effects and interactions.  The \code{poLCA::poLCA} function is of the multinomial-type even though it is philsophocially different from multinomial regression, as it has a latent variable as its response rather than an observable multinomial response. 

The \code{ordinal-type}  is used to fit a multinomial response whose categories are ordered, initially applied to results from \code{MASS::polr}.  The other functions shown in Table~\ref{tab1} do similar fitting, but allowing for some generalization in defining cutpoints between categories, and in allowing for random effects.

\section{Regression Models of the Glm-type}
Effect plots for models of the \code{glm-type} are drawn by collecting information from the regression model of interest and then using that information in the code for drawing generalized linear model effect plots.  The required information is summarized in Table~\ref{tab2}. 

\begin{table}
\caption{Values that must be supplied from a regression object to draw effects plots for a \code{glm-like} regression model.  In the table, the regression model object is called \code{m1}.\label{tab2}}
\begin{center}
\begin{tabular}{|l|p{4.5in}|} \hline
Argument & Description \\ \hline
\code{call} & The call that created the regression model is generally returned by either \verb+m1$call+ or \verb+m1@call+.   The call is used to find the usual \code{data} and \code{subset} arguments that Effects needs to draw the plots.  See the example in Section~\ref{secgls} for an example where the \code{call} must be modified.\\
formula &  The formula for the linear predictor is required for Effect plots. By default Effects uses \verb+insight::find_formula(m1)$conditional+  (see \url{https://easystats.github.io/insight/}) that will return the fixed-effect formula for many regression models.\\
\code{family} & Many \code{glm-type} models include a family, including an error distribution and a link function.  These are often returned by the default \code{stats::family(m1)}.\\
\code{coefficients} &  The vector of fixed-effect parameter estimates is required for Effect plots. The default value is \verb+effect::effCoef(m1)+ that calls \verb+insight::get_parameters(m1)+ and then reformats the result from a two-column data frame with a names in the first column and values in the second column to a vector of named values, as is expected by the effects package.\\
\code{vcov} &  The estimated covariance of fixed-effect estimates is required.  Effects uses \code{stats::vcov(m1)} by default.\\
\hline
\end{tabular}
\end{center}
\end{table}


For a regression model of the \code{glm-type} for which the defaults don't work we provide a simple mechanism that may allow the model to be used with \pkg{effects}.  We illustrate by a few examples that are included in the effects package.

\subsection{glmmPQL}
Objects of type \code{glmmPQL} do not respond to the generic \code{family} function, but the name of the family can be obtained from the call:
\begin{alltt}
effSources.glmmPQL <- function(mod) 
 \{list(family = mod$family)\}
\end{alltt}

\subsection{gls}
The \code{weights} argument has a different meaning in \code{nlme::gls} and \code{glm}, the \code{call} must be modified to set \code{weights=NULL}
\begin{alltt}
effSources.gls <- function(mod)\{
  cl <- mod$call
  cl$weights <- NULL
  list(call = cl)
\}
\end{alltt}

\subsection{betareg}
The \code{betareg::betareg} function fits data similar to a binomial regression but with beta errors adapting these models for use with Effects is considerably more complex than the two previous examples.

\begin{alltt}
effSources.gls <- function(mod)\{
  coef <- mod$coefficients$mean
  vco <- vcov(mod)[1:length(coef), 1:length(coef)]
# betareg uses beta errors with mean link given in mod$link$mean.  
# Construct a family based on the binomial() family
  fam <- binomial(link=mod$link$mean)
# adjust the variance function to account for beta variance
  fam$variance <- function(mu){
    f0 <- function(mu, eta) (1-mu)*mu/(1+eta)
    do.call("f0", list(mu, mod$coefficient$precision))}
# adjust initialize
  fam$initialize <- expression({mustart <- y})
# collect arguments
  args <- list(
    call = mod$call,
    formula = formula(mod),
    family=fam,
    coefficients = coef,
    vcov = vco)
  args
\}
\end{alltt}



\bibliography{methods-supported-by-effects}
\end{document}


