mixed effect models

Funnily, mixed effect regression was the first type of regression analysis I learned (I was given a huge complex data set with no prior R experience as an analysis task). I compiled a collection of papers and link and books that I used to self teach. My goal is to provide the links and a description as to why they were useful/why I needed that information so one can follow along and self teach also.

I am also continually updating it as new sources arise.

To begin, the following are MUST READ books or papers or tutorials. They really set the foundation for understanding and building multilevel models and what they capture beyond regular ordinary least squares regression (they aren’t necessarily in reading order, links may need to be updated too).

The following are extra materials that are highly relevant but that I didn’t interact much with. They may (or may not) be useful.

Finally, here are some pages that go over some of the basic questions I had during implementation. I will try to cluster them into overarching topics.

Understanding the analysis

Of course, the first search I did was to understand mixed regression in general. What does it do? Why do I need it? How is it different from other analyses?

Confidence Intervals

After running your regression, how do you get confidence intervals for your betas? Typically you use confint(model) or if you want wald (asymptotic and fast but less precise) confidence intervals, you use confint(model, method=’Wald’). However, here are some links for comparing confidence intervals through other packages or the difference between prediction intervals and confidence intervals.

(Restricted) Maximum Likelihood Estimation

An important aspect to understanding these models is how the parameters are estimated (hint: not using least squares). They use Maximum Likelihood (ML) or Restricted Maximum Likelihood (REML).


I understand the lack of p values in these models, but I come from traditional labs, so I had to learn how to draw p value based inferences from these models. There are many methods for this: likelihood ratio test (lrt) for model comparison, lmerTest for both anova and predictor style inference, bootstrapping, etc.

Logistic Regression

I ended up modeling trial accuracy data, which is a binary outcome variable and thus requires logistic regression models. The implementation wasn’t difficult, but interpreting the results takes practice and care. These links are general tutorials that helped me understand implementation and coefficient interpretation.

Model Building

I keep getting mixed advice about this approach and its varieties. I was taught by a statistician who said stepwise approaches were ok but I read otherwise. For exploratory work this may be ok (as compared to confirmatory), but do what you want. I’ll just post the materials I used to understand these methods.

Model Complexity

When I first started, I wondered how crazy these models can get. Can I just throw every variable in? Are there costs/benefits/limitations to parsimony vs complexity?

Model Fits

Diagnosing whether the model fits well and how to do so is important. This typically involves some form of checking unexplained variance along with examining assumptions.


When the data is not robust enough for the model or the model is too complex, it will not converge. This tends to render your estimates unreliable. So this is an important issues to either fix or look into to see how bad it is.

Variance Components

I’m currently working on projects that are more interested in the variance components than the betas. The variance components tell you how much the means vary across units of your random effects, e.g., if participants is a random effect, how much their intercepts vary. Important to this topic are intraclass correlations (ICC) and variance partitioning coefficients (VPC) and their interrelations.


Related to variance components, the within/between subject variance can give you a sense  about the reliability of your measure. The within subject variance would be the residuals that aren’t captured in the model, the between would be the random effect groupings. Not all links are necessarily mixed model related, but may be useful. Note: this is intimately related to variance components/icc above so those sources will also help.

Power Analysis

The hardest part (for me) about starting a study is determining power, especially when your analyses consist of complex mixed models. I haven’t fully read through all of these links,  but I am aggregating them to read soon.


This is an approach I’m slowly starting to look into, how to make my multilevel models bayesian. Here are some packages that are helpful.


Turns out analyzing your own data is only half the applications. Sometimes it’s worth it to be able to simulate data sets with specific variance components, or even specific crossed random effects. Here I’ll be adding links that help thinking through data simulation, especially when it gets complicated in terms of correlations between participants and/or stimuli.


This is just stuff I learned through the process that may not be directly related to mixed models.

So these are the links I found most useful, and I will update as I continue forward. And when I have more time I will make the links more descriptive as they are cryptic at the moment.

(un)documentation & (dis)identification

Where and for whom is the undocumented immigrant*?

*Here I mostly speak of myself (even if I use plural indicators) and this is not intended to generalize to every other experience, although if it does, great!


To discuss where I am, I want to situate myself as both within and between boundaries of social identity and geographic space.

One of the clearest truths I felt growing up was that documentation (i.e., a green card or citizenship) was a very salient and crucial form of recognition. Legitimized as a human member of society not just in the eyes of the law, but socially too. Lacking documentation has acted as the removal (or better yet, withholding) of recognition, forcing a state of disidentification where one must manage (and in doing so, co-opt) invisibility while living outside of the boundaries of legitimacy. This state is felt as a loss, as a long-term disorientation where identity is ungrounded and shifting and illegible within current modes of racio-ethnic-national understandings. The only constant is social illegibility, but even this itself is changing through the national construction and proliferation of deservingness and merit (e.g., DREAMers and DACA) that draws a tangible benchmark for receiving a piece of humanity and the societal rewards that follow.

However, in that space between legality, merit, and recognition, there is freedom to re-orient the boundaries of recognition. Current strategies in the fight for immigrants accept these benchmarks by fighting for indiscriminate assimilation. They contort and bend and fold themselves to embody the model minority, to live a recognizable life. But what good is a recognizable life if it requires you to cover up the people behind you? We must take advantage of this unstable state of disidentification to make a nonDREAMer life livable, to pry open eyes and national discourses to the value of the unseen. We must shake off any doubts we have when we say that we value the life of the “criminal“, that we value the life of those who invisibly survive and thrive in the U.S. in ways that aren’t considered excellence, that we value the life of the deported and the future undocumented.

And as we negotiate identity through discourse, undocumented identity is also geographically ungrounded and seems to be nowhere and everywhere at once. I’m not in Mexico, but I am. I’m am in the United States, but I’m not. I may step on the same physical ground as you, yet I wade through alternate social matrices that place me everywhere and nowhere. I believe some line of research has deemed this third culture, but if I don’t belong to the first two, how can this be my third?

For Whom?

If intersectionality refers to the invisibility and vulnerability of living within the overlaps of oppressive structures, then (un)documentation is a battleground where legality, nationalism, racism, sexism, queerantagonism all collide to produce highly invisible yet malleable targets. This malleability is a product of a group that by design is not meant to be publicly heard or seen. This means we’re mainly constructed by media think pieces on legality (towards whichever image is useful at the time) and by the few times our stories get told on a wider scale. Yet why is this invisibility by design? What is the function of having a physically recognizable yet socially unrecognizable body of people?

The answer I’d like to emphasize is that we form the basis of identity for the documented. We solidify others’ identities by being The Other. If they can point to what they are not, they can recognize each other as worthy of societal participation. You can see this by the many existing technologies for enforcing otherness. It happens legally through I.C.E raids, deportation, criminalization, and the impossible bureaucracy of formal citizenship. It happens psychologically through aggressive stereotypes and mistreatment and by stripping our ability to fully ground ourselves to previous and current cultural frameworks and geography. It happens politically through disenfranchisement – the literal instantiation of silence – and economic depravation. And while we exist for others, we don’t get to exist for ourselves. We cleave away at our being, cutting and cutting until we’re recognized by everyone but ourselves.


Where is the undocumented immigrant?
Mangled through the glorifying lens of merit AND rendered invisible through violent acts of forced miscrecognition.

For whom is the undocumented immigrant?
For those who seize recognition.

whiteness in statistics

I’ll set the context so people see where I’m coming from. I’m an undocumented immigrant from Mexico, have lived here in the U.S. for 21 years, and my family learned english as a second language. You can say I understand the immigrant/foreign experience intimately. This morning I scrolled through Facebook and came across a question on a statistics forum I follow Psychological Methods Discussion Group. You can read the transaction here (I scrubbed the Middle Eastern woman’s name but kept the repliers’ names for accountability).


There’s lots to deconstruct here.

Here we have a Middle Eastern woman who asked for help about a vague problem in a group whose title invites these sort of questions. Could her question have been more wordy, sure. Is it often that people who are non native english speakers ask questions that can’t directly lead to an answer? Sure. But that’s not the issue at hand. Daniel Lakens and colleagues (Robin Kok, Jazi Zilber, etc; all white presenting, many northern European) took it upon themselves to be the arbiters of the English language and deride this woman’s approach to finding resources. (Jazi Zilber is particularly troublesome given his self-pronounced experience with non native speakers).

When I read through their disgusting mocking of the lack of english proficiency, I couldn’t believe what I was looking at. Not only is it racist to assume that a person of color is lacking proficiency in language (as insinuated by their comments on her punctuation not being used in any language), but the accusation of a mental disorder for a simple question is gross. Yes, I was very aggressive -but think about the context: I’m a graduate student publicly fighting established researchers on their racism + I grew up in poor neighborhoods where physical violence and pointed call outs was the way to handle these problems + it’s sickening to see individuals disrespected. I follow Daniel Lakens and that group because I think they make great points about statistics in psychology. But here is some more context: this particular group has already been embroiled in the big fight about tone policing, with its members being particularly sharp about their academic critiques of others’ work. Generally, I agree with their main points about being vocal of unsound science, but this thread has fallen into an unacceptable domain. See, groups like this are full of psychologists and statisticians, who are “liberal”, and mainly (from what I see – and I read most posts) white. This means there usually aren’t checks and balances to keep their ignorant racial insensitivity in check. If there are people of color, they are far and few in between and clearly unwelcome unless they abide by white academic’s classist and racist rules of engagement. Not everyone in a non-english-dominant country can afford or has access to english education. And even when they do, it doesn’t mean they can hold it above others. Let’s take a look at the english of the man who cast his stone about english proficiency, here a video of one of his talks.

So to recap, we have a white northern european, with that english proficiency, and his fellow bullies jeering a middle eastern nonnative speaker’s english. I learned english as a child, so I grew up highly proficient, but it infuriates me to imagine my mom or my younger self coming into a room titled “HELPFUL DISCUSSIONS” asking a question only to be mocked for a) coming to ask for help and b) how we asked for it. If they didn’t know what she was asking or if they knew of a better forum, they could have asked in a non condescending manner or pointed her in a better direction. Instead, they took it upon themselves to completely ostracize the woman and subjugate her to shitty unnecessary comments. They clearly lack the humility it takes to provide help to those who most need it, I would hate to see him teach minorities. Uli was right, you can ignore the question or patiently engage. Don’t be an ass and try to masquerade it as benevolence.

For those who asked for evidence of how an instance like this could be racist, please read the following paper of the year in the scientific journal Language by Sharese King that delineates how racism manifested in the judgments of a witness’ testimony in the Trayvon Martin case who spoke in African American Vernacular English.

Here is an excerpt:
Screen Shot 2016-12-29 at 2.07.04 PM.png

I also highly recommend the ending of that paper delineating steps for how to fight linguistic racism (in the courtroom), stay woke. In addition, there is emerging field of raciolinguistics.

Here is an excerpt from the blog post “Why we need raciolinguistics“:

Screen Shot 2016-12-30 at 10.13.19 AM.png

The other aspect of this is something I have wanted to write about for a long time. The tone policing and suppression of minority voices. There’s been plenty of times on these forums that I have seen sexist (a man sarcastically dumbing down his remarks to a female professor as if she didn’t understand, it was clear he was wrong), transphobic (a man repeatedly enforcing gender binaries when transgender individuals were also relevant to the topic), and racist comments that go unchecked (like above). Or if they are checked, they are fought back against. Whether these instances are due to explicit prejudicial mindsets is unknown, but what is clear (to me) is often people don’t understand (or care to understand) how what they are saying is oppressive (which further necessitates for them to shut up and listen to critique).

Here are the replies I received:


Now we have a white northern european telling an undocumented Mexican in the United States what counts as racism. So I guess his motto is he can criticize others on their academic work, but he’s immune to critiques on his smug elitism, racism, and classism online. I knew when I commented I was jumping into their bro-den (they usually stick up for each other there), so I was ready for the backlash. However, what I received was far more unsettling. Daniel Laken’s threat to my academic future for calling out a problem he’s propagating is 100% unacceptable. Personally, I’ve fought so many damn obstacles to get to where I am, I don’t have a narrow minded view of success – if a post like this where I expose racism in my field limits my future jobs, then those are places I wouldn’t want to work at. What is the point of a forum or of all these surveys regarding the diversity within the field if not to bring up these issues? How is a graduate student supposed to address what they see as an injustice from established researchers? When it’s not about the research itself, but rather their toxic character, should a student stay silent? Should targeted established individuals be threatening students into silence? I don’t have the answers, but I’m willing to find out.

As a psychologist, this type of public interaction is an embarrassment to the field that studies human minds and needs to be addressed. Is this the reputation we want? Stuck up elitist white psychologists/statisticians who would rather spend time making fun of a help seeker than being mindful and patient or at the very least just not engaging?

To conclude, can someone with clinical developmental psychology experience help this woman? It may not be fruitful, but at this point she’s owed this much. (Update: she has been helped and apologized to)

Also, and this is actually the main point: fellow scientists, do better.

Joel Martinez

For a look at how whiteness defends itself (disguised as scientific inquiry) when a person of color speaks out, take a look at many of the responses in the following thread:

Makes you wonder about the state of this field. Some loud people on that thread (actual practicing psychology researchers) lack basic knowledge of racism, whiteness as a structure and mentality, and minority experience. They lack self awareness to take a step back and see what they’re defending. They were happy to disregard accountability for their friend. They flaunted complicity by pretending there was no issue, that we minorities just need to toughen up. They felt entitled to belittle/discredit/attempt to mechanically quantify the perceptions of racism of a minority. They transferred blame to me as the aggravator for calling out an injustice with a quickness. They gladly fought over whether it was racism or classism as a theoretical issue or whether I was calling him racist rather than focusing on the real experience that occurred. One of my favorite replies accused sensitive academics like me of inciting right wing populism across the world.

These are the tactics used by whiteness to maintain superiority and gas light minorities into submission. (With regards to charges of sexism, there is definitely something to be said about the lack of inclusivity in my approach and it’s something I learned from this situation and will correct in the future. Still doesn’t negate any of the points I have made here).

I limited my engagement in the debate because I had no idea where to even begin, there was so much ignorance of the critical conversations we have in the U.S. regarding race relations, the large and small forms of racism, and the improvement of minority treatment – no easy place to start. The moderators tried to keep it middle of the road, a “sanitized” look at the situation, but people got nasty defensive. Some responders prioritized their “objective” need for evidence over the fact that as minorities, we perceive microaggressions differently (otherwise their legitimacy wouldn’t be questioned). This ends up becoming a colorblind analysis, which may explain why they can see a bunch of white people make fun of a POC’s English and not register it as racism. This also places their lack of experience with racism as the benchmark and minority experiences needing “evidence” of racism to be taken seriously (ironically, the kind of evidence that one builds through personal experience with racism). What an eye opening insight into the nonprogressive mindset of some fellow psychologists. For social psychologists (the field that discovered implicit prejudice), I’m surprised that how they spoke about racism sounded like they expected it to only look like a confederate flag wearing white guy yelling the n word. I clearly struck a chord, and I have a feeling this won’t be the last discussion about this topic given their lack of understanding. There’s a LONG way to go.

One final point, many people supported my post through likes but generally stayed silent (though big thank you to the ones that did jump in). I received private messages of support for bringing this topic to light and if you look at where the support of most people of color lay, it was in likes for this post. Interestingly, they could easily see the racism that the people mentioned above couldn’t. That’s what mattered to me, that it resonated with them. However, to my fellow researchers of color and allies, you need to find confidence in your voice and fight for each other. Social norms matter, and currently they suck bc they allow situations like this, but we can change that. We shouldn’t let people tell us what constitutes racism when they’ve never been on the receiving end of it and we will not be told that our perceptions of racism are imaginary. We finally have a voice and this is the growing pains of battling for visibility.


social mindfulness

Peer-Reviewed Research + Books

My goal here is to aggregate academic resources on social justice. It will be a one page source for scholarly perspectives on social justice. Keep in mind this page is under construction, when I have more time I will fill it with content. Also, it has a focus on issues that I have had immediate experience with or a need to learn, but it will be expanded to include other topics.

The following are links to pages that have already aggregated important publications in list form.

Below I will list individual papers/books and summarize the take away message. I’ll try to find open links to them.

Racial Discrimination


Forms & Norms


Colorblindness & implicit



Incarceration & cop interactions

Politics, Housing, & Income




Costs & Consequences


Conflict Management

Queer Issues





Social Action


International Considerations


Immigrant Health

Immigrant Crime

Immigrant Education

Field Research

Economic Class







Other Sources

Here I will aggregate different sources like blogs or fiction that I have found insightful on the same issues above. Given that these are more author-driven pieces, doesn’t mean I agree 100% with everything they write, but rather, they are on this list because they provide insights that help conceptualize an issue.

Race/Ethnicity & Discrimination


Forms and Norms




Conflict Management


Queer Issues





International Considerations




Economic Class

Critical Theory


On this page, I will aggregate issues related to statistics proper.  For a developed resource page on mixed regression models, go here. This page will cover facts about other statistics – effect sizes, bayesian analyses, homo/heteroskedasticity, etc. Given the need to run high powered studies to output worthwhile science, I will also be collecting links and papers here regarding: how to run power analyses, the concepts behind them, simulations, and general information about them along with issues of replication and reproducibility of analyses.This page is currently under construction.

Error management

  • Family-wise error rate: Link

T Test

Degrees of Freedom

Testing Variance homo/heterogeneity

Effect sizes



Confidence Intervals

Likelihood statistics

Multivariate methods





  • The following site is a tutorial for how to structure your analysis stream to make it more reproducible. Link



P Hacking/QRPs

Machine Learning

  • Machine learning in neuroscience: Link

Signal Detection Theory



hidden minority

I participated in a campaign for the Princeton Hidden Minority Council aimed at raising awareness for what it means to be a first-generation or underprivileged minority going to Princeton. I think it’s important to be exposed to our perspective and the typically unspoken problems we face here in this privilege bubble. Here is my submission:



You should check out the website

You should also check out the album and the other submissions (link). I think they raise important points regarding family, identity, achievement, struggle.



My studies will require faces of all kinds, so I think I’ll just congregate face databases here.

Here’s a database others are working on with more information and sources.

Here’s another that includes more than just faces.

AgingMind – lifespan faces with a wide array of emotions: Link Paper
FACES – young, middle, old with variety of emotions: Link Paper
CAFE – children, many emotions, many races: Link Paper
KDEF – emotional adult faces: Link
Chicago Face Database – different ethnicities, normed faces: Link
10K Adult faces – adult faces, not really normed: Link
CVL – young adult faces, neutral, many angles: Link
FEI – brazilian faces ages 19-40, variety of angles: Link
Georgia Tech – frontal faces: Link
PICS – variety of angles, colors, ages (old looking though): Link
USCD – variety of angles, 28 faces: Link Link2
Bogazici – Turkish face set
FRL: neutral/smiling, diverse ethnicities + ages Link

Less Useful
Youtube faces: Link

If you want a fuller list: Link

compute & model

Being at Princeton, I’m surrounded by computational modeling, so I will aggregate sources here.

Reading list

Neural network

Reinforcement Learning

Machine Learning

  • techniques: Link
  • Easyml: Easily Build And Evaluate Machine Learning Models

Drift Diffusion

  • Drift Diffusion model is a computational model of decision making that decomposes reaction times and responses into cognitive components. The linked package is a specific python package I have used before.
  • Heirarchical Bayesian method: Link
  • hBayesDM: Link

Social Modeling

  • David Rand’s behavioral economics code: Link
  • Social Network Analysis class: Link
  • Bootnet: Link