Rationalists Try Investigative Journalism

If you’re not steeped within the broader rationalist/effective altruism ecosystem, this story might not make total sense unless you keep in mind one person: Sam Bankman-Fried. When he was exposed as a fraudster last year, the effective altruism (EA) movement was impugned by association because SBF was easily the world’s most notable face of the movement. If you were already of the mind that EA was full of naive and out-of-touch techie optimists, that they were hoodwinked by SBF served as further confirmation. The EA community naturally became much more vigilant about unrooting other nefarious actors within the movement.

So back in September, Ben Pace (admin of LessWrong, the central rationalist congregation space) published what he described was the result of an extensive 6-month long investigation into an EA startup called Nonlinear. The post relayed allegations of mistreatment from two former employees (pseudonymously Chloe & Alice) concluding with “if Nonlinear does more hiring in the EA ecosystem it is more-likely-than-not to chew up and spit out other bright-eyed young EAs who want to do good in the world.” Nonlinear frantically denied many of the allegations and asked Ben to delay publication for a week to give them time to compile counter-evidence but Ben refused. It wasn’t until almost three months later that Nonlinear provided a lengthy response, complete with a 134 page appendix of evidence.

The aim here isn’t to bog you down into examining each and every one of the indictments against Nonlinear, but TracingWoodgrains provides a summary of the chasm between some original claims and what the evidence from Nonlinear shows instead:

• accusations that when one employee, "Alice", was sick with COVID in a foreign country and nobody would get her vegan food so she barely ate for two days turned into "There was vegan food in the house and they picked food up for her, but on one of the days they wanted to go to a Mexican place instead of getting a vegan burger from Burger King."

• accusations that they promised another, "Chloe", compensation around $75,000 and stiffed her on it in various ways turned into "She had a written contract to be paid $1000/monthly with all expenses covered, which we estimated would add up to around $70,000."

• accusations that they asked Alice to "bring a variety of illegal drugs across the border" turned into "They asked Alice, who regularly traveled with LSD and marijuana of her own accord, to pick up ADHD medicine and antibiotics at a pharmacy. When she told them the meds still required a prescription in Mexico, they said not to worry about it."

May Allah have mercy on you if you ever draw any attention from Trace’s obsessive investigative impulse, because he followed it up with a thorough and damning takedown of the various ways this investigation failed. The failure isn’t just according to the standards of professional journalists, but also serves as a disturbing reminder of how rationalists can confidently delude themselves into mistakes. I highly recommend reading Tracing’s essay in full.

---

I don’t have much to add except to highlight some other instances where people led themselves astray. For example, fellow LW admin Oliver Habryka emphatically defended the decision not to delay publishing the investigation, despite growing concerns over its accuracy:

I don't have all the context of Ben's investigation here, but as someone who has done investigations like this in the past, here are some thoughts on why I don't feel super sympathetic to requests to delay publication

But now in response to Trace’s investigation into the investigation, Habryka appears to have changed his mind completely:

Given that Ben is working on a response, I think it's clearly the right call to wait a week or two until we have another round of counter-evidence before jumping to conclusions.

To be clear, the decision to delay publication will always be a judgement call subject to reasonable disagreement. As pointed out by Helen Lewis’s response to Trace, sometimes the targets of an investigation will request a publication delay under false pretenses either as an indefinite delay tactic or to break the story themselves under much more favorable framing.

Even the question of whether to notify the targets of an investigation is subject to reasonable disagreement. To give an intentionally trivial example from a journalistic amateur, the YouTube channel Gamers Nexus published an expose a few months ago about what it saw as shoddy practices by Linus Tech Tips, a wildly popular tech review channel. GN's video was very well received, but the most notable criticism was around their intentional choice not to contact LTT for comment prior to publication. They defended their decision, arguing it is not necessary when there's either a pattern of misbehavior or a significant risk of a cover-up.

One of the criticisms against LTT was how they failed to return an expensive prototype they received for testing purposes from Billet Labs, opting instead to auction it off without notice to or permission from Billet. GN's concerns about notifying LTT appears vindicated, because less than 3 hours after GN's video was posted, Linus quickly sent an email to Billet asking for an invoice (after months of radio silence) and then publicly proclaimed "we have already agreed to compensate Billet Labs for the cost of their prototype" falsely implying that Billet was in agreement. This was solid evidence that had LTT been notified in advance, they would've scrambled towards similarly dishonest attempts at public relations damage control.

So yes, reasonable people can disagree on whether or not to delay publication or even notify the targets of an investigation. No denying that. What should guide our decisions here should be adherence to generally-applicable principles, and I struggle to discern what principles are guiding Habryka’s decisions here.

For example, Ben Pace vaguely cited what he thought were credible threats of retaliation against Chloe and Alice for speaking out. Habryka speculates on several other possibilities:

My guess is Ben's sources were worried about Emerson [founder of Nonlinear] hiring stalkers, calling their family, trying to get them fired from their job, or threatening legal action.

By his telling there are ample reasons to discount the fears in this instance (though not conclusively so). Ben wrote that Emerson “reportedly” had plans to hire stalkers, and though this allegation is not impossible it strikes me as too inherently absurd to take seriously (How does one find stalkers to hire? What instructions would these stalkers receive? Would this be in person or online? How would Emerson guard against being linked to these stalkers? etc). The other fears he outlines fall under a similar penumbra in that had Emerson pursued the plans, they would only serve as the best confirmation of the allegations against him as a vindictive and vengeful character (but also what exactly would he even say to their families?).

I don't know what evidence Ben saw (and apparently neither does Habryka) but absent specific details, retaliation is a meaningless metric to consider because anyone saying anything negative about someone can plausibly cite retaliation as a potential risk. But assuming the threats are 100% legitimate, how exactly does hewing to a specific publication date mitigate against any of them? Habryka says that having things out in the open provides a defense, I admit I don't understand how that works exactly, nor do I understand why public disclosure would cease to be an option had Emerson actually followed through on his hypothetical retaliations before the post was published. We all know about the Streisand effect by now.

Habryka’s argument against publication delay I found the most shocking was this one:

Separately, the time investment for things like this is really quite enormous and I have found it extremely hard to do work of this type in parallel to other kinds of work, especially towards the end of a project like this, when the information is ready for sharing, and lots of people have strong opinions and try to pressure you in various ways. Delaying by "just a week" probably translates into roughly 40 hours of productive time lost, even if there isn't much to do, because it's so hard to focus on other things. That's just a lot of additional time, and so it's not actually a very cheap ask.

This was a giant blaring red alarm to me. When I heard about “40 hours of lost productive time” I initially parsed its meaning as “lost productivity because I was flooded with tons of irrelevant information that took 40 hours to sort through”. I never would have guessed that Habryka was instead referring to a mental fixation so severe that it occupies nearly half his waking hours. I would like to think that this should serve as a warning, a caution that perhaps one is too psychologically invested to adequately pursue truth, not as a justification to further accelerate.

---

The other aspect of the community response I found particularly puzzling was the overwhelming negative reaction to Emerson threatening a defamation lawsuit. The mysterious and prolific Gwern offered an unusually combative denouncement, tarring defamation lawsuits as overwhelmingly frivolous and claiming “they are solely a threat to destroy someone financially by running up costs.”

I confess my biases in that I am a lawyer, but also a free speech maximalist who used to work at the ACLU (back when they were cool) and an emphatic supporter of anti-SLAPP statutes. At the same time, I have only a vague idea of how many lawsuits are filed in general (~40 million), an extremely vague theory about what portion are defamation suits, and a hopelessly speculative guess of how many of those are frivolous. The normally fastidious Gwern is expressing a negative opinion about defamation lawsuits with an extremely high level of epistemic certainty, but the only evidence he offers are bare assertions and anecdotal examples from high-profile cases.

I suspect that defamation lawsuits have a poor reputation in part because of a selection bias. There are significantly more threats to sue than actual suits in our universe, and the threats that will shine brightest on the public's memory will necessarily be the most outlandish and least substantiated. Threats are further proliferated because they're very cheaply deployed (anyone with a bar card can type out a cease & desist letter on their phone on the toilet and still have time to flush) and — crucially — authentically terrifying regardless of the underlying merits or lack thereof. As Gwern points out, there is no question that lawfare is often levied as a war of financial attrition.

The closest corollary would be the bevy of tort abuse stories. Before it was widely and thoroughly vindicated, the McDonald's hot coffee story served as the lodestar condemnation that the American tort system was fucked beyond repair. But again, we're going to deal with a selection bias problem here. Unless you're trawling through every civil court docket in the country, the only time any layperson would hear about a personal injury story is when it's blatantly ridiculous. The same issue exists with defamation lawsuits.

So just because defamation lawsuits are used as a tool of abuse, does not mean that every defamation claim is baseless. I would hope that this statement is self-evident. Instead of picturing a scorned celebrity siccing their horde of rabid lawyers against any whiff of criticism, I'd want everyone to consider that sometimes random nobodies are accused of quadruple homicide by TikTok psychics, or accused of election fraud by the former mayor of New York City. I'd hope that everyone can appreciate how terrifying it can be to be the subject of a malicious smear campaign, how daunting the prospect of initiating a defamation suit can be, and how uncertain any potential vindication might be.

Ideally we'd have some way to discern which grievances are valid and which ones aren't besides just declaring all defamation lawsuits as inherently suspicious.

---

Lastly, there was an interesting question regarding who has the right to anonymity in this story. Nonlinear’s co-founders Emerson and Woods were already publicly outed by Ben, but even Nonlinear kept Alice and Chloe’s anonymity. But in their response, they allege Alice has a concerning history of levying delusional allegations of persecution, very likely the result of an untreated mental illness. So what is the principle behind who gets to stay private?

There's a worthy journalistic norm against naming victims of sexual assault, and a norm in the other direction in favor of naming individuals charged with a crime. You could justify this by arguing that a criminal 'forfeits' the right to remain anonymous, that society has a transparency interest to know who has committed misdeeds. Whereas a victim has not done anything to diminish their default right to privacy.

How you apply these principles to Nonlinear depends entirely on who you view as the malefactor (or none/both) I suppose, but there is obvious disagreement on this question. So who decides?

---

Yes, a lot of this can be dismissed as petty drama and hot gossip. But I agree with Trace that this should serve as a constructive learning episode. I adore the ethos behind rationalism and I have immense admiration and gratitude for the rationalist community, but this saga should serve as a stern reminder that we’re all humans after all. No matter how much of the Sequences we have memorized, we shouldn’t delude ourselves into thinking we can toss out any of society’s guardrails and replace it with something we re-derived out of thin air.