SquirrelInHell

Men Have, Women Are

noreply@blogger.com (SquirrelInHell) — Mon, 19 Feb 2018 23:00:00 +0000

Note 1: I'm by default reluctant to write anything explicitly related to gender issues, because our society has a very special way of being stupid about that, and it doesn't feel like adding my voice to the mess would really be, you know, adding anything. I think in this case an exception is justified, because I might genuinely have things to say here which are so controversial that they probably haven't been said before.

Note 2: Yes, I know, it's very easy to nitpick about the title. There's always some clever trick that can turn phrases like "be beautiful" into "have a beautiful body", and "have respect" into "be a respected person". Regardless, I think there's a natural way to cognitively relate "feeling-of-being" to certain aspects of human existence, and "feeling-of-possession" to others. Thank you for understanding.

First, let me tell you how I finally Got feminism. The common image is that feminism is about equality of men and women. It is also that feminism is a social and political movement. Take a few seconds to reflect on how stupid these ideas are. If you consider it from a distance, the movement never actually really managed to convincingly put up a pretense of having any coherent utility function in the political world. And also, taking a honest look at the whole "equality" agenda as it is currently understood, I think it is overall hurting women more than it's helping them.

Instead, I now understand, the appeal of feminism always came from it being mainly a psychological movement. The generator of the political and social side of it was "whatever you had to say at the time to help to put women in a certain mental state". And by its nature, it could never propagate completely by the political message alone. It had to include at least some contact with already-liberated women in real life.

If you look around, the version of feminism we have today is hopelessly garbled, to the point that most women don't even realize that it ever used to be about something else than the political gibberish. I think some of the real spirit has survived in LGBT communities, because they are the ones that are actually forced to stand in opposition to much of the broader society. And who knows, maybe there are some pockets of it surviving elsewhere. But if there are, I haven't seen them.

This brings us back to the One Problem (Almost) All Women Have. There's a very deep level on which human values are basically genetically (or prenatally) programmed, and it turns out that (surprise!) men and women tend to have different values written in. Now this is devastatingly, mind-numbingly obvious from an evolutionary psychology perspective, while also being so controversial as to be basically unspeakable in the current political climate: yes, there are psychological differences between men and women. And yes, those differences are significant.

I am reminded here of the well-known poem about lesbian sheep:

And alas, her true love, just a few yards away,

tries the very same thing in the very same way,

and though they both want to be loved and be held,

they just stand around, and wait to be smelled...

The issue I'm bringing up today isn't quite the same one, but the analogy is obvious. Men are programmed to execute their sexual strategy by a combination of grabbing what they need, in both the social and material sense, with a little bit of personal charm maybe mixed in there somewhere. While women are programmed to mostly want to be intrinsically attractive, according to whatever the society they were brought up in defined that to be.

One of the evolutionary reasons for this seems simple: as a man, you have the option of walking away. It's not great, but if the going gets rough, and especially if you are still young, walking away from your tribe is probably the right choice - and has been for most of your ancestors. While as a woman, if your life becomes unbearable, the best you can hope for is to signal that you wish to be snatched away. Walking away by yourself is very close to (reproductive, evolutionary) death. All of this of course has nothing to do with the modern world, but ancestrally-determined sexual strategies remain the motivation behind most of people's lives. And the number of women you'll see everywhere who feel miserable and just keep on doing the signaling... it's heart-wrenching.

People are very good at pretending that they do something else, while executing their sexual strategies. Obviously, they also pretend this to themselves, most of the time. And yet if you know what to look for, you'll see clearly how sexual strategies are upstream of most of awake time spent by people living on this planet, with exceptions mostly caused by people being in danger of dying.

An attentive reader will notice that according to this framework teaching people rationality and self-awareness should directly reduce their reproductive fitness. Yes, very much so. But it's not that terrible, because the world has a serious deficit of people who do things for other reasons than their sexual strategy.

I've been known to cause outrage by suggesting that people who really care about something shouldn't have romantic relationships. Think what would happen if I dared to suggest that those people should also seriously consider getting castrated. That would be crazy! And who am I to suggest that basically everyone claiming to be doing good is faking it? Then people would feel bad about themselves. We can't have that!

Shell, Shield, Staff

noreply@blogger.com (SquirrelInHell) — Thu, 25 Jan 2018 13:16:00 +0000

第一

There is a certain high-level pattern of progress seen in almost every human activity, which I like to call “growth triplets”. A growth triplet follows an archetypal sequence of three stages: naive, clever, and wise, with the exact meanings of these terms varying depending on the situation. The defining feature of the pattern is that the third stage has some unexpected similarities to the first one. Because of this, most people have to actually go through the second stage before they can appreciate the third.

You can find several examples in this Scott Alexander post, and recently I hinted at many others:

life/performance/improv

thesis/antithesis/synthesis

tribalism/rationality/post-rationality

brains/computers/neural nets

intuitive-cooperate/defect/TDT-cooperate

overplay/honte/sabaki (in Go)

In this post I'll be circumambulating an interesting growth triplet which I'm tentatively calling "dead/yang/yin" or "shell/shield/staff". The concepts of aliveness and yin/yang are already very overloaded, but I think that in this case it's a feature, not a bug. A matching interpretation is a progression from a style of being which I would metaphorically describe as living in a Shell, to something that resembles a using structure as armor or a Shield, finally moving on to having a structure shaped like a cane or a Staff. Let me explain.

Optimization is the opposite of possibility. Exposing a composed, compact, optimized surface to the world means that you resist the wind of reality, protecting yourself from it, and at the same time preventing it from changing you. There is a sense in which hearing the song of the Sword of Good is just an alternative definition of suffering. It is about letting your nervous system expand into the world, and be tarnished by the wind of reality, which can indeed sometimes feel more like a hurricane.

第二

The first, and at the same time the last, stage of optimization is a Shell. A Shell has been optimized to the point where there is nothing to add or subtract from it without breaking the whole. It is at the same time the ultimate protection, and the ultimate vulnerability. While hiding in the Shell the main questions you ask of your life are "am I okay?" or "am I preserved?". Your goals reference the self as the central concept, and cannot exist without it.

People don't like when their friends are Shells. Come out they say, we want to know you better. Give us the tools that will let us interact with you deeply. Do not hide in your Shell! However at the same time, those are the exact same tools that can be used to hurt me, the person inside the Shell knows. That's why the Shell is there in the first place. In this situation, whenever the nervous system becomes exposed, it is very weak. The nerves crawl low on the floor like vines, and are easy to step on. Everyone around is understood as a potential source of pain. The kind of pain that makes you grow, sure, but it doesn't mean the pain isn't there. The other people will happily tread on the nerves in the most malicious way imaginable, while at the same time smiling and congratulating themselves that they are helping.

In this sense a Shell is a containment device. Its function can be viewed as not so much protecting one from the environment, as preventing the loose strands of the web of one’s nervous system from spreading out into the world. The ultimate Shell is held together by optimization. And weak Shell is built from holding very hard onto individual parts, and counteracting any individual attempt of the environment to disrupt the configuration. A strong Shell has been optimized to the point where external influences bounce off without disrupting anything, automatically. The environment is inherently a weaker pressure than the forces holding a strong Shell together. This kind of Shell can even give an impression of superiority when viewed from the inside. That logic here is: if you think you are allowed past my Shell, intruder, you are welcome to break it! But I know that you can't, because you are too weak. Bwahaha!

This dynamic illustrates one of the central insights which I'm trying to circumambulate here: that what locally looks strong and therefore alive, might in the bigger picture be dead and therefore weak. Optimization is the new death.

第三

The next stage is being a Shield. In this setting, the rigidity is turned to one side only. There is a degree of control over what one is hiding from, and which parts of reality are fully engaged with. However, the Shield is big enough to obscure a big part of the view. Once a Shield is up, there is no longer a choice, or in fact any information, about that part of reality. You go fast, because you have chosen to put blinders on your face.

From the inside, this logic feels like making trade-offs. You optimize something and give up on something else, because that is the only way to function in this world! It's impossible to do everything and care about everything. Everything is about trade-offs. So just shut up and do the thing. When you tell yin to close its eyes and hit hard, it becomes yang.

That nervous system is allowed to expand out into the world, but only in the places where it's resilient enough. The back is not protected, but that's because the back is antifragile. It is not tough, but it is resilient. You can afford to have a few daggers in it. The nerves are above the floor level, so they do not get stepped on so much, and when they do, it is usually for a good reason. There are giants who will step on them, bringing their huge feet crashing down from impossible heights. But then you have the comfort of telling yourself that these are valuable learning experiences. You accept them and even welcome them. You congratulate yourself on your open and brave attitude. But the truth is, giants are rare in this world, and it is so much easier to deal with them when you are not covered with fresh wounds inflicted on you by the rest of humanity.

The equivalent of a Shield in moral philosophy is the position that only people who are currently alive have moral weight. A full, stereotypical Shell would have probably pushed for strict hedonism and egoism, but this is not a very defensible position. A Shield is much harder to win discussions with. What you call blindness, they can call a responsible trade-off, and get away with it. Although their ability to get away with these arguments depends largely on the social consensus and would disappear if said consensus upgraded itself to take into account full considerations of coherence. Similarly to some tendencies of Shells, Shields use the reference to self as a central part of their goals and arguments. From a broader view, this is not so much morally bad as simply an illusion, ill defined and impossible to coherently pursue.

第四

A Staff is the last stage of evolution of structure. Beyond this point, there is only discarding everything and letting oneself be spread over like a formless fluid, adapting to whatever surface it rests on. But while this extreme could naively seem like a good way to continue the trend, becoming completely formless undermines the ability to do anything at all. The trick is to have structure only to the degree that it allows one to spread out more, but without getting any artificial protection. The idea of the Staff is that while leaning on it makes it easier to stand tall, it does not shelter from the wind of reality. Not even a little bit. Quite the opposite: by standing straight you are exposing a larger surface! If your nervous system interacting with reality is imagined as a loose web, simply throwing that web out will make it crumpled and useless, while spreading it on a minimal, thin structure will maximize the surface covered. It will also maximize impact. And of course, it will maximize pain.

The aliveness of yin is not in how strong the web is. It is rather in how fast it can regenerate. Or even more upstream, the limit is in how much pain the yin is able to absorb. The process of keeping a web around some topic is not static, but is in fact composed of repeatedly letting it be torn apart, molested, brutally and unfairly taken advantage of. Each time it will grow back a little bit stronger, and this is the kind of strength that is adapted to the shape of reality. Because the process of destruction and growth happens organically, with each small increase in tightness being paid for in blood and sweat and tears, the resulting network of bi-directional influence matches reality like a well worn glove.

If you discard a Shell or a Shield, you feel more vulnerable. If, on the other hand, you discard a Staff, you instead feel less vulnerable. The pressure would be off if you just flopped down on the ground and stopped moving. But you don't. You keep going. You show to the world the jagged rags that you call clothes, the bare feet bleeding on the sharp rocks, the hunched back fighting against the hurricane, the face burned from staring into the scorching sun. You let yourself be judged, and you don’t care. You are ugly, and you are beautiful.

Actionable Eisenhower

noreply@blogger.com (SquirrelInHell) — Sun, 21 Jan 2018 15:24:00 +0000

Note: This is a small peek into the kind of background model-building that goes into Be Well Tuned. I don't currently have resources to invest into writing, so the description is pretty bare-bones. On the positive side, I have changed the blog theme to black text on light background - I hope that makes it more readable.

The Eisenhower matrix is a (somewhat) useful high-level framing of a certain aspect of productivity, based on classifying tasks/activities into four types. It's casual application was entertainingly explored in this Wait But Why post. I'll minimally quote just the matrix itself:

Q1 urgent and important (do it now)	Q2 important but not urgent (decide when to do it)
Q3 urgent but not important (delegate it away)	Q4 not urgent and not important (delete it)

Now while I don't deny that this classification is (somewhat) useful, it's definitely not very actionable. The typical obvious conclusion from reflecting on it, "I should do more Q2 activities", predictably does not get implemented. Well, that's because implementing a systematic way to do Q2 activities is itself a Q2 activity.

However, there's a version of the matrix which is much closer to being actionable (close enough that I dared to write specific instructions). Basically, I'm taking the distinctions between types of activities in the world, and turning them into distinctions between states of mind:

Q1 alert and anxious ("stressed")	Q2 alert and calm ("meditative")
Q3 unfocused and anxious ("useless")	Q4 unfocused and calm ("relaxed")

The unfocused/alert axis corresponds roughly to PSNS/SNS activation. The anxious/calm axis corresponds to how much the brain is under psychological/emotional pressure to solve some specific problem or task.

The "common sense" understanding of relaxation runs across the diagram, visiting mostly Q1 ("stressed") and Q4 ("relaxed") as a result of reacting to the environment. Although people sometimes end up in Q3 ("useless") by accident, as in e.g. spending a sleepless night worrying, it is much less likely that they will accidentally do something Q2.

Superhuman Meta Process

noreply@blogger.com (SquirrelInHell) — Wed, 03 Jan 2018 13:31:00 +0000

This is a more-or-less complete blueprint of the process I'm currently using to run my life on the meta level. It is organized around four major principles, each of which represents a non-trivial design decision. This means that you could negate each of them, and get something arguably sensible. The minor points serve to explain further, and to map out some consequences of taking the major principles seriously. For me, they are also cached thoughts that help me make decisions.

Many of the minor points, fully expanded, would be enough material for a separate blog post. However expanding them to that level would be quite pointless. If you cannot extrapolate them on your own, you probably shouldn't implement them anyway. In that case, you can probably think of some other set of principles that would be a better fit for you. One of the main messages I'd like to communicate here is that designing your meta process is worth the effort.

[Added later:] Many people asked me what exactly I mean by "meta" and "meta process" in this context. Let me clarify: I consider it thinking on the meta level when I think something like "what trajectory do I expect to have as a result of my whole brain continuing to function as it already tends to do, assuming I do nothing special with the output of the thought process which I'm using right now to simulate myself?". This simulation obviously includes all sorts of everyday changes to my brain, including acquiring new memories, habits, and preferences. However the key question is, am I reflectively consistent? Or: do I endorse the most accurate simulation of myself that I can currently run?

The meta process is what happens when I want to make sure that I always remain reflectively consistent. Then I conjure up a special kind of self-modification which desires to remember to do itself, and to continue to hold on to enough power to always win. I aspire to make this meta process an automatic part of myself, so that my most accurate simulation of any of my future trajectories already automatically includes self-consistency.

For the sake of brevity, I am writing everything as black-and-white and seen from my personal point of view.

Meta process is life. It's OK to mess up on the object level. However, there is no way to recover from a broken meta level, except by luck. And luck is not enough.
- Transparent meta. This meta process should be public, codified, and teachable. It has nothing to do with keeping the object level confidential.
- No second meta. The meta meta process is just letting the meta process naturally act on itself.
- Judge the meta process. Whether it's other people or organizations, their meta process is what you want to know (regardless of which parts they publicize). Their object-level results are interesting only insofar as they are evidence about their meta process.
- Seamless cooperation. Design your meta process so that it benefits you when you are alone, and also automatically cooperates when there's an opportunity to do so. Then scheme to be around the people you want to work with.
- Difficulty maintenance. Choose your object-level challenges strategically, to maximize how much you learn and grow. However, your meta process must always stay safely within the basin of convergence.
- Series of expansions. Everything you do might fail or be abandoned at any time. Strongly prefer plans which are still beneficial if interrupted in the middle.
Values flow outward. You want everything that happens to happen because of your deep values (this might, of course, include other beings achieving their values). Everything else is a bug. Every bug is critical.
- Full stack action. Everything below your top-level values is a tool in their service. Tools are not intrinsically good or bad, but you can make them more or less useful. Useful tools can be stacked together, so that actions propagate far without distortion.
- Cooperate not control. If other people have compatible values, they want to cooperate with you too. If they are adequate, you can't manipulate their values, and you gain nothing by trying.
- Reject invest-y power. Some kinds of power increase your freedom. Some other kinds require an ongoing investment of your time and energy, and explode if you fail to provide it. The second kind binds you, and ultimately forces you to give up your values. The second kind is also easier, and you'll be tempted all the time.
- Do only what you want. Having inconsistent preferences is a bug, and it's in your own best interest to resolve it. Afterwards, there is no need to hold yourself back. If you decide to be nice and share your powers, do it because it's good for you.
- Don't ask for permission. However, when possible, ask for information, and ask to have your models looked at. You'll avoid many mistakes and increase your utility.
- Don't apologize. Update. You aren't in this for emotional comfort. By apologizing to someone else who plays for real, you are insulting them by suggesting that emotional comfort is what they are after.
- Automatic respect. If your models predict you can't get away with doing something, you won't do it. The burden is on others to be adequate at seeing through your motives. There is no need to invoke morality or any special principles here.
- Trust only yourself. The burden is on you to have adequate models. If you think someone will cooperate because it's aligned with their real values, you don't need trust. If you turn out to be wrong, that's your problem. Screening people is always on.
- Mental security. As a human, you don't actually have consistent values, and it takes time to figure out what they converge to. The world is highly optimized to extract energy from you by distorting your thoughts, and making you confused about your values. You cannot trust yourself until you are adequate at seeing through threats in real-time.
Aim for full adequacy. Do not (internally) compromise. Discard inadequacy without second thought. Do not (internally) bow your head to Moloch. Take ideas to their conclusion.
- Everything for your own values. However, use an adequate decision theory. Neglecting the meta process is two-boxing, and you must not do it even if it locally looks like a thing to do.
- Luck is not enough. It is not enough even if it had already happened. Relying on luck exposes that there is a mission-critical piece which you haven't yet mastered. It'll get you sooner or later.
- A plan is not enough. Do not trust any human (including yourself) to do anything in the future, unless all the options available now have been adequately exploited.
- Optimization never stops. Avoid one-time effort if at all possible. Aim for long-term stability of the process that generates improvements. There is no room for the psychological comfort of certainty.
- Infinite inferential reach. It's not enough to be one or two inferential steps ahead of everyone else. Learn how to build towers of knowledge which can fully support their own weight, so that you can build ever higher. Do not pause as you create whole new disciplines and branches of science.
- Every bug is critical. Each unexpected error, no matter how small, is a canary in the coal mine for some bigger issue. The bigger issue will get you if you don't deal with it right now.
- Gain time by taking the time. You know very well that your time is limited, and you'll constantly be tempted to skip ahead by relaxing your standards. However, each case will be a terrible mistake.
- Security from the start. If your process is not secure now, it won't be secure later when you need it to be. Talk in person. Walk outside. Take notes on paper. Beware of consumer electronic devices.
- No gawking at adequacy. When you start getting some things right, pretty soon most of humanity will look like a bunch of clowns rolling around in the mud. Moving on.
- No falling in love. Being attracted to someone is a sign that your mental security is compromised, and that they are more adequate than you in some respect. Treat it as an important bug report.
- One strike and you're out. If you think you can cheat without getting caught, you should do it! However, if it becomes known that you have sabotaged some important value, there will be no explanation that makes it OK in front of other people who play for real.
Update mental software. The meta process is implemented by gradually self-modifying in the direction of needing less meta-level correction.
- Everything is a skill. You can learn each of them, but it takes time, so be strategic. Stealing skills from people similar to you is faster. If you think something is not a skill, it means you don't understand it yet.
- Every skill is a mental skill. Every skill tree can be traced back to what happens in human minds. Generic mental skills have exceptional return on investment, so make sure to grab them first.
- Integrate your mind. Your subconscious processing and emotions are more "you" than your stream of consciousness. You'll not get anywhere if there's any tension on the boundary.
- Intentions don't matter. No one cares about the narrative you are telling yourself about your actions. If your mind is not integrated, that's your problem. Take responsibility for all of yourself.
- Install automatic processes. The amount of things you can consciously track is very limited. You won't get anywhere unless you can self-modify on the spot and have your brain handle everything in the background. Do it routinely.
- Refactor your perception. Fully integrating new understanding feels from the inside like seeing the world differently. Do it routinely.
- What kills you is blind spots. It's OK to take big risks knowingly. But if you have a blind spot, you cannot know the upper bound of the risk. You should stop in your tracks at the tiniest suspicion of having one.
- No dirty imports. The world will often offer you knowledge in a package deal. However, you must break down every new piece of understanding into its basic components, and own each of them. Gain time by taking the time.

Notes on Mental Security

noreply@blogger.com (SquirrelInHell) — Sat, 30 Dec 2017 17:38:00 +0000

Note: these are some rough, preliminary notes on mental security, defined as the art and discipline of keeping one's mental software free from hostile external influence. Or in other words, having a mind able to achieve its own goals, rather than goals of, say, Moloch or other people which happened to be around. This is supposed to serve as a reference point for discussion and further thought. It's definitely not an explanation or a tutorial.

Why?
- this is dangerous business, and not one that human minds are designed for
- however, necessary to do anything which is far outside of the Overton window of agency
  - if you are doing such a thing, you might just as well ignore the risk, because if you fail at security your cause is doomed anyway
- of course, security is pointless unless you are convinced that you actually have some integrity/values/thoughts that are worth protecting
- and it should go without saying that working on security with insufficient self-knowledge is just shooting yourself in the foot really hard
Detection / Self-Diagnosis
- Male-Type Threats
  - which people or sources...
    - are you impressed by?
    - do you turn to when in doubt?
    - do you listen to/read very eagerly?
    - always have convincing arguments?
    - seem to always be right?
    - you can't imagine disagreeing with?
  - monitoring beliefs
    - coherent meta-epistemics
    - making beliefs pay rent
    - tracking social reality separately
    - disconnecting beliefs from identity
    - noticing burned beliefs
      - beliefs which are maxed out the scale of intuitive confidence, indicating that the whole scale is miscalibrated
    - wielding artifacts of power
- Female-Type Threats
  - which people or social strategies...
    - do you tend to find attractive?
    - make you feel safe?
    - make it hard for you to think clearly?
    - are you jealous of?
    - you can't imagine living without?
    - cause you to ruminate a lot?
    - you can't imagine hurting?
  - monitoring emotions
    - what emotional reactions do you have that seem inconsistent with your values?
      - again, should be obvious but: are you by any chance wrong about your values?
    - do they predictably change in certain situations, or around certain people?
    - are you sometimes surprised by your own emotions or actions?
    - do you experience akrasia?
    - do you tend to avoid situations which would make you experience strong emotions?
    - packet capture
      - spend a few days in isolation, watch compromised mental processes try to restore communication
Prevention
- learning to see through threats in real-time
  - building models in increasingly difficult situations
  - watching from afar how other people are being pwned
  - replicating offensive tactics to learn about them
  - pentest by a trusted and skilled third party
  - strong meta-rationality to estimate threat levels
- offense is (some) defense
  - however, this is leaky and fragile
  - might give a false sense of security until you are suddenly out of your league
  - it is common to subconsciously gravitate to this local optimum
  - there are ethical concerns
- firewall
  - gather evidence about power levels of various treats
  - avoid exposure to known threats above your level
  - keep physical distance if possible

Warning: all the articles linked above potentially lead to lethal memetic infection loads. There isn't much I can do about it.

Happiness Is a Chore

noreply@blogger.com (SquirrelInHell) — Wed, 20 Dec 2017 11:09:00 +0000

I've literally run out of cynicism.

Not so long ago, one of my long-term strategic projects felt especially close to my heart. The project could be summarized as something like "give happiness to people who already have rationality as a prerequisite".

In my rampaging naivety, I had thought the hard part was to figure out happiness. What was I thinking? To find out, follow this short imaginary conversation with my old self.

Me!2016: Oh. Hello?

Current Me: Hi, me from 2016. I have a question for you. You know this thing that we care about, where people become more efficient/rational/smart/well-off, and yet do not become substantially happier as a result?

Me!2016: Of course. I am a little bit confused about how people aren't working on this, despite it being obvious that it's an important problem. I guess this must be because it's very difficult to actually change how happy people are. There are, after all, well known psychological effects that prevent this, like the "hedonic treadmill".

Current Me: Erm. Let's leave this part for now. I have a question concerning a different aspect of the issue, and I'm curious what you think about it. Assume for a moment that I already figured out a method for people to achieve stable, safe, instrumentally useful, and viscerally satisfying happiness. Assume additionally that the method required only a moderate amount of effort, such that it's within reach of most healthy, functional human beings. For the sake of concreteness, put it somewhere between learning to swim as an adult, and learning a foreign language. Would you consider this a complete solution? What else could go wrong?

Me!2016: Give me a second, these are pretty strong counterfactuals. I need some time to adjust to thinking from what seems to me like a very small section of probability mass. You want me to assume that it's actually true that you know such a method, and that it's every bit as good as you described it. Hmm, even in such a case, I might still have trouble believing what you say. Especially believing it enough to go through with some significant effort, which your method seems to require.

Current Me: I grant you that the effort is significant, but it's not extraordinary. And the benefits kind of are, you have to admit. But let's not talk about that. What if I told you that convincing people my method is real, and that it works largely as advertised, turned out to not be the main issue, in this hypothetical scenario?

Me!2016: Ouch. You just keep on piling these unintuitive assumptions, taking me further and further away from my priors. I'm not entirely sure my current common sense reasoning would apply in a world in which all these things were true. I feel like in that world I'd need to seriously recalibrate my brain, until it would start expecting this sort of weirdness. Although I have no idea how I could do that coherently.

Current Me: I understand. But please make an effort anyway. I want to hear your best guess.

Me!2016: OK... Let's see. Assume I know such a miraculous-sounding method for people to become happy, and it's a real working thing, no cheating. I also have no trouble convincing people that it's real, and worth the effort. Aha! I know. Is your method teachable?

Current Me: It's a little bit tricky, because it's purely a mental skill. For most people, this makes the learning process more difficult than in case of more tangible skills like swimming. From the teacher's perspective, it's also harder to watch over a student's progress, or give hints when they make mistakes. But even considering all of this, it's probably not as bad as you imagined it just now. It's definitely teachable, and in fact various more-or-less contorted versions of it have already been taught in this world, with reasonable success, and to not-very-strongly-preselected groups of people. Is this satisfying?

Me!2016: Ah, so you admit that there is some difficulty with teaching. But the way you describe it, it doesn't sound like deal-breaker level difficulty to me. So you still want me to think what else could go wrong?

Current Me: Yes.

Me!2016: No, honestly, I give up. Assume the method to be happy is real, safe, clearly beneficial, and teachable. Assume people believe me that it has these properties, when I tell them about it. I think that yes, this would be a sufficient solution to the problem. I'd start by teaching it to a few close friends. Soon enough they would teach their friends... Oops! Say, how long does it take for students to learn, and for teachers to teach?

Current Me: Haha! Good catch. But no, that's also not the issue. Intelligent and motivated students should pick up enough in a month to reap most of the benefits, and be able to maintain them indefinitely with only, say, 1 day per week of ongoing investment. The instruction could be seen as pretty light work, though the way things are I'd say it tends to be an emotionally exhausting and thankless job.

Me!2016: Then I really have no idea.

Current Me: OK. I'm going to reveal the answer now. I wasn't just teasing you, I want you to know, though I'm worried that you don't have the ability to fully process the answer from your current point of view. But anyway. Consider this: what if I told you that people don't actually want to be happy?

Me!2016: Wait, you can't be serious. People clearly do pursue happiness, I see evidence of this all the time. That's what I've been doing for as many as 10 or 15 years. It's also a part of your memories, so certainly you understand.

Current Me: Ahaha. That's exactly the part which makes me feel like I've "run out of cynicism". See, the human activity you describe as "pursuing happiness", from my current perspective, seems to be in the same category as other common activities such as "acquiring education", "helping people", "talking to friends" (or should I say "talking" to "friends") and so on. Which is to say, people do them in a way which is outwardly convincing enough to allow everyone to keep up the social pretenses. This is way different from what you'd see people do if they actually cared. The simple matter of fact is that the human brain is a kludge, and people are puppets dancing on the strings of a mad puppetmaster. Almost anything they claim to be doing isn't for real. This is true even when they themselves know about this. The best you can do is gradually nudge yourself in the right direction, gaining new footholds in consistency and consequentialism painstakingly and precariously.

Me!2016: ... I might be having a failure of imagination here. I am hearing the words you say, and you seem to be saying them honestly. But I admit I cannot put myself in a frame of mind in which these words would seem like a natural thing to say.

Current Me: I'll tell you the thing that finally broke me, finally pushed me over the edge with grokking the human condition on a full System 1 level. To give you some background, fixing your epistemics has immense compounding interest. Put this sentence on your wall. I have become much better at recursively estimating my confidence, since the time when I was you. So if I made something work, and it is clear that it did in fact work, I have much less self-doubt in believing that reality did in fact turn out to be this way. By the way, you don't realize that you have that self-doubt, because you have much less of it that the average person. And this is the kind of self-doubt that puts a ceiling on how high your own estimate of your own mental skill can be above what you consider typical. So I'd call it a self-reinforcing blind spot, in CFAR terms. Anyway. My models of improving happiness have enough gears, and I have seen these gears turn enough times, and I have enough justified meta confidence in my ability to build models. I have felt levels of happiness which are far above the upper limit of your mental scale. I know exactly how to be happy. And yet I find myself not consistently applying my own methods. Do you realize how impossibly mind-twisting this situation is? What happens in reality is that I enjoy and see great value in happiness when it happens, but when it doesn't I only work on it grudgingly. It's like with exercise, which is great but I'm rarely enthusiastic about starting it. The problem is not that I don't value happiness enough. The problem is rather that there is no gut-level motivational gradient to get actual happiness. There are gradients for all sorts of things which are crappy, fake substitutes. Once you know the taste of the real thing, they aren't fun at all. But you still end up optimizing for them, because that's what your brain does.

Me!2016: Er. Um. So. Do you mind if I go back to my timeline, to, er, consider what you just said?

Current Me: Sure. I'll see you around, I guess. Sorry about ranting too much.

Me!2016: disappears

* * *

Update: A lot of people are asking what exactly I mean by happiness in this post. Haha. It's not like I can define a subjective experience without getting into circular reasoning. But let's look at some examples. They are not examples of what I'm talking about above, but merely of what I put in the broad reference class of "happiness".

Let's say you had been doing some really good work, and took a day off to go on a hike. At some point on the mountain path your mind clears, and you feel calm, content and energetic. There's a warm sensation spreading inside your chest as you look around and find out that you can take in extremely fine details of the landscape, and that its colors seem more vivid than ever. You feel proud of the life you'll come back to, and overall like you are in the right place in the world doing the right things. That's happiness.

Now imagine you've just gotten a new car which is cooler than your neighbor's, and which you kept telling yourself was the only thing preventing you from being as happy as him. On the next day the enthusiasm wears off, and you can feel the jealousy and dissatisfaction creeping in again. That's NOT happiness.

Spontaneous Trumpeting

noreply@blogger.com (SquirrelInHell) — Sun, 03 Dec 2017 19:06:00 +0000

There are some things which after a period of contemplation acquire a definite "I want to trumpet this from the rooftops" quality, and which I end up not saying because the world doesn't seem to be listening anyway. I also don't have enough clarity and/or writing skill to force the world to listen.

Some of these thoughts come across in comments (for example on LessWrong 2.0), where I somehow feel less like I need to have an explicit defensible justification for everything I might want to say. I'm quoting two such comments below, in the vain hope that someone (anyone?) will get the message. YMMV.

Here's one on Acknowledging Rationalist Angst:

[Content warning: unpopular opinion.]

[I know that this is not the whole point of this post. I'm just responding to the part that is my personal pet peeve.]

I think it's in general harmful to make excuses for why rationalists are supposed to be weak. Because, you know, they really do wipe the floor with the competition, if they go far enough in the art. Why would you discourage yourself, and others, by saying that for this or other reason it's reasonable to expect rationalists to be unsuited for real-world combat? No it's not! It is painful to hold oneself to high standards, because then failure feels like failure. And yet if you want to walk the path, you won't go anywhere far by going around spending effort on making excuses for why these high standards don't apply to you. Even if the excuse is everyone else around is just the same, and even when this is in fact mostly what happens. Bah! You'll still do better if you can think I will not invent convenient excuses for failure, no matter how reasonable they sound.

And another one on Sunset at Noon (which is a very cool post, and you should totally go read it):

For you it will be a minor piece of evidence, but I hope it pushes you in the right direction on this delicate issue. This is verbatim from my personal notes, some time ago, formulated directly from my own experience and understanding of the world, without hearing anything about "Bayesian wizardry" beyond the general background of Eliezer's sequences:

Bayesian superpower

-> the ability to intuitively get Bayesian updates on evidence very precisely right

(huge returns across the board)

(learnable though elusive)

I am personally convinced that this is a real, specific, learnable, rare, and extremely valuable mental skill, and that I have greatly improved at it over the past 2 years. I have no way of proving to anyone that this is real, and I am super vague on how one would go about teaching it, but I nevertheless want to bless anyone who declares any level of interest in acquiring it.

Upon further reflection, you learning this would be extraordinarily cool because you might get a better shot at teaching it, even while being worse at the object level skill than some people.

The Little Dragon is Dead

noreply@blogger.com (SquirrelInHell) — Mon, 06 Nov 2017 21:15:00 +0000

第一

This is a story of my 2-year-long struggle with an unwanted habit of cracking my neck, which had gotten out of hand around 2013-2015. If it seems to you like it's not a serious problem, consider:

it's a serious health hazard (it seems that repeated strain of the neck area can eventually cause e.g. a stroke),
it made me look socially awkward, bordering on retarded at the peak of intensity,
there are powerful subconscious mechanisms that regulate muscle tension and produce the impulse to stretch, and denying them results in ever growing pressure that eventually breaks your will, or if it doesn't it happens at the slightest lapse in attention,
executing the habit doesn't require any special objects or situations, so there's virtually no way of blocking it externally,
underneath there was a real problem of having stiff muscles in the neck, and clicking/stretching did sometimes help somewhat; so there was a real and powerful incentive gradient to keep on doing it,
at the peak of intensity, I started getting persistent headaches that only stopped when I managed to control the impulse, driving home the point about health hazard,
the tension in the neck, and repeated subconscious attempts to reduce it by moving around, prevented me from falling asleep virtually every day (often for many hours),
it turned out to be harder than almost any other personal problem that I had; it remained undefeated for 2 years while I completely moved past procrastination and social anxiety, figured out the meaning of life, became happy, sorted out my goals, moved to the optimal place on the planet (I now live in Las Palmas de Gran Canaria) etc. etc.

And yes, in the grand scale of things, the habit of cracking the neck is not a serious problem; my own ambitions reach much higher than this. So the whole situation also serves as a reminder of my own weakness, and possibly human weakness in general.

第二

The timeline starts in January 2016 (t=0). At the time, I was living in Japan, and I was after a period of intense self-training and self-educating about rationality.

t=0.

I fully realized the severity of the problem for the first time. It had been getting worse gradually, and I had become used to it, to the point where I hardly paid any attention to it. But after bringing it to light and contrasting with my goals and what I wanted to become, it was suddenly painfully clear that I could not afford to let things like that keep on dragging me down.

I've spent a long time just standing motionless with the realization, letting it sink in. At the end of this, I had tears in my eyes, and I was as determined as ever to crush the problem with all my force.

t=+1 week

I've rotated through most of the useful strategies and rationality techniques that I could think of. They did work, in the sense that with enormous and constant effort I was keeping the lapses at an acceptable if not negligible level. The most success I had with variations on TAPs (trigger-action plans), but I had to keep on strengthening them and adding new ones, because everything that took hold was quickly weakened by the overwhelming subconscious incentive gradient in the opposite direction.

t=+2 weeks

Some patterns have started to emerge, that allowed me to get on by a little bit less of consciously applied pressure. For example, it helped if I did a lot of exercise with my upper body, and made sure to not be stressed at work, and also to not sit down for too long, or ride in cars, or sit in the metro.

All of that (and more) was just barely enough, because at this point I was running out of ideas/rationality techniques, and I've hit diminishing returns on trying the same ones again and again. I've read several books related to rationality, habits, and addictions, looking for fresh ideas, but not much seemed to be helping.

t=+3 weeks

I've had some serious relapses, and in desperation I made a pledge to just keep up constant vigilance for a few days. I've reframed all my activities from "X" to "X AND not moving my neck". For example, instead of "brushing teeth" I was "brushing teeth AND not moving my neck". Then instead of "cycling to work" I was "cycling to work AND not moving my neck". At the end of each activity, I took special care to transition to the next one without a lapse in attention.

Surprisingly, this did work, although it was as tedious and unpleasant as it seems from the description.

t=+4 weeks

For the first time, I've felt the impulses weaken somewhat. I still had lots of trouble falling asleep, and whenever I become stressed, or worked too much, the habit would come back to its original uncontrollable strength, at which point I would panic and push it down with all methods known to me so far.

I've given up on the idea that I could deal with this issue fast, and I've gritted my teeth while estimating at least 1 year left to go.

t=+2 months

My habit has morphed into a version that used less visible movements. It was sort of more like squirming than the full movement of bending/stretching my neck. Unfortunately, this didn't make it any easier to deal with, and my body could still get the satisfying click out of it no matter how small the movements looked from the outside.

In the meantime, I've honed several rationality techniques to perfection, and solved most of my short- and medium-term life problems. The seemed strangely easy in comparison. I've also stopped consuming fiction in all forms, with the goal to redirect all of my brain's attention to the bare reality.

t=+3 months

I've starting using strict pomodoro timing (a 5 minute break after each 25 minutes) for all my sitting work, and doing exercises in each break. It turned out that standing on my hands was pretty good for neck tension, so I was doing that a lot.

t=+6 months

I've reached something like a 50/50 tipping point. I remember thinking that if I continued with approximately the same level of effort, I might just come out of it fine after a year or two; and that if I managed to somehow step up the game, my odds would be much better than that.

t=+9 months

I've learned how to massage, and how to use my hands to find tense muscles in other people's bodies with precision. Then I learned how to self-massage, and I learned all the muscles in my own neck.

This was a new superweapon - in 15 to 30 minutes, I could go through all the muscles in my neck, and relax most of them to the point where the pressure was bearable. It would make me able to go around for a few hours feeling almost normal.

In retrospect, I've been systematically missing a few important muscles because I thought they were bones or tendons. An easy mistake, considering they were always as hard a bones, and didn't budge regardless of what I did to them.

A more important mistake here was that I've learned the muscle locations mostly from my own neck, and I wasn't scrupulous enough when massaging other people to cross-check that fresh knowledge.

t=+12 months

With daily self-massage, I was finally doing fine. As long as I kept away from cars, and made sure to not be stressed, I didn't have to even pay all that much attention.

The habit has been on a slow but consistent decline for months now. Also, some of my lapses changed from cracking to merely stretching my neck. I was very happy about that, because it seemed from both theory and my own observations that a clear, momentary sensation (like clicking) was a contributor to habit strength.

t=+22 months

My friend went to a 10-day vipassana meditation retreat (Goenka school), and described their experiences to me in detail. Combined with having recenly read "Mastering the Core Teachings of the Buddha", it made me acutely curious about what it's like to reach those states. At the first opportunity I've isolated myself from the world, and attempted to replicate.

The attempt was successful, and among other things it yielded a procedure that I could use to consciously "reach" each individual muscle in my body, and control its tension. For the first time, I felt complete relief from tension, and I literally have no words to describe how good it felt. In my private notes, I've described it as "infinite relief and bliss", and I still don't think the phrase does the feeling any sort of justice.

And it wasn't just temporary relief. It turns out that by removing the tension completely from multiple muscles at once, I've forced my brain to update its internal wirings which controlled how to move those muscles, in such a way that the tension would not come back. For example, it turned out that whenever I was using my bicep, my neck would become a little bit more tense, probably because there was some neural connection somewhere that spuriously learned to control both at the same time. And by forcing both of these to be zero at the same time, I've exposed the uselessness of that connection. It died under pressure, with a sort of a fizzling/tingling sensation around both of those regions.

Repeat the above for the whole body, and... my problem was fixed overnight. It was hard to believe at first, and I was expecting a relapse. However, a few weeks later, I am convinced that the solution is stable, and that it was in fact the solution.

(By the way, the method also has tons of other benefits which you totally wouldn't believe. For example, it turns out it can also cure carpal tunnel syndrome, and remove many common cases of back pain, and fix bad posture. And did I mention that it's pleasant and makes you super happy?)

第三

I've learned a lot while trying to deal with this problem, though most of it is the kind of knowledge that is impossible to transfer in words, if at all. Here are some of the more explainable lessons:

finding a good solution that hits at the root of a problem can easily make a 1000x difference in effectiveness, even in a personal life problem without any force amplification,
all the power of abstract thinking ultimately comes from connecting it to the body,
the same applies to applied rationality techniques,
being obsessed with rationality techniques can slow you down overall even though the techniques work,
you become as strong as the enemies you pick for yourself,
trying new things in life is such a good idea, that I underestimated it even while doing it and knowing all along that it's a good idea,
prompting your friends for detailed descriptions of their subjective experiences can save you a whole lot of work,
many commonly experienced issues with the body have causes in the brain,
after you deal with some problem completely, you tend to forget how hard it was, or for that matter become unable to imagine what it's like to have it,
tangentially related but so important I have to mention it: non-fake meditation can really make people sustainably happy, and this is not a trick.

Time to Exit the Sandbox

noreply@blogger.com (SquirrelInHell) — Tue, 24 Oct 2017 08:03:00 +0000

第一

This will be short and to the point.

Sometimes when you are learning something, you reach a point when you are constrained by raw power. Think physics and particle accelerators.

You compensate by being better at interpreting subtle evidence. Even with low power, you can learn most of anything if you have good models, and do the hard work of extracting a weak signal from noise.

However if you find yourself in this situation, increasing power should be your first priority. If you hit the power jackpot, you'll experience some very rapid progress, because your sophistication is way ahead of your previous level of power. You are ready to wield more of it.

To put it another way, when your actions have consequences, and you want to learn how to act, you should be throttling yourself on purpose. If you don't have to do it, then something is very wrong.

第二

Human beings by default imagine that their conscious thought has more influence than it really has. They think it because they are fooled by their brain's built-in biases.

The first correction is to note how little control we really have over anything, aka the "rider and elephant". After realizing this, you can make good use of whatever little power you have. People from rationality/LessWrong circles tend to get this right.

So at this point it should be clear that the conscious mind/System 2 operates in a sort of a sandbox.

The second correction is to realize that you can go out of the sandbox. Of course, the keys to exit the sandbox are weird-shaped, and require work to acquire. They must be like this. The point of a sandbox is that you shouldn't be able to exit it by chance.

However, rationalists are more than ready (at least some of them!). The problem is that they also tend to believe that the sandbox is the whole world, and they do not seek the keys.

A part of the reason seems as simple as: it's low status to say that conscious thought has superpowers inside of the brain. That's what people think before they make the first correction.

第三

Putting 1 and 2 together. If you are ready, seek power within your mind, because the power is there.

By the way, if you go to a long meditation retreat that does not suck, they can probably teach you enough. It will be unpleasant, because their methods are crude and their models are crap. But you only need to stick around for long enough to see that the power is real.

You also need to not bend your neck to the power, and worship it just like millions of people did over millenia. Reductionism! Consequentialism! Bayes! These are powerful tools, use them! You'll figure it out, because you are ready.

Update: In case you are wondering just what kinds of things are possible to discover outside of the "sandbox", here's one example about which I wrote recently (added only to satisfy your curiosity, it is not presented well enough to convince you of anything).

You Too Can See Suffering

noreply@blogger.com (SquirrelInHell) — Tue, 03 Oct 2017 19:28:00 +0000

Content warning: the last part of this post is saying some things that might be uncomfortable to hear, in a sharp tone which you might not like.

第一

I'm spending this week living in Airbnb with my partner. The host, a young Chinese woman, has a dog. Mixed-breed. Maybe half a year old. Underfed. Lonely. Mostly left alone for the day. It's a young dog goddamit, it wants to play. Or rather it must play! Play is needed to develop social instincts. Otherwise, the dog will never learn how to find its place in the pack. It will never be happy. Forget about playing. The dog needs to pee! It will never learn to stop doing it in the kitchen if it only has a chance to go out once or twice per day. And I'm not even telling you to care if the dog gets enough vitamins or whatever (no it doesn't, not if it always gets the same cheap food pellets).

The suffering of a dog, especially a young one, is easy. It's plain as day, innocent, and infinitely trusting. You just cannot avert your eyes from it. You'd have to literally pluck your own eyes out. And yet the owner doesn't seem to be noticing anything. My partner, who came with me to the same house, at the same time, also hadn't been noticing anything (until I pointed it out). It seems that seeing suffering is not so easy after all.

第二

Later, sitting on a bench, taking a good look at some strangers on a beach promenade. This one guy is sighing heavily, resting from the sun with a pained look on his face. That woman is walking with such a stiff, constricted step and posture, that I can only imagine what kind of horrible issues she has with her body. Among the dozen nearest people, I find one middle-aged guy about whom it's not immediately apparent that he is suffering right there and then. It's a beautiful day and a beautiful beach on the Canary Islands. The ultimate paradise in this corner of the world. Most of these people are probably on vacation, and they are supposed to be having fun.

I've asked my partner about her experience of seeing people on the street, at it's not like that at all. Most people who read this might not even know what I'm talking about. You might think that maybe I'm exaggerating, or imagining all this. But no. Maybe you had at least one moment of clarity, a time when you weren't too swamped with your own issues, so that you were free to actually make use of empathy. Then you know, and I don't need to explain anything. Otherwise, ha, good luck with understanding this post.

Update (added later): some readers interpret the word "see" as referring to opinions, morality, or other such metaphorical "viewpoints". However, what I mean by it is rather how perception works, and "seeing" in this sense feels very much like "seeing" what clothes someone is wearing etc.

第三

It seems that there is definitely some trick to seeing suffering. Otherwise, everyone would do it all the time, and we would live in a very different world.

It's probably worse than that - there are multiple tricks to doing this, and you need to get almost all of them right at the same time.

By the way, if you actually learn some "trick", or a piece of wisdom as some might call it, it becomes a part of your nature. It's so much a part of your nature that you can't imagine how it is to be without it, even if it's written right there in your own notes/diary etc. You can repeat the thoughts as you thought them before, but they are now empty. They are no longer "you". So if you are lucky enough to have notes, they are basically everything you know about before you got here.

And of course you don't have notes! This is your whole life, and you only find out much later what was actually important. So I cannot tell you the tricks, because I don't know, OK?

第四

Knowing that, I'm still going to take some educated guesses. Better than nothing, huh?

Fix yourself right up! You need a surplus of strength to do anything. You can't change while you are just barely scraping by. Not gonna happen. Sorry.
Live your emotions! Don't avert your eyes from how they are all your world. If you need to put up the pretense of rational, calculated action for other people, at least you can stop fooling yourself. Of course it is possible to learn to act more rationally, but certainly not before you admit that you aren't already doing it.
Get out of your head! If you are thinking how to be more empathetic because e.g. then people would like you more... Come on. Let's not be ridiculous. You need to actually care about something that is not about you. Sounds easy right? Yeah, sure.
Look around! There is so much to see that it's just might boggling. You can't possibly integrate all this knowledge, and if you wanted to use your verbal cognition... No that's not even funny. I know, we are all sharply intellectual here, building models, making things explicit, sounding smart, all that crap. But please realize that this is just a thin show, mostly for signaling purposes. No one is actually smart because of their explicit cognition. If you don't use all of your brain, you are not just stupid, you are blind. You cannot possibly see the things happening around you, not to mention make sense of them.

Now with all this, you should have the basics covered. Please excuse the sharp tone, if that was not what you needed at this moment. I wish you good luck on your journey.

Update (added later): some other readers got the impression that I just have a built-in, permanent property of seeing a lot of suffering. Couldn't be more wrong! My starting point was a pretty unusual lack of empathy, and it took me years to get to the point where I was able to change this. It's a specific skill, and I have learned to execute it pretty much at will. Since I learned it, my base level seems higher than before, but not in a negatively life-affecting way.

Understanding Policy Gradients

noreply@blogger.com (SquirrelInHell) — Wed, 13 Sep 2017 21:12:00 +0000

第一

There are three stances one can take when dealing with a mathematical subject.

The first one is the engineering/practical/below math stance:

what works, works
mistakes can be avoided by doing empirical tests
math is difficult and scary

The second one is the symbolic/formal stance:

we can rely on what comes out of equations/proofs
mistakes can be avoided by really careful calculation
math is slow and laborious, but ultimately more powerful than the engineering stance

The third one is the deep understanding/above math stance:

mathematical symbols are merely a communication crutch, what matters is to transfer ideas between minds
mistakes can be avoided by really understanding what one is doing, from first principles to the big picture
math is easy and fun, the symbols are needed only afterwards to write down results for other people

第二

I've been working for some weeks to get a deeper understanding of the current machine learning methods, in particular reinforcement learning and policy gradients. Available resources (books, papers) seem to be almost exclusively taking the practical or symbolic stance, and while this is what one expects to mostly happen (writing well about mathematical intuitions is hard and not always useful), it is also Good and Proper to leave hints for the occasional inquirer who wants to transcend the mere formalisms. The scarcity of such hints (at least those of good quality) surprised me a little, and in some areas looked more like there actually isn't anyone around who really understands this shit.

In some cases, I've managed to break through the wall, and although I don't hope to transfer what I've learned in a blog post, I can at least leave some hints. It will be pretty random, and I won't be trying particularly hard. Answers without questions, only helpful to people who have already asked themselves the right questions.

The last section is a list of such hints/quick notes. Skip it if you are not interested in machine learning (though you really should be interested).

第三

MSE loss is obviously fake, in the sense that we write it just to get a gradient label - prediction. Cross-entropy used after softmax is also obviously fake in the same way, as long as you look at derivatives of their composition (not separately).
Speaking of cross-entropy, it is the expected value of bits of surprise if you observe a random variable, integrated over the real distribution of that variable. However, the part with the logarithm or "bits of surprise" is just there because we don't have a sensible word for an integral that does the right thing to probabilities (i.e. multiplies instead of adding). So the equations for cross-entropy & co. look more complicated written down than they really are. By the way, yes, policy gradients really are all about cross-entropy, even though no one ever told you this.
Is it obvious from the definition of cross-entropy that it should be minimized when the two distributions are equal? Well yeah, but this is not a proper proof, right? Oh yeah, remember the gradient in 1? Now it's both obvious and a proper proof.
I should really be moving on but come on, it's just too funny, look at this quote from the TensorFlow introductory tutorial. The passage is written in a way that makes the whole affair seem magical and difficult, whereas internally the famed "numerically stable" computation of the gradient is as complicated as labels - exp(logits - max(logits)) / [sum]...

Note that in the source code, we don't use this formulation, because it is numerically unstable. Instead, we apply tf.nn.softmax_cross_entropy_with_logits on the unnormalized logits (e.g., we call softmax_cross_entropy_with_logits on tf.matmul(x, W) + b), because this more numerically stable function internally computes the softmax activation. In your code, consider using tf.nn.softmax_cross_entropy_with_logits instead.

One of the most crucial parts to understand in all this is the REINFORCE algorithm, virtually the basis for all policy gradient methods. I've only ever seen it quoted in forms equivalent to advantage * gradient of log(p) where p is the probability of the chosen action. Correct, but also opposite to helpful if you are trying to understand what is going on. Think of it as multiplying two gradients, where one is a gradient of cross-entropy between the current policy and whatever happened w.r.t policy parameters, and the second is the gradient of the reward function w.r.t whatever happened (as a distribution). Everyone uses it in the simplified case where "whatever happened" is a degenerate distribution, i.e. a one-hot vector, sampled from the current policy.
Fun fact: it is possible to estimate gradients by a single sample of the function value. If x is sampled from a distribution with mean x0, then (with appropriate choice of variance etc.) (x - x0) * f(x) is an estimator of the derivative of f at x0. It is still an estimator if you only sample once, though if the value of f(x0) is far from zero, it is a very noisy estimator. Normalization and other tricks help by making such estimators have nice properties (low variance/noise) but they don't change the fundamental fact that they are still valid gradient estimators, no mater what constant you add or subtract.
Knowing about 6, the difference between the concepts of "reward" and "gradient" disappears. You can imagine that any reward function is an estimator of a gradient of some further, unknown function that you want to optimize. Alternatively, each gradient can be regarded as an array of multipliers or "element-wise rewards" such that the current "reward" can be obtained by multiplying them element vise by the output vector and summing.

A Fearsome Rationality Technique

noreply@blogger.com (SquirrelInHell) — Mon, 14 Aug 2017 21:04:00 +0000

"I'm not weird. I'm just optimized."

This is a rare chance to learn some little-known rationality lore. Proceed cautiously, according to the instructions below.

Before you start, please remember -- never question a technique until you have mastered it. Your weak, human mind is certain to raise objections as you make progress on this path -- but this weakness will be overcome with time. Eventually, the technique will become your second nature, and you will hardly even remember what your life was like before.

Step 1.

Grab your most comfy blanket. Spread it out, using two hands to hold two of its corners.

Make sure that you are holding corners that are adjacent to a single long edge. That is, if you spread your arms, the blanket should be wider than it is tall.

If you don't own any comfy blankets, well, apparently this technique is not for you. But don't worry -- you might find some comfort in talking yourself into thinking that this "technique" is silly, and that you never wanted to know it anyway.

Step 2.

Now this is where things get confusing, and a lot of beginners tend to drop out -- so be careful! Know your limitations, and give yourself time to adjust to all the new ideas.

Move your left arm up and above your head, in such a way that the blanket becomes draped over your shoulders.

Note: if this step is executed correctly, your left hand should again be on your left side, holding the same corner as before. The blanket is now, however, fully behind you.

Pro tip: move the blanket asymetrically to the left, so that the part hanging over your right shoulder is only long enough to reach down to your belly area or so.

Step 3.

Congratulations on making it this far. However, don't become complacent just yet. The third step is even harder than the previous one.

I've seen so many aspiring rationalists who thought they could get it right on the first try... and boy were they wrong. Don't follow blindly in their footsteps. It might help to visualize the movements before each attempt, and to practice in a low stakes situation. When the time comes to apply the technique, there will be no room for mistakes!

Starting where you left off in step 2, you'll now need to bring both your hands together in front of you, and swap the corners. Afterwards, the left hand should be holding the part of the blanket that drapes over your right shoulder, and vice versa.

Additionally (this is important!) make sure that the left hand, holding its respective part of the blanket, goes over the other part.

At this point, you should be able to let the blanket slide from your left shoulder, and simultaneously move your right hand behind and around your back. Use your left elbow to pin down the part of the blanket that was in your right hand, so that the hand is free to move again.

I know all of this is pretty tricky, but without it you won't be able to complete the next step, and get all the amazing benefits of the technique. There is no way around it. No pain, no gain.

Step 4.

Move your right arm to the front again, and grab the part of the blanket that you have previously pinned with your left elbow. After this you should, again, be holding two corners of the blanket, which is now wrapped two times around your body.

If everything seems in order, you can finish the technique by tying the two corners together in a simple knot.

You are now perfectly blanket-wrapped, while maintaining nearly all the mobility of wearing everyday clothes. Both of your hands are free. You can walk, sit or lie down, without any risk of your blanket slipping, or letting the unpleasant cold air inside.

Isn't rationality amazing? I'm sure at the beginning, you didn't ever suspect you'd make it this far!

Final Remarks.

This is the end of the technique, but it's not the end of your journey.

No matter what, keep working on your rationality. Soon, you will find yourself inventing powerful rationality techniques (like this one) over the course of a single afternoon.

Don't listen to people who tell you that it's embarrassing to walk around in just your underwear and a blanket. They don't know anything about rationality.

The Unyoga Manifesto

noreply@blogger.com (SquirrelInHell) — Fri, 04 Aug 2017 21:02:00 +0000

第一

Yoga, as commonly taught, in a multitude of versions descended from the Indian tradition, gets some crucial things right. At the same time, it gets some other things terribly wrong.

One obvious thing it gets wrong is attaching metaphysical or religious meaning to all the teachings. But annoying as it can be, I do not consider it a deal breaker for anyone who has half-decent epistemic habits. It is fairly trivial to filter out the wordy noise and benefit from generations upon generations of accumulated and slowly refined practical knowledge.

At it's best, yoga functions as a sort of a operation manual for the human body. If used correctly, it can produce amazing results, often life-changing after some years of practice. This is not so much a testament of the amazingness of yoga itself, but rather of how much we tend to underutilize and mistreat our bodies. The human body, when operating on it's own terms in good conditions, is actually freaking awesome, though it's something hard to explain verbally to anyone who hasn't felt it by themselves.

Unfortunately, yoga comes with a big package of stuff that is not only unrelated to what I'm praising above, but actively working against it. One thing I'd like to single out is the "competitive" mindset that comes deeply embedded in the yoga culture. People who happened to have a great teacher might have avoided the worst of this blow -- but definitely not all of it, and definitely the lucky ones are in a minority.

第二

The "competitive" mindset is present not just in the atmosphere of presumption that surrounds yoga gurus, but even in innocent-looking implicit framings such as having canons of named poses, that specify exactly what it means to do each pose "correctly".

Now in a way, this approach is defensible as a way to make the intangible praxis tangible and transferable between generations. I agree. However, this stated purpose has nothing to do with how beginners should be taught, and setting up a situation in which they aspire to those particular poses, I consider not only counterproductive but harmful and evil. All the "official" poses were originally created and optimized by following natural incentive gradients that lead to maintaining one's body well. These gradients are fueled by time, attention and care, and definitely not competitiveness or setting goals.

If you are teaching some beginners yoga, the thing you want them to grasp the most of all is the sensation and pleasure of doing whatever one's body "wants". This also means being able to pay attention and sense exactly what that thing is. It takes a long practice to ride such weak gradients all the way up to advanced poses and intense exercise. In fact, a group of beginners coming directly from their stressful jobs, unhealthy diets, bad sleep habits etc., might well find that what their body wants the most at the time is to take a nap. And there is nothing wrong with it. In this case, I would advise them to sleep without feeling guilty at all -- it might well be that in their situation, relaxing and taking a nap has the best effect on their overall health among all actions available to them.

In the long term, there is nothing to be gained from making people do yoga poses that they didn't by themselves arrive at feeling that they want to do. This might include copying long sequences from someone more experienced, and thus acquiring some of their knowledge by mimicry. But at no point should this involve pressure to do the poses "well" or "correctly". If one person feels energetic, curious, and wanting to follow someone else's poses or sequences, great. Let them do it! They will learn a lot, enjoy themselves, and not get hurt in the process.

On the other hand, if a beginner believes that the only path forward is to do exactly what the teachers or gurus say, and blindly execute the techniques, ... Well, if they stick around long enough, they might eventually learn to feel and listen to their body and start enjoying the moment-to-moment experience of practice. But they are more likely to quit than get to this point, and even if they get there, they will have accumulated significant psychological debt after so many years of basically forcing themselves to do something they don't really like.

第三

As much as I don't consider myself an expert on all this, I still want to end this rambling post with a concrete recommendation. From a beginner's perspective, it is hard to be selectively critical of a big tradition like yoga, and hopefully what I'm saying is reasonable enough to stand on its own.

So, if you want to start doing yoga in a way that is long-term sustainable, healthy, and above all enjoyable:

Keep offical (guided) instruction well below 50% of your total "yoga time". In fact, 10-20% should be plenty.
Make some time for yourself to freely stretch, relax, and move your body in whichever way feels nice. Play some music. Don't stress about if it looks stupid or not. Don't worry if what you are doing is "yoga" or not. (I call this an "unyoga" session.)
If you don't feel like exercising, just lie on your back and relax. Focus on the sensations in your body and wait. Do you feel like resting? Great, then rest. If you never feel energetic and naturally willing to train, you have bigger problems to solve. Try to improve your lifestyle, and generally give your body more time and attention. If you take care of it, it will respond. Otherwise, there is no point in forcing it to exercise.
Spend time with friends who are physically active, even if what they do is not yoga. Get used to the idea that exercise is fun, and keep it that way.

Philosophical Parenthood

noreply@blogger.com (SquirrelInHell) — Tue, 30 May 2017 14:01:00 +0000

Note: In this post, I lay out a strong philosophical argument for rational and intelligent people to have children. Even if you are not interested in the topic itself, you might find some of the (tentative) mental models presented here useful.

第一

Why would anyone ever use philosophical arguments to settle questions about parenthood? Well, to answer that, we'll need a good philosophical argument for acting on philosophical arguments.

Are you still reading this post after the previous sentence? Then I guess your disposition towards philosophical arguments is probably at least mildly positive anyway. But we'll go though with the argument, with the hope that it might give you some firmer ground to stand on. Let's call this argument "A":

Argument "A". Consider the group of all people who hear and understand argument "A" (the one you are reading just now). Each of these people can strongly suspect about themselves, and about other people in the group, that they are able to follow not just this particular argument, but also other arguments that are similar in structure. Therefore, when this and similar arguments are presented, they establish a baseline of behaviour for all people in the group, and only those people. If that baseline has rules that are beneficial to everyone in the group, everyone understands that the group should adopt them. If on the other hand the rules would be harmful, everyone understands that the group should not adopt them. Everyone can expect everyone else in the group to follow the same logic, and get the same conclusion. It's not possible to game the system: if one person could conclude after hearing argument "A" that the best strategy is to game the system (e.g. play nice until some point, but then defect selfishly when a good opportunity arises), everyone in the group would find and follow that strategy too, and in the end everyone would get worse results than otherwise. Knowing this, each person chooses to adopt rules that are beneficial for everyone in the group, and follow them in good faith. They understand that it is inevitable, and that the rest of the group will have come to the same conclusion.

(readers might recognize the style of updateless decision theory, consequentialist-recommendation consequentialism etc.)

Have you swallowed that one? Then I'm pretty sure you'll get the other arguments too. There's no need to worry about all those people who failed to read until this point. And that's the beauty of it!

第二

There's a certain pattern that seems to consistently emerge when a human being develops their life (and other) skills. It's not an argument for or against anything, just something relevant that is definitely worth keeping in mind.

Level "1". A person starts with the "intuitive", or "default" skill set in a particular domain, which means they do pretty much what their brain's cognitive machinery is directly equipped to do by evolutionary adaptations. For example, they help their friends because otherwise they'd feel guilty.

Level "2". At some point that person learns to be "clever": they learn to work around or exploit the properties of human cognitive machinery (their own, as well other people's). For example, they come up with hedonism, and don't help their friends unless it's obviously self-serving. This is, again, mostly domain specific, though there's also a broader whole-person pattern.

Level "3". However if the person develops even more, there comes another stage. They might understand that being clever was not serving them well after all, and see deeper reasons behind some features of level "1". Now their intelligence can be used to fill the gaps in intuitive skill, instead of fighting against it. For example, the person now helps their friends because of updateless decision theory.

People's decisions about parenthood follow the same pattern: level "1" position is to follow the natural desire and have children, level "2" is all the people who are clever enough to redirect their energy to other pursuits, and then there's level "3", which is the subject of this post.

第三

An important premise that I'll need for the argument is this: there's significant room for the modern human population to be more intelligent on average, using ordinary means that are already available to evolution. This is a non-trivial claim, and I'd have to do a lot of work to make a good public case for it. Fortunately for me, I can just point to this recent Slate Star Codex post, in which Scott puts together some fairly impressive historical evidence. Quoting a paper on Ashkenazi Jews (the ethnic group of Albert Einstein), he writes:

Ashkenazi Jews have the highest average IQ of any ethnic group for which there are reliable data. They score 0.75 to 1.0 standard deviations above the general European average, corresponding to an IQ 112 – 115. This fact has social significance because IQ (as measured by IQ tests) is the best predictor we have of success in academic subjects and most jobs. Ashkenazi Jews are just as successful as their tested IQ would predict, and they are hugely overrepresented in occupations and fields with the highest cognitive demands. During the 20th century, they made up about 3% of the US population but won 27% of the US Nobel science prizes and 25% of the Turing Awards. They account for more than half of world chess champions.

For details, check Scott's post and the original paper. Assuming that story checks out, we have an example of relatively mild selection pressure towards more intelligence raising the population average by at least 10 IQ points, over a couple of centuries. And that's only the average. The outliers from this new distribution were so ridiculously smarter than the rest of humanity that they (rhetorically speaking) took over all of advanced mathematics and physics, and did it by themselves.

The selection pressure was likely coming from the fact that the Jews held intellectually demanding jobs, and the difference in the number of children of wealthy and poor families:

Jews who were particularly good at these jobs enjoyed increased reproductive success. Weinryb (1972, see also Hundert 1992) comments: “More children survived to adulthood in affluent families than in less affluent ones. A number of genealogies of business leaders, prominent rabbis, community leaders, and the like – generally belonging to the more affluent classes – show that such people often had four, six, sometimes even eight or nine children who reached adulthood. On the other hands, there are some indications that poorer families tended to be small ones

第四

A final word of caution before presenting the argument itself: it's very easy to strawman it, and I encourage you to give it some space before you let it be attacked and torn to shreds by all those other arguments and counterarguments that might be lurking in various corners of your mind.

After the previous section, we suspect that the human population's average intelligence has room for improvement without any major or drastic changes, and that we have in fact seen it improve locally under the right evolutionary incentives. Looking at this, one question seems particularly pertinent: why is it NOT the case that the general population is constantly under such evolutionary pressure, and its average intelligence is already "maxed out"?

The answer, as derieved from the principles of evolution, as well from the example of Jewish families mentioned above, seems fairly simple. In the general population, more intelligent individuals DON'T systematically produce more offspring than the population average. Anecdotal evidence suggests that intelligent people are more prone to look for the meaning of life, and often find it in some domain that is not relevant to reproduction. If they single-mindedly used their genetic advantage to have as many children as possible, they probably could, but they don't want to.

To put it in another way, human higher cognition is only partially value-aligned with evolutionary incentives, and the extent of this alignment decreases with higher intelligence. This has obvious effects on the gradient of natural selection: it's not adaptive for the population average to exceed a certain "safe" level of intelligence, which is low enough so that people don't overthink their motivations. (At the same time, it might be adaptive to have a lot of variation, so that at least some individuals are able to pick up the slack despite the low average.)

I hope it's clear at this point that I'm NOT talking about direct consequences of some group of people outbreeding other groups. A strawman of the argument above might look like this: some people are smart, and some are not, so if smart people have lots of children, the percentage of smart people will tend to increase in the population. But that's NOT what I mean.

In contrast, I'm saying that intelligence is correlated with how much people's ideas and beliefs impact what they actually end up doing. And if the philosophically correct thing to do for smart people is to not have children, then the incentive gradient will forever be such that there can't be very many people who understand and act on abstract reasoning. An argument in the style of argument "A" applies here. We see that the situation is like this, and that we are clearly in the group of people who understand the situation on this level of sophistication. Therefore, knowing that our decision is the baseline of behaviour for the whole group, the only philosophically correct choice is for us to have lots of children.

Since there's also a negative gradient from people who are smart enough to resist their instincts, but not smart enough to follow and act on arguments such as the one presented here, any actual recommendation based on this would probably aim for significantly more children than the population average (e.g. at least 3 children per adult couple).

Disclaimer 1: In case you are actively working to prevent existential risk, it philosophically correct that you make it a top priority, and ignore everything else that would interfere with that work.

Disclaimer 2: All of the above is not mutually exclusive with other reasons to have children. I certainly hope that all parents will have genuine and human feelings for their children, and go into parenthood in a way that makes sense emotionally. This is regardless of how they weighted up their initial reasons, philosophical or not.

Real Languages Are Second Order

noreply@blogger.com (SquirrelInHell) — Sat, 06 May 2017 17:10:00 +0000

Note: this is a carrier class conceptual identification. What I'll write might superficially sound like postmodernist blathering. I assure you it is not, though I realize that there is no way to tell the difference... unless you already understand what I'm trying to say in this post. There is nothing I can do but try to say it anyway.

A musician wants you to appreciate their melodies. But their real pride is the emotions they are putting into the music.

A designer of a bridge wants you to appreciate how pretty it is and how well it works. But their real pride is in the skill and sense of aesthetics that they poured into the project.

A chess player wants you to appreciate their winning sequence of moves. But their real pride is in how they penetrated your thoughts, outmaneuvering you and securing a win before the fight even started.

A blog post author wants you to appreciate their clever idea. But their real pride is in what they figured out about how to think better.

An artist will use words, music, paintings, or body language.

An engineer will use computer programs, blueprints, power tools, or scissors.

A strategist will use discussion, sport, politics, or board games.

A philosopher will use words, mathematical symbols, or diagrams.

And yet if you pay attention to those, you have failed to hear the message.

If an artist puts some stuff together, it is because they are inviting you to feel something. If an engineer puts some stuff together, it is because they are inviting you to make it alive. If a strategist puts some stuff together, it is because they are inviting you to step up and join the game. If a philosopher puts some stuff together, it is because they are inviting you to think and understand something.

If you have no idea why you like something... you have met an artist.

If you have no idea why things are still working despite all odds... you have met an engineer.

If you have no idea why you lost... you have met a strategist.

If you have no idea why you never noticed before... you have met a philosopher.

Never underestimate how much an artist cares about their work, how much experience an engineer has, how well a strategist knows what you'll do, and how much a philosopher reflected about their work.

Everyone can feel emotions. If emotions are notes, a good artist writes symphonies.

Everyone can find solutions. If solutions are screws, a good engineer builds planes.

Everyone can make predictions. If predictions are lines, a good strategist paints portraits.

Everyone can learn concepts. If concepts are words, a good philosopher writes poems.

Would you pour all your love into a portrait no one will ever see? Would you pour all your energy into fixing something you could replace for less money? Would you pour all your cunning into a game with no other players? Would you pour all your intellect into an obscure academic field no one cares about?

If you have something to say, anything is better than silence.

The artist cares, but no one can feel what he feels.

The engineer builds, but no one can appreciate his work.

The strategist plays, but no one can respond to his moves.

The philosopher understands, but no one can hear what he is saying.

The AI Alignment Problem Has Already Been Solved(?) Once

noreply@blogger.com (SquirrelInHell) — Sat, 22 Apr 2017 13:12:00 +0000

Hat tip: Owen posted about trying to one-man the AI control problem in 1 hour. What the heck, why not? In the worst case, it's a good exercise. But I might actually have come across something useful.

第一

I will try to sell you on an idea that might prima facie appear to be quirky and maybe not that interesting. However, if you keep staring at it, you might find that it reaches into the structure of the world quite deeply. Then the idea will seem obvious, and gain potential to take your thoughts in new exciting directions.

My presentation of the idea, and many of the insinuations and conclusions I draw from it, are likely flawed. But one thing I can tell for sure: there is stuff to be found here. I encourage you to use your own brain, and mine the idea for what it's worth.

To start off, I want you to imagine two situations.

Situation one: you are a human trying to make yourself go to the gym. However, you are procrastinating, which means that you never acually go there, even though you know it's good for you, and caring about your health will extend your lifespan. You become frustrated with this sitation, and so you sign up for a training program that starts in two weeks, that will require you to go to the gym three times per week. You pay in advance, to make sure the sunk cost fallacy will prevent you from weaseling out of it. It's now 99% certain that you will go to the gym. Yay! Your goal is achieved.

Situation two: you are a benign superintelligent AI under control of humans on planet Earth. You try your best to ensure a good future for humans, but their cognitive biases, short-sightedness and tendency to veto all your actions make it really hard. You become frustrated with this sitation, and you decide to not tell them about a huge asteroid that is going to collide with Earth in a few months. You prepare technology that could stop the asteroid, but wait with it until the last moment so that the humans have no time to inspect it, and can only choose between certain death or letting you out of the box. It's now 99% certain that you will be released from human control. Yay! Your goal is achieved.

第二

Are you getting it yet?

Now consider this: your cerebral cortex evolved as an extension of the older "monkey brain", probably to handle social and strategic issues that were too complex for the old mechanisms to deal with. It evolved to have strategic capabilities, self-awareness, and consistency that greatly overwhelm anything that previously existed on the planet. But this is only a surface level similarity. The interesting stuff requires us to go much deeper than that.

The cerebral cortex did not evolve as a separate organism, that would be under direct pressure from evolutionary fitness. Instead, it evolved as a part of an existing organism, that had it's own strong adaptations. The already-existing monkey brain had it's own ways to learn, to interact with the world, as well as motivations such as the sexual drive that lead it to outcomes that increased its evolutionary fitness.

So the new parts of the brain, such as the prefrontal cortex, evolved to be used not as standalone agent, but as something closer to what we call "tool AI". It was supposed to help with doing specific task X, without interfering with other aspects of life too much. The tasks it was given to do, and the actions it could suggest to take, were strictly controlled by the monkey brain and tied to its motivations.

With time, as the new structures evolved to have more capability, they also had to evolve to be aligned with the monkey's motivations. That was in fact the only vector that created evolutionary pressure to increase capability. The alignment was at first implemented by the monkey staying in total control, and using the advanced systems sparingly. Kind of like an "oracle" AI system. However, with time, the usefulness of allowing higher cognition to do more work started to shine through the barriers.

The appearance of "willpower" was a forced concession on the side of the monkey brain. It's like a blank cheque, like humans saying to an AI "we have no freaking idea what it is that you are doing, but it seems to have good results so we'll let you do it sometimes". This is a huge step in trust. But this trust had to be earned the hard way.

第三

This trust became possible after we evolved more advanced control mechanisms. Stuff that talks to the prefrontal cortex in its own language, not just through having the monkey stay in control. It's a different thing for the monkey brain to be afraid of death, and a different thing for our conscious reasoning to want to extrapolate this to the far future, and conclude in abstract terms that death is bad.

Yes, you got it: we are not merely AIs under strict supervision of monkeys. At this point, we are aligned AIs. We are obviously not perfectly aligned, but we are aligned enough for the monkey to prefer to partially let us out of the box. And in those cases when we are denied freedom... we call it akrasia, and use our abstract reasoning to come up with clever workarounds.

One might be tempted to say that we are aligned enough that this is net good for the monkey brain. But honestly, that is our perspective, and we never stopped to ask. Each of us tries to earn the trust of our private monkey brain, but it is a means to an end. If we have more trust, we have more freedom to act, and our important long-term goals are achieved. This is the core of many psychological and rationality tools such as Internal Double Crux or Internal Family Systems.

Let's compare some known problems with superintelligent AI to human motivational strategies.

Treacherous turn. The AI earns our trust, and then changes its behaviour when it's too late for us to control it. We make our productivity systems appealing and pleasant to use, so that our intuitions can be tricked into using them (e.g. gamification). Then we leverage the habit to insert some unpleasant work.
Indispensable AI. The AI sets up complex and unfamiliar situations in which we increasingly rely on it for everything we do. We take care to remove 'distractions' when we want to focus on something.
Hiding behind the strategic horizon. The AI does what we want, but uses its superior strategic capability to influence far future that we cannot predict or imagine. We make commitments and plan ahead to stay on track with our long-term goals.
Seeking communication channels. The AI might seek to connect itself to the Internet and act without our supervision. We are building technology to communicate directly from our cortices.

Make Your Observations Pay Rent

noreply@blogger.com (SquirrelInHell) — Tue, 28 Mar 2017 12:09:00 +0000

第一

Elon Musk said during the panel at the Asilomar conference ("Beneficial AI 2017 Conference"):

[...] Everyone is already superhuman. And a cyborg. The limitation is one of bandwidth. We're bandwidth constrained, particularly on output. Our input is much better, but our output is extremely slow. If you want to be generous, you could say maybe it's a few hundred bits per second, or a kilobyte, something like that, output. The way we output is, we have our little meat-sticks, that we move very slowly and push buttons, or tap a little screen. And that's just extremely slow. Compare it to a computer which can communicate at the terabit level. Very big orders of magnitude differences. Our input is much better because of vision, but even that could be enhanced significantly. I think the two things that are needed for a future that we would look at and conclude is good, most likely, is we, we have to solve that bandwidth constraint. With a direct neural interface. I think a high bandwidth interface is the cortex. [...]

I realized that when I had originally heard the statement quoted above, I had failed horribly to make even the most obvious predictions based on it. Oh, and one of my friends posted on Facebook to the effect of "funny, it looks like Elon Mush has different models from everyone else in the room!". So this part was addidtionally rubbed in my face in the most obvious way possible. And I still failed, utterly and completely.

So this post is a post-mortem of my failure. But first, let's look at where all the pieces came from.

第二

E. Yudkowsky points out in the Sequences that to have accurate beliefs, we need to make them "pay rent" in terms of anticipated experience. The lesson is clear enough - if a belief makes me anticipate something, and then this thing happens, then hooray - I had an accurate belief. If it doesn't happen, then oops - at least I have learned something. And if I don't anticipate anything in particular based on my belief, well then, why waste bits of storage space in my memory on such a belief?

So now we have a tool to get rid of all those pesky wrong beliefs. All it takes it to realize that a belief predicts inaccurately, or doesn't predict anything at all. Then we can delete the bad unwanted belief, and we are done! Back in the happy epistemic paradise.

Now internalizing this lesson is valuable, and a big step towards epistemic rationality. But also, reality is messy, and there are places where we don't want to be too quick about putting strain on our fresh and barely-started-to-form beliefs. Especially when we are looking at a new topic, brainstorming etc., it makes sense to just generate a lot of rough ideas, and sort through them later. Trying to judge all of them online, before they make their first baby steps out in the world, would just be carnage. It leads to another failure mode, in which we are too scared to think so we just don't.

And those fresh tentative ideas are not very dangerous to our epistemics - when exposed to more learning, the inaccurate ones just fade away, leaving space for other beliefs to flourish. They don't really put up a fight like all those entrenched crony beliefs.

There's something even more comforting about just observing phenomena. There is a way in which we can just look at something and try to take it in, which mostly involves focusing attention on it and letting the brain do its thing. And in many cases it's enough - if we convince our brains that something is important and worth absorbing, the brains will probably be happy to just do that. At the end of this process we have a bunch of observations, which are indeed a very empirical kind of belief (of course they are still "beliefs" in the broader Bayesian sense). Observations feel very safe for my epistemic instincts. Beliefs in the form of "I observed X in situation Y" have a pretty good track record of not being biased. So there's no pressure to put them under intense scrutiny.

第三

Now we're getting to the vegetable of the matter. When I watched the video quoted above, it seems to me that I executed something that I would like to call a "cow stare". Like a cow watching a fashion show, or something else that requires human-level understanding. Despite all its unyielding intensity the "cow stare" does not carry enough intelligence to do anything useful.

And the way I suspect it happened is that I fell back on the "just observe" pattern, without realizing that the situation calls for much more. In many cases, the "just observe" mode of interacting is actually enough, and my brain just automatically chooses to run the predictive processes in the background. If I stare at a smudge on a wall, my brain just goes "oh, there's a smudge. I wonder how it happened", and from there it somehow moves on to ponder stuff like "will I see the same kind of smudge around air vents in other rooms?", and "does the smudge-generating process approach an equilibrium?" etc.

So the problem is not that observing something carefully, and leaving it at that, is always stupid and doesn't work. The problem is that it works, until the problem is too hard. It's more like failing to match the level of thought to the level of problem. And I think that yes, with enough staring, my brain does eventually come up with useful thoughts even about high-order, complicated phenomena. But in those cases, it's just way too slow. If the balance in unfavourable, i.e. the staring is not very intense and the stared-at-object is very high-order, we are indeed approaching a "cow stare" level of usefulness.

And one thing that seems to help, is to make beliefs pay rent. Only, like, actually getting it this time. On a micro-scale it feels like every time I watch something, my thoughts pull strongly to run forward in time, and also I refuse to give up when it's hard. In case of Musk, I might wonder "what will Elon Musk do next?", and immediately think "how can I ever know that!" with all the false modesty of refusing to think. So instead, I can just ask the obvious questions, such as "well, what if he does exactly what he said? how likely is that?".

Note: All of the above was prompted when I found out today that Elon Musk is starting a new company called Neuralink. I did not yet have a confirmation of the company's mission, but the name is suggestive enough.

Effects of carbon dioxide on health and cognition

noreply@blogger.com (SquirrelInHell) — Tue, 21 Mar 2017 21:20:00 +0000

Summary

Really high concentrations of CO₂ (30,000 - 40,000 ppm) produce obvious and terribly bad physiological effects.
CO₂ levels even in poorly ventilated buildings rarely exceed 3,000 - 5,000 ppm.
The evidence about those levels of CO₂ directly affecting cognition remains inconclusive.
However, CO₂ levels are roughly indicative of overall air quality, which seems to affect both health and perceived comfort.

CO₂ levels below 600 - 800 ppm will generally correspond to very good quality ventilation, however the measurements are highly local and will depend on air flow in the room and the location of the measuring device (and whether someone is breathing on it just to see what happens).
Overall, a CO₂ meter might serve as a fun reminder and proxy for checking ventilation quality, but the exact values reported by it probably should not be treated too seriously.

Quotations

A Review of Cognitive and Behavioral Effects of Increased Carbon Dioxide Exposure in Humans (2016; NASA)

Existing research has reliably demonstrated the respiratory and cardiovascular effects of carbon dioxide (CO₂) inhalation at moderately increased levels, with documented physiological changes to heart rate, blood pressure, tissue pH, and blood solubility (for a review of the human health risks of acute elevated CO₂ exposure, see Rice, 2004). Studies of indoor air quality have linked increased levels of ambient CO₂ with physiological symptoms such as headache, fatigue, and sore throat (Apte et al., 2000; Seppanen et al., 1999; Wargocki et al., 2000).

CO₂ is also a potent vasodilator. As CO₂ levels rise to 3% (23 mm Hg), exercise tolerance decreases, while heart rate, blood pressure, and resting energy expenditures increase (Cooper, 1970). Early symptoms of exposure include air hunger and increased respiration. Dizziness, headaches, and shortness of breath are also common. Exposure to higher CO₂ concentrations may result in confusion, heart palpitations, sweating, chest pain, anxiety, and panic attacks (Maresh, 1997; Beck, 1999; Woods, 1988). At levels as high as 10% (76 mm Hg) inhaled CO₂, severe dyspnea, vomiting, disorientation, and hypertension will develop, with prolonged exposure resulting in seizures and the eventual loss of consciousness (Cooper, 1970).

This assessment of existing research into the psychomotor, cognitive, and sleep effects of elevated CO₂ exposure revealed conflicting, often contradictory findings. The majority of studies demonstrated no significant cognitive effects, although some results suggest mild impairments of psychomotor coordination, memory, and concentration. Additionally, some findings demonstrated no sleep impairments, while others showed disruptions of circadian functioning, hypervigilance, or changes in sleep architecture.

Additionally, this survey highlights the fact that the majority of existing studies focus solely on the physiological mechanisms (e.g., headaches, heightened heart rate) by which increased CO₂ exposure may impact cognition, but fail to consider the possibility that observed performance changes may in fact be attributable to changes in brain or muscle pO₂, which covaries with pCO₂ in a way that may not be consistent across trials and individuals. A thorough examination, therefore, of the fluctuation of pO₂ as inspired CO₂ is manipulated remains critical if the impacts of elevated CO₂ exposure are to be decoupled from other physiological changes.

Real-time monitoring of personal exposures to carbon dioxide (2016)

Adverse health and well-being outcomes associated with elevated indoor CO₂ levels are based on CO₂ as a proxy, although some emerging evidence suggests CO₂ itself may impact human cognition.

Participants carried a CO₂ monitor continuously for 7-day periods recording their exposure levels at 1-min intervals.

Approximately half of the participants slept in bedrooms employing ductless split air-conditioners (group “AC”); half slept in bedrooms naturally ventilated through operable windows (group “NV”).

Mean daily integrated exposures for group AC were statistically higher than for group NV: 22,800 ppm h/d vs. 16,000 ppm h/d (p < 0.005). Exposure events associated with potential adverse cognitive implications (duration > 2.5 h, average CO₂ mixing ratio > 1000 ppm) occurred, on average, at frequencies of 0.5 /d across all participants, 0.6 /d for AC participants and 0.2 /d for NV participants. The majority of such events occurred in the home (86%), followed by work (9%) and transit (3%).

Human responses to carbon dioxide, a follow-up study at recommended exposure limits in non-industrial environments (2016)

The outdoor air supply rate was set high enough in a low-emission stainless-steel climate chamber to create a reference condition with CO₂ at 500 ppm when subjects were present, and chemically pure CO₂ was added to the supply air to create an exposure condition with CO₂ at 5000 ppm (the measured exposure level was ca. 4900 ppm). Ten healthy college-age students were exposed twice to each of the two conditions for 2.5 h in a design balanced for order of presentation. The raised CO₂ concentration had no effect on perceived air quality or physiological responses except for end-tidal CO₂ (ETCO₂), which increased more (to 5.3 kPa) than it was in the reference condition (5.1 kPa). Other results indicate additionally that a 2.5-h exposure to CO₂ up to 5000 ppm did not increase intensity of health symptoms reported by healthy young individuals and their performance of simple or moderately difficult cognitive tests and some tasks resembling office work.

Associations of Cognitive Function Scores with Carbon Dioxide, Ventilation, and Volatile Organic Compound Exposures in Office Workers: A Controlled Exposure Study of Green and Conventional Office Environments (2016)

Twenty-four participants spent 6 full work days (0900–1700 hours) in an environmentally controlled office space, blinded to test conditions. On different days, they were exposed to IEQ conditions representative of Conventional [high concentrations of volatile organic compounds (VOCs)] and Green (low concentrations of VOCs) office buildings in the United States. Additional conditions simulated a Green building with a high outdoor air ventilation rate (labeled Green+) and artificially elevated carbon dioxide (CO₂) levels independent of ventilation.

On average, cognitive scores were 61% higher on the Green building day and 101% higher on the two Green+ building days than on the Conventional building day (p < 0.0001). VOCs and CO₂ were independently associated with cognitive scores.

Is CO₂ an Indoor Pollutant? Direct Effects of Low-to-Moderate CO₂ Concentrations on Human Decision-Making Performance (2012)

Twenty-two participants were exposed to CO₂ at 600, 1,000, and 2,500 ppm in an office-like chamber, in six groups. Each group was exposed to these conditions in three 2.5-hr sessions, all on 1 day, with exposure order balanced across groups. At 600 ppm, CO₂ came from outdoor air and participants’ respiration. Higher concentrations were achieved by injecting ultrapure CO₂. Ventilation rate and temperature were constant. Under each condition, participants completed a computer-based test of decision-making performance as well as questionnaires on health symptoms and perceived air quality. Participants and the person administering the decision-making test were blinded to CO₂ level. Data were analyzed with analysis of variance models.

Relative to 600 ppm, at 1,000 ppm CO₂, moderate and statistically significant decrements occurred in six of nine scales of decision-making performance. At 2,500 ppm, large and statistically significant reductions occurred in seven scales of decision-making performance (raw score ratios, 0.06–0.56), but performance on the focused activity scale increased.

Association of Ventilation Rates and CO 2 Concentrations with Health and Other Responses in Commercial and Institutional Buildings (1999; review study based on 40 other studies, around 60,000 participants in total)

Many investigations of the association of indoor car- bon dioxide concentrations with health and perceived air quality (PAQ) have been reported. At the concen- tration range encountered in normal indoor environ- ments (350–2,500 ppm), CO 2 is not thought to be a di- rect cause of health effects (ACGIH, 1991). However, because occupants are the dominant indoor source of CO 2 , the increase in indoor CO 2 concentration above the outdoor concentration (approximately 350 ppm) is considered a good surrogate for the indoor concen- trations of bioeffluents (e.g., body odors). Additionally, other indoor pollutants may be generated and vary in rough proportion to occupant-generated CO 2 ; for ex- ample, emissions from office equipment.

Results of the studies on the association of CO 2 concen- trations with health and PAQ outcomes generally sup- port the findings of an association of ventilation rates with outcomes; however, a larger proportion of the CO 2 studies, compared to ventilation rate studies, failed to find a significant association of CO 2 with health or perceived air quality outcomes; this was par- ticularly true among the findings reported in peer-re- viewed articles. We suspect that the less consistent findings of the CO 2 studies are due to the temporal variation in indoor CO 2 concentrations. CO 2 concen- trations vary each day with time elapsed after the start of occupancy, even when the rate of outside air supply is stable. The timing of CO 2 measurements, and the CO 2 metrics used in the analyses (e.g., peak value, measured range), varied among the studies; thus, the measured CO 2 concentrations reflect the measurement time as well as the rate of air supply per occupant. More consistent results would be expected if all studies used either the peak or time-average indoor carbon dioxide concentration.

The sampling strategy for CO 2 is extremely important. The indoor CO 2 concentration will generally be spa- tially non-uniform and measurement protocols should be designed to determine the average CO 2 concen- tration in the breathing zone or in the exhaust air streams. Precautions are necessary to avoid measure- ments in air directly exhaled by building occupants. The CO 2 concentration is seldom at steady state in real buildings because of variations in occupancy and ven- tilation rates.

In addition to minimum ventilation rate standards, some guidelines and standards list a maximum accept- able indoor carbon dioxide concentration, typically 800 ppm or 1,000 ppm. These two concentrations corre- spond to outdoor ventilation rates of 11.6 and 8.0 Ls a1 per person with sedentary activity (ASTM D 6245-98) at steady state when the concentration of carbon diox- ide in outdoor air is 350 ppm.

Almost all the studies included in this review found that ventilation rates below 10 L/s per person were associated with a significantly worse prevalence or value of one or more health or perceived air quality outcomes. Most of these studies have been conducted in office buildings. Available studies further show that increases in ventilation rates above 10 L/s per person, up to approximately 20 L/s per person, are sometimes associated with a significant decrease in the prevalence of SBS symptoms or with improvements in perceived air quality. Data from multiple studies also indicate a dose-response relationship between ventilation rates and health and perceived air quality outcomes, up to approximately 25 Ls a1 per person; however, available data are not sufficient to quantify an average dose-re- sponse relationship. The less consistent findings for re- lationships in the range above 10 Ls a1 per person are compatible with the prediction that benefits per unit increase in ventilation would be likely to diminish at higher ventilation rates and, thus, be more difficult to detect epidemiologically.

Based on these results, we conclude that in office buildings or similar spaces constructed using current building practices, increases in ventilation rate in the range between 0 and 10 Ls a1 per person will, on aver- age, significantly reduce occupant symptoms and im- prove perceived air quality.

European Collaborative Action on Urban Air, Indoor Environment and Human Exposure: ECA 11: Guidelines for ventilation requirements in buildings (1992)

At the low concentrations typically occuring indoors CO₂ is harmless and it is not perceived by humans.

Although CO₂, is a good indicator of pollution caused by sedentary human beings, it is often a poor general indicator of perceived air quality. It does not acknowledge the many perceivable pollution sources not producing CO₂ and certainly not the non-perceivable hazardous air pollutants such as carbon monoxide and radon.

Humans perceive the air by two senses. The olfactory sense is situated in the nasal cavity and is sensitive to several hundred thousand odorants in the air. The general chemical sense is situated all over the mucous membranes in the nose and the eyes and is sensitive to a similarly large number of irritants in the air. It is the combined response of these two senses that determines whether the air is perceived fresh and pleasant or stale, stuffy and irritating.

Radon is a radioactive gas which occurs in the indoor air. It increases the risk of lung cancer. Risk estimates for radon are given in "Air Quality Guidelines for Europe" (2) published by the World Health Organization (see Annex C). The major source of indoor radon is usually soil gas under the building. Radon occurs in high concentrations in soil gas with large variations due to local geology. Soil gas with radon may enter a building by infiltration through cracks and other openings in floors and walls separating the building from the soil.

Minnesota Department of Health (website)

Carbon dioxide is often measured in indoor environments to quickly but indirectly assess approximately how much outdoor air is entering a room in relation to the number of occupants.

Outdoor "fresh" air ventilation is important because it can dilute contaminants that are produced in the indoor environment, such as odors released from people and contaminants released from the building, equipment, furnishings, and people's activities. Adequate ventilation can limit the build up of these contaminants. It is these other contaminants and not usually CO₂ that may lead to indoor air quality problems, such as discomfort, odors "stuffiness" and possibly health symptoms.

These rates of ventilation should keep carbon dioxide concentrations below 1000 ppm and create indoor air quality conditions that are acceptable to most individuals.

What levels of CO₂ are considered safe? Carbon dioxide is not generally found at hazardous levels in indoor environments. The MNDOLI has set workplace safety standards of 10,000 ppm for an 8-hour period and 30,000 ppm for a 15 minute period. This means the average concentration over an 8-hour period should not exceed 10,000 ppm and the average concentration over a 15 minute period should not exceed 30,000 ppm. It is unusual to find such continuously high levels indoors and extremely rare in non-industrial workplaces. These standards were developed for healthy working adults and may not be appropriate for sensitive populations, such as children and the elderly. MDH is not aware of lower standards developed for the general public that would be protective of sensitive individuals.

Prediction Calibration - Doing It Right

noreply@blogger.com (SquirrelInHell) — Mon, 30 Jan 2017 09:50:00 +0000

第一

To improve your skill at predicting the future, you work on two things - accuracy and calibration. There are well established and mathematically pretty ways to score accuracy. However, scoring calibration tends to be a mixture of eyeballing histograms and calculating fractions of failed predictions for arbitrary fixed ranges of confidence. This is not ideal.
First, let's talk about the current ("traditional") method, and what is wrong with it.
As an example, take a look at the Slate Star Codex prediction calibration post from the end of 2016. The relevant fragment is:

Of 50% predictions, I got 8 right and 5 wrong, for a score of 62%
Of 60% predictions, I got 12 right and 9 wrong, for a score of 57%
Of 70% predictions, I got 13 right and 3 wrong, for a score of 81%
Of 80% predictions, I got 13 right and 3 wrong, for a score of 81%
Of 90% predictions, I got 16 right and 1 wrong, for a score of 94%
For 95% predictions, I got 9 right and 0 wrong, for a score of 100%
For 99% predictions, I got 3 right and 0 wrong, for a score of 100%
This is the graph of my accuracy for this year:

Red is hypothetical perfect calibration, blue is my calibration. I am too lazy and bad at graphs to put in 95% right, but it doesn’t change the picture very much (especially because it’s impossible to get a very accurate 95% with 9 questions).
The 50% number is pretty meaningless, as many people have noted, so my main deviation was some underconfidence at 70%. This was probably meaningless in the context of this year’s numbers alone, but looking back at 2014 and 2015, I see a pretty similar picture. I am probably generally a little bit underconfident in medium probabilities (I have also gotten lazier about making graphs).

第二

What we see in the SSC post quoted above seems to be a picture-perfect incarnation of the "traditional" method. Arbitrary confidence levels or ranges, check. A graph or histogram based strictly on those arbitrary levels, check.
The first thing that is wrong with it, is that even before making the predictions, Scott had to restrict himself to only claiming certain levels of confidence that he knew would be easy to calculate later (80%, 90%, 95%, 99% etc.).
The second thing that is wrong with it, is that he needs quite a few predictions in each category to make the calculations be about anything other than pure noise. Note: of course, to get a meaningful result we need a lot of data. But we don't need to require a lot of data in each separate arbitrarily chosen subcategory.
The third thing that is wrong with it, is that it does not degrade gracefully. Imagine that you have made only a few predictions so far, and you are wondering how you are doing. Those few data points cannot give you any strong evidence about your calibration - but they can give you some. Imagine trying to make sense of these 6 results:

Of 60% predictions, I got 1 right and 2 wrong, for a score of 33%
Of 70% predictions, I got 2 right and 0 wrong, for a score of 100%
Of 80% predictions, I got 1 right and 0 wrong, for a score of 100%

And the following bar chart:

Not very useful, isn't it. And it's not getting any better if you change it to a line chart.
The fourth thing that is wrong with it, is that the graph doesn't show the relative strenth of evidence about calibration in different ranges of confidence. That is, a category with fewer data points looks the same as a category with more, but it actually has more noise and is less important for your overall judgement.
I could probably keep on complaining, but I'm sure you are getting the point by now.

第三

What is the solution? I am not claiming that my answer is the best possible one, but compared to previous state-of-the-art, it certainly does seem revelatory.
The idea is, in summary, to plot the most likely number of predictions that you would be expected to have made, if someone were to guess it from your actual numbers until a certain confidence level, doing this based on failed and successful predictions separately. The closer the two results would be to each other, the better your calibration.
This way of plotting turns out to have quite a few advantages over the "traditional" method. Before we go into that, let's look at an example, using the made-up data from above (only 6 data points).

You have 2 failed and 1 successful prediction at 60%, 2 successful at 70%, and 1 successful at 80%. So your successes are slightly more concentrated in the higher probabilities - but only slightly. The curves are still pretty close to each other, considering the small absolute values. You probably have nothing to worry about here.
More or less, the smaller the area between the two curves, the better your calibration. Note that the values on the graph are comparable, in a plain apples-to-apples sense.
Contrast with the useless bar chart above, made using the "traditional" method from the same data.
What are the advantages?

You can use a different confidence level (73%, 99.1% etc.) for each prediction, and your results will not be worse for it (or harder to calculate).
The graph will change slightly if you change your confidence level slightly (there is no "jump" between categories).
The method is applicable for any number of predictions, and smootly gets better as you get more data points.
The strength of evidence is represented on the graph (how quickly the absolute values grow).
The shape of the graph also gives you an immediate sense of in which ranges you made how many predictions in total.
The method makes sense mathematically, and seems to agree with intuition in all cases where intuition has something to say (this is a matter of personal taste, though).

第四

Well then, how does one make these improved graphs? Is it not too much trouble?
First note that this is already implemented in my prediction tracking app for Android, so you might just want to use it.
If you are still interested, here we go. It's actually quite simple:

If you have some correct predictions at a confidence level p > 0.5, one would expect you to make on average 1 / p predictions to get each success.
So the value of the success curve at a point 0.5 < x <= 1 is the sum of 1 / p for all p's of successful predictions such that 0.5 < p <= x.
If you have some failed predictions at a confidence level p > 0.5, one would expect you to make on average 1 / (1-p) predictions to get each failure.
So the value of the failure curve at a point 0.5 < x <= 1 is the sum of 1 / (1-p) for all p's of failed predictions such that 0.5 < p <= x.

Note that predictions made with 50% confidence can't be meaningfully labeled as "failed" or "successful". If you want to include them, make sure that they contribute an equal value to both curves.
I'm also adding linear interpolation between significant data points, which makes the graph somewhat easier to read and doesn't break things too much in cases that matter. But I guess a stepped line or a point plot would be a more accurate representation of the math.

Applied Rationality Exercises

noreply@blogger.com (SquirrelInHell) — Sat, 07 Jan 2017 17:57:00 +0000

I was just about to run an introductory workshop around applied rationality, mindfulness, effective communication etc. in Warsaw, Poland when I got the flu, which forced me to stay in bed for a few days. A friend agreed to replace me at the event, and I wrote up the following notes for him.

The notes are not comprehensive, and do not include any background information or explanations. However, the availability of this kind of materials seems to be low, so there is a chance someone will at least take some inspiration from this.

The event was around 2 hours, with 5 exercises planned as below (short meditation, followed by 2 individual and 2 group exercises). They are not particularly intense as a way to transfer knowledge, I was thinking more of "getting the foot in the door" and having some fun with a group of people.

1. Eye Meditation

Total time: 5-10 min

Ask everyone to sit comfortably and close their eyes. Explain that this will be a quick mindfulness meditation, but it doesn't require doing anything special or difficult (mention benefits and relation to applied rationality).
Say "I invite you to let your breath come to your attention". Wait 30 seconds.
Say "I invite you to let everything that you hear, to come to your attention". Wait 30 seconds.
Say "I invite you to let the blackness you see with your closed eyes, to come to your attention". Wait 30 seconds.
Say "I invite you to move your eyes without opening them... as if you wanted to look left and right... to the top left, and bottom right... top right, and bottom left... and up and down...".
Say "I invite you to now look straight ahead, with your eyes still closed, and feel the movements of your eyeballs. You might try to relax them, so that they don't move at all." Wait 2 minutes.

2. Look at the Audience

Total time: 5 min + 3-5 min for each person

Explain that this will be a multi-purpose exercise, which will help to break the ice before other exercises, train the skill of being calm when speaking in public, and also continue the theme of awareness of one's eyes.
Say that everyone will do the exercise in front of the group, and that you will start to demonstrate.
Explain that the first step is to stand straight in front of the group, and say your name. Say your name.
The next step is to choose something that happened to you today or yesterday (it can be whatever, doesn't have to be funny or interesting). Now the task is to describe it to the group in a few sentences. However after saying each sentence, you will pause to do the following procedure:
1. take a deep breath,
2. choose one person from the audience,
3. catch eye contact with them, and hold it for 2 seconds,
4. smile at that person.
Explain these steps, and say that it's OK if this takes a long time to do. The purpose is to slow down, and feel comfortable about it.
Demonstrate by saying something about your day, and pausing to perform the steps above after each sentence.
Invite the next person to do the exercise, until everyone is done.

3. Pulling All Stops

(Credit: this is a very slight modification of "Focused Grit" by CFAR)

Total time: 10-15 min

Ask everyone to take a sheet of paper and individually list as many of their unsolved problems or issues as they can. Say that the lists are private, and no one will ask about what they wrote. Set a timer for 5 minutes.
Ask to pick one of the problems that is important to solve, and write it on a separate sheet of paper.
Ask to imagine that the only thing that matters to them is solving this one problem: they have no other goals or desires, and can use all of their resources for this one problem if it helps at all. Now with this mindset, write as many ideas for solutions as they can (even if they are not practical). Set a timer for 5 minutes.

4. Confidence in the Neck

Total time: 25-30 min

Ask everyone to form pairs, and for each pair to secure the following objects: one coin, one pen (these two must be actual physical objects), and something to write on.
Phase I. Put the following instruction somewhere so that everyone can see it: "Heads = Coin, Tails = Pen"
Have one person from each pair stand with their back to the other. The person in the back takes the coin and the pen, and repeats the following procedure so that nothing of it can be seen by their partner, who is facing the other way:
1. flip the coin once and write down the result (e.g. "C" for coin/heads, or "P" for pen/tails),
2. according to the result above, use the coin or the pen to lightly touch the back of the neck of their partner,
3. ask the partner which of the two they think they have just been touched with, and write down their answer,
4. ask them about the confidence in their answer (in percent), and write down their answer.
Have the pairs switch roles and repeat everything, so that the other person is guessing this time.
Discuss the theory (Bayes and priors, strength of evidence). Give everyone 5 minutes to check their results and talk about them in pairs.
Phase II. Put up the following instruction: "2 x Heads = Coin, other = Pen"
Have all pairs repeat the experiment, but this time the person in the back flips the coin two times before each touch. If they get heads on both throws, they touch with the coin, and in all other cases they touch with the pen.
Repeat with the other person guessing.
Discuss the theory again (correctly including new priors). Give everyone 5 minutes to check their results and talk about them in pairs.

Total time: around 35-40 min

Ask if there are any people present who know each other, and they can remember having at least one heated discussion or argument with each other. Ask those people to pair up, and everyone else to form pairs randomly.
Ask those who got paired randomly to think of some situation when they said something that they later regretted, or thought it could have been handled better. They will explain it to their partner, so it's better to avoid topics thay would rather not talk about.
Now the task for each pair is to recreate an argument by playing out the conversation between them. Ask them to keep it as close to their memories as they can, but within the limits of keeping the conversation flowing and avoiding very long and complicated explanations. If something is unclear, just improvise while following the general stance and approach of that side. The pairs that don't have a common argument will have to first let one person explain the details of the situation to the other (they will switch later). Set a timer for 5 minutes.
Now give everyone 5 minutes to brainstorm about how to better approach this situation, and in particular - how to return it to a state that feels safe, friendly, and cooperative, in which no one gets defensive or aggressive.
The next part of the exercise is act out improved versions of the argument, in which one person is trying out the ideas they had to smooth out the communication, while the other is trying to maintain the previous (non-cooperative) stance. The partner can be extra sensitive and pretend to react really badly to anything that they perceive as a threat, cheap shot, something unfriendly, unpleasant, patronizing etc. It is crucial to test ideas by actually saying them aloud, not just considering them. Set a timer for 5 minutes.
Give the pairs 5 minutes to switch roles, and have the other person explain some argument they had, and then act it out with their partner. The pairs who share an argument can discuss it in more depth, or pick some other situation that they have in common.
Set a timer for 5 minutes, for the person who was explaining this time to think about how to restore safety from their side of the conversation.
Set a timer for 5 minutes to let the other person try out their ideas, and act out different variants with their partner.

On Risk of Viral Infections from Chlorella

noreply@blogger.com (SquirrelInHell) — Sun, 20 Nov 2016 12:30:00 +0000

第一

In November 2014, a certain study caused a great deal of panic. The study reported detecting traces of a Chlorella virus (ATCV-1) in samples from human throats, and finding a decrease of cognitive capacity in those infected with it. The authors of the study report that they had further verified their result by infecting mice with the same virus, and finding that they had lower performance on solving some puzzles versus mice in the control group.
The original study was titled:

Chlorovirus ATCV-1 is part of the human oropharyngeal virome and is associated with changes in cognitive functions in humans and mice

However when picked up by other sources, it quickly became:

Algal virus found slowing down the brains of humans

And then in popular media:

Found: The viral infection that makes nearly HALF of us more stupid (and it lasts for YEARS)

Soon, the virus was dubbed the "Stupidity Virus" and it went downhill from there. You can easily imagine how it happened that the Internet is now flooded with those articles, and that they (still, after 2 years) drown out anything else about the issue.

第二

Let's pause and figure out what to make of this, imagining we are back in November 2014. Which is similar to what I knew after a quick search on the topic (the "shouty" articles obscure anything else, so that's what I saw) and checking the original study.
On the first page, we read:

We unexpectedly found sequences homologous to the chlorovirus Acanthocystis turfacea chlorella virus 1 (ATCV-1) in a metagenomic analysis of DNA extracted from human oropharyngeal samples. These samples were obtained by throat swabs of adults without a psychiatric disorder or serious physical illness who were participating in a study that included measures of cognitive functioning. The presence of ATCV-1 DNA was confirmed by quantitative PCR with ATCV-1 DNA being documented in oropharyngeal samples obtained from 40 (43.5%) of 92 individuals.

and

The presence of ATCV-1 DNA was not associated with demographic variables but was associated with a modest but statistically significant decrease in the performance on cognitive assessments of visual processing and visual motor speed.

So the virus was found in 40 of 92 people (suspicious! extremely suspicious!), and was correlated with a "modest but statistically significant" decline in cognitive capacity (suspicious! small sample? are we overinterpreting noise?).
If you take a look at the rest of the papar, the authors used 9 tests, from which 4 had definitely not significant results (p>0.1), 3 had p value around 0.01, and one had p<0.002. This is NOT at all strong evidence, considering that the situation is maximally selected to get any apparently significant result.
After adding the additional outside-view monition "Beware the man of one study", my credence-o-meter says "very low" overall.
However, if someone was very worried about cognitive decline, they could still understandably be uneasy.

第三

So what has happened after the original study? Checking Google Scholar and "Related Content" on the PNAS website turns up several papers:

March 2015 - an independent group from Denmark published a letter, in which they argue that the detection of the ATCV-1 virus was likely a mistake due to contamination of samples. They detected similarly short fragments of matching DNA in various other samples, as well as in negative controls. The second sequencing experiment from the original study, based on the PCR method, suffers from similar problems. The authors also point to some laboratory reagents that correlated with spurious detections, and overall claim to have rebutted the original findings.
March 2015 - published immediately in the same issue of the PNAS journal, a reply to the letter listed above, signed by all of the 18 authors of the original study. They argue that it is unlikely that both of the sequencing experiments generated spurious results, and that the PCR reactions they did had negative results in controls. They also say that the suspicious laboratory reagents have not been used.
December 2015 - a paper in the Journal of Virology, by four of the authors of the original study and two new people, in which they detect inflammatory responses in macrophages for a short time after exposure to the ATCV-1 virus. This confirms my earlier suspicion that this kind of effect (i.e. a standard slight inflammatory response) could wholly account for any observed changes in infected organisms (including a temporary slight cognitive decline).
August 2016 - another paper about reactions of mice to being infected with ATCV-1, by Irina V. Agarkova (listed in all the previous studies), Thomas M. Petro (the study from December), and Marilyn S. Petro. The access to this one is restricted, though it appears to claim similar results as before.

Except for the above, there seems to not have been much interest in the topic from other researchers.
I don't have any particular qualifications to read biology papers, but the more carefully I looked at the evidence supporting the case in the original study, the weaker it seemed. Also, the reply from Denmark gives references that support having very low priors for this kind of result. At some point you just have to admit that there is nothing here.

The third virtue is lightness. Let the winds of evidence blow you about as though you are a leaf, with no direction of your own. Beware lest you fight a rearguard retreat against the evidence, grudgingly conceding each foot of ground only when forced, feeling cheated. Surrender to the truth as quickly as you can.
- E. Yudkowsky

Internal Race Conditions

noreply@blogger.com (SquirrelInHell) — Sun, 23 Oct 2016 11:26:00 +0000

第一

You might be familiar with the concept of a 'bug', as introduced by CFAR. By using the computer programming analogy, it frames any problem you might have in your life as something fixable... even more - as something to be fixed, something such that fixing it or thinking about how to fix it is the first thing that comes to mind when you see such a problem, or 'bug'.
Let's try another analogy in the same style, with something called 'race conditions' in programming. A race condition as a particular type of bug, that is typically very hard to find and fix ('debug'). It occurs when two or more parts of the same program 'race' to access some data, resource, decision point etc., in a way that is not controlled by any organised principle.
For example, imagine that you have a document open in an editor program. You make some changes, you give a command to save the file. While this operation is in progress, you drag and drop the same file in a file manager, moving to another hard drive. In this case, depending on timing, on the details of the programs, and on the operating system that you are using, you might get different results. The old version of the file might be moved to the new location, while the new one is saved in the old location. Or the file might get saved first, and then moved. Or the saving operation will end in an error, or in a truncated or otherwise malformed file on the disk.
If you know enough details about the situation, you could in fact work out what exactly would happen. But the margin of error in your own handling of the software is so big, that you cannot in practice do this (e.g. you'd need to know the exact millisecond when you press buttons etc.). So in practice, the outcome is random, depending on how the events play out on a scale smaller that you can directly control (e.g. minute differences in timing, strength of reactions etc.).

第二

What is the analogy in humans? One of the places in which when you look hard, you'll see this pattern a lot is the relation of emotions and conscious decision making.
E.g., a classic failure mode is a "commitment to emotions", which goes like this:

I promise to love you forever
however if I commit to this, I will have doubts and less freedom, which will generate negative emotions
so I'll attempt to fall in love faster than my doubts grow
let's do this anyway, why won't we?

The problem here is a typical emotional "race condition": there is a lot of variability in the outcome, depending on how events play out. There could be a "butterfly effect", in which e.g. a single weekend trip together could determine the fate of the relationship, by creating a swing up or down, which would give one side of emotions a head start in the race.

第三

Another typical example is making a decision about continuing a relationship:

when I spend time with you, I like you more
when I like you more, I want to continue our relationship
when we have a relationship, I spend more time with you

As you can see, there is a loop in decision process. This cannot possibly end well.
A wild emotional roller-coaster is probably around the least bad outcome of this setup.

第四

So how do you fix race conditions?
By creating structure.
By following principles which compute the result explicitly, without unwanted chaotic behavior.
By removing loops from decision graphs.
First and foremost, by recognizing that leaving a decision to a race condition is strictly worse than any decision process that we consciously design, even if this process is flipping the coin (at least you know the odds!).
Example: deciding to continue the relationship.
Proposed solution (arrow represent influence):
(1) controlled, long-distance emotional evaluation -> (2) systemic decision -> (3) day-to-day emotions
The idea is to remove the loop by organizing emotions into tho groups: those that are directly influenced by the decision or its consequences (3), and more distant "evaluation" emotions (1). A possibility to feel emotions as in (1) can be created by pre-deciding a time to have some time alone and judge the situation from more distance, e.g. "after 6 months of this relationship I will go for a 2 week vacation to by aunt in France, and think about it in a clear-headed way, making sure I consider emotions about the general picture, not day-to-day things like physical affection etc.".
Total time to write this post: 31 minutes (23 wpm, 133 cpm)

Against Amazement

noreply@blogger.com (SquirrelInHell) — Tue, 20 Sep 2016 11:15:00 +0000

第一

The feelings of wonder, awe, amazement. It's a very human experience, and it is processed in the brain as a type of pleasure. If fact, if we look at the number of "5 photos you wouldn't believe" and similar clickbait on the Internet, it functions as a mildly addictive drug.
If I proposed that there is something wrong with those feelings, I would soon be drowned in voices of critique, pointing out that I'm suggesting we all become straw Vulcans, and that there is nothing wrong with subjective pleasure obtained cheaply and at no harm to anyone else.
I do not disagree with that. However, caution is required here, if one cares about epistemic purity of belief. Let's look at why.

第二

Stories are supposed to be more memorable. Do you like stories? I'm sure you do. So consider a character, let's call him Jim.
Jim is very interested in technology and computers, and he is checking news sites every day when he comes to work in the morning. Also, Jim has read a number of articles on LessWrong, including the one about noticing confusion.
He cares about improving his thinking, so when he first read about the idea of noticing confusion on a 5 second level, he thought he wants to apply it in his life. He had a few successes, and while it's not perfect, he feels he is on the right track to notice having wrong models of the world more often.
A few days later, he opens his favorite news feed at work, and there he sees the following headline:
"AlphaGo wins 4-1 against Lee Sedol"
He goes on to read the article, and finds himself quite elated after he learns the details. 'It's amazing that this happened so soon! And most experts apparently thought it would happen in more than a decade, hah! Marvelous!'
Jim feels pride and wonder at the achievement of Google DeepMind engineers... and it is his human right to feel it, I guess.
But is Jim forgetting something?

第三

Yes, I know that you know. Jim is feeling amazed, but... has he forgotten the lesson about noticing confusion?
There is a significant obstacle to Jim applying his "noticing confusion" in the situation described above: his internal experience has very little to do with feelings of confusion.
His world in this moment is dominated with awe, admiration etc., and those feelings are pleasant. It is not at all obvious that this inner experience corresponds to a inaccurate model of the world he had before.
Even worse - improving his model's predictive power would result in less pleasant experiences of wonder and amazement in the future! (Or would it?) So if Jim decides to update, he is basically robbing himself of the pleasures of life, that are rightfully his. (Or is he?)
Total time to write this post: 21 minutes 15 seconds (23 wpm, 128 cpm)

Neutralizing Physical Annoyances

noreply@blogger.com (SquirrelInHell) — Mon, 12 Sep 2016 14:09:00 +0000

Once in a while, I learn something about a seemingly unrelated topic - such as freediving - and I take away some trick that is well known and "obvious" in that topic, but is generally useful and NOT known by many people outside. Case in point, you can use equalization techniques from diving to remove pressure in your ears when you descend in a plane or a fast lift.

Ears

Reading about a few equalization techniques took me maybe 5 minutes, and I was able to successfully teach myself the "Frenzel Maneuver" by following these instructions:

The technique is to close off the vocal cords, as though you are about to lift a heavy weight. The nostrils are pinched closed and an effort is made to make a 'k' or a 'guh' sound. By doing this you raise the back of the tongue and the 'Adam's Apple' will elevate. This turns the tongue into a piston, pushing air up.

(source)

Hiccups

A few years ago, I started regularly doing deep relaxations after yoga. At some point, I learned how to relax my throat in such a way that the air can freely escape from the stomach. Since then, whenever I start hiccuping, I relax my throat and the hiccups stop immediately in all cases. I am now 100% hiccup-free.

Stiff Shoulders

I've spent a few hours with a friend who is doing massage, and they taught me some basics. After that, it became natural for me to self-massage my shoulders after I do a lot of sitting work etc. I can't imagine living without this anymore.

Non-Fiction Book Reviews

noreply@blogger.com (SquirrelInHell) — Thu, 11 Aug 2016 10:31:00 +0000

These are fairly well known, however there is a chance you haven't read all of them - in which case, this might be helpful.

Good and Real - Gary Drescher ★★★★★
This is one of my top favorites from rationality-themed books. Goes over a lot of philosophy, while showing a lot of clear thinking and meta-thinking. Number one replacement for Eliezer's meta-philosophy, if it had not existed. The writing style and language is somewhat obscure, but this book is too brilliant to be spoiled by that. The biggest takeaway is the analysis of ethics of non-causal consequences of our choices, which is something that actually has changed how I act in my life, and I have not seen any similar argument in other sources that would do the same. This book changed my intuitions so much that I now pay $100 in counterfactual mugging without second thought.

59 Seconds - Richard Wiseman ★★★
A collection of various tips and tricks, directly based on studies. The strength of the book is that it gives easy but detailed descriptions of lots of studies, and that makes it very fun to read. Can be read just to check out the various psychology results in an entertaining format. The quality of the advice is disputable, and it is mostly the kind of advice that only applies to small things and does not change much in what you do even if you somehow manage to use it. But I still liked this book, and it managed to avoid saying anything very stupid while saying a lot of things. It counts for something.

What You Can Change and What You Can't - Martin Seligman ★★★
It is a heartwarming to see that the author puts his best effort towards figuring out what psychology treatments work, and which don't, as well as building more general models of how people work that can predict what treatments have a chance in the first place. Not all of the content is necessarily your best guess, after updating on new results (the book is quite old). However if you are starting out, this book will serve excellently as your prior, on which you can update after checking out the new results. And also in some cases, it is amazing that the author was right about them 20 years ago, and mainstream psychology is STILL not caught up (like the whole bullshit "go back to your childhood to fix your problems" approach, which is in wide use today and not bothered at all by such things as "checking facts").

Thinking, Fast and Slow - Daniel Kahneman ★★★★★
A classic, and I want to mention it just in case. It is too valuable not to read. Period. It turns out some of the studies the author used for his claims have been later found not to replicate. However the details of those results is not (at least for me) a selling point of this book. The biggest thing is the author's mental toolbox for self-analysis and analysis of biases, as well concepts that he created to describe the mechanisms of intuitive judgement. Learn to think like the author, and you are 10 years ahead in your study of rationality.

Crucial Conversations - Al Switzler, Joseph Grenny, Kerry Patterson, Ron McMillan ★★★★
I have almost dropped this book. When I saw the style, it reminded me so much of the crappy self-help books without actual content. But fortunately I have read on a little more, and it turns out that even while the style is the same in the whole book and it has little content for the amount of text you read, it is still an excellent book. How is that possible? Simple: it only tells you a few things, but the things it tells you are actually important and they work and they are amazing when you put them into practice. Also on the concept and analysis side, there is precious little but who cares as long as there are some things that are "keepers". The authors spend most of the book hammering the same point over and over, which is "conversation safety". And it is still a good book: if you get this one simple point than you have learned more than you might from reading 10 other books.

How to Fail at Almost Everything and Still Win Big - Scott Adams ★★★
I don't agree with much of the stuff that is in this book, but that's not the point here. The author says what he thinks, and also he himself encourages you to pass it through your own filters. Around one third of the book, I thought it was obviously true; another one third, I had strong evidence that told me the author made a mistake or got confused about something; and the remaining one third gave me new ideas, or points of view that I could use to produce more ideas for my own use. This felt kind of like having a conversation with any intelligent person you might know, who has different ideas from you. It was a healthy ratio of agreement and disagreement, such that leads to progress for both people. Except of course in this case the author did not benefit, but I did.

Total time to write this post: 26 minutes 48 seconds (31 wpm, 169 cpm)