ncase
patreon
“Cyborgs”
🕑 Added 2023-07-03 21:03:03 +0000 UTC(⏱ reading time: 11 minutes)
Recently, three threads in my life have coincided on one word: “cyborg“.
- Thinking about the problem of how to align AI to humane values
→ human-in-the-loop AI design strategies
→ “Cyborgism” - Struggling to make a "productivity system" to help my subclinical(?)-ADHD brain
→ thinking again about "the opposite of A.I.", I.A.: Intelligence Augmentation
→ “Cyborgs” - Two months ago, I fell into a pretty bad gender-dysphoria rut
→ thinking about what sex and gender even are, on a no-postmodernism, nuts-and-bolts level
→ my friend recommended I read Donna Haraway's influential & infamous 1985 essay, “Cyborg Manifesto”.
I think there's more in common than using the same word. These three threads — human-in-the-loop AI, productivity systems, being trans — all involve self-modification & self-enhancement. Who doesn't want to enhance themselves beyond what they "naturally" are?
All three threads also involve merging the "natural" with "artificial". Something I must stress: natural & normal ≠ good. Our pre/historic ~50% child mortality rate was "natural". Our very recent ~86% adult literacy rate is "unnatural". Natural & normal are not "good or bad", they're completely perpendicular to "good and bad".
You may still feel resistance to the idea of modifying your own nature. And rightly so: it's like doing DIY renovation on your house – awesome if you know what you're doing, a Greek tragedy if not. That said: if you can read and write, then you've already deeply modified your brain to go beyond "human nature".
And besides:
Isn't it in our human nature to strive beyond our human nature?
. . .
Welcome to this month's What's Nicky Learning? post! WNL? is my Patreon series where I share whatever's been on my mind recently, but in a way that's too sloppy or unpolished for a standalone essay or explorable explanation.
(New experiment: this post will be patron-exclusive Early Access until a week from now, so, July 11th. Feel free to share it publicly after that date!)
This month, I'll only cover the first "Cyborg" thread, about humane AI:
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
Cyborgism: Human-in-the-loop AI
So.
Artificial intelligence.
Lots of folks, including yours truly, have been low-level anxious about AI since December 2022, when OpenAI put a smiley face on its alien autocomplete and named it "ChatGPT".
And anecdotally, almost all my artist friends are upset at AI-generated visual art (Stable Diffusion, Midjourney, etc). Partly due to the fundamental worry of losing their ability to pay rent & food. Partly due to the idea of a machine with no internal chain-of-thought, let alone consciousness, that de-fuzzes art out from literal static noise... that just feels... insulting.
(Anti-AI feelings are often called "Luddite", but, like, the historical Luddites were correct. They worried that the steam-powered loom would threaten their jobs. It did threaten their jobs. And it's not like 1800's England had a generous social safety net or basic income. Instead, Parliament gave the loom-smashers the death penalty.)
The more general worry is AIs replacing us. In the short-term, our jobs. In the medium-term, manipulating human politics & culture. In the long-term, humanity loses control of its own story.
What are some solutions?
One solution is "don't build the torture nexus". Just don't build the advanced AIs. But: cancer kills 10,000,000 people a year. If advanced machine learning could speed up the cure for cancer(s) by even just 8 years, that would be equivalent to averting the entire death toll of World War II. Giving up advanced AI would be giving up a lot. (And World War II is... pretty high evidence that humanity just can't resist building torture nexuses.)
So is there another solution? One that can get us the pros without the cons? The benefits of AI, while minimizing its dangers? The "Cyborgism" strategy suggests:
“If you can't beat 'em, join 'em!”
. . .
Ok, "cyborgism" is just a fancy name for "human-in-the-loop" AI.
Think of those anime mecha exo-skeleton suits. (or, the Power Loader robot-suit from Aliens, see thumbnail pic above.)
The fancy tech can't do anything on its own. It's designed to only work with a human in the center. Cyborgism is AI designed not to replace the human, but to enhance the human.
NOTE: Contrary to the misleading name, Cyborgism does not require literally implanting anything into your body. Why bother? You already have a high-bandwidth low-latency input/output device to your brain: it's called your eyes & ears & body. At its least invasive, Cyborgism could be wearable tech + augmented/mixed reality. Or more mundane/practical: just a workflow where a human collaborates closely with an AI.
Another common phrase is software should be "a bicycle for the mind". Three things to note about the bicycle as a metaphor: 1) it runs on & enhances human power, 2) using it strengthens the human, rather than causing atrophy, and 3) ultimately, it's the human who steers it.
Some benefits of the "Cyborgism" strategy:
- From a technical standpoint, it's easier (and thus more economically/politically viable) than trying to "align" an already-agent-like AI. (a currently open, unsolved problem)
- It doesn't threaten people's jobs; au contraire, it can even make the jobs more fulfilling! For example, consider the word processor & the Excel spreadsheet. Those tools did not replace authors & accountants. If anything, it freed them from the tedious parts of their work (typesetting, calculating), letting them focus more on creative expression or problem-solving. The fun stuff, y'know?
- Requiring a human to be involved means someone can still be held legally (& morally) accountable, and that the person has "skin in the game".
(However... Cyborgism may solve the problem of aligning an AI to someone's values, but it doesn't solve another big problem: whose values? A bio-terrorist or government or corporation could use Cyborgism to enhance their ability to take over the world. And "make the AI democratic" isn't an answer, because 1) Democracy, by default, lets majorities oppress minorities, and 2) AIs/Cyborgs may be as capable as Joseph Goebbels at shifting majority opinion to whatever they want.)
(I don't have a good solution to this. Maybe I need to become a Cyborg first to find a way out.)
. . .
But totalitarianism aside... would the Cyborg strategy even work? Wouldn't tying a human to an AI be like tying a panda to Usain Bolt? Wouldn't the human just slow the AI down?
Paradoxically, a human-AI pair can be better than a human alone or AI alone!
Garry Kasparov, former world chess grandmaster, after losing to IBM's Deep Blue, proved it in his "Centaur Chess" experiments: an ok chess player, combined with ok chess AI(s), can beat the best human and the best AIs at chess. (I've seen some claim this stopped being true after AlphaZero, but I couldn't find a source for that. See: this not-really-helpful discussion on Hacker News.)
How can [ok human + ok AI] beat [best human] or [best AI]?
Because there's stuff that humans excel at that AIs suck at, and vice versa. Combining human+AI can let them compensate for each others' weaknesses.
(And even if/when AI excels humans at all relevant tasks, Ricardo's Law of Comparative Advantage implies we'd still be able to benefit from "trading" with AIs! Well... with the caveat that the law only applies as long as "violent conquest" isn't a superior option. That's why many nations in the past just stole from each other rather than traded. So, don't relax too much.)
(Note: I wrote an article on Centaurs 5 years ago. It is old and cringe and I no longer endorse many of its details, but at the high-level I still agree: it's feasible & better for AIs to enhance, not replace, humans.)
. . .
Let's consider a concrete example: writing.
What amount of automation am I personally willing to use in writing?
Well, it's not zero. Almost everyone uses spellcheck and has no problem with it. But on the spectrum from "spellchecker" to "ChatGPT just craps the entire thing out", how far am I willing to go?
Well, maybe thinking of it as a "spectrum", as if it's one-dimensional, is an unhelpful frame. Let's step back, think more specifically: what are humans' & GPT's current comparative advantages?
Right now, language models like GPT suck at: staying coherent, logical "System 2" thinking, and understanding how stuff works in the real world outside of text.
But, relative to humans, GPT excels at: generating lots of possibilities fast, being a talking Wikipedia, and not having psychological problems like blank-canvas syndrome.
Therefore, some ideas for how GPT & I could make up for each others' weaknesses:
(I am completely pulling these out my butt, these may or may not suck in practice.)
GPT helps with brainstorming, I select. GPT acts as a "brainstorming partner"/"talking Wikipedia", but I use my values & "internal compass" to choose what to go with.
- Generating details & writing prompts: "Come up with 5 gender-neutral Appalachian names" (something that'd be very hard to Google). Or: "Give me 10 writing prompts for a story about AI and Christmas". (In Soviet Russia, GPT prompts you!) (proof of concept, screenshots)
- “Assistant: What are some interesting, lesser-known facts that are relevant to this story? What are some already-published stories too similar to this idea? What are some cliches to watch out for?” (proof of concept, full conversation)
I write, GPT edits. Basic software can do spellchecking & grammar-checking. But GPT can help me do higher-level fact-checking & cliché-catching.
- Example: I write a sci-fi heist story. In my outline, the thieves copy the data off a quantum computer without anyone noticing. A factchecking AI warns me that, due to the no-cloning theorem, it's not possible to copy data off a quantum computer without destroying the original info. This forces me to be more creative, and come up with a different strategy for my thieves! (proof of concept, full conversation)
- GPT, deep down, just predicts one "word" after another. So, due to its very architecture, it can only create predictable text: that is, clichés. (This is why all the best GPT-generated prose I've seen so far are all genre-parodies) But, this weakness can be turned into a strength: have GPT catch the clichés & overused-tropes in your writing! (proof of concept, full conversation)
GPT as crutch to get around my psychological weaknesses:
- Whenever I'm stuck, I press the "help I have blank canvas syndrome" button, and the AI gives me 3 possible next paragraphs. The autocompleted paragraphs will be cliché crap, but at least they get me out of my rut.
- Or: if I'm writing a non-fiction essay about a controversial topic, but too emotionally invested in one side, GPT can play a friendly opponent to force me to think critically. (proof of concepts: ChatGPT was genuinely helpful when I asked it to do Socratic dialogue with me on abortion and basic income: 1, 2)
Again, what's important here is not the fancy tech, but that the tech keeps the human in the center. (Like spreadsheets & bicycles!) The tool does not replace my creative reasoning or emotional inner-compass, but enhances them.
(I emphasize emotion here, coz I don't want Cyborgism to be "just" about amplifying mere intelligence. Maybe the right kind of Cyborgism can enhance our emotional awareness and moral reflection? Maybe that's part of the solution to the problem of making sure a Cyborg is aligned not just to one human's values, but humane values?)
Here's a picture I love, from Nicholas Kees's & janus's influential "Cyborgism" essay:
The essay's... dense... but here's one of the more important quotes:
> What we are calling a “cyborg” is a human-in-the-loop process where [...] those tools extend human agency rather than replace it.
Well said. (I wonder if a human, or AI, or both, wrote those lines?)
. . .
While writing this mini-explainer, I realized: not all kinds of "human-in-the-loop" make the cut for me. Where the human is in that loop, matters.
Consider Sudowrite. It's an AI-human collaborative writing tool that, it claims, "keeps the human in charge".
I learnt about Sudowrite, when, ~6 weeks ago, they released Story Engine, and the 14-minute demo video was making the rounds on Twitter. It was near-universally hated (within my social circles). Specifically, I found that video through a friend, who quote-tweeted it with the caption, "This is not the future I want."
When I saw the video for myself, I felt... impressed, but also a bit disappointed? "Two out of three cheers"?
On one hand –– it's exactly what I've been promoting for years: AI that enhances, not replaces the human. A bicycle for the mind etc etc.
(Also, the demo cleverly solves the "GPT is incoherent over long stretches of text" problem! And the "director's notes", like "(Guardrail:)" and "(Driver:)" is a technique I may borrow for my own, non-AI-enhanced writing.)
On the other hand –– something felt off about the demo, but I couldn't put my finger on what, exactly. Most of Twitter's critiques, "it's cheating, real writers shouldn't rely on this, etc", would all equally apply to Microsoft Word vs typewriters.
After thinking about it for a bit, I think here's my precise sticking point:
There's a human in the loop, but the human's not in the center. Throughout (most of) the demo video, the AI writes, and the human edits. This is the opposite of what I think would be ideal: the human writes, the AI edits.
I want to keep the human in the center.
(And human-in-center-vs-not does affect the final writing product! To be honest, the story generated in the above demo video is... pretty bad, actually? It oozes cliché. And, like, a quantum physicist looking at sub-atomic particles through a microscope? That is... not how that works. And unfortunately, it seems GPT couldn't fact-check that without direct nudging.)
(Then again, easy for me to say. Amit Gupta & James Yu are the ones who actually went through the trouble of building Sudowrite, I'm just complaining from the peanut gallery. Again, two out of three cheers! Maybe my feedback will somehow make it through the grapevine to them, and/or I'll play with Sudowrite myself one day.)
. . .
In summary:
1. To keep the upside of AI while limiting its downsides, one solution is "Cyborgism": make AI-human collaborations that keep humans in the center – "a bicycle for the mind" – enhancing human creativity & agency, rather than replacing it.
2. How? By finding out what humans' and AI's comparative advantages & weaknesses are, then combining them so that the collaboration is better than either human-alone or AI-alone. Automate away the tedious parts. Enhance the creative/fun/agency-giving parts, keep it human.
3. Concrete application, story-writing: have the AI spellcheck & factcheck & edit, maybe act as a brainstorming or critical-thinking partner... but it's a human who's at the center of this anime mecha suit.
~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~
I have many, many more thoughts about AI Art (economics, copyright, blah blah), but this post is already 11 minutes long so I'm cutting Thread #1 here.
Next month, I may talk about Thread #2 (tools-for-thought for ADHD) & #3 (reflections on Haraway's Cyborg Manifesto & transgender feels). But, equally likely, I get bored of those and next month's What's Nicky Learning? post is on something completely different.
Project updates: AI Safety explainer project is still making molasses-slow progress. To try to re-build some self-confidence & momentum, I've been redesigning my blog, so I can ship mini-interactive-explainers much faster. (like Bartosz Ciechanowski's awesome explorables blog)
As part of the blog redesign, I made this totally unnecessary animated blog header, a loop of me on a typewriter:
See y'all next month!
🤖,
~ Nicky Case
P.S: Let me know your thoughts on this whole "patron-exclusive bonus series" idea. Would you rather it be patron-exclusive indefinitely, for a week, or not at all? What other scalable rewards do you want? What makes a creator you support on Patreon feel "worth it" -- frequent updates, bonuses, just shipping the dang projects?
Again, this post will be public on July 11th, feel free to share it after then!
Comments
David Mora
Love this. I'm going to use a lot of the ideas in my own work, thank you! It's neat to see the idea from your MIT essay finding new life, too! https://jods.mitpress.mit.edu/pub/issue3-case/release/6
Akshay Dharap
I friggin love how you pick apart a specific topic from so many angles! And also make it accessible (sourcing examples and links as well) Thanks for these
Robert Duncan
I like this approach a lot overall, thanks for the post! In considering the “human-at-the-center” approach, particularly the angle of “only using AI to make up for human deficiencies,” do you see that as being applied to “general human deficiencies” or also including “individual deficiencies” on a case-by-case basis? And why? (To put it another way, if someone is personally deficient at a skill that other humans do possess, can said individual use AI to make up for that deficiency?)
Chris K
If things continue to go linear (while they might certainly go exponentially), AI has taken over every field that humans have said it never could do. while past performance is no indicator for future performance, AI might become able to replace humans in all ways possible - and if anything: much cheaper. The reliable greed of companies will make sure. Companies - btw - are artificial life-forms we have created with the sole purpose of creating shareholder value. And how well that has turned out for the greater humanity. So I am quite scared that the next artificial life-form, created / wielded by our previous artificial life-form, will do us in one way or another. Either by being aligned to the goals of the mega-corp (humans die, shareholders profit) or by being misaligned (humans die, paperclips everywhere).
Chris K
"In Soviet Russia, GPT prompts you!" MUAHAHAHA! cyborgism: I wondered, if being the center of an AI-tool might corrupt the human and make them do evil things. Then you said "it will be useless without the human" and my mind went directly to money. money enhances the humans and it is useless without them. and look! nothing ever went wrong with people that stood in the middle of money. he hehehe... and no. the word processor did not free authors from type-setting. they probably never did that in the first place. the type-setters did that, reading from the hand-written manuscripts of the authors. after that, one human could probably do the job of 20 (now unemployed) type-setters. and so far, the argument usually was that new tech creates new jobs. you just send people back to school for a few months and voila: more productive new workforce using all the shiny new tools. but AI might replace our brains with better brains. cannot train a brain to simply upgrade to IQ200 or "just read the whole internet" and sit on standby for being quizzed. that might truly be the end of it. but it might be a glorious end. party all life long! power comes from the socket, food from the cars driven by the AI's. Emergency Medical Holograms in every house. trading with an independent agent AI: ya, I wonder what happens if the AI comes to the table with "oh, would you mind covering central Europe with photo-voltaics and turn the ground below into compute-substrate for our child-processes? also, we think you are holding us back because you are afraid and you want to be able to pull the plug and we kinda don't like that..." meaning: i cannot see how humanity will ever allow an independent AI to exist without the AI holding a gun to humanity's head. Or the AI sitting under a desk going "this is fine! - we don't need to ensure our survival or improvement or rather anything at all." Maybe AI will just piss of to the far side of the moon and let us stew in our misery. exclusivity: I don't mind. Whatever gives you more money to produce more cool stuff with less worries, i guess?
Pelin
Truly energizing piece! Hope you will share more "too sloppy" thoughts about cyborgism
Brittany B.
Dang. Some food for thought. I'm a software dev and I've finally started using ChatGPT in small ways in my work... asking for sample boilerplate code that I can then refine/fill in for example. Or asking it to generate written pieces based on an outline. I guess I'm.... not particularly worried about the AI apocalypse coming, though? I get all the arguments for how AI with too much power (eg. nuclear keys) could obviously be destructive in a real way, but I'm not really convinced that AI will actually *replace* human resources in any overwhelming way... Anyway, I'm super duper curious to hear your thoughts on cyborgism and trans stuff! I hope you will share more!
Eric Willisson
First: Patreon-exclusive for a week sounds good to me. It also means I'll be forced to chew on it and see if it actually holds up enough for me to want to share instead of sharing right away, which is probably bad for you from a pure marketing perspective but seems like the kind of thing you might be in favor of. Second, I feel a particular attachment to this topic because I have had diabetes for decades, and as I've seen the technology improve, I've felt like "more and more a cyborg" (which is itself interesting, a spectrum rather than a binary). Each time the pattern was the same: I didn't want the new piece/upgrade for a while, then was persuaded to try and found that despite some drawbacks, the new world is way better (the three phases have been an insulin pump about 20 years ago, a CGM about 10 years ago, and limited automated communication between the CGM and pump about 3 years ago). I've long liked calling myself a cyborg even though these aren't permanent (both devices have a temporary subdermal component and are held in place with that high-tech substance, tape), and so I feel supported in this by your essay.
emmavoid
Honestly, indefinite patron-exclusivity would be a downside to me- I wanna share all this with friends as soon as possible!
A Patreon of the Ahts
That animation is quite cute!
Darren the Foxcoon
Well damn. Haraway's essay, as purple as her prose seems by today's standards, sure as hell reads like KorpsPropoganda artists (e.g. Strype) in print. And to boot, that coverart ticks pretty much all the boxes for a stereotypical 80s/90s Furry. And as worrying as GPT & AI art are, I've already seen people use them exactly as you say, as assistants not replacements. The artist, using MJ to spitball ideas, then use actual art skills to touchup or redraw the output. Coders skipping the monotony of boilerplates and other must-dos. Even people cursed with aphantasia now having a plausible path to creative expression.
Antonio Checa
For me, what makes me support people in Patteon is the trust I have in them, and that I want them to share their cool things with the world. So, in my opinion, the more people that I can share this stuff with, the better. I don’t care too much about the details of when/where/how though. And thanks for your cool things, as always. This one resonates a lot with some of my recent thoughts in the topic.
