ncase
patreon
March Updates: I made a neural network from scratch!
🕑 Added 2024-03-30 23:18:39 +0000 UTC(β± 6 min read)
Three updates! Two big updates, and one kinda-update!
π§ Update #1: I'm making a neural network toy!
I'm making an interactive visualization of an Artificial Neural Network, from scratch! Look, you can directly click & drag to change its inputs, weights, and biases... and see all the intermediary activations. GIF below:
(Not pictured above: it can also learn by backpropagation, and you can even see the "errors" propagating backwards through the network!)
What's Left To Do: A sidebar with more settings to edit the network. I also want to visualize the cost function as a nice spinning 3D graph, so you can actually see gradient descent in action.
Mockup of the sidebar so far:
Motivation for this project: As Feynman's blackboard said when he died: "What I cannot create, I do not understand". I've seen a lot of theory & math of neural networks before, and I have played with code libraries that handle the neural networks for you... but I'd never made my own neural network fully from scratch. It forced me to learn... stuff.
(The three stages of learning about backpropagation; 1) First impression: Woah what is this complicated nightmare. 2) Learning the theory: Oh, it's just the chain rule! That sounds simple! 3) Implementing in practice: I HATE INDICES AND I HATE SADDLEPOINTS.)
Anyway, what with all the hype and/or fear around neural networks β some justified, some not β I thought I'd actually learn, very concretely, what they are. Then I can share those lessons with you, too!
Future Plans: I may release this first as a standalone toy, no "explanation" attached, then make it into an "explorable explanation".
Alternatively, I'd like to try making "Educational VTuber" videos: I play with a simulation I've made, while explaining ideas, all while I'm represented on-screen like a Twitch streamer's cartoon avatar. Then, I publish that video as the "product".
It'd be a more novel format for educational stuff, video does "better numbers" than text/websites, and it might be more fun / sustainably-motivating for me to make!
. . .
π€ Update #2: AI Safety for Fleshy Humans: Part One is reeeeeeeeeeally close to launching
Part One: The Past, Present & Possible Futures (15 thousand[!] words total) has been fully written, illustrated, and 90% polish-draft edited.
A few random illustrations from the Intro:
(There's also ~70 spaced repetition flashcards, so you can remember these facts long-term! The cards are embedded in the text with Orbit, but there'll also be a downloadable Anki deck.)
What's Left To Do: Finish editing to send it to a dozen reviewers (Laypeople & Domain experts); Incorporate reviewer feedback; Animate & draw a bit more art; Finish setting up the website.
Future Plans: As mentioned a few updates ago, Part Two is already ~70% done, because I realized very late, that the series worked way better Part One became Part Two and Part Two became Part One and... anyway, that caused a delay, but the next instalment should not take a frickin' year this time.
If all goes ideally, Part One ships late April, Part Two ships late May, Part Three (final part) ships in late June or early July.
. . .
π§ "Update" #3: My first Explorable Explanation is now 10 years old???
[RAMBLE INCOMING. There's no structure or polish, sorry.]
. . .
On February 2014, my videogame's crowdfunding campaign was failing, so I needed a last-week promotion. I was torn between either A) making a My Little Pony parody-mod, or B) making an interactive explanation of how I coded the lighting effect in my game.
Alas, I didn't have enough time to do the more fun Option A, so I settled for Option B.
That was my first ever explorable explanation, Sight & Light (Feb 2014). It hit the frontpage of Hacker News, got me in contact with Bret Victor, who then connected me with Vi Hart, who I collaborated with on Parable of the Polygons (Dec 2014), which was a huge hit, and
-- several dominos later --
I indirectly affected a few major Western news site? Washington Post's MOST-viewed article was their interactive(ish) thing on Covid/epidemiology models; its creator, Harry Stevens, told me he's been a fan of mine for a while.
Aaaaand that's ten years, folks.
. . .
On one hand, wow? And thank you folks β (and specific thanks to Bret Victor, Vi Hart, also hi Alan Kay we met only a few times but you were very encouraging) β for supporting me in this strange experiment for the last 10 years. <3
On the other hand, ugh, I feel like we could've done so much more? I'll admit, I kinda dropped the ball on Explorables, due to:
Some mental health stuff, partly exacerbated by the pandemic & gender-dysphoria-related chronic throat pain.
Some interpersonal bullshit in my Explorables-related community group, that led me to be sad and slightly angry and detached.
Also, I got a bit blackpilled on the value of "increase the public's scientific literacy". First, it's not clear that changing one's deepest beliefs even changes one's actions much. (a study, replication 1, replication 2) Second, increased scientific literacy leads to polarization on hot topics β to believing more strongly in whatever your political/religious identity already says to believe β not to understanding & agreement. (paper 1, paper 2)
. . .
"Explorable Explanations" was first coined by Bret Victor, who recently updated his original Explorable Explanations webpage with this note:
1) Aw thanks Bret <3 and 2) Yeah, fair, "Explorables" drifted a lot, for... better and worse and neutral? (Some of my interactives do give the reader a full sandbox, to edit the models & challenge my conclusions. Some of my interactives, uh, don't.)
"Explorables" is/was a mix of two dreams:
A tool for thinking: A computational medium to critically think in, for generating new ideas, not "just" transmitting ideas. (Bret's original direction; notebooks like Observable & Jupyter are in this vibe.)
A tool for teaching: Explorations in how to best use digital/interactivity for pedagogy & sharing ideas/research. For example: Simulations to let you play with scientific/computational models (most of what's on the Explorables site, Distill.pub); Embedded spaced repetition flashcards (e.g. Quantum Country); Sandboxes with live feedback to help you actually train & practice skills (e.g. the now-defunct Try Ruby by the now-vanished _why). And so on.
Of course, thinking & teaching aren't enemies, or anything. But I think the lack of precision on Explorables' purpose led to...
...actually I'm not sure what it did or didn't lead to. There's no way to see the counterfactual, but I can't shake off the feeling I missed some kind of opportunity. But I'm not even sure if that opportunity, whatever it is, is right for me specifically, anymore.
It's been ten years. Someone else can take this challenge? I don't like being tied to one thing. I want to feel playful & experimental in my creation again.
There's so much I want to do.
π£,
~ Nicky Case
Comments
Pietro Peterlongo
Thanks a lot for the behind-the-scenes story about Explorable Explanations, it is really interesting! 𧑠I think the drift on explorables is probably natural and the original and intended meaning probably had a narrower scope. Life has its course and I think part of the role of Bret has been to inspire all of us and we should do what we want with that inspiration, which is what you and the others wonderfully did and which is now acknowledged correctly in the essay. If we wanted to recover the original meaning we should probably find another term now. If I had to take a stab at it, they Victoriginals sound more like "Debatable Explanations" to me :) Letting others pick up the baton on Explorables and let yourself run free and playful and experimental sounds like a great idea, wishing you the best in that direction (or whatever you decide to do next)! Since this is my first message here after becoming a patron recently, let me also say thank you for everything you have done so far. You have been an incredible source of inspiration and important information packaged in a delightful way (and driven by a very honest and transparent ethics). π
Michael Huff
Honestly I super duper can't wait for your neural network toy! I'm thinking about dedicating some time in the near future to coming up to speed on this as well and I remember your work has always helped me feel more comfortable with a topic. I'm really sorry to hear you were disheartened on explorable education not leading to the hoped for effects. Looking closer at the studies I had to giggle that the "Moral Behavior of Ethicists" studied mostly academics as that is a population I wouldn't call an average sample by any means. Also I admit to being a biased party since I volunteer at a science museum so I don't know if that sentiment will ever be displaced. (Oh no, this possibly supports Kahan et. al !?)
Brian Handy
I think Explorables set a new standard for interactive blog education - anywhere from linear to branching to sandbox interactivity, just like the games industry as a whole. Regardless of how formalized a final model we get out of your decade exploring the space, weβre lucky to now have so many examples of quality works this way which the rest of us can study and create pieces in dialogue with for years. I think you deserve to feel proud of that regardless. Itβs possible that it just feels incomplete to you because you broke so much ground on a new undiscovered genre. I think if we are lucky, people will work on this format for years to come.
