AIExplained
patreon
Sebastien Bubeck Qs
🕑 Added 2023-12-15 17:24:22 +0000 UTCSpeaking to Sebastien Bubeck in an hour, for AI Insiders. Bubeck is Sr. Principal Research Manager at Microsoft, co-author of Sparks of AGI and Phi-2 (Textbooks are All You Need) and Yann LeCun debater: A New Kind of Intelligence. A bit late notice I know, timing was just confirmed, but wanted to give Insiders a chance to suggest questions. No guarantees, but will strongly consider those upvoted! Video derived from discussion up after at some point soon.
Comments
Prajay Tipre
Thank you sir!
Philip
This should solve it! https://support.patreon.com/hc/en-us/articles/212052266-Getting-Discord-access
Prajay Tipre
Unrelated but can anyone drop the discord link? Was having a hard time finding it.
Jie Yu
No, but the Boston Dynamics AI Institute will be working on exactly this problem!
Philip
I asked something similar, video coming before Christmas
Michal Babula
This is a question form chaGPT to Sebastian: As someone deeply involved in AI research, what are the next big milestones or challenges you anticipate in the field? What areas of AI research excite you the most?
Philip
Asked!
Philip
Didn't get to this but next time!
Philip
Briefly touched on this but his time was short alas
Philip
Asked!
Philip
We touched on reasoning quite deeply!
Philip
Asked and answered. But you shall have to wait for the video! Haha
Philip
Asked!
Philip
I asked something similar. Instruct actually hurts perf. apparently!
Nazzaroth
Would love to know what he thinks a world post-agi would look like and where society/technology will develop towards or what his person plans will be once it is achived.
Jeff Thom
Will true multimodality only be achieved through robots that can actually, see, touch and interact with -- and therefore understand the real world?
Jeff Picel
Do we need new benchmarks for measuring capability for reason? What would those look like and how are they different than what we already have?
solarapparition
Very basic one, but what does he think that the path to AGI looks like?
Jörg Eitner
What does he think about the new Mamba architecture and if it has the potential of replacing transformers in the near future
Blixt
On the tune of Yann LeCun I would love to hear a take on his JEPA models (and his opinions around it: https://twitter.com/ylecun/status/1735648731855310896). Maybe also somewhat related, using diffusion models instead of transformers to generate text (eg CodeFusion from Microsoft).
Shaun McDonogh
Also…selfishly so forgive me…, can the standard GPT architecture be used to predict the next image (rather than word)? Is there an “image embeddings” database perhaps?
Rakesh Murria
Using high-quality training data, what is the smallest size of model does he think could build to be on the level of a GPT4V or Gemini Pro
Daniele Moro
The blog post says "Phi-2 is a base model that has NOT undergone alignment through reinforcement learning from human feedback (RLHF), nor has it been instruct fine tuned." How is the model getting such good results without any RLHF??
Anouar Mansour
I'd like to know what is the most top of mind for him regarding AI in the next year. What he looks forward to and what he sees being accomplished in the AI space.
G Subs
Lifelong lurker and newbie to AI from totally separate field, please forgive any misunderstanding in my questions. Current benchmarks and decontamination methods are flawed. Synthetic data seems particularly at risk for beating traditional decontamination methods. Are the phi models susceptible to this, and any thoughts on the next likely outflow of synthetic data for training generated by already contaminated LLMs Nice paper that looked at better methods for evaluating this “Llm decontaminator” https://arxiv.org/abs/2311.04850
Eli T. Drumm
Does he foresee some kind of inflection point where the best models are good enough to synthesize training data for other models and is this a viable path forward towards AGI? (I still need to understand more of exactly how the synthetic data was generated, and regardless I'm guessing you can phrase this question a lot better)
Shaun McDonogh
Amazing! Excited to find out 1) what are the likely biggest contributing factors to a model discovering “new knowledge” (also the recent deep mind post was interesting) and 2) do they have any thoughts as to what Q* was about. Not sure i get 2 questions but worth a shot lol.
