The Model That Changes Everything: Alpaca Breakthr…

2023/3/17

The Model That Changes Everything: Alpaca Breakthrough (ft. Apple's LLM, BritGPT, Ernie and AlexaTM)

8 years of cost reduction in 5 weeks: how Stanford’s Alpaca model changes everything, including the economics of OpenAI and GPT 4. The breakthrough, using self-instruct, has big implications for Apple’s secret large language model, Baidu’s ErnieBot, Amazon’s attempts and even governmental efforts, like the newly announced BritGPT.

I will go through how Stanford put the model together, why it costs so little, and demonstrate in action versus Chatgpt and GPT 4. And what are the implications of short-circuiting human annotation like this? With analysis of a tweet by Eliezer Yudkowsky, I delve into the workings of the model and the questions it rises.

Web Demo: https://alpaca-ai0.ngrok.io/

Alpaca: https://crfm.stanford.edu/2023/03/13/alpaca.html
Ark Forecast: https://research.ark-invest.com/hubfs/1_Download_Files_ARK-Invest/Big_Ideas/ARK%20Invest_013123_Presentation_Big%20Ideas%202023_Final.pdf
Eliezer Tweet: https://twitter.com/ESYudkowsky/status/1635577836525469697

Dear silly Internet saying "but knowledge distillation is already known": the key idea here is that the fine-tuning above the base model is comparatively much *easier* and *cheaper* to extract and re-imbue, if you have a new base comparable to the earlier base model.

(Spelling… https://t.co/en28X1PBxk

— Eliezer Yudkowsky (@ESYudkowsky) March 14, 2023