Llama 2: Full Breakdown

2023/7/20

Meta have released Llama 2, their commercially-usable successor to the opensource Llama language model that spawned Alpaca, Vicuna, Orca and so many other models. I read the full 76 page technical paper, the Responsible Use Guide, each of the release pages, the terms and conditions and have also run many of my own experiments on HuggingFace. This video covers the most noteworthy bits, including:

– The benchmark results vs other open-source models and GPT 3.5 and GPT 4.
– The decision to release in the face of the US Senate
– Why Zuckerberg opensourced Llama 2, including the Mistral AI Factor
– The opensource signatory list
– How Orca and Phi-1 compare
– Why Llama 2 might soon be everywhere

Llama 2 Release & Download: https://ai.meta.com/resources/models-and-libraries/llama/
Llama 2 Technical Paper: https://scontent-man2-1.xx.fbcdn.net/v/t39.2365-6/10000000_6495670187160042_4742060979571156424_n.pdf?_nc_cat=104&ccb=1-7&_nc_sid=3c67a6&_nc_ohc=GK8Rh1tm_4IAX-m2jNJ&_nc_ht=scontent-man2-1.xx&oh=00_AfCu5KuIzANjp_1LZ-rGcXGJR6TL9JvoZHgKOgl41cCCCw&oe=64BBD830
Llama 2 Demo: https://huggingface.co/blog/llama2#demo
Ts and Cs: https://ai.meta.com/resources/models-and-libraries/llama-downloads/
Responsible Use Guide: https://scontent-sjc3-1.xx.fbcdn.net/v/t39.8562-6/361643215_1004219997281331_6332933766797859993_n.pdf?_nc_cat=111&ccb=1-7&_nc_sid=ae5e01&_nc_ohc=4bD2ixIWrlEAX9z_dvZ&_nc_ht=scontent-sjc3-1.xx&oh=00_AfBBl2AGeJzsQOpLEjO849AHIZ1P1MY68TTXBwcMwrybtQ&oe=64BB0C07
Open Source Signatories: https://about.fb.com/news/2023/07/llama-2-statement-of-support/
SIQA: https://arxiv.org/pdf/1904.09728.pdf
BoolQ: https://arxiv.org/pdf/1905.10044.pdf
Orca Paper: https://arxiv.org/pdf/2306.02707.pdf
(my video on Orca): https://www.youtube.com/watch?v=Dt_UNg7Mchg&t=842s
PHI-1: https://arxiv.org/pdf/2306.11644.pdf
(my video on Phi-1): https://www.youtube.com/watch?v=7S68y6huEpU
US Senate Letter: https://www.hawley.senate.gov/hawley-and-blumenthal-demand-answers-meta-warn-misuse-after-leak-metas-ai-model
Mark Zuckerberg on Lex Fridman: https://www.youtube.com/watch?v=Ff4fRgnuFgQ&t=8710s
Mistral Tweet: https://twitter.com/GuillaumeLample/status/1681346701766934543
And Memo: https://drive.google.com/file/d/1gquqRqiT-2Be85p_5w0izGQGgHvVzncQ/view
+ Yi Tay: https://twitter.com/YiTayML/status/1681352594416087040
Qualcomm: https://twitter.com/nonmayorpete/status/1681384734910484480?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet
Dr Jim Fam: https://twitter.com/DrJimFan/status/1681372700881854465
MIT Leak: https://www.technologyreview.com/2023/07/18/1076479/metas-latest-ai-model-is-free-for-all/