By imitating the logic and explanations of GPT 4 (and using GPT 3.5 as an assistant), as well as by training on diverse tasks and an order of magnitude more examples, we have Orca. I will showcase it on a dozen benchmarks and go through in detail how it works and why.
I will also end on comments from Sam Altman and Ilya Sutskever on whether Opensource will catch-up…
Orca Paper: https://arxiv.org/pdf/2306.02707.pdf
False Promise Paper: https://arxiv.org/pdf/2305.15717.pdf
https://sharegpt.com/
FLAN: https://arxiv.org/pdf/2301.13688.pdf
Vicuna: https://lmsys.org/blog/2023-03-30-vicuna/
No Moat memo: https://www.semianalysis.com/p/google-we-have-no-moat-and-neither
LLM Leaderboard: https://chat.lmsys.org/?leaderboard
AGIEval: https://arxiv.org/pdf/2304.06364.pdf
BIG-Bench Hard: https://arxiv.org/pdf/2210.09261.pdf
Language Models as Tool Makers: https://arxiv.org/pdf/2305.17126.pdf
Altman Interview: https://www.youtube.com/watch?v=VWUhASix9ws
DERA Paper: https://arxiv.org/abs/2303.17071
Let’s Verify Step by Step: https://arxiv.org/abs/2305.20050
