GPT-4

George Hotz says GPT-4

• has 220B parameters
• is a mixture of experts with 8 experts
• is doing 16 times inference

(See also this tweet)