GPT-4 is non-deterministic…

… and MoE is the likely reason why

Intro

Outputs of GPT-4 are non-deterministic.

This means that the model doesn’t give the same output for the same input. This is easy to observe, especially for longer outputs. This is odd, as setting the temperature to 0 should lead to consistent results.