only if temperature is 0, but are they truly determinstic? I thought transformer based llm's where not

temperature does not affect token prediction in the way you think. The seed value is still the seed value, before temperature calculations are performed. The randomness of an LLM is not related to its temperature. The seed value is what determines the output. For a specific seed value, say 42069, the LLM will always generate the same output, given the same input, given the same temperature.

Thank you, I thought this wasn't the case (like it is with diffusion image models)

TIL