I’m trying to use replit.ai with python and I’m surprised to have the exact same results several times in a row.
For example, if I try to ask “Give me a number between 1 and 1000” 10 times in a row with 5 seconds sleep between them, it will always give me the same result.
Do you think it’s normal ? is there some kind of cache on replit.ai.modelfarm ?
Example here :
Result is always 342 even if i change the temperature.
Do i need to change something to avoid some caching ? or is bison model completion always returns the exact same value
(it is the same behaviour if i ask for a word at random, always the same word will appear)
This behavior is correct for neural networks. They are so arranged that they give the same answers to the same questions. The AI gives the answers it is trained to give. Try asking him what day it is, and he will always give the same answer, regardless of the date, and most likely his answers will always be wrong.
Yes I would have said that first, but when I ask a value “at random” with the TopK (randomness) at the maximum level, I feel it should try to give different answers …
At least that’s what other completion models like openai’s are doing …
If I ask : “Tell me a small story in one sentence with random words”, it always gives me exactly the same result… There should be some randomness somewhere … i feel that’s not normal
This would be possible only if there was a neuron with a random value in the neural network model. I thought about this when I was interested in creating neural networks. But in ReplitAI, randomness is not provided.
There is the topK parameter : Top-K changes how the model selects tokens for output. Specify a lower value for less random responses and a higher value for more random responses. By default it is at the higher value.
Maybe this doesn’t work in Python implementation but asking for a random value in all other models always gives a random value. Even if the neural network doesn’t really know what random is, the models always add values to add more or less randomness to the results