mmhamdy/speecht5-finetuned-fleurs-it-it
Text-to-Speech • Updated • 1
Mohammed Hamdy
mmhamdy
AI & ML interests
AI4Sci | NLP | Reinforcement Learning
Recent Activity
reacted to theirpost with 🧠 about 12 hours ago
It was supposed to be a failed experiment. Instead, it led to the discovery of one of the most intriguing phenomena in neural networks, simply because a researcher forgot to turn it off and left it running....for a week!
In 2022, researchers at OpenAI were studying how neural networks generalize from their training data. For this task, they were training small transformer models to perform modular arithmetic.
The thing is, neural networks are weird. When a model has an abundance of parameters (like neural nets), it can easily overfit. It essentially memorizes its training data, scoring a perfect 100% accuracy when tested on it, but remains completely clueless when faced with any new instances not present in the training set (close to 0 accuracy). It is like memorizing 1 + 2 = 3 without understanding the concept of addition, so if 2 + 3 wasn't in the training set, the model fails miserably!
Usually, when a model overfits like this, people just cut their losses, turn off the experiment, and move on with their lives.
But sometimes they forget. And that is exactly what happened to our researchers at OpenAI. A week later, they checked back in, and a miracle had happened!
They discovered Grokking (And no, this has nothing to do with xAI's Grok , the term was originally coined by sci-fi author Robert Heinlein to mean understanding something so deeply that it becomes part of you). Grokking is when a neural network suddenly and abruptly learns to generalize long after it has overfitted. Just take a look at the graph in the image below!
Spooky, right! I told you neural nets are weird! reacted to danielhanchen's post with 🚀 about 14 hours ago
Google releases Gemma 4 QAT. ✨
You can now run Gemma 4 at 3x less memory with near original performance.
QAT makes it possible to run Gemma 4 26B-A4B on 16GB RAM.
GGUFs: https://huggingface.co/collections/unsloth/gemma-4-qat
QAT Guide: https://unsloth.ai/docs/models/gemma-4/qat repliedto their post about 16 hours ago
It was supposed to be a failed experiment. Instead, it led to the discovery of one of the most intriguing phenomena in neural networks, simply because a researcher forgot to turn it off and left it running....for a week!
In 2022, researchers at OpenAI were studying how neural networks generalize from their training data. For this task, they were training small transformer models to perform modular arithmetic.
The thing is, neural networks are weird. When a model has an abundance of parameters (like neural nets), it can easily overfit. It essentially memorizes its training data, scoring a perfect 100% accuracy when tested on it, but remains completely clueless when faced with any new instances not present in the training set (close to 0 accuracy). It is like memorizing 1 + 2 = 3 without understanding the concept of addition, so if 2 + 3 wasn't in the training set, the model fails miserably!
Usually, when a model overfits like this, people just cut their losses, turn off the experiment, and move on with their lives.
But sometimes they forget. And that is exactly what happened to our researchers at OpenAI. A week later, they checked back in, and a miracle had happened!
They discovered Grokking (And no, this has nothing to do with xAI's Grok , the term was originally coined by sci-fi author Robert Heinlein to mean understanding something so deeply that it becomes part of you). Grokking is when a neural network suddenly and abruptly learns to generalize long after it has overfitted. Just take a look at the graph in the image below!
Spooky, right! I told you neural nets are weird!Organizations
mmhamdy/whisper-tiny-finetuned-minds14-en-us
Automatic Speech Recognition • Updated • 2
mmhamdy/whisper-tiny-finetuned-gtzan
Audio Classification • Updated • 3
mmhamdy/poca-SoccerTwos
Reinforcement Learning • Updated
mmhamdy/vizdoom_health_gathering_supreme
Reinforcement Learning • Updated
mmhamdy/ppo-LunarLander-v2-2
Reinforcement Learning • Updated
mmhamdy/a2c-PandaReachDense-v2
Reinforcement Learning • Updated
mmhamdy/a2c-AntBulletEnv-v0
Reinforcement Learning • Updated
mmhamdy/Reinforce-Pixelcopter-PLE-v0
Reinforcement Learning • Updated
mmhamdy/ppo-Pyramids
Reinforcement Learning • Updated • 29
mmhamdy/ppo-SnowballTarget
Reinforcement Learning • Updated • 2
mmhamdy/Reinforce-CartPole-v1
Reinforcement Learning • Updated
mmhamdy/dqn-SpaceInvadersNoFrameskip-v4
Reinforcement Learning • Updated • 3
mmhamdy/q-Taxi-v3
Reinforcement Learning • Updated
mmhamdy/q-FrozenLake-v1-4x4-noSlippery
Reinforcement Learning • Updated
mmhamdy/ppo-Huggy
Reinforcement Learning • Updated
mmhamdy/ppo-LunarLander-v2
Reinforcement Learning • Updated