Under certain circumstances, nutmeg can produce hallucinogenic effects much like the ones from Marijuana. The reason for this is that it contains a compound called myristicin. It's found in it's seeds, and large quantities of it provide deliriant effects and can cause hallucinations if consumed in excess.
So why isn't there an uproar and ban on the use of nutmeg? Well the reason is that it's a pretty unpleasant way to get high, so no one really does it. For starters, nutmeg doesn't particularly taste well by itself. Only a little bit is used for cooking. However, for drug use, large amounts of nutmeg are required. Also, it takes a long time to feel the effects of the drug and also doesn't wear off for a few days. So if you're thinking about it... don't do it