They weren’t stoners. Tolkien makes it quite clear he’s talking about tobacco, not weed - the only reason he called it “pipe-weed” is that he was attempting to avoid having New World anachronisms in Middle Earth. (Ignore the potatos 😉).