That’s the best and simplest way to go!
Instead of asking him out, you invite him to go for a drink.
You say something like:
“Whaou! What a day! – I am going downtown for a drink – Join me?”
It is that simple.
What if you have him on the phone?
“I’ll be with my friends at café … on Saturday evening. Meet me there ?”
These approaches work wonders because they are non committing for him.
One of the key challenges if you try to invite him for a “formal” date is that he might find it already too committing for him.
So, simply being very casual and adding this invitation in the flow of your conversation is what works best.