Yup, that's my method. They always seem quick & temporary at the time but somehow they are still in the shop many years later. If you change your mind or add to the series, its not like another block of wood is a big expense or effort. I also have a block for common drill/clearance drills series which gets a lot of use & quick to grab. Like 2-56, 4-40, 6-32...
I like MDF a bit better than grained wood which can tend to distort over time & grip the shanks. You may have to experiment with the drill size for the right fit to go slightly oversize. Thats better than tugging on the sharp end of an end mill. I'm actually thinking of making some blocks for my Dremel tools. I have officially run out of room on my plastic ones that I cant seem to find a source for anymore.