The problem is that it's difficult to know ahead of time which ones will be empty. Consider recursion, advanced wildcards, ranges.... You kind of have to create directories which don't exist as you go.
I would suggest a different approach. Remember the names of all directories which are created, say in a linked list. Then, after the COPY or MOVE finishes, try to remove all of them — in reverse order, inside-out. Newly created directories which are still empty will be removed. Newly created directories which are not still empty will fail, but that's okay; just silently ignore the error and carry on.