Add a third of flour, followed by a third of milk. Beat to combine. Repeat until you run out of flour and milk. Add vanilla extract and stir to combine.
Line 24 pieces cupcake tin with paper liners. Fill each one 2/3 of the way full. Bake at 360°F for about 20 minutes. Let cupcakes completely cool at room temperature.
Beat butter with one cup of powdered sugar. Continuously beating, add the remaining powdered sugar to the butter mixture until you run out. Add vanilla extract and milk.