Yes, cupcakes are allowed.
When you bake a cake, the egg gets cooked into the batter and that’s allowed. Basically, if your product needs refrigeration, as would be the case with a custard, cheesecake, or cream pie, then it isn’t allowed. Standalone milk and dairy products, like yogurt, cheese, etc. are not allowed, but you can incorporate certain kinds of dairy (limited amounts of milk, eggs, etc.) into your baked goods. Incorporating any amount of cheese would not be allowed.
It does appear that candy is not allowed in WA, unless it’s a baked candy, but hardly any candies fall into that category. So fudge, chocolate, truffles, etc. are not allowed. However, chocolate frosting is allowed, and I believe that chocolate buttercream frosting would also be allowed, but you’ll need to confirm with your health dept on that one. Also, when you mix a small amount of milk with powdered sugar, it does not require refrigeration and is therefore allowed.
Hope that helps.