Mitchell, it depends on your state but the short answer is technically, probably yes, you do have to get a permit even for one day. Some states don’t require sellers to need a permit to produce cottage foods, while others totally prohibit this kind of sale — they don’t even allow home sellers at all.
In almost all states, you can sell your goods at certain types of events without a permit. Church bake sales, school fundraisers, or other non-profit, charitable events allow one-time sellers in this way. Also, food swaps don’t really fall under the category of sales and aren’t seen as being illegal, if the fair is like that.
If the fair charges admission, and there is an application process to become a vendor, then they probably won’t let you in if you’re not allowed. My advice would be to just try it and see how far you can get in the process. Except for some states, it probably wouldn’t be worth it to get a permit just for a day. If you get into the fair as a vendor and you are illegal, just be aware that many states do have health officials walk around events like that and check vendors. But, illegal sellers are relatively common, and the health departments are used to that fact, so if they found you out, then they’d likely just shut down your stand and inform you of the law and ask you not to do it again. It’s unlikely you would be fined.