Basically, your final product needs to be non-potentially hazardous. When you add milk to cake batter and then bake it, it is no longer considered potentially hazardous. When you mix milk into frosting, the sugar stabilizes it and the final frosting product is non-perishable. Ice cream is a potentially hazardous food and cannot go in a cottage food product.
Unfortunately you can’t own both a commercial food business and a cottage food operation. Ironically, you could buy the cookies from a class B CFO and buy a commercial-grade ice cream, but you’d still need a commercial kitchen to make them into sandwiches and package them. You could still try talking to your health dept and see if you can own both kinds of businesses, but I don’t think it’s possible.