Movie Kyle's "beast" form isn't exactly that hideous. Kendra's spell was to make him as ugly on the outside as he is on the inside. The form shows he wasn't that ugly on the inside and is a clue that he can be redeemed.
Also many people find his beast form more attractive than his original form. The movie's Aesop is about accepting people for who they are. Boom! The audience just has to accept Kyle the way he really is when he transforms back into his old self.
Another theory is that while Kyle may not be incredibly ugly, he and his dad think of him that way.
Kendra's role at the end. She's not just punishing Kyle's dad for being a dick. It could be part of a calculated move to help repair the relationship between father and son.
In the beginning of the book, during her rant about how looks aren't important, Kendra glares at Kyle when she says "Blue eyes and blonde hair is always easy to recognize". It took a while to realize, if she's really old enough to have experienced many historical events, then she must've lived through the Holocaust. Adolph Hitler's belief in the "Aryan Race" must've cemented Kendra's resentment towards people focused on looks. In hindsight, her speech in the book could be viewed as a Take That!.
