Kurt is an atheist because of his experiences of being picked of for being gay.
There were many memorable songs on the installment, as Finn also discovered a burn in his grilled cheese sandwich that resembled Jesus. He prayed to it as a result, and a couple of his wishes came true. For example, Sam got injured and Finn took the starting quarterback job again.
But Finn then lost his faith after Emma talked a bit of practical sense into him.
Also, we learned that Sue didn't believe in God, due to everyone being mean to her older sister when they were growing up.
Mercedes took Kurt to church and everyone there prayed for his dad. This still didn't convince Kurt that God existed, but Burt Hummel woke up to conclude the episode. In the end, no real statement was made regarding religion, except that we should tolerate everyone's beliefs.
Source: TV Fanatic