Sometimes it can be necessary
Often, after a patient has finally finished with orthodontic treatment, they are happy to be done with it, are happy with the results, and just want things to go back to normal. This is why retainers are so often neglected at the end of treatment. Let me be absolutely clear about this - retainers are just as mandatory for the success of the alignment as wearing the braces themselves. After the braces have painstakingly pulled the teeth into the correct alignment, it is the job of the retainer to make sure they stay that way.
Not wearing the retainer means that the natural tendency of the teeth to be in the position that they were originally in is not hindered by anything and they will return, or try to return, to their original position. Your orthodontist will tell you how long you have to wear the retainer, and, if your conditions permits, a retainer can be made that you only have to wear during sleep.
If you have already undergone the financial and physical pain of orthodontic treatment, it simply does not make sense to let it all go to waste, or even worse to begin the treatment again, just because of a small piece of plastic. Retainers are nearly invisible, and should be perfectly comfortable as well. If your retainer is causing you discomfort, have your orthodontist adjust it to fit your needs. But not wearing it at all is a very silly move, and possibly a very costly one.