The one bad thing about fantasy football.

im new to fantasy football this is my first year and glad to say im off to a 3-2 start. however i did this about 2 weeks into it and saw what an android peyton manning was being. so sorry but i picked him over drew. so naturally the day would come i would have to play against drew. that day is this weekend. but its even worse then that he has darren sproles and colston! but i got jimmy , PT(23), saints D, and garret hartly. ive convinced myself that say if drew,sproles,closton break records this weekend i cant get mad. there my saints and i should be happy for them. anyone else have a similar problem?