Peterborough started brightly and passed the ball well, forcing the best early chances as Sagi Burton went close twice in the first seven minutes.
After just two minutes, Burton rose well to meet an Andy Legg corner but the header drifted just wide.
Then, five minutes later, an identical move saw Burton's header destined for the top corner until a late, great save from City keeper Gavin Ward.
On 11 minutes Richard Logan deserved better when he lobbed Calum Davenport before rattling the inside of the post.
Coventry had taken to the field with three strikers, two of which - McSheffrey and Dele Adebola - have been linked with moves to Peterborough in recent weeks.
However, neither did enough in the first half to ignite any fresh transfer rumours.
McSheffrey did come close on 34 minutres but his 25-yard effort was deflected wide by Simon Rea.
From the resulting corner Mo Konjic rose well but his header dropped inches wide of the upright.
Posh defender Tom Williams provided the highlight of the first half with a screaming 30-yard volley that forced an excellent save from Ward.
Coventry had a penalty appeal turned down in the second half when Mark Arber appeared to handle Julian Joachim's cross.
McSheffrey came to life, though, after the break and stepped up on 58 minutes to send his home-town team through to the fourth round.
Mark Arber had fouled Joachim and his protests saw the ball moved forward to the edge of the box, from where McSheffrey calmly slotted the ball past the otherwise excellent Mark Tyler.
Minutes later the game was over as Joachim ran onto a Davenport flick and scored from eight yards.
Posh were reduced to ten men when Burton picked up his second yellow card on 63 minutes, but still managed a late consolation through Andy Clarke.