Harry Kane Photo: PHOTOSPORT
A day devoid of shocks also had Premier League leaders Manchester City crush second-tier Fulham 4-1 while relegation strugglers Norwich City enjoyed a morale-boosting 1-0 win at top-flight rivals Wolverhampton Wanderers.
Harry Kane scored twice as Tottenham Hotspur overcame Brighton & Hove Albion 3-1 in the day's late game.
After Middlesbrough's incredible penalty shoot-out win at 12-time winners Manchester United on Friday, the stage was set for sixth-tier Kidderminster Harriers to supply some FA Cup magic as they hosted high-flying West Ham United.
And how close they came.
Kidderminster led for 72 minutes thanks to Alex Penny's side-footed goal in the 19th minute but with one of the great FA Cup shocks brewing, Declan Rice levelled in stoppage time.
Then, as penalties loomed at the end of extra time, Jarrod Bowen broke Kidderminster's hearts to earn West Ham a 2-1 win.
It was cruel on the side 121 places below West Ham in the standings and visiting manager David Moyes, while delighted at his side's let-off, reserved some sympathy for the minnows.
"In truth I think Kidderminster deserved the opportunity to take it to penalty kicks," Moyes said.
"Kidderminster were excellent today, they prepared well for us. They've had games when we haven't so we should have come into it as the fresher team but they looked more ready for the whole situation."
Kidderminster manager Russell Penn said his side had put their "heart and soul" into the game.
"Of course we're disappointed by the timing of the goals, but I couldn't be any more proud," he said.
Chelsea did not leave it quite so late but also had to come from behind to beat third-tier Plymouth Argyle 2-1 after extra time at Stamford Bridge.
Plymouth shocked the Premier League side when defender Macaulay Gillesphey nodded the ball home from a free kick by former Chelsea player Jordan Houghton in the eighth minute.
Captain Cesar Azpilicueta equalised in the 41st minute with a neat flick and fellow Spaniard Marcos Alonso then struck in the 16th minute of extra time.
Even then Plymouth were not done and they were awarded a penalty five minutes from time when Malang Sarr brought down Ryan Hardie in the box but Spain's Kepa Arrizabalaga dived low to his left to save Hardie's shot.
Chelsea coach Thomas Tuchel was not at the game after testing positive for COVID-19.
"We put ourselves in trouble with the penalty at the end but everybody would say in the FA Cup the most important thing is to go through and that's what we did," Tuchel's stand-in for the day Arno Michels said.
Man City fell behind to an early goal by Liverpool target Fabio Carvalho at the Etihad Stadium but Ilkay Gundogan quickly equalised and John Stones put them ahead before halftime.
Riyad Mahrez scored twice in the second half.
Everton fans gave Lampard a rousing reception at Goodison Park and went home happy as the Toffees easily beat Brentford.
Yerry Mina powered in a header in the 31st minute and Richarlison doubled Everton's advantage three minutes later.
Ivan Toney got Brentford back in the tie with a penalty but Mason Holgate restored Everton's two-goal lead and Andros Townsend wrapped up the win in stoppage time.
Crystal Palace beat fourth-tier Hartlepool United 2-0 while in a match between former FA Cup winners, Southampton beat second-tier Coventry City 2-1 after extra time.
Kenny McLean was on target for Norwich as they beat Wolves to reach the fifth round for only the second time in 10 seasons.
The Cup action continues tomorrow with holders Leicester City at Nottingham Forest after Liverpool host Cardiff City.
-Reuters