At launch, Apollo 11 weighed 6,000,000 pounds, now all that was left of Columbia weighed in at a mere 11,000 pounds.
The next critical event in the Apollo 11 mission was the Trans-Earth Injection burn. The burn involved firing the big service propulsion engine for two and a half minutes on the back side of the moon.
The ascent of the Eagle was strikingly swift compared with the liftoff of the huge Saturn V rocket from Cape Canaveral. Of course for the Moon launch, there was no atmosphere resisting Eagle, and there was only one-sixth gravity to overcome.
Until now they had been focused on reaching the moon, landing, taking a walk on its surface, setting up experiments, exploring, and gathering evidence. With those tasks completed and their lunar bounty was board, the top priority was to fly back to Earth.