"Scholar, orator, poet, writer, administrator and a great leader has left us today. This is an irreplaceable loss for the nation," Baijal said in a tweet.
"His contribution to the nation and its people has been immense. I convey my deepest condolences on demise of Bharat Ratna Atal Bihari Vajpayee ji," he added.
Kejriwal, who turned 50 on Thursday, had earlier visited AIIMS along with his deputy Manish Sisodia to enquire about the health of Vajpayee, who was critical.
"Am deeply saddened. A great loss for India," Kejriwal said in a tweet.
The 93-year-old former Prime Minister, who was admitted in the All India Institute of Medical Sciences (AIIMS) on June 11, was "on life support".
Kejriwal had requested Aam Aadmi Party (AAP) volunteers and his well-wishers not to celebrate his birthday in view of Vajpayee's condition.