The 73-year-old had last been seen in Coldingham on Friday and his family reported him missing the following day.
It sparked a search by the Borders Search and Rescue Unit and he was eventually found on Sunday.
Rescuers said a fence which stopped him reaching a nearby burn and heat from his dog may have helped him survive.
The man had last been seen on Friday picking up his newspaper but searches on Saturday night failed to locate him.
Seymour HaughRescue unit team leader By the time the rescue unit was called in on Sunday morning it said "hopes for his survival were slim".
We have seen previous cases where the close proximity of a casualty's dog has helped to maintain body temperature, and this may well hold true in this instance”
Several days of heavy rain meant local burns were in spate and there were fears he had fallen into a waterway. However, the man was eventually found in a gully by a local farmer.
The pensioner was "still alive but only just" with his terrier dog by his side. He was said to be "deeply hypothermic" and in need of urgent medical attention.
He was tended to by members of BSARU until an ambulance arrived to take him to Edinburgh Royal Infirmary.
Rescue unit team leader Seymour Haugh said: "The fact that he had been missing, along with his dog, for 48 hours, during which the weather had been pretty unrelentingly foul made the family and everyone involved in the search fear the worst.
"That he was found alive is testament to his tenacity.
"We have seen previous cases where the close proximity of a casualty's dog has helped to maintain body temperature, and this may well hold true in this instance."
He said the team had been assembling for a training exercise when it was called out which enabled 18 people to divert to the scene.
"That, the quick location of the casualty and the prompt first-aid we were able to administer almost certainly saved a man's life," said Mr Haugh.
"It's very gratifying - it's what we train for after all."
Mr Haugh said there were now "hopeful signs" the pensioner would make a full recovery.