Well, I have decided to spend as much time outside as possible, and after around 6 hrs or so and a lot of water (he has incredible bladder control for a puppy!) he finally peed outside, and got a jackpot of treats and praise. And it seems that was all it took! Since then he only had one accident inside, at night, which is to be expected. He will hold it until we go outside, and pee as soon as we're out! He is a perfect puppy now!!
Unfortunately I will only have him for a few more days.
Thank you for your advice!