In a canoe I keep the keys in a dry box and the phone in a phone pouch around my neck. It’s totally waterproof and I can operate it through the pouch. I can even talk on the phone with it.
I do the same as you have done. I have to have my phone for my iBobber. The box is cumbersome to get out and put it back in. But for me it works. Mine also gives a warning that it is getting too hot and will shut down without warning. The box is not ideal but the best option I have found.
Save your empty plastic jars with screw on lids. " Jiff Peanut Butter" you can easily attach a line to it. They float and are waterproof. A sponge on the inside is always a good idea.
Got a waterproof case for the phone. Phones are lasting so long now that it really makes sense to take the hit and get a quality case. I always keep it on me. If I get separated from the boat somehow or get injured, I'll need the phone to be able to call out to get help.