I've tried all manor of cleaning solutions in my ultrasonic cleaner. Including some rather strong bases and acids. But not phosphoric acid, as I thought it would be too caustic (although it really isn't that strong). Oxalic acid however is mildier and might work well in it.
Be careful with rust remover products though. I had a bottle of commercial rust remover soak that was given to me by the makers of it (to try out and report my results, as part of some product testing/evaluation consulting I do at times). The label said not acidic, safe for all metals, won't harm your skin, items can be left soaking for days, only removes the rust and no damage to the substrate, etc. So I tried it on a lock according to the directions; a Fiat key cylinder for the rear trunk/engine cover lock levers on the door post. The lock was rusted frozen and I was hoping to loosen the rust enough to work it free and be able to use the lock (it matched the key for the sister locks on the car). After a couple hours soaking I checked on it. The solution was so murky I could not see the lock, so I poured it through a strainer into another container. All that was left were the tumbler pins. It had completely dissolved everything else. Don't trust the labels and trial test things of importance before letting them soak.
I've also tested a couple of other rust soak products with very poor results. After days of soaking they hardly touched the rust. But others work better, so it is a bit of trial and error.