2nd the wood glue over epoxy. Even though it will be sealed later I would use a good waterproof glue like titebond III. Add glue to sawdust slowly and mix into a paste. Try to get like a peanut butter consistency, not too wet. Do a little test spot or two to see how it behaves before going whole hog.First, what @wedge2 said.
Second, I’ve seen that done with white glue and sawdust. Since it will be varnished to seal later that might be easier than jacking around with epoxy.
As for the caulk lines I kind of like the block plane idea if the caulk is rock hard (giggity). It always depends on the consistency which tool will work best. If the stuff is at all gummy it will clog up the mouth of the plane.
If you wanna spend a few bucks get a good battery powered multi tool and a sealant remover blade. That seems like by far the easiest option and those tools are useful for many other things.