So I've removed the water pump, and found no debris in the pump nor in the hoses. Further as I turn the pump I don't hear any noise so it doesn't seem to be coming from the pump at all. I see the small hole in the pump housing, but it's just a hole. No screw in it nor valve as far as I can tell. Seems like a problem, and the likely explanation for the leak, which seemingly has nothing to do with the noise.

Potentially dumb question: with the serpentine belt and water pump off, is there any danger starting the engine for just a few seconds to reconfirm the noise? If there is, I'm not seeing it.

 

No danger for a few seconds. The hole is in there to alert you that the bearing is bad and fluid is weeping by, which then runs out the hole. It does not get any screw or valve, its sole purpose is to leak when you need to replace the pump. No harm in starting the engine however since you need a pump anyway as evidenced by the leak alone, I would replace that to see if it's maybe your issue. Spinning it by hand and spinning it a several hundred RPM is a different thing. While it may or may not be the source of the noise, it's definitely bad and a potential culprit.

 

The same weep hole that lets fluid out when the shaft seal wears out, can let air in as the engine cools. The pump might have air in it when first started. The amount of coolant shown in the photo is more than a small "weep" by the way.