The pooch went in near Veteran's Field and was scooped up about a quarter-mile south by the current to behind Mariner's Tower, borough DPW Director Tom Quinton told Daily Voice.
Members of North Hudson Regional Fire and Rescue, who were having lunch at the marina with some Edgewater colleagues, jumped into their boat with Quinton and others and headed out.
A New Jersey State Police boat also pulled up.
"At first, we couldn't see him," Quinton said. "Once we did, we called out, and he swam right over to the boat. We pulled him right in.
"His teeth were chattering -- you could hear them."
The rescuers gave him a sweater -- and secured him this time with a leash -- while they sought his owner.
"He was cold and unhappy," Quinton said. "But luckily he was fine."
Click here to follow Daily Voice Cliffside Park-Edgewater and receive free news updates.