tl dr: It sounds like you have a bad caliper.
The way calipers work is the seal doubles as the retracting device. When the piston (which is basically a cylinder inside the caliper body) is pushed out by the hydraulic force of the brake fluid as you press on the pedal, the piston moves out of the housing and sort of takes the seal with it. The seal gets sideways a little bit in the housing. The piston will continue to slide out of the housing until it meets resistance, which is usually the pad against the rotor face. When the brake pedal is released, the hydraulic force is reduced and the deformed seal will now return to its regular shape. When doing this, it pulls the piston back with it. If when the piston is pushing out against the seal, it isn't meeting any resistance from the seal, the seal will not become deformed as it's supposed to and then won't retract the caliper piston. This would also be why it's very easy to push the piston back into the caliper body with a screwdriver. I usually use a screwdriver to collapse the piston with the caliper still on the vehicle (depending on the style of caliper, of course). It usually takes a lot of force to get it to go back past the seal. If you aren't getting much resistance, this would be another clue which may validate my theory.
My suggestion? Take the caliper back and tell them it doesn't work. Either get a new one or a refund.