Why do Heavy Objects Actually Fall Faster?

Why heavy objects theoretically falls a bit faster. By using newton i'm looking at what happens when a 1 kg object is dropped. then what happens with bigger objects. It turns out, that the earth is not stationary, and will accelereate upwards to meet the falling object. Hence the fall is faster or at least the collision time is lower.