Well, if anyone could do it, it would be you. ;)
We have, ourselves, never managed to wear one out in our demo units, or through anything resembling normal use. (We did manage to kill one by driving it directly at 9 V for a while…)
But, thinking about it for a moment, we would actually expect a reduced lifetime because of the way that you’re using it. As you say, because of the rubber band, the motor is having to work harder than it usually does. You might see if you can reduce the force that it needs so that the next one lasts a little longer.