Sure, but even recycled batteries have a shelf life, presumably not as long as new ones. So disposal issues will be a real problem when the time comes.
Eventually, but there's some cool stuff going on with getting really good usable material from recycled batteries. The biggest issue is consumption... all energy consumption contributes to entropy in some way - either mining for battery materials, burning coal, etc. The nice thing about solar is the bulk of the entropy is occurring billions of miles away, but we still need to store that energy here, which again leads to entropy.
The only way to slow that down, really is to consume less.