The source of fossil fuels in an electric car comes through the larger power grid in America. In order to charge your electric car, you have to plug it into the power grid. So, in that sense, your toaster, blow dryer, phone charger, etc. all use fossil fuels too.
As the power grid itself becomes less dependent on fossil fuels and moves toward renewable energies, the electric car will continue to become greener - as well as your toaster, etc!