Bing Toolbar is one of the most removed programs using Toolbar Cleaner. Toolbar Cleaner makes removing the Bing Toolbar easy. Start by finding all instances of Bing in Toolbar Cleaner (you can get Toolbar Cleaner here) and click to remove. Once you have removed Bing, Toolbar Cleaner will give you the option to run your control panel. Look in your control panel to see if there were any other unwanted programs installed on your computer.
Bing probably changed your homepage when you installed it. If you need to change your homepage settings you can read about that here.