As of SQL Server 2012 you no longer have to go through the hassle of deleting the bin file (which causes other side effects). You should be able to press the delete key within the MRU list of the Server Name dropdown in the Connect to Server dialog. This is documented in this Connect item and this blog post.
Note that if you have multiple entries for a single server name (e.g. one with Windows and one with SQL Auth), you won't be able to tell which one you're deleting.