The most likely cause is that you are using forms-based authentication (FBA), which in SharePoint 2010 means you are necessarily using Claims, and you have misconfigured one of the several web.config files you need to set up for FBA. Check them carefully for typos.
This message can also arise if your Claims to Windows Token Service (C2WTS) is not running. Check the Services on Server page to make sure that it is Started. The error may occur when you try to navigate to a service application management page where the service is dependent on C2WTS.