Currently, vCenter Server 5.1 is not supported on Windows Server 2012. Trying to install usually results in several errors, but they can be resolved with some workarounds and things to consider. This post is a gathering of (some of) those things.
First of, be sure to install .NET 3.5 using the Server Manager. It can be found as a feature. The installation delivered in the vCenter installation cannot be installed.
It's best to install the required components (SSO and Inventory Service) separately to ensure the installation of the different components goes smoothly. You will have to type in and remember the SSO passwords though.
Upon installation of the vCenter Server, it will install completely but will then return an error. Although the installation is complete, manually starting the services does not work. The reason is that the Protected Storage service no longer exist in Server 2012, but the VMware vCenter Server service still depends on it. The solution is to manually remove the dependency through the registry.
I don't know if it's required, but after the above fix, I re-ran the vCenter Server installation, which now completes without an error.
On a side-note, it's also important to have DNS resolution correctly set up. Unless you changed it, SSO and other services use DNS names to connect to the components, even when they are all installed on a single server. The DNS name you used must resolve to the IP address of the local machine, or else the various components won't be able to interact with each other. Test DNS resolving by pinging the FQDN entered in SSO and others; it should resolve to the proper IP address.