The timestamps of the depot files (the RCS and binary archive files under P4ROOT) do not affect the operation of the Perforce server. Restoring files from backup -- and updating the archive file timestamps in the process -- does not have any adverse effect on the Perforce server.
However, the operating system timestamps do affect the operation of older versions of the Perforce Proxy server. Beginning with version 2007.2, the Proxy uses MD5 comparisons, rather than file timestamp comparisons, to determine whether or not to serve a file from the cache. Versions of the Proxy prior to 2007.2 use timestamp comparisons. As a result, attempting to restore Perforce Proxy cache files from backup might lead to cache files with misleading timestamps (either older or newer than the primary server files). Misleading timestamps can result in files being unnecessarily fetched from the server instead of from the Proxy cache.
Rather than attempting to restore the Proxy cache from backup, it is recommended that the Proxy cache simply be "re-seeded" by syncing the entire server through the proxy, or copying the primary server archive files into the appropriate Proxy cache location. More information on this process can be found in KB article Installing a Proxy Server.