All we know is that your OS is malfunctioning. The issues you're describing are too general to reach a conclusion. It could be a hardware issue, a software issue or malware of some sort.
It sounds like your OS is too unstable to troubleshoot from within. The following will allow you to troubleshoot from outside your OS:
Kaspersky Rescue Disk 10 Ultimate Boot CD - Overview
Sometimes a RAM issue is related to heat or another factor not present when running a memory test. Is the computer in a hot room? Is airflow in and out of the computer restricted in any way?