We suggest you upgrade your system kernels as soon as possible, putting a higher priority on shared hosting servers is advised.
Most systems will simply upgrade via yum, however your datacentre may have a local yum repository, so the new kernels may not be available yet (This has been reported by a few users already). We suggest you add a standard CentOS mirror to your yum configuration file to overcome this situation.
OpenVZ have a patched kernel available in there yum repository (2.6.18-194.8.1.el5.028stab070.5) so a yum update should allow you to install the new kernel, if not, the kernel is available for download here: http://wiki.openvz.org/Download/kernel/rhel5/028stab070.5
CentOS also have a new kernel release that patches the vulnerability in the standard and Xen kernels (2.6.18-194.11.4.el5), which is available via yum also.