Modern Graphics Processing Units (GPU) have reached a dimension with respect to performance and gate count exceeding conven- tional Central Processing Units (CPU) by far. Many modern computer systems include - beside a CPU - such a powerful GPU which runs idle most of the time and might be used as cheap and instantly available co-processor for general purpose applications. In this contribution, we focus on the efficient realisation of the com- putationally expensive operations in asymmetric cryptosystems on such off-the-shelf GPUs. More precisely, we present improved and novel imple- mentations employing GPUs as accelerator for RSA and DSA cryptosys- tems as well as for Elliptic Curve Cryptography (ECC). Using a recent Nvidia 8800GTS graphics card, we are able to compute 813 modular ex- ponentiations per second for RSA or DSA-based systems with 1024 bit integers. Moreover, our design for ECC over the prime field P-224 even achieves the throughput of 1412 point multiplications per second.