GPUs can be used to perform parallel computation. The idea is to use the thousands of CPUs the GPUs have nowdays. The two APIs used for this purpose are CUDA (developed by nVidia) and OpenCL. Both APIs are witten in C/C++, but you can also use them from Python through PyCUDA and PyOpenCL. Here are some examples: