Diffie Hellman vs RSA
RSA is two algorithms, one for asymmetric encryption, and one for digital signatures. These are two distinct beast; although they share the same core mathematical operation and format for keys, they do different things in different ways. Diffie-Hellman is a key exchange algorithm, which is yet another kind of algorithm. Since the algorithms don’t do the same thing, you could prefer one over the other depending on the usage context.
Asymmetric encryption and key exchange are somewhat equivalent: with asymmetric encryption, you can do a key exchange by virtue of generating a random symmetric key (a bunch of random bytes) and encrypting that with the recipient’s public key. Conversely, you can do asymmetric encryption with key exchange by using the key resulting from the key exchange to encrypt data with a symmetric algorithm, e.g. AES. Moreover, Diffie-Hellman is a one-roundtrip key exchange algorithm: recipient sends his half (“DH public key”), sender computes his half, obtains the key, encrypts, sends the whole lot to the recipient, the recipient computes the key, decrypts. This is compatible with a one-shot communication system, assuming a pre-distribution of the public key, i.e. it works with emails.
The shared key is an asymmetric key, but, like all asymmetric key systems, it is inherently slow and impractical for bulk encryption. The key is used instead to securely exchange a symmetric key, such as AES (Advanced Encryption Standard) used to encrypt subsequent communications. Unlike Diffie-Hellman, the RSA algorithm can be used for signing digital signatures as well as symmetric key exchange, but it does require the exchange of a public key beforehand.
RSA and Diffie-Hellman are both based on supposedly intractable problems, the difficulty of factoring large numbers and exponentiation and modular arithmetic respectively, and with key lengths of 1,024 bits, give comparable levels of security. Both have been subjected to scrutiny by mathematicians and cryptographers, but given correct implementation, neither is significantly less secure than the other.
The nature of the Diffie-Hellman key exchange does make it susceptible to man-in-the-middle attacks since it doesn’t authenticate either party involved in the exchange. This is why Diffie-Hellman is used in combination with an additional authentication method, generally digital signatures. When using RSA, a 1,024-bit key is considered suitable both for generating digital signatures and for key exchange when used with bulk encryption, while a2048-bit key is recommended when a digital signature must be kept secure for an extended period of time, such as a certificate authority’s key.
Getting back to the question at hand, you can’t really merge the two algorithms because of the unique attributes and complexity that each one has. Most encryption systems offer a choice between them rather than combining them. SSL 3.0 supports a choice of key exchange algorithms, including the RSA key exchange when certificates are used, and Diffie-Hellman key exchange for exchanging keys without certificates and without prior communication between client and server.