RE: Failed to decrypt



This is most probably because of the default key size difference. For RC2
(and RC4) please always explicitly set the key size via
CryptSetKeyParam(KP_EFFECTIVE_KEYLEN).

BTW you should not use RC2 or RC4 or even DES. If you can control both sides
at least use DES3, but preferably AES.

Laszlo Elteto
SafeNet, Inc.

"Charles Zhang" wrote:

I have a server application and client application.

The server creates a exchange key pair and the public exchange key is
transferred to client side.

The client side creates a session key then export it using the server's
public exchange key.

The server import the session key using its private exchange key.

Then the server and client use the session key for encryption.


However, when the session key handle is used to decrypt I get "Bad Data"
error in the case as outlined below

1. The client runs on Windows 2000 and the server runs Windows XP or
Windows 2003.
2. The CSP is "Microsoft Strong Cryptographic Provider"
3. The encryption algorithm is "CALG_RC2"
4. The session key is more than 40 bits ( tried 40 bits, 56 bits and 128
bits. 40 bits worked fine when 56 bits and 128 bits did not work).

By the way, when using the session key handle to encrypt a message on
the client side, no errors are returned.

When both client and server runs on Windows 2000, no errors returned.

When client runs on Windows XP and the server runs on Windows XP/Windows
2003, no errors returned.

I would like someone to point out about what I am doing wrong.

By the way, I am using "Cryptography API"

I use "CryptImportKey" to import session key then use CryptDecrypt to
descrypt messages.


Charles Zhang

Thanks

.