CryptEncrypt() buffer limit?

From: Michael Prendergast (MichaelPrendergast_at_discussions.microsoft.com)
Date: 10/28/04 

Date: Thu, 28 Oct 2004 12:29:01 -0700

Hello,

I have a strange little bug that keeps occurring whenever I make a call to
CryptEncrypt():

It seems that whenever I call CryptEncrypt() to encrypt a plaintext buffer
of greater than 53 bytes, it fails and returns NTE_BAD_LEN through
GetLastError(). This seems independent of the actual buffer size I pass in.

For example, I tried getting the size required to encrypt a 58 byte buffer,
and gave CryptEncrypt() a size of 1000 for the actual buffer length, and
received NTE_BAD_LEN in response.

Here's some sample code:

---------------------------------------------------------------------
        ULONG ulEncryptedDataSizeInBytes = 58;

        if (CryptEncrypt(m_hRemotePublicKey, 0, TRUE, 0, NULL,
&ulEncryptedDataSizeInBytes, 1000) == FALSE)
        {
            hrReturnValue = GetLastError();
        }
    }
---------------------------------------------------------------------

I'm using MS_DEF_PROV as the cryptographic provider name, with PROV_RSA_FULL
and CALG_RSA_KEYX as the encryption algorithm (I'm using this encryption for
a public key exchange sequence).

Does anyone have any idea what could be wrong? Is there something I need to
set with SetKeyParam() first?

Also, as a side note, this works if I make repeated calls to CryptEncrypt()
with temporary copy buffers of less 32 bytes each.

Any help would be greatly appreciated.

Thank you very much for your help.

Ciao,
Michael Prendergast

Relevant Pages

  • RE: Decryption using private key from cert store failing with err
    ... also do not know why you recommended adding the cryptencrypt call. ... sent to the server which will call encrypt using the ... and clients, and we are trying to do the same for Windows using some of the ... when calling cryptdecrypt with a certs private key on a 117 byte buffer. ...
    (microsoft.public.platformsdk.security)
  • CryptEncrypt 3DES Encrypted Buffer Size
    ... I'm using CryptEncrypt to encrypt a buffer that's 16 bytes in length. ... another hardware implementation that doesn't use the CryptoAPI. ...
    (microsoft.public.platformsdk.security)
  • Re: Cryptographic Exception - Bad Data (DESCryptoServiceProvider)
    ... Encrypt the buffer (with an 8 byte DES key) and store the result in the ... the crypto stream is doing something that I do not want it to do ...
    (microsoft.public.dotnet.framework)
  • Re: CryptEncrypt 3DES Encrypted Buffer Size
    ... > I'm using CryptEncrypt to encrypt a buffer that's 16 bytes in length. ... > another hardware implementation that doesn't use the CryptoAPI. ...
    (microsoft.public.platformsdk.security)
  • [PATCH 03/04] Add encryption ops to the keyctl syscall
    ... +asmlinkage long sys_keyctl(int cmd, unsigned long arg2, ... + be encrypted/signed using the key payload. ... + to encrypt or sign the data and to return the result in outputbuf. ... + userspace if the buffer pointer is not NULL. ...
    (Linux-Kernel)