Crypto API and Windows 98 SE

From: Daniel (no_spam_sonofsmog_at_no_spam_hotmail.com)
Date: 10/11/05 

Date: Mon, 10 Oct 2005 16:41:15 -0700

I have written a DLL in VB6 that performs key pair decrypt operations
followed by a key pair signature verification. The DLL performs these
operations repeatedly on various pieces of data which need to be decrypted
and their signature checked. The DLL works flawlessly on Windows 2000.
However on the target system, Windows 98, it behaves unpredictable.
Sometimes the operatios are successful, but about half the time the
CryptImportKey fails when importing a private key for decryption or more
often a public key for signature verification and the whole thing GPF's.
After that the CSP CryptAcquireContext often fails when try to create a new
or use an existing context (with an error that indicates there is a problem
with the underlying DLL) I have tried everything I can think of as far as
CryptAcquireContext goes, creating a new container, grabbing a reference to
a new container, passing NULL as the provider, specifying the base and the
enhanced providers, setting the container name to NULL. You name it.

The relavent snips of code look like this.

-- acquire a context

' try to open the existing container
        lngReturnValue = CryptAcquireContext(hCryptProv, KEY_CONTAINER,
SERVICE_PROVIDER_ENH, PROV_RSA_FULL, CRYPT_VERIFYCONTEXT)
        If lngReturnValue = 0 Then
               lngReturnValue = CryptAcquireContext(hCryptProv,
KEY_CONTAINER, SERVICE_PROVIDER_ENH, PROV_RSA_FULL, CRYPT_NEWKEYSET)

-- import a private key for decryption or a public key for signature
checking

Public Sub Import_KeyPair(Optional ByVal PrivateKeyPassword As String =
vbNullString, Optional ByVal UseExchangeKeyPair As Boolean = True)
'import keys
Dim lngReturnValue As Long
Dim lngLength As Long
Dim lngParams As Long

'check to see if there are no keys available at all
If strPublicPrivateBlob = vbNullString And strPublicBlob = vbNullString Then
        Err.Raise vbObjectError + 1, , "One of the ValueXXXKey properties
must hold a valid key"
End If

'release old key pair handle
If hKeyPair <> 0 Then
        CryptDestroyKey hKeyPair
End If

If UseExchangeKeyPair Then 'set the type of key pair (Exchange key pair or
Signature key pair keys)
    lngType = AT_KEYEXCHANGE
Else
    lngType = AT_SIGNATURE
End If

If strPublicPrivateBlob = vbNullString Then 'must be a Public key
    'import the key
    lngLength = Len(strPublicBlob)
    lngReturnValue = CryptImportKey(hCryptProv, strPublicBlob, lngLength, 0,
0, hKeyPair)
    If lngReturnValue = 0 Then
                Err.Raise Err.LastDllError, , "DLL error code shown above.
Could not import the Public key"
    End If

Else 'must be a PublicPrivate key
    'get a session key handle from the password to unlock the
    'private key, which is encrypted

    SessionKeyFromPassword PrivateKeyPassword

    'import the key
    lngLength = Len(strPublicPrivateBlob)
    lngParams = CRYPT_EXPORTABLE Or CRYPT_NO_SALT
        lngReturnValue = CryptImportKey(hCryptProv, strPublicPrivateBlob,
lngLength, hSessionKey, lngParams, hKeyPair)
    If lngReturnValue = 0 Then
        Err.Raise Err.LastDllError, , "DLL error code shown above. Could not
import the PublicPrivate key. Most likely an incorrect private key password
was entered."
    End If
End If

'release session key handle
If hSessionKey <> 0 Then
        CryptDestroyKey hSessionKey
    hSessionKey = 0
End If
End Sub

-- get a session key handle from the password to unlock the private key,
which is encrypted

Private Sub SessionKeyFromPassword(ByVal Key As String)
'This sub takes a string key as input and sets the module-level
'hSessionKey variable to a new session key handle.
'This sub is used by EncryptDecrypt, Export_KeyPair and Import_KeyPair.
Dim lngParams As Long
Dim lngReturnValue As Long
Dim strHash As String
Dim lngHashLen As Long
Dim hHash As Long 'the handle to the hash object

'Create a hash object to calculate a session
'key from the Password (instead of encrypting
'with the actual key)
lngReturnValue = CryptCreateHash(hCryptProv, CALG_SHA, 0, 0, hHash)
If lngReturnValue = 0 Then Err.Raise Err.LastDllError, , "DLL error code
shown above. Could not create a Hash Object (CryptCreateHash API)"
'can use CALG_MD5 to get a 128-bit hash. CALG_SHA returns a 160-bit hash
(more secure).

'Hash the Password
lngReturnValue = CryptHashData(hHash, Key, Len(Key), 0)
If lngReturnValue = 0 Then Err.Raise Err.LastDllError, , "DLL error code
shown above. Could not calculate a Hash Value (CryptHashData API)"
'A hash is a 'fingerprint' of any string.
'Hashes are extremely useful for determining whether a
'transmission or file has been altered. This code can use
'one of two algorithms (see note above). No matter what the
'length of input data, the hash will be a fixed length and
'will be unique for that string of data. The same hash is produced for
'the same input data every time. This is useful here to
'produce a fixed-length, unique password for any length password entered.

'Get the actual hash value
lngReturnValue = CryptGetHashParam(hHash, HP_HASHVAL, vbNull, lngHashLen, 0)
'get the hash length
strHash = String(lngHashLen + 1, vbNullChar)
lngReturnValue = CryptGetHashParam(hHash, HP_HASHVAL, strHash, lngHashLen,
0) 'get the hash value
If lngReturnValue = 0 Then Err.Raise Err.LastDllError, , "DLL error code
shown above. Could not lngReturnValuerieve the hash value"

'Set certain values to add more flexibility and security.
'Make the key exportable. (I don't export the key in this sample code)
lngParams = CRYPT_EXPORTABLE 'use this when you generate your own SALT,
which is recommended (see 8 lines below)
'Make the key exportable and add a system-generated SALT.
'use this line of code instead of the one above if you want the API to set
the SALT...
'but the SALT is the same every time so this shouldn't be used!
'Note that I generate my own random SALT below.
'lngParams = CRYPT_EXPORTABLE Or CRYPT_CREATE_SALT

'release old session key handle if one exists
If hSessionKey <> 0 Then
    CryptDestroyKey hSessionKey
'hmmm hSessionKey = 0
End If

'Derive a session key from the hash object
lngReturnValue = CryptDeriveKey(hCryptProv, CALG_RC4, hHash, lngParams,
hSessionKey)
If lngReturnValue = 0 Then Err.Raise Err.LastDllError, , "DLL error code
shown above. Could not create a session key (CryptDeriveKey API)"

'Destroy the hash object
If hHash <> 0 Then
    CryptDestroyHash hHash
'hmmm hHash = 0
End If

End Sub

I have tried releasing a getting a new Crypto context between decrypting and
checking the signature, but still no luck. Any help would be GREATLY
appreciated! 

-- 
Daniel Blackmon
Project Lead - Software Engineer

Relevant Pages

  • Re: rsa implementation question
    ... > There is a notion of blocks in many public-key ciphers, ... It's not about decrypting to sign, encrypting to ... as it would mean that you'd have to find hash collisions. ... I generate a signature for a string "some string" with SHA. ...
    (comp.lang.python)
  • how to verify signature with DSACryptoServiceProvider
    ... computer) and DSACryptoServideProvider for signature of the hash (my ... signed hash to the end of the encrypted file. ... int securedSaltLength = bReader.ReadInt32; ...
    (microsoft.public.dotnet.languages.csharp)
  • Re: Sign with RsaCryptoService Provider Verify with win32 Crypto A
    ... Normally the signature should contain ... hash), so I think the .Net version of the code always put the hash id into ... What flags are you using in CryptSignHash and CryptVerifySignature ... I specify CRYPT_NOHASHOID in CryptSignHash and CryptVerifySignature ...
    (microsoft.public.platformsdk.security)
  • Re: PGP clearsign question
    ... > outlined that would make cracking that key easier? ... You are anyway using the hash of the message and not the message ... > appears to be signed by the private key, ... to be unique for every signature - for example when using DSA or ElGamal ...
    (sci.crypt)
  • Re: Can you use ECC to produce digital signatures? It doesnt see so.
    ... Hash: SHA1 ... it doesn't appear to me that you can use ECC ... | key on a SHA-1 hash of the message to produce a digital signature. ... You have a private key k which is just an integer and a public ...
    (sci.crypt)