eBay Digital Signatures for APIs signature header generation – Ebay API

Having read through eBay’s guide for including digital signatures to certain of their REST API calls, I am having trouble with generating the signature header. Rather than including all of the documentation here (there is a lot!), I’ll provide links to the appropriate pages and some of the documentation. The following page it the starting point provided by eBay:
https://developer.ebay.com/develop/guides/digital-signatures-for-apis
The next page is where I am lead to from the previous page describing how to create the signature:
https://www.ietf.org/archive/id/draft-ietf-httpbis-message-signatures-13.html#name-eddsa-using-curve-edwards25
Which leads me onto the following :
https://www.rfc-editor.org/rfc/rfc8032#section-5.1.6

5.1.6.  Sign

   The inputs to the signing procedure is the private key, a 32-octet
   string, and a message M of arbitrary size.  For Ed25519ctx and
   Ed25519ph, there is additionally a context C of at most 255 octets
   and a flag F, 0 for Ed25519ctx and 1 for Ed25519ph.

   1.  Hash the private key, 32 octets, using SHA-512.  Let h denote the
       resulting digest.  Construct the secret scalar s from the first
       half of the digest, and the corresponding public key A, as
       described in the previous section.  Let prefix denote the second
       half of the hash digest, h[32],...,h[63].

   2.  Compute SHA-512(dom2(F, C) || prefix || PH(M)), where M is the
       message to be signed.  Interpret the 64-octet digest as a little-
       endian integer r.

   3.  Compute the point [r]B.  For efficiency, do this by first
       reducing r modulo L, the group order of B.  Let the string R be
       the encoding of this point.

   4.  Compute SHA512(dom2(F, C) || R || A || PH(M)), and interpret the
       64-octet digest as a little-endian integer k.

   5.  Compute S = (r + k * s) mod L.  For efficiency, again reduce k
       modulo L first.

   6.  Form the signature of the concatenation of R (32 octets) and the
       little-endian encoding of S (32 octets; the three most
       significant bits of the final octet are always zero).

I have some Python code from the appendix from this same web page (https://www.rfc-editor.org/rfc/rfc8032#section-6):

## First, some preliminaries that will be needed.

import hashlib

def sha512(s):
    return hashlib.sha512(s).digest()

# Base field Z_p
p = 2**255 - 19

def modp_inv(x):
    return pow(x, p-2, p)

# Curve constant
d = -121665 * modp_inv(121666) % p

# Group order
q = 2**252 + 27742317777372353535851937790883648493

def sha512_modq(s):
    return int.from_bytes(sha512(s), "little") % q

## Then follows functions to perform point operations.

# Points are represented as tuples (X, Y, Z, T) of extended
# coordinates, with x = X/Z, y = Y/Z, x*y = T/Z

def point_add(P, Q):
    A, B = (P[1]-P[0]) * (Q[1]-Q[0]) % p, (P[1]+P[0]) * (Q[1]+Q[0]) % p;
    C, D = 2 * P[3] * Q[3] * d % p, 2 * P[2] * Q[2] % p;
    E, F, G, H = B-A, D-C, D+C, B+A;
    return (E*F, G*H, F*G, E*H);


# Computes Q = s * Q
def point_mul(s, P):
    Q = (0, 1, 1, 0)  # Neutral element
    while s > 0:
        if s & 1:
            Q = point_add(Q, P)
        P = point_add(P, P)
        s >>= 1
    return Q

def point_equal(P, Q):
    # x1 / z1 == x2 / z2  <==>  x1 * z2 == x2 * z1
    if (P[0] * Q[2] - Q[0] * P[2]) % p != 0:
        return False
    if (P[1] * Q[2] - Q[1] * P[2]) % p != 0:
        return False
    return True

## Now follows functions for point compression.

# Square root of -1
modp_sqrt_m1 = pow(2, (p-1) // 4, p)

# Compute corresponding x-coordinate, with low bit corresponding to
# sign, or return None on failure
def recover_x(y, sign):
    if y >= p:
        return None
    x2 = (y*y-1) * modp_inv(d*y*y+1)
    if x2 == 0:
        if sign:
            return None
        else:
            return 0

    # Compute square root of x2
    x = pow(x2, (p+3) // 8, p)
    if (x*x - x2) % p != 0:
        x = x * modp_sqrt_m1 % p
    if (x*x - x2) % p != 0:
        return None

    if (x & 1) != sign:
        x = p - x
    return x


# Base point
g_y = 4 * modp_inv(5) % p
g_x = recover_x(g_y, 0)
G = (g_x, g_y, 1, g_x * g_y % p)

def point_compress(P):
    zinv = modp_inv(P[2])
    x = P[0] * zinv % p
    y = P[1] * zinv % p
    return int.to_bytes(y | ((x & 1) << 255), 32, "little")

def point_decompress(s):
    if len(s) != 32:
        raise Exception("Invalid input length for decompression")
    y = int.from_bytes(s, "little")
    sign = y >> 255
    y &= (1 << 255) - 1

    x = recover_x(y, sign)
    if x is None:
        return None
    else:
        return (x, y, 1, x*y % p)

## These are functions for manipulating the private key.

def secret_expand(secret):
    if len(secret) != 32:
        raise Exception("Bad size of private key")
    h = sha512(secret)
    a = int.from_bytes(h[:32], "little")
    a &= (1 << 254) - 8
    a |= (1 << 254)
    return (a, h[32:])

def secret_to_public(secret):
    (a, dummy) = secret_expand(secret)
    return point_compress(point_mul(a, G))


## The signature function works as below.

def sign(secret, msg):
    a, prefix = secret_expand(secret)
    A = point_compress(point_mul(a, G))
    r = sha512_modq(prefix + msg)
    R = point_mul(r, G)
    Rs = point_compress(R)
    h = sha512_modq(Rs + A + msg)
    s = (r + h * a) % q
    return Rs + int.to_bytes(s, 32, "little")

## And finally the verification function.

def verify(public, msg, signature):
    if len(public) != 32:
        raise Exception("Bad public key length")
    if len(signature) != 64:
        Exception("Bad signature length")
    A = point_decompress(public)
    if not A:
        return False
    Rs = signature[:32]
    R = point_decompress(Rs)
    if not R:
        return False
    s = int.from_bytes(signature[32:], "little")
    if s >= q: return False
    h = sha512_modq(Rs + public + msg)
    sB = point_mul(s, G)
    hA = point_mul(h, A)
    return point_equal(sB, point_add(R, hA))

Now, the problem that I am having is that this code insists on the "secret" consisting of a 32 byte array:

if len(secret) != 32: raise Exception("Bad size of private key")

However, the secret is described as being the private key provided by eBay’s Key Management API (https://developer.ebay.com/api-docs/developer/key-management/overview.html), which is not a 32 byte array, but a 64 character ASCII string (see https://developer.ebay.com/api-docs/developer/key-management/resources/signing_key/methods/createSigningKey#h2-samples):
"privateKey": "MC4CAQAwBQYDK2VwBCIEI******************************************n"

When I try to generate a signature with the eBay private key using this Python code, it gives me an error saying it is a "Bad size of private key". If I convert the private key from eBay to a bytearray, it is 64 bytes long. How can I use the Python code to generate the signature header using the private key supplied by eBay?

To further complicate things, I am actually using Excel VBA (Visual Basic) to make the API call after using Python to generate the signature (simply because Python is better at this kind of thing!). eBay’s PAID FOR technical support has confirmed that the following headers are correct and that there is no "message" as described in https://www.rfc-editor.org/rfc/rfc8032#section-5.1.6, but they have not yet been of any further help other than suggesting that there may be a "bug".

http.setRequestHeader "signature-input", "sig1=(""x-ebay-signature-key"" ""@method"" ""@path"" ""@authority"");created=1667386210"
http.setRequestHeader "x-ebay-signature-key", "<jwe returned by eBay>"
http.setRequestHeader "x-ebay-enforce-signature", "true"

The remaining header would be as follows once I can generate a valid signature:

http.setRequestHeader "signature" "sig1=:<signature>:"

Everything I have tried results in the same response:

{
  "errors": [
    {
      "errorId": 215122,
      "domain": "ACCESS",
      "category": "REQUEST",
      "message": "Signature validation failed",
      "longMessage": "Signature validation failed to fulfill the request."
    }
  ]
}

Here are some example keys like the ones generated by eBay. https://www.ietf.org/archive/id/draft-ietf-httpbis-message-signatures-11.html#appendix-B.1.4

"The following key is an elliptical curve key over the Edwards curve ed25519, referred to in this document as test-key-ed25519. This key is PCKS#8 encoded in PEM format, with no encryption."

-----BEGIN PUBLIC KEY-----
MCowBQYDK2VwAyEAJrQLj5P/89iXES9+vFgrIy29clF9CC/oPPsw3c5D0bs=
-----END PUBLIC KEY-----

-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIJ+DYvh6SEqVTm50DFtMDoQikTmiCqirVv9mWG9qfSnF
-----END PRIVATE KEY-----

This is the format of private key that I believe that I need to convert to a 32-byte array to work with the above Python code. I believe that there is a typo on the linked to web page and it should be "PKCS", not "PCKS".

UPDATE:
If I run the following command:

openssl ec -in test.pem -text

Where test.pem is a text file containing:

-----BEGIN PRIVATE KEY-----
MC4CAQAwBQYDK2VwBCIEIJ+DYvh6SEqVTm50DFtMDoQikTmiCqirVv9mWG9qfSnF
-----END PRIVATE KEY-----

It displays private and public keys as 32 byte hex dumps, but even when using these values I get the same response as above with the 215122 error. When I verify using the Python "verify" method in the code above with these 32 byte hex dump keys, validation is successful.