NUT-13 specify keyset ID integer size: 32 bits #189

2024-11-12T02:09:27Z

vnprc commented

2024-11-12 02:09:27 +00:00

(Migrated from github.com)

Is the intention of this integer representation of keyset ID to fit it into 32 bits? I ran into this implementation in cdk that produces a u64 output which seems wrong to me.

https://github.com/cashubtc/cdk/blob/main/crates/cdk/src/nuts/nut02.rs#L117

impl TryFrom<Id> for u64 {
    type Error = Error;
    fn try_from(value: Id) -> Result<Self, Self::Error> {
        let hex_bytes: [u8; 8] = value.to_bytes().try_into().map_err(|_| Error::Length)?;

        let int = u64::from_be_bytes(hex_bytes);

        Ok(int % (2_u64.pow(31) - 1))
    }
}

I am opening this PR to get some clarity.

Is the intention of this integer representation of keyset ID to fit it into 32 bits? I ran into this implementation in cdk that produces a u64 output which seems wrong to me. https://github.com/cashubtc/cdk/blob/main/crates/cdk/src/nuts/nut02.rs#L117 ``` impl TryFrom<Id> for u64 { type Error = Error; fn try_from(value: Id) -> Result<Self, Self::Error> { let hex_bytes: [u8; 8] = value.to_bytes().try_into().map_err(|_| Error::Length)?; let int = u64::from_be_bytes(hex_bytes); Ok(int % (2_u64.pow(31) - 1)) } } ``` I am opening this PR to get some clarity.

a1denvalu3 (Migrated from github.com) approved these changes

2024-11-12 16:43:58 +00:00

13.md

					
				@ -45,3 +44,3 @@

				The integer representation `keyset_id_int` of a keyset is calculated from its [hexadecimal ID][02] which has a length of 8 bytes or 16 hex characters. First, we convert the hex string to a big-endian sequence of bytes. This value is then modulo reduced by `2^31 - 1` to arrive at an integer that is a unique identifier `keyset_id_int`.

				The 32 bit integer representation `keyset_id_int` of a keyset is calculated from its [hexadecimal ID][02] which has a length of 8 bytes or 16 hex characters. First, we convert the hex string to a big-endian sequence of bytes. This value is then modulo reduced by `2^31 - 1` to arrive at an integer that is a unique identifier `keyset_id_int`.

I am noticing something else here that is not per-se an issue but it's awkward: why reduce by 2^31-1 instead of 2^31? But I think I know why:

q % 2^31 == q & (2^31-1). Normally when reducing modulo a power of 2 you can skip division and just use a mask to get the desired bits. Whoever wrote this first must have confused this or made a typo.
Now we can't change this back without breaking the protocol but I thought it was funny.

I am noticing something else here that is not per-se an issue but it's awkward: why reduce by `2^31-1` instead of `2^31`? But I think I know why: `q % 2^31 == q & (2^31-1)`. Normally when reducing modulo a power of 2 you can skip division and just use a mask to get the desired bits. Whoever wrote this first must have confused this or made a typo. Now we can't change this back without breaking the protocol but I thought it was funny.

prusnak commented

2024-11-12 18:31:06 +00:00

(Migrated from github.com)

NACK. This is implementation specific (some languages do not have 32-bit int) and the intention is obvious from the provided examples.

vnprc commented

2024-11-12 20:17:14 +00:00

(Migrated from github.com)

I disagree that it's obvious. I opened this PR because I found the language confusing. It seems to assume the term 'integer' means a 32 bit number. The python example is clear to me but the javascript example uses BigInt() twice to arrive at a number that fits into a regular int.

I think the language does a good job explaining that the input is 8 bytes or 16 hex chars but does not explicitly say the size of the output. The size of the container of the function output is an implementation specific detail but it would be helpful to explain that the output of this function fits into 32 bits or 4 bytes or 8 hex chars.

@prusnak would this language be better?

The integer representation keyset_id_int of a keyset is calculated from its hexadecimal ID which has a length of 8 bytes or 16 hex characters. First, we convert the hex string to a big-endian sequence of bytes. This value is then modulo reduced by 2^31 - 1 to arrive at a unique identifier keyset_id_int that can be stored in 4 bytes or 8 hex chars.

I disagree that it's obvious. I opened this PR because I found the language confusing. It seems to assume the term 'integer' means a 32 bit number. The python example is clear to me but the javascript example uses BigInt() twice to arrive at a number that fits into a regular int. I think the language does a good job explaining that the input is 8 bytes or 16 hex chars but does not explicitly say the size of the output. The size of the *container* of the function output is an implementation specific detail but it would be helpful to explain that the output of this function fits into 32 bits or 4 bytes or 8 hex chars. @prusnak would this language be better? > The integer representation `keyset_id_int` of a keyset is calculated from its [hexadecimal ID](https://github.com/cashubtc/nuts/blob/main/02.md) which has a length of 8 bytes or 16 hex characters. First, we convert the hex string to a big-endian sequence of bytes. This value is then modulo reduced by `2^31 - 1` to arrive at a unique identifier `keyset_id_int` that can be stored in 4 bytes or 8 hex chars.

clarkmoody commented

2024-11-12 20:25:43 +00:00

(Migrated from github.com)

I like the idea of specifying that the keyset ID should be able to fit into a 32-bit integer. Maybe the spec should just constraint the range of valid values?

prusnak commented

2024-11-12 20:26:58 +00:00

(Migrated from github.com)

... that can be stored in 4 bytes or 8 hex chars.

The value is not stored anywhere - it is just used as an input to BIP32 child key derivation (CKD) function. Therefore I find it irrelevant whether the value is stored in 32-bit int, 64-bit int, 4 bytes etc. Because all that matters is what is the data type that CKD is expecting (and yeah, in statically typed languages it is usually uint32, but can be literally any int type).

> ... that can be stored in 4 bytes or 8 hex chars. The value is not stored anywhere - it is just used as an input to BIP32 child key derivation (CKD) function. Therefore I find it irrelevant whether the value is stored in 32-bit int, 64-bit int, 4 bytes etc. Because all that matters is what is the data type that CKD is expecting (and yeah, in statically typed languages it is usually uint32, but can be literally any int type).

👍 1

vnprc commented

2024-11-12 21:16:51 +00:00

(Migrated from github.com)

Ok, sure. My goal in suggesting this change is to make it more clear to devs implementing this spec.

👍 1

callebtc (Migrated from github.com) reviewed

2024-12-03 11:55:30 +00:00

13.md

					
				@ -45,3 +44,3 @@

				The integer representation `keyset_id_int` of a keyset is calculated from its [hexadecimal ID][02] which has a length of 8 bytes or 16 hex characters. First, we convert the hex string to a big-endian sequence of bytes. This value is then modulo reduced by `2^31 - 1` to arrive at an integer that is a unique identifier `keyset_id_int`.

				The 32 bit integer representation `keyset_id_int` of a keyset is calculated from its [hexadecimal ID][02] which has a length of 8 bytes or 16 hex characters. First, we convert the hex string to a big-endian sequence of bytes. This value is then modulo reduced by `2^31 - 1` to arrive at an integer that is a unique identifier `keyset_id_int`.

Each extended key has 2^31 normal child keys, and 2^31 hardened child keys. Each of these child keys has an index. The normal child keys use indices 0 through 2^31-1. The hardened child keys use indices 2^31 through 2^32-1.

https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki

> Each extended key has 2^31 normal child keys, and 2^31 hardened child keys. Each of these child keys has an index. The normal child keys use indices 0 through 2^31-1. The hardened child keys use indices 2^31 through 2^32-1. https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki