Cashu v1 – The Great Cleanup #55
No reviewers
Labels
No labels
breaking change
bug
documentation
enhancement
needs discussion
needs implementation
new nut
ready
wallet-only
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
forgejo-admin/nuts!55
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "the_great_cleanup_v1"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The Great Cleanup
This is a PR that encapsulates a collection of changes to the protocol that will make our lives easier in the future. It cleans up early mistakes that were made as the protocol grew organically and removes implicit assumptions about the funding sources (e.g. Lightning) and currency units (e.g. "sats") used for Cashu mints that are inherent in the current protocol.
New
v1APIWe are introducing the
v1API where we clean up many input and output models.Click to show new endpoints
New endpoints
GET /v1/keysandGET /v1/keys/{keyset_id}(newKeysetmodel)GET /v1/keysets(newKeysetsmodel)POST /v1/swap(newBlindedMessagemodel)POST /v1/melt/quote/bolt11(replacesGET /checkfees)POST /v1/melt/bolt11POST /v1/mint/quote/bolt11(replacesGET /melt)POST /v1/mint/bolt11POST /v1/check(no changes)GET /v1/info(no changes)POST /v1/restore(newGetInfoResponse)Click to show error response
Error responses
Error responses of the mint are now defined as a HTTP 400 response with a JSON body
Click to show deprecated endpoints
Deprecated endpoints
GET /keys,GET /keys/{keyset_id}GET /keysetsPOST /splitGET /mintPOST /mintGET /checkfeesPOST /meltPOST /checkGET /infoPOST /restoreNUTs
GET /checkfeesendpoint (formerly NUT-03) is now retiredPOST /v1/swapendpoint is now in NUT-03 (formerlyPOST /splitNUT-06).New request and response models
Click to show all changed models
NUT-01
GetKeysResponseNUT-02
GetKeysetsResponseNUT-03
PostSwapRequestPostSwapResponseNUT-04
PostMintQuoteBolt11RequestPostMintQuoteBolt11ResponsePostMintBolt11RequestPostMintBolt11Response
NUT-05
PostMeltQuoteBolt11RequestPostMeltQuoteBolt11ResponsePostMeltBolt11RequestPostMeltBolt11ResponseQuotes
We are also introducing
quotes, a general way to register a mint and melt transaction with the mint that will work across different payment methods (bolt11, bolt12, on-chain, ...) and different currency units (sat, msat, usd, ...). Quotes add the ability for Lightning backends to decide the amount and currency of the ecash they need in order to receive or pay a Lightning payment. The flow remains very similar to before and will be illustrated with two examples in the following.Mint Quotes
To mint ecash (output) via Lightning (input), the wallet first requests a
MintQuotefor a given amount of sats the wallet wants to mint. TheMintQuotehas anquoteid and includes a Lightning invoice. The user pays the Lightning invoice and then calls/v1/mintreferencing the previousquoteit corresponds to.Melt Quotes
To melt ecash (input) and make a Lightning payment (output), the wallet requests a
MeltQuotefor a given Lightning invoice it likes to pay. In theMeltQuote, the mint tells the wallet how many sats it needs to supply and what the fee reserve is in order for the mint to fulfill this request.Diagram
Hexadecimal keyset IDs
Our keyset IDs are ugly and need special treatment for HTTP (base64 urlsafe). We switch to hexadecimal keyset IDs that are generated much like the previous ones. We also add a version byte as a prefix.
An example implementation in Python:
BlindedMessagenow has keysetidfieldOutputs (
BlindedMessages), now also have a keysetidfield, likeProofs(inputs) andBlindSignaturesdo. With theid, the wallet tells the mint which keyset the client is expecting a signature from during a/v1/swap(created outputs),/v1/melt(change outputs), or/v1/mint(minted outputs). The requestedidMUST be from anactivekeyset (part of the/v1/keysresponse and the/v1/keysetsresponse). If the wallet uses anidthat is not existent or not active (rotated-out of), the mint MUST refuse the transaction.The
BlindedMessagebecomesClick to show additional remarks
Additional remarks
Adding an
idfixes a race condition we previously created workarounds for (see https://github.com/cashubtc/cashu-ts/pull/64 for example). Essentially, the mint's keys could have rotated between the wallet sending the outputs to sign to the mint, and the mint responding with a signature. We can now get rid of this code by making it part of the protocol.Another critical issue that results from the same race condition is deterministic secret derivation: If a wallet deterministically derives secrets for keyset A, sends
BlindedMessagesto the mint and the mint rotated keys in the mean time, it would respond withBlindedSignatureskeyset B. That means the wallet has incremented its deterministic secret derivation counter on the wrong keyset ID.Standardized secrets
Wallets should use standardized secrets (32 bytes of randomness) in lowercase hex. Closes https://github.com/cashubtc/nuts/issues/54
@ -27,4 +32,4 @@With curl:```bash👀
@ -53,0 +61,4 @@"unit": "sat","active": True},{This would imply that a single token can have proofs from multiple keysets.
Is there a use case for this? We could reduce the payload if we don't repeat the keyset id in a response.
While we're refactoring you know..... 😅
@ -128,2 +84,3 @@##### Example JSON:### ErrorsIn case of an error, mints respond with the HTTP status code `400` and include the following data in their response:Since the old tokens are removed from the nut I think saying "new" is confusing.
@ -5,3 +4,4 @@`mandatory`---Since keyset is being changed to support multiple units will this request always be in satoshis?
@ -5,3 +4,4 @@`mandatory`---Being able to specify unit is interesting... it can allow denominations of millisats and maybe even (don't shoot me) bits.
@ -53,0 +61,4 @@"unit": "sat","active": True},{Yes, this is possible according to the TokenV3 encoding. Some wallets use this to export their entire balance from multiple mints with a single token. Not sure how useful that is, something we might deprecate for a next encoding version.
@ -5,3 +4,4 @@`mandatory`---The new
GET /v1/mintallows wallets to specifyamount(example:1234), currencyunit(example:usd), and payment methodmethod(example:bolt11) in the request body.@ -5,3 +4,4 @@`mandatory`---In that case "in satoshis" should be removed I think.
A few overall comments:
Overall some great improvements in there (also lots of new code to write for us! 😅). I suggest that if there is intention to eventually move forward with a solution to #54, it should be included here as part of this fairly wide-ranging set of breaking changes.
@ -126,3 +84,1 @@This token format includes information about the mint as well. The field `proofs` is like a V1 token. Additionally, the field `mints` can include an array (list) of multiple mints from which the `proofs` are from. The `url` field is the URL of the mint. `ids` is a list of the keyset IDs belonging to this mint. It is important that all keyset IDs of the `proofs` must be present here to allow a wallet to map each proof to a mint.##### Example JSON:### ErrorsGood call on removing those.
@ -192,4 +133,4 @@```json{"token": [Not sure where this should be mentioned but I don't want to forget it so I'm commenting here:
Because the keyset IDs are now expressed as hex but the wallets are likely to use them as string identifiers, the capitalization of the hex matters (as opposed to the secrets, where only the underlying bytes matter). We should have somewhere in NUT-02 a line expressing that the keyset ids should always be lowercase (or uppercase if we prefer, but my personal preference is lowercase) when stored/sent over the wire.
@ -27,4 +32,4 @@With curl:```bashMay I suggest the spec use a valid keyset json object? This would allow for initial cross-checking of one's understanding (even though at this point you could also test/check using the test vector associated with NUT-01. All that's required here is to remove the ellipsis and calculate the actual keyset ID of this small keyset and replace the current one with it's correct counterpart.
This PR is called the great cleanup so I'll point this out: the files include links that are not valid, and I think that could be cleaned up here. My preference would be to add links at the bottom as needed in the document above instead of pre-emptively adding an arbitrary number (20) of them, for which many are non-existent NUTs.
@ -53,0 +59,4 @@{"id": "009a1f293253e41e","unit": "sat","active": TrueI don't think the examples in other files are showcasing this version byte (they don't start with
00). Maybe adding a line as to why this version byte is useful might be good. Or you could just state it in the PR maybe so that we have a reference for later as per the thinking behind it.My preference would be to hash the concatenated bytes of all public keys. That way it's all bytes until the very end, where you pull the first 8 bytes of the hash and use their hex-encoded format.
Should this be
.hexdigest()[:16]instead?Should these use the
v1/paths? Wondering if communication with the mint using the old non-v1 paths are expected to stay with the base64 and only the v1 paths are expected to migrate to the new keyset ids.Because this is the response to the
keys/<keysetid>GET request, I expected this response to not be an array of"keysets"since there can only be one (IIUC). Is there a reason for it to be that way? I'm thinking maybe this would work just as well:@ -53,0 +61,4 @@"unit": "sat","active": True},{I agree that the "multiple mints per token" makes things more complicated than they could be, particularly because the use case for it is unclear. If you export your entire balance from multiple mints in a single token... at this point you're just exporting a data bundle; I think the word token looses its meaning and strength if it is too wide of a definition. Just my 2 cents. But like is stated above, this doesn't need to be solved here of course. Might just be something to think about for future refactorings.
@ -12,3 +25,3 @@## ExampleRequest of `Alice`:**Request** of `Alice`:Typo:
stealingsteal.I think this part was forgotten. Did you mean to finish this or remove it?
@ -19,1 +32,4 @@With the data being of the form `PostSwapRequest`:```jsonI think it would be good to add an example of the v1 request body.
@ -15,2 +14,3 @@# Mint quoteRequest of `Alice`:To request a mint quote, the wallet of `Alice` makes a `POST /v1/mint/quote/{method}` request where `method` is the payment method requested (here `bolt11`).This might be a v1 path instead of the old one. Or does that change apply to the old requests as well?
@ -36,0 +54,4 @@Response of `Bob`:```jsonAgain just a reminder that the keyset ids don't have the
00prefix mentioned in NUT-02.Great discussions over the dev call the other day. I figured I'd add a small todo list of things that might need to be addressed here before I forget. Note that not all of those might be required, and I'm going off of memory so hopefully not missing anything.
/info/endpoint is future-proof and potentially adds messaging around breaking changes and supported featuresmint/andmelt/endpoints and their definition in the NUTs (are they good as is or do they now require their own spec file?)unitfield toProofobject, requiring a new token format version (cashuB)v1routes@ -53,0 +59,4 @@{"id": "009a1f293253e41e","unit": "sat","active": TrueThis is my preference as well but the example code below it looks like it is concatenating the string of hex encoded pubkey.
+1
@thunderbiscuit commented this previously but it was removed when the change to bytes was pushed. Step 4 states to take the first 16 characters of the hex encoded hash but the example implementation seems to take the first 14. I think it step 4 should take the first 14 characters, making the id 16 after the version is added.
@ -53,0 +59,4 @@{"id": "009a1f293253e41e","unit": "sat","active": TrueThis is my understanding as well, and I would prefer this response unless there is a reason for the list.
@ -39,0 +80,4 @@```bashcurl -X GET http://localhost:3338/v1/mint/quote/bolt11/DSGLX9kevM...```Can't comment on the correct line as it wasn't changed in this PR, but on line 77 [NUT-0] should be [NUT-00]
I've completed most of the Todos! I would suggest deferring these remaining suggestions to later since the cleanup in this round focusses more on breaking changes in the API.
I think these can be added as we go. What we should probably do already now is to indicate which (method, unit) pairs a mint supports.
We can do it in cashuA and add a
unitfield in the JSON without breaking the formatI would love this but I'm afraid it needs to be in another PR
Thanks for the detailed comments and many errors you've found!
@ -7,3 +7,3 @@This describes the basic exchange of the public mint keys that the wallet user `Alice` uses to unblind `Bob`'s signature.This document outlines the exchange of the public keys of the mint `Bob` with the wallet user `Alice`. `Alice` uses the keys to unblind `Bob`'s blind signatures (see [NUT-00][00]).Should a mint have only on active keyset per unit?
@ -122,4 +149,2 @@Note that the mint needs to convert the URL-safe id back from `L3zxxRB_I8uE` to `L3zxxRB/I8uE` before it can look up the keys and respond to the request.[00]: 00.mdI think a test vector for id generation should be included as part of this PR
@ -1,40 +1,85 @@NUT-03: Request mintNUT-03: Swap tokensinvalidates should be invalidate
@ -3,3 +3,3 @@`mandatory` `author: calle``mandatory`I agree with limiting this nut to this. However, should a note be added on how this maybe expanded in the future or wait and define that when it happens?
@ -3,3 +3,3 @@`mandatory` `author: calle``mandatory`Is a call to the check fees endpoint still needed as the
fee_reserveis included in the quote?@ -1,7 +1,7 @@NUT-08: Lightning fee returnI wonder if this should be made more general. I think most payment methods with fees will work in a similar way where the request is over paid and the change is returned as in cashu
How is this derived?
@ -7,3 +7,3 @@This describes the basic exchange of the public mint keys that the wallet user `Alice` uses to unblind `Bob`'s signature.This document outlines the exchange of the public keys of the mint `Bob` with the wallet user `Alice`. `Alice` uses the keys to unblind `Bob`'s blind signatures (see [NUT-00][00]).yep. more than one just splits the anon set.
special case could be multiple currencies in the future (even with sidechains it's technically a floating exchange rate)
A general observation: in any context that is not obviously a keyset structure naming the keyset_id id is not intuitive at all.
ksid or keyset or keyset_id would probably be more appropriate. the other field names are pretty self explanatory except for the single letter stuff.
@ -198,3 +139,3 @@"proofs": List[Proof]"proofs": Proofs},...Think the example proof secrets here should be updated to the recommended 32 byte hex secret, even though that is not enforced.
@ -1,40 +1,85 @@NUT-03: Request mintNUT-03: Swap tokensSame comment as above, secret should be the recommended 32 byte hex
The descriptions says to take the first 16 characters, but in the python implementation the first 14 characters are used.
The example response is missing a pair of curly braces. Keysets returns a list of objects like in the example above (Response
GetKeysResponseofBob:)@ -13,2 +12,3 @@## Multiple keysetsA wallet can ask the mint for all active keyset IDs via the `GET /keysets` endpoint. A wallet **CAN** request the list of active keyset IDs from the mint upon startup and, if it does so, **MUST** choose only tokens from its database that have a keyset ID supported by the mint to interact with it.#### Active keysetsBoth endpoints /keys and /keysets are very similar. What was the general idea of having two endpoints that almost return the same data? Now /keys and /keysets both return
unitandidand can therefore drift apart. In my opinion it would be easier to just have one endpoint that returns all key related data. If the client is only interested in active keysets, this could be accomplished by a query parameter. Having just one endpoint would be easier to maintain and result in less redundant code.@ -1,40 +1,85 @@NUT-03: Request mintNUT-03: Swap tokensI think the term split doesn't fit well anymore since we got rid of the amount field. This endpoint should be renamed to /swap, because it is less specific. Not every call to /split is indeed a split, but it transforms the inputs to outputs.
@ -3,3 +3,3 @@`mandatory` `author: calle``mandatory`typo: The word "fullfil" should be spelled as "fulfill".
Tons of great work in here. I left some more comments. I see you have a few more todo items in your PR description so I don't want to approve it before you're actually done, but this is looking good IMO.
One thing I think doesn't cause problem but I want to make sure I ask: are the v1 routes in any way impacting the way the tokens were before and are now built? As in, I don't think mixing up proofs from old routes and proofs and new one from this newer cashu workflow can cause trouble inside a single cashu token, but can you confirm this to be true?
Lastly, I don't know where this might fit but there is a high-level mental model that took me a few weeks to really grasp I don't know why, and that was the key for me to not get lost in the sauce with all that new jargon/verbiage. Here is how I would write it (again don't know where it fits or if it's even needed, maybe just split into parts at the top of the split/melt/mint files?)
The Cashu protocol defines 3 types of interactions that can happen between a client and a mint, where the client can exchange:
@ -54,4 +57,4 @@```json{"amount": int,"C_": hex_str,I agree with @moonsettler here and would rename to
keyset(orkeyset_id, but the fact that it's an id is self-explanatory IMO).@ -198,3 +139,3 @@"proofs": List[Proof]"proofs": Proofs},...Just a note to fix this token once the version is updated to
Band the secrets are updated to be hex-formatted.I think this line should mention that the current token prefix is
B.Same as above. The latest token version is
B.@ -7,3 +7,3 @@This describes the basic exchange of the public mint keys that the wallet user `Alice` uses to unblind `Bob`'s signature.This document outlines the exchange of the public keys of the mint `Bob` with the wallet user `Alice`. `Alice` uses the keys to unblind `Bob`'s blind signatures (see [NUT-00][00]).Typos:
with his active keysets->with its active keysetsif the mint will sign promises with.->if the mint will sign promises with it.Typo:
identified by its keyset id can be computed->identified by its keyset id, which can be computed@ -42,1 +43,3 @@..."keysets": [{"id": <keyset_id_hex_str>,What happens if the keyset requested with the
GET /v1/keys/{keyset_id}endpoint is not in the mint's old keysets? We might want to have a line on this (with the error returned if there is a specific one?).@ -88,0 +107,4 @@def derive_keyset_id(keys: Dict[int, PublicKey]) -> str:sorted_keys = dict(sorted(keys.items()))pubkeys_concat = b"".join([p.serialize() for p in sorted_keys.values()])return "00" + hashlib.sha256(pubkeys_concat).hexdigest()[:14]If you end up changing the name of the field to
keysetorkeyset_id, reminder to change it here as well.@ -3,3 +3,3 @@`mandatory` `author: calle``mandatory`You refer to a call to the
/checkfeeendpoint and imply an example but there isn't one anymore. I think an example of the checkfee interaction could be added back.There are no
prandproofsfields anymore.@ -89,1 +139,4 @@"amount": 8,"secret": "4f3155acef6481108fcf354f6d06e504ce8b441e617d30c88924991298cdbcad","C": "0278ab1c1af35487a5ea903b693e96447b2034d0fd6bac529e753097743bf73ca9",}How does the mint return overpaid fees? I think we should add an additional
changefield to the response like we did before the cleanup@ -3,3 +3,3 @@`mandatory` `author: calle``mandatory`The term proof is ambigous in this context, because it doesn't refer to a cashu Proof but a bolt11 payment preimage.
payment_preimagemight be a better name since this is a specific bolt11 response we can use the lightning terminology.@ -3,3 +3,3 @@`mandatory` `author: calle``mandatory`typo: bolt11 payment preimage
@ -30,0 +44,4 @@"expiry": <int>}```Where `quote` is the quote ID, `amount` the amount that needs to be provided, and `fee_reserve` the additional fee reserve that is required. The mint expects `Alice` to include `Proofs` of *at least* `total_amount = amount + fee_reserve`. `paid` indicates whether the request as been paid and `expiry` is the Unix timestamp until which the melt quote is valid.How long is a quote valid? Can a wallet request a quote wait for 1 day and then call /melt? Or is the mint using the expiry of the bolt11 invoice? I think there should be an expiry field in the reponse of the mint that tells the wallet how long this quote is valid.
@ -121,7 +121,7 @@ If the `locktime` is in the past and a tag `refund` is present, the `Proof` is stypo: and
@ -53,26 +53,26 @@ The mint produces these DLEQ proofs when returning `BlindedSignature`'s in the rJust to stay consistent
jsonAll Todos are now closed and with that the PR is officially ready for review (and thanks for all reviews already 🙏)
Rocket emoji!
ACK
e59f284f0b.Major improvement of the protocol. I'm happy to see that we have a few implementations that were able to put this into code already, and looking forward to seeing what gets built on top of this. 🚀
One little thing: the test vectors should be updated (if not in this PR in a very close follow-up PR).
Looks good to me. Agree with tb that it would be best to include test vectors as it helps to ensure implementations are correct.
ACK e59f284
Just to confirm my understanding format
Ahas been changed to include the unit, and there is no versionByet.@ -3,3 +3,3 @@`mandatory` `author: calle``mandatory`Since its not stated that mints cannot. Mints can have multiple keysets for one unit, for example 2 keysets for sat?
Believe this should be
payment_preimage@ -93,3 +145,3 @@```**Response** `PostMeltResponse` from `Bob`:Response of `Bob`:payment_preimagewill be null if the payment has failed so should this be optional?@ -81,0 +114,4 @@"proof": "c5a1ae1f639e1f4a3872e81500fd028bece7bedc1152f740cba5c3417b748c1b","change": [{"id": "009a1f293253e41e",The proof field here should be named the same as it is in NUT-05 since its serves the same function. I believe we decided to go with payment_preimage since it is bolt11 specific.
ACK
e1568fdadcThis is a great improvement of the cashu protocol.
@ -75,3 +78,3 @@````amount` is the value of the `Proof`, `secret` is the secret message (no encoding standard), `C` is the unblinded signature on `secret` (hex string), `id` is the [keyset id][02] of the mint public keys that signed the token (string).`amount` is the amount of the `Proof`, `secret` is the secret message (no encoding enforced, 32 byte random hex string recommended to prevent fingerprinting), `C` is the unblinded signature on `secret` (hex string), `id` is the [keyset id][02] of the mint public keys that signed the token (hex string).Super small nit that can totally be addressed in further changes: in the other 2 models you have the order as "amount, id, others", but in this one you have "amount, other, id". The order shouldn't break anyone's code but it feels cleaner to have them all the same (here the
idfield should precede theC_field).@ -75,3 +78,3 @@````amount` is the value of the `Proof`, `secret` is the secret message (no encoding standard), `C` is the unblinded signature on `secret` (hex string), `id` is the [keyset id][02] of the mint public keys that signed the token (string).`amount` is the amount of the `Proof`, `secret` is the secret message (no encoding enforced, 32 byte random hex string recommended to prevent fingerprinting), `C` is the unblinded signature on `secret` (hex string), `id` is the [keyset id][02] of the mint public keys that signed the token (hex string).I was wrong, it does break in small ways the tests (not the validity of the tokens but their serialization).
For example note the example at the end of NUT-00 which has the order "id, amount, secret, C" (different from the model for
Proofdescribed above):This means that a library that implemented the order described in the model section:
Will not be able to reproduce your resulting serialization in base64 (even though both tokens would be valid).
I suggest 2 things:
@ -96,4 +89,1 @@[18]: 18.md[19]: 19.md[20]: 20.mdCan this be used by the mint to fingerprint a wallet? If some wallets are checking by sending the full proof and others are only sending the secret? Should a recommendation be made to do one or the other similar to the ordering of the proofs to avoid fingerprinting?
@ -84,4 +70,3 @@`BlindedSignatures` is a list (array) of `BlindedSignature`'s (see [NUT-0][00]).[00]: 00.md[01]: 01.mdThe trailing comma from 12 should be removed.