Meeting minutes
Results of Stripe SPC experiment
[Slides presented by Benjamin Tidor]
btidor: Welcome. I'm a payments engineer at Stripe. Today I'll talk about the pilot project using Secure Payment Confirmation (SPC) with 3DS.
...SPC at this time focuses on authentication, not yet instrument selection preceding authentication
… SPC intends to be reliable low-friction authentication
… features biometric confirmation of transaction details in browser-native UI
… details of transaction presented in trusted UI
… SPC protects user privacy by requiring explicit consent before confirming identity (to the merchant Web context)
… we also get the myriad goodness of FIDO authentication (e.g., phishing resistant)
… SPC signs transaction details in a way that should satisfy PSD2 requirements for dynamic linking
… when used with 3DS, could be used as an alternative to a traditional 3ds redirect for step-up
… gracefully degrades to vanilla 3ds for unenrolled users or unsupported devices
... Zero-UI fallback is an important property for gradually upgrading systems
... SPC in the experiment has the following superpowers:
- browser binds payment details into cryptographic signature
- a credential can be provisioned on the issuer domain, but can be used by other origins for specific transactions
- that signature can be used as proof (verifiable by the issuing bank) that the user confirmed transaction details
[We review mockups of the flow. The mockups were those shared in 2020 by Stripe and Google]
btidor: Stripe built an enrollment page (in lieu of the issuer)
Gerhard: Was enrollment part of the origin trial?
btidor: Yes, enrollment was part of the pilot, but we built the enrollment UX ourselves.
… SPC doesn't mandate who does the enrollment
… we did a full 3DS flow in a payment handler secure modal window
Chris: Is enrollment per merchant?
btidor: The big idea for SPC is that issuer-registered credentials can be used cross-merchant.
… in the pilot, merchants could re-use the credentials enrolled with the relying party, but only the merchants participating in the pilot could do so.
IJ: The SPC pilot showed browser-native UX at transaction time. We have also heard some interest in browser-native UX for enrollment; I expect that to be a topic we will discuss at the remote meeting later this month.
[Experiment background]
btidor: We ran a pilot with certain merchants and certain customers on MacOS with a platform authenticator
… good question for more discussion: should a user-verifying platform authenticator required, or should we accommodate more types of authenticators?
… we also had some restrictions on which cards were eligible...had to support 3DS2 (some version)
… we only wanted to enroll users going through the 3DS2 challenge flow
… on the merchant side we used predominantly small and medium sized merchants in a variety of countries
… ran the experiment for 3 months
… in the experiment, Stripe is standing in for banks. We did not hand data to banks. Moving forward, we hope to get communication wired up so that this is used as an SCA-compliant mechanism.
… we looked at three cases: (1) Explicit 3DS2 challenge (2) Vanilla 3DS with both frictionless and challenge, and (3) SPC
… we wanted to benchmark the 3DS conversion compared to an existing challenge flow
… the Vanilla 3DS arm depends on the rate of frictionless flows...we expect this arm to be more variable because there are so many context-specific considerations for determining whether to step up.
btidor: Hypotheses were about: (1) conversions and (2) time spent authenticating. We will also say more below about fraud rates.
[Results]
btidor: All-inclusive conversion for the 3DS2 challenge flow (for our sample set) was 84.7%
… one caveat in the experiment...we requested a challenge from the issuer and ran it through an iframe
… a lot of time we asked for the challenge flow but the bank actually gave us the frictionless flow
... In those cases MORE THAN HALF were dropped from the experiment.
… this skews the results a bit compared to the vanilla flow
… the all-in conversion rate for vanilla flow was 91.4%
… the conversion rates not exactly comparable, but at a high level drop-off decreases in the vanilla flow
… SPC flow with fallback had end-to-end conversion rate of 92.7%.
btidor: ...if you prompt the user and they complete with biometric OR if they cancel and continue through fallback, we count that as success.
... So uplift was about 8% comparing SPC to the ordinary 3DS2 challenge flow.
jalpesh_: This is very encouraging. Thanks very much for the work!
… some questions about the experiment .... how many transactions were run across the three flows?
btidor: I don't have that number in front of me. We did get enough transactions to be statistically significant.
jalpesh_: A lot of issuers don't force the challenge. Do you feel that by including India and Europe where SCA was not enforced you will get better 3DS2 challenge rates?
… a lot of issuers in the US will do frictionless, but in countries where challenge is expected by the consumer, there might be different results
btidor: We did get challenges in a number of markets including in the US
… it was a comparable number to the number of challenges in Europe, so we think we can draw conclusions.
… we do see some regional effects, but I don't have that information here.
jalpesh_: It would be great to monitor where the challenge was actually presented.
btidor: The 3DS2 challenge arm data I shared does reflect that point.
thierry: This is only looking at authentication success rate?
btidor: Yes.
thierry: Is 8% what we would pick up if we were to offer SPC? Because 92.7 includes fallback. What portion went through the new flow?
btidor: This was preliminary experiment, so there are some unknowns. But the overall idea is that we see the end-to-end success rate going up
thierry: Do you think it has to do with consumer non-readiness?
btidor: I think that convenience is definitely a factor, but it's too soon to explain behavior choices
Chris: To clarify the 92.7% figure: does that number include enrollments, or just the return payers?
btidor: The 92.7% was only for "previously enrolled, but we expect the difference to be small
Chris: What we haven't measured then, probably, is the hurdle to getting people enrolled.
btidor: We discuss that in a moment
Gerhard: What about user who pays with same card on two browsers? Are people enrolling a second time?
… are they registering frequently...that could indicate a positive experience the first time around
btidor: I think for this pilot the number of re-registrations was low since we only offered SPC on one platform (MacOS)
<benoit> (chrome already saves that kind of information across devices with autofill... would be interesting to see if they would do the same with this type of authentication as well)
btidor: One of the things to talk about is the transaction confirmation window. We want to look at how many people authenticate biometrically v. traditional flow.
… we found that the biometric click-through rate was 86.3%, which is a bit lower.
… they canceled out. That was not a surprise; we have heard that in general as soon as there is a change in UX from what people recognize, there is drop-off. However, the drop-off in the experiment was less than the drop-off compared to other UX changes that people have experienced. So this is encouraging. But we think we can make further improvements.
… for instance, the enrollment screen and the authentication screen are not harmonized.
… we think that people who authenticate multiple times, however, succeed more frequently. They do biometric more once they are familiar with the UX.
...About 20% of people enroll, but we think that can go up
… Furthermore, we think that if the enrollment message is presented by the issuing bank (which is the expectation longer term), enrollment rate should go up a lot.
...Regarding fraud, there were a negligible number of fradulent transactions (and were low across all three flows)
JeffH: So 7.3% of people in SPC...what is there experience?
btidor: They would probably have seen the biometric prompt (canceled), and then had one of the usual failures with the SMS prompt (e.g., don't have phone)
… some of the fallback was also caused by API errors. In a small number of cases, users may have scanned their fingerprints but something when wrong and we had to fall back.
JeffH: So of 92.7%, 86.3% completed biometric flow?
btidor: Yes.
lawrence: In the pilot, what as the average transaction value?
btidor: I don't have that information to share, but small and medium merchants participated in the pilot. We don't think there were any extremely small or extremely large transactions.
Gustavo: Looking at the "Touch ID to verify purchase". Is the amount and merchant data coming from the Web site?
… how confident are we that the information is accurate.
btidor: The origin shown in the transaction window is the top-level origin seen by the user (the merchant site).
… the origin is not modifiable; determined by the browser.
… the total and currency come from the merchant origin (via Payment Request)
… the important part is that the data that the user authorized is signed, so issuer can detect if modified
Ian: SPC leverages both Payment Request API and Web Authentication. So:
- Amount and currency come from PR API.
- Instrument (card) art is registered with the browser during enrollment so it comes from the relying party.
- The origin of the merchant (the beneficiary) is determined by the browser
Gustavo: Will SPC work in a WebView?
btidor: That was not part of the experiment.
Ian: (A next piece of work will be expanding the discussion to pursue cross-browser/platform interop.)
AdrianHB: Is the transaction dialog from the browser or OS?
btidor: Not all authenticators have display. So it's the browser that gathers the info and passes to the authenticator for signature
… any attack on authenticators would still be possible with SPC
AdrianHB: I believe that we'd need support from platform authenticators for the tx-simple extension to WebAuthn
IJ: I recall Chrome folks saying the transaction dialog is a good way to display information for more interop
Danyao: Browser is more trusted than merchant, and we can run a prototype. Wwe can still discuss whether it makes sense to push this UI to the platform level.
btidor: The next interesting result was that authentication "duration" went down. The mean time for 3DS2 challenge was 52 seconds. Compare to mean of 15 seconds for SPC (all inclusive). That was on same order of magnitude of vanilla 3DS
… with SPC, fallback to 3DS happens infrequently enough to keep the time low
… the mean in vanilla case is 22 seconds...the challenge flow drags up the mean.
… so SPC mean is less even than vanilla flow
… that's because the biometric flow is fast, and happens a lot
… that's why we think it's useful to look at the biometric and fallback flows together
… in the long term, we can imagine people only using the biometric flow, but even with some fallback, the rates are very good
Summary
btidor: So the key results were that conversion went up by 8% and reduced authentication duration by at least a factor of three; both compared to the 3DS2 challenge flow.
… we'd like to increase the enrollment rate
… we'd like to better measure how much people are going through biometric flow or why are falling back.
...We'd like to see how SPC can work with 3DS and other protocols
… next work is API design. Hope to hear interest from other browser vendors as well
… we are optimistic about these results. Many thanks to the Google chrome team for enabling this origin trial. Thanks!
nicktr: We'll be discussing this more at the remote meeting at the end of the month; see the remote meeting agenda. So next steps are API design, getting more payment methods supported, more support on more platforms and browsers, more experimentation
THANKS BENJAMIN AND GOOGLE!
PAG update
<nicktr> ian: PAG outcome: continue work.
Next meeting
29 March remote
<nicktr> ian: please register for the F2F meeting