Test of Bitcoin lightning wallets

Lightning network enables fast and inexpensive Bitcoin payments. I wrote about how to use it in my book Cryptocurrencies – hack your way to a better life (my e-shop, Amazon). In this article, I want to put all parts of the statement in the first sentence to the test – whether it enables the payments (do they work?), whether they are fast, and whether they are cheap (at least compared to on-chain payments). This is because, unlike on-chain payments, issues of liquidity, routing, availability, etc come into play with Lightning. With on-chain payments, if we have the recipient’s address and pay a sufficient fee, the payment will always go through. The other party doesn’t have to be online, doesn’t have to have any special liquidity, etc. In lightning, not only the sender’s wallet is important, but also the receiver’s.

TL;DR

Phoenix and Breez wallets are both good choices. The reasons for using custodial wallets have vanished. If you can accept some fees, you don’t have to worry about opening any channels, and you simply use your lightning wallet to send and receive with two buttons. That’s it, it just works.

Phoenix occasionally has a problem connecting to the Electrum node, Breez has a limitation to a maximum balance of 4M sats. Breez has more transparent fees, Phoenix is more reliable. Payments up to 1.5M sats almost always go through, for higher amounts I would still use on-chain.

What and how I tested

I tested the custodial lightning wallets Blue Wallet (which is since susetting the custodial lightning node, so I removed the tests from here completely as of february 2023) and Wallet of Satoshi. In the early days of lightning, before the second generation of lightning wallets, they were a good way to onboard new users. I wanted to check if this advantage still applies or if there is no longer a reason to use custodial wallets at all. 

What is a custodial wallet? It is somewhat similar to a bank account – the keys to the bitcoins are held by the wallet operator (or node) and you just control them using the wallet interface.

Blue Wallet is a very good non-custodial on-chain wallet and if you run lndhub on your node you can have private keys to Lightning as well, but that’s for advanced users (similar to wallets such as Zeus that control your own node).

Of the non-custodial second-generation wallets, I picked the PhoenixBreez, and Muun. Muun is technically not a lightning wallet, it only allows you to swap on-chain to lightning and vice versa, but from a user perspective you would only need to scan the QR code of a lightning invoice and pay, similarly it can create a lightning invoice to receive bitcoins.

I tested both Android and iOS versions of all wallets. There are two reasons for this. First, there are rumours that iOS or Android works better (due to notifications or OS features etc.) or one OS is generally worse with payments than the other. This has not been confirmed in this test, the wallets on both platforms worked practically the same. The second reason is that, for example, Phoenix has a completely different code on iOS and Android, so they’re basically two different wallets.

The phones were connected via WiFi (with GSM turned off) using Starlink as an uplink, without VPN. The receiving wallet was always open and in the foreground.

I installed all wallets from the App Store and Play Store (no beta versions) on a fresh device, created a new wallet, created a back-up using whatever mechanism for backups of a particular wallet. Yes, we should back up our sats with temporary wallets too!

The test then proceeded by first trying to send 500k sat from my lightning node to each wallet using wallet-generated lightning invoice (no on-chain, no lnurl). For Breez and Muun wallets on both platforms I was unable to do this, so I then topped them up with the same amount of 500k sats from the Phoenix wallet (from another device that wasn’t part of the test). I’m not evaluating this initial funding of the wallets as my node may have a connectivity issue and it also has direct channels to some providers, so it would not be fair, although with Muun in particular the problem of receiving proved more frequent.

In this test I did not try any on-chain payments directly – everything was done purely via lightning, although of course with non-custodial wallets, there were some on-chain transactions going on in the background, but from the user’s point of view they were only reflected in the fees (and in case of Muun in some waiting time).

The next step was sending from all wallets to all wallets of the other platform. This means that, for example, Breez on Android was sending sats to all iOS wallets (including Breez). The testing was always across both platforms, so that I could have both wallets open on the foreground. No Breez iOS to Phoenix iOS, I had only one testing device for each OS..

First payment I tested was 20k sats, then 200k sats, then 1.5M sats. Finally, I did a limited test of 3M sats between non-custodial wallets (I don’t want to hold even for a second more than 1.5M sats on a custodial wallet).

Summary of results

Start-up speed

I didn’t measure the startup speed, so I’ll just pass on the feelings.

Breez often had a problem with the speed of the launch. While I was doing a test within one day, Breez decided a couple of times that it was going to synchronize with the network when I was receiving a payment. While I understand what’s going on and why it does this (Breez uses the neutrino SPV wallet and thus does block header synchronization and verification), this is a bit unpleasant from a user’s perspective. Especially if you haven’t run your wallet for a while, it can take longer.

One possible solution is to use the Electrum protocol as Phoenix does, for example. Here, however, I ran into another problem – the connection to the Electrum server often fails depending on the Internet connection used. This did not happen to me during the test (Starlink does not block any ports), but during the HCPP in Prague, for example, users regularly restarted Phoenix while switching on the VPN. The selection of the Electrum server can also be a problem – although it is probably random, it can happen that the backend server is down and the app needs to be restarted to choose a different server. While this can be done faster than waiting for sync when you are standing t the checkout in the store, you need to understand what’s going on in the first place – sometimes restarting doesn’t help because the exotic port in question is blocked by the provider of data connection. In that case, you need to either connect to local wifi or turn on VPN. An improvement would be to use a standard protocol and port (e.g. wrap Electrum in HTTPS on port 443), and more quickly detect that the Electrum server is responding slowly and switch to another one. This is a problem that should be solved by the wallet authors, not the users. Wrapping Electrum in HTTPS and switching in the background to a different server when Electrum does not connect in three seconds (while trying the first connection in the background as well) would be a much better user experience.

Speed of payments

I measured the payment speed with a “stopwatch” on another device. I considered a payment complete when at least one of the parties said that the payment had been made – the way Lightning works should suggest that the payee (who submits a prehash to the Lightning invoice) is the first to know that a payment has been made, but especially with custodial wallets (Wallet of Satoshi), the payment is often marked as confirmed by the sending wallet first. Occasionally, however, Breez and Muun also responded with a delay. With custodial wallets, this is understandable, if a bit annoying – after all, the server confirms the payment (both sending and receiving) and only informs your mobile wallet of the balance change. This information for Wallet of Satoshi often took minutes (even tens of minutes once for WoS), and the user often has to refresh or even kill the app and launch it again in order to update the balance and transactions.

In general, payments went through quickly, with the exception of Muun on Android, where even tens of seconds were common. Since the measurement was done with a “stopwatch”, I only wrote down approximate times; I was concerned with whether the payment was instantaneous (less than two seconds), somewhat slow (within about ten seconds), or downright slow (Muun on Android had a record for acceptance – 80 seconds).

Muun has a rather serious problem with sending higher amounts. Since sending lightning payments is “backed” by a on-chain transaction and the user has all the keys, a double-spend attack is theoretically possible until the transaction is mined. The Muun node that makes the payments has to address this risk somehow – some payments will go through immediately, even if the underlying on-chain transaction is not confirmed by a miner. But, for example, with the 3M payment to Phoenix, the wallet told me that it was already too risky for it and I had to wait for the transaction to be mined before the lightning invoice was paid. So the “Lightning” payment took 18 minutes, not really “lightning fast”. This is one of the reasons that I do not call Muun a lightning wallet. To my surprise, however, after 18 minutes the payment did indeed arrive in Phoenix, but if you were standing at the checkout in a store and wanted to pay for a purchase this way, you wouldn’t be very happy – especially since there’s no longer an option to say “screw it, I’ll pay another way” and cancel transaction, you just have to wait for the block, the money has already left your wallet.

Breez is also slow at times, with the user interface reflecting a successful payment only later after it has occurred. The other party often confirms it and the interface shows the payment processing or payment in a pending state for a few seconds. As much as I like the Breez wallet, I paid with it during last year’s HCPP and while I was waiting for “processing”, I was jokingly saying out loud “lightning is lightning fast, but Breez is like breeeeeeeeeeeeeeeeeeeeze”. Since the terminal in Paralelní Polis also takes a long time to mark a successful payment as processed (sometimes it doesn’t even mark it, but that’s for another story), two “slow” technologies have collided – the payment is long processed between nodes, but the user interfaces of both sides won’t show it until later. I have to say, though, that Breez has improved quite a bit on this as well, and is probably a bit more nimble than last year. It still feels a bit slower than Phoenix for example.

Fees

One of the goals of the lightning network is to reduce transaction fees on transactions. The way lightning achieves this is that multiple payments can be made over a single channel, and thus lightning saves the number of transactions that must be written to the bitcoin blockchain. However, lightning itself also has fees.

In on-chain transactions, the number of bytes that are written to the blockchain is a scarce resource. The median transaction is around 250 bytes, and the unit in which the fee is determined is the number of satoshis per byte. If we want the transaction to be confirmed as soon as possible, we can pay more satoshis for each byte, on the other hand, if we want a cheap transaction and are ok with waiting (which is usually my case, I am a cheap guy 😅), we can just put 1 sat/vB. The fee does not depend on the amount of Bitcoins we send, but on the number of inputs and outputs. So we can easily send even 100 BTC for 500 sat. The median fee is around 10 sats/vB, so we can currently reasonably expect fees for onchain transactions to be somewhere in the range of 250-5000 sats, depending on how long we want to wait and how big the transaction is.

Channel capacity is a scarce resource in Lightning transactions. Thus, in lightning, the fee depends on the amount we send. In addition, for non-custodial wallets, sometimes an on-chain transaction needs to be done and thus there is also an additional fee that is often paid by the receiver.

For the initial wallet funding (500k sats) the average sending fee was 1429 sats, if you include the fees for receiving, it was 2829 sats. Here, however, the average is not so indicative, the fees vary considerably. Custodial wallets and Muun do not charge receiving fees, Phoenix charges 1% for opening a channel, minimum 3000 sats, so in this case they charged 5000 sats. Breez charges 0.4% and a minimum of 2000 sats, but will often open a larger channel than the payment we accept (making subsequent receiving cheaper). Breez is much more transparent about fees – when you create an invoice (or even when you make an on-chain payment), it will tell you when it will charge you (for example, if you receive more than 100k sats, it will charge you a fee). If you enter the amount when creating an invoice and confirm it, then under the QR code, Breez will show you the fee it will charge you for receiving the payment. This is a significantly better user experience than with Phoenix, which while it will tell you what the fee will be if you need to open a channel, it doesn’t tell you when the channel will need to be opened, you will only find out about the fee after the payment is received. Phoenix also has higher fees for receiving payments and will never open a much larger channel than the amount you are receiving.

After the initial funding of the wallet, fees for sending were already dominating, although with 1.5M payments, receiving fees kicked in again as the initial channel was not large enough to receive this payment.

For a 20k sat payment, the average fee was 56 sat. Since Muun is effectively an on-chain wallet, the highest fees were when sending from Muun (144 sats average – counting both Android and iOS version). There were also higher fees when receiving into Muun (but these were also paid by the sender) – the average was 97 sats, although there was more variability in fees when receiving.

When sending 200k sats, the average fee is 331 satoshi, which is often already more than the cheapest on-chain payment (if you are willing to wait – but with Lightning the confirmation is instant). The Wallet of Satoshi clearly has the cheapest payments, the average payment was only 5.5 satoshi if you don’t count sending to Muun Wallet.

At the time of writing, the dollar value of 1.5M sats is just over $300. The cheapest wallet remains the Wallet of Satoshi. Phoenix paid an average of 940 sats if it didn’t send to Muun. Breez 1748 sats. Muun paid an average of 269 sats to send. This is very interesting because apparently sending doesn’t depend that much on the amount you send (it’s an on-chain payment). So the lightning payments to Muun are paid extra by the users who send, but the Muun wallet user doesn’t see the higher fees. Although this might have been because of relatively low on-chain fees at the time I was testing, maybe it would have been higher when blocks are full.

Comparing what payments are worthwhile on-chain vs via Lightning, at 10sats/vB it breaks somewhere around 1.5M sats, where I would still pay with Lightning because of the speed of confirmation (and better privacy). At 3M sats, even with respect to reliability, it’s already worth going on-chain instead of lightning.

Payment reliability

At 20k and 200k sats all payments went through seamlessly, regardless of wallet. At 1.5M sats, Breez had a problem sending the payment to Muun (both Breez iOS to Muun Android, and Breez Android to Muun iOS). Since I’ve been using Muun payments to Breez more often outside of this test, I noticed the problem a while back. I can’t tell if this is an issue with Breez or with receiving to Muun. 

Tip for anonymizing UTXOs

If you want to anonymize your on-chain UTXOs and ensure they are not easily linkable, it is possible to use Breez to accept on-chain transactions up to the amount of the open channel for free (you only pay a miner fee to send the UTXO to Breez). If you need the balance then in on-chain form, you can send the funds via Lightning to Muun, this will turn it into an on-chain balance. The fee for this Lightning payment is a bit higher, but still not as high as other swap-out services. You will receive a brand new UTXO that is in no way related to the original UTXO – it will be provided by the Muun service, which does not have access to the original Breez UTXO. So you can send different UTXOs one at a time to Breez and receive it as a consolidated UTXO unrelated to original UTXOs through Muun. Note that the maximum balance in Breez is 4M sats. If you send more, it will do an on-chain refund.

With 3M sats, the reliability of lightning payments went drastically down. Payments from Phoenix wallets went correctly to Phoenix, Breez and Muun (I have not tested custodial wallets with 3M sats). Breez failed the first time when sending, the second time a payment from Breez iOS to Phoenix Android succeeded.

The payment from Muun to Phoenix Android went through after the block was mined (more than 10 minutes), so I decided not to test further payments at that amount, as it’s not a good user experience and I don’t recommend such large payments from Muun via Lightning – our expectation of lightning payments is that they don’t take more than a few seconds. If it was, for example, a payment through a payment gateway that has an expiration (due to exchange rate volatility), the payment would fail completely – with an on-chain payment, the gateway would know that the money is on its way and the exchange rate is locked and it would just wait for confirmation. So in this case the user experience of on-chain payment is better than Muun through lightning. 

A few words about the individual wallets

Blue Wallet

Blue Wallet was my first mobile lightning wallet. I also wrote about its possible use in the chapter of my book Cryptocurrencies – Hack your way to a better life, which still applies. You can also find this chapter on my blog: How to build local payment systems on top of Bitcoin. This use-case still applies, but you cannot use it as a lightning node anymore.

While I admire the creators of Blue Wallet for pioneering the user-friendly Lightning wallet, and of course the on-chain wallet was one of the first multi-platform mobile wallets to have coin control (and unlike the Lightning wallet, it’s not custodial), I can’t recommend it for new users.

Wallet of Satoshi

This wallet is “subjectively” the ugliest. I shouldn’t rate the design, you’re reading this article on WordPress site with a default theme, but the wallet looks more like a 90’s video game and not something we should entrust our money to. The verb “entrust” is used correctly in this case because it is a custodial wallet, so you have to trust the authors that you won’t lose your money and that they will take sufficient care of the keys to your Bitcoins.

Aside from the design issues, the wallet worked reliably, it clearly had the lowest fees. Often, the payment information was not updated right away, sometimes payment notifications came with a delay of several minutes. Backing up the wallet works by pairing it with an email, and the confirmation email to protonmail also came with a delay of a few minutes.

If there were no good non-custodial alternatives, I would recommend it for small payments. Right now, however, I’d rather go straight to a non-custodial solution and swallow the fees for creating channels.

Muun Wallet

Muun is a great on-chain wallet. It uses taproot and has a rather interesting security model. The fact that it also allows lightning payments is a nice bonus. Sending lightning payments to Muun is quite expensive (for the sender). Since it can’t be relied on for speed of sending payments, I don’t recommend it as a primary lightning wallet. However, it is a good option to interact between the on-chain and lightning world.

I definitely recommend keeping a look at this wallet occasionally, but if you want to install a true second-generation Lightning wallet, I’d go with the Breez or Phoenix.

Payment reliability issues – unreliable acceptance of higher amounts via Lightning and having to wait for block confirmation when sending put it at a disadvantage in normal day-to-day use. At the same time, this model of on-chain payments for every Lightning invoice (both received and sent!) stops working the moment on-chain fees increase again.

I personally use it for privacy along with the Breez wallet as I wrote above.

Breez

The Breez wallet was a pleasant surprise. It worked quite reliably, even if it was a bit slower. At higher amounts the reliability was a bit worse, even the initial funding from my node didn’t work (although in further tests it worked).

Since I use this wallet quite often also outside of this test, mainly due to the cheap swap-in from on-chain to Lightning, I have quite a bit of experience with it. So I have to say that it has its “moments” when it fails to handle payments, especially when it comes to higher amounts. On the other hand, it is packed with features – for example, using it as a payment terminal (PoS) or as a podcast player that allows you to reward podcast creators. It is a so-called “superapp” that integrates various other lightning apps. So you can top up your mobile phone credit directly from your wallet via Bitrefill or participate in lnmarkets. You can thus “handle” many of your needs from one place.

I look at many of these features in my course Lightning network for private bitcoin payments among friends and for products and services, where I demonstrate each feature, including using it as a PoS terminal.

A minor drawback, but at the same time a good reality check, is the limitation to 4M sats (at the time of writing approx $800). This is a software wallet and you should not trust it with your Bitcoin savings. So it might actually be a good thing that it forces you to move your savings to a hardware wallet from time to time.

Phoenix

Phoenix wallet is currently my choice for new users. I’m alternating this choice with Breez, though, so don’t take this sentence as a clear winner of the test. Phoenix is a non-custodial second-generation Lightning wallet that’s easy on the user. In the test, it was the most reliable for payments and had relatively reasonable, though not the lowest, fees. Unlike Breez, Phoenix is a minimalist app that is dedicated to Lightning payments and strives to be good at that use case (and just that use case).

In addition to lightning payments, it allows receiving and sending on-chain payments using the integrated swapping service. Everything else in the lightning economy should be handled by integration – mainly through the web browser.

One drawback of Phoenix is the uncertainty in fees. While Phoenix is transparent in what the fees are if you open a channel (and swap-in), it doesn’t tell you when the channel will need to be opened in advance (not even when you type in the amount in the invoice).

When we (as Paralelná Polis) organized an event in someone else’s premises, I installed Phoenix at the bar’s phone, so that the barista could accept lightning payments. I did the traditional thing – I sent him a higher amount (over 300k satoshi) and he sent most of it back. 

Since the fee is 1% and a minimum of 3000 sats (but check the settings as fees can change from time to time), it is a good idea to make the first payment of at least 300k sats. The fee is still 3000 sats even if it was lower, but ACINQ (the company behind this wallet) will open a channel in the amount of the payment and not more. So my “onboarding ritual” is to send 300k or more sats to the new wallet and then get most (not all) sent back.

However, the wallet soon needed a new channel (sales through Lightning were higher than I expected thanks to the amazing community around Paralelná Polis) and the owner of the bar soon called me to tell me that there is some relatively high fee with every payment. Someone bought a drink and suddenly the charge was 70 cents to accept payment. ACINQ was in fact creating new channels for each payment and charging 3000 sats each time. I explained to him what happened and did the “ritual” again – send 1M sats (and pay the 10k sat fee, so about 2$), send them back to me and the sales could continue without any more fees.

By the way, we didn’t use Breez with PoS mode because I was expecting (and hoping) sales to be more than 4M sats. The other problem with Breez is that with swap-in, you can only process one incoming on-chain payment at a time (with Phoenix, you can create multiple swap-in on-chain addresses at a time). In the end, nobody paid on-chain. Whether the sales were more than 4M sats I don’t know, so maybe Breez would have been a better choice, but with “the ritual”, Phoenix also worked well.

A minor drawback of Phoenix is that it is connecting to “random” Electrum servers on weird ports. This doesn’t always work – either because the ports are blocked or because the randomly chosen Electrum server is not responding. I hope that ACINQ will be able to solve this problem as soon as possible. 

Conclusion

The lightning ecosystem is already ready for everyday use. As with any open-source tool and open protocol, the quality of implementations varies. However, I consider this an advantage because we are in a phase of rapid innovation. For payments between friends, family or payments for goods and services up to about 1.5M sats, lightning is reliable (with the exception of filtered internet from some mobile operators who don’t give you access to all ports – use a VPN in this case).

If you’re willing to accept the variability of fees, which in my opinion are slightly less predictable than most on-chain transactions, then most wallets just work. If you want to have more visibility into fees, that will be a problem with Phoenix.

We are at a stage where I personally would not use custodial solutions, nor would I recommend them to anyone. You can’t go wrong by choosing Breez and Phoenix. When using Phoenix to accept payments, I recommend studying how fees work. With Breez, you need to be mindful of the maximum wallet balance that Breez will allow – 4M sats.

If anyone tells you that Lightning wallets work better on one platform or the other, at least according to this test, nothing like that was confirmed – both platforms worked basically identically.

Using lightning will eliminate coin history (“tainted coins”) issues, give you more privacy, you don’t have to wait for payment confirmations (except for Muun), and you might even save on fees (especially with Wallet of Satoshi and Breez).

Support yourself

This headline was originally called “support me”. I wanted to showcase my great book Cryptocurrencies – hack your way to a better life (my e-shop, Amazon) and my course Lightning network for private bitcoin payments among friends and for products and services. In reality, however, by buying and studying them, you are mainly supporting yourself. You’ll learn how to use lightning, swap between on-chain and lightning, use lightning to pay payment requests from other payment networks, learn how to use PoS mode, and how lightning technically works on the inside. There’s much more in the book, you’ll learn how to use the fascinating crypto-economy to improve your lives. No book like this has been published before, so I highly recommend it.

Cryptocurrencies - hack your way to a better life

Both products are great, and besides me, you will support yourself with new knowledge and, if you apply the knowledge, useful experience. Use coupon code LIGHTNINGTEST for a 11% discount on both products in my e-shop (the coupon does not work for Amazon unfortunately).

Other wallets

There are, of course, other wallets that I did not test. With many you just control your node (e.g. Zeus). Such wallets don’t make that much sense in a test like this, because the success of payments depends mostly on how you configure your lightning node, what channels you open, and so on. Payment speed, success rate and amount of fees depend more on how you can work with the channels. Both custodial wallets and non-custodial wallets that I have tested work by hiding the channel experience from the users. In this way they are more standardized and it makes sense to test and compare them in terms of fees and payment reliability.

Another interesting wallet is Blixt Wallet. I didn’t include it for the reason that I didn’t find out about it until after the test. However, it has some limitations that would make it difficult to do a fair comparison. First of all, automatic channel opening on incoming payments needs to be explicitly turned on in the settings, and even that only goes up to 400k sats (so just the initial funding I started the test with wouldn’t work because I started with a payment of 500k sats). In addition, this wallet is a hybrid between a wallet that allows you to manage channels and an automatic channel opening. Therefore, the test would not be completely standard, but in this case I consider it more of an advantage. In the end, I figured I’d at least install it, but the iOS version didn’t even start. I’ll take a look at it in a few months when it matures a bit.

Technical details

The test took place on October 26, 2022. The latest versions of the wallets and operating system current on that date were used.

Versions of tested wallets

Android

  • Phoenix 1.4.23
  • Breez 0.15-beta
  • Muun 49.10
  • Blue Wallet 6.3.1
  • Wallet of Satoshi 1.17.12

All were tested with Android 11, ColorOS 11.1 2022.04.15 on OPPO A74.

iPhone

  • Phoenix 1.5.3
  • Breez 1.0.16
  • Muun 2.6.3
  • Blue Wallet 6.3.1
  • Wallet of Satoshi 1.17.12

All were tested with iOS 16.0 20A362 on iPhone 12 Pro.

Test results and notes

My test notes in ods and xlsx formats: