Hi all,
Since the outset of Matrix, our aim has always been to provide a protocol that lets you build open, decentralised,
secure communication apps which outperform the mainstream centralised alternatives. It’s been a twisty journey - first
focusing on making Matrix work at all (back in 2014), and then getting it out of beta with Matrix 1.0 in 2019, and now
focusing on making Matrix fast, usable and mainstream-ready with Matrix 2.0.
Meanwhile, the pendulum of decentralisation continues to accelerate in our direction. Our friends at Bluesky have shown
that it’s possible to build decentralised social apps which are mainstream friendly enough for
Presidents to
recommend them; Elon continues to destroy Twitter and showcase
the importance of decentralisation to everyone, and even Meta is dabbling in decentralised social media (and
decentralised communication!)
So, where does Matrix sit in all this? Well, in order to make the transition to mainstream, we’ve been beavering away to
implement four main pillars in Matrix 2.0:
- Instant login, instant launch, and instant sync (aka Simplified Sliding Sync, MSC4186)
- Next Generation Auth (aka Native OIDC, MSC3861)
- Native Matrix Encrypted Multiparty VoIP/Video (aka MatrixRTC, MSC4143)
- Invisible Encryption (MSC4153 & friends).
Between these, we believe that Matrix can now be used to build apps which genuinely outperform the mainstream
alternatives - and what’s more, these MSCs now all have implementations you can use today. The MSCs themselves are
not all finalised and ready for merge (but are getting close), and when they pass FCP (Final Comment Period) to merge
into the spec, we will formally bump the spec release to version 2.0.
We actually declared Matrix 2.0 as ready for action back at The Matrix Conference last month, and now that the
videos have been published you can watch the launch right here:
Today's Matrix Live:
https://youtube.com/watch?v=ZiRYdqkzjDU
Since the conference talk things have already moved on a bit, though, and we’ve landed a bunch of tweaks to address
teething issues - and so here’s the current state of action:
1. Simplified Sliding Sync
Simplified Sliding Sync is the final version of Sliding Sync - the API which provides instant login, launch & sync in
Matrix 2.0. To say that the API has been through a lot of iterations is an understatement, but we’re finally there
with MSC4186, which simplifies the original Sliding
Sync API (MSC3575) by removing the concept of a
server-determined room list ordering entirely, and instead lets the client sort the roomlist as needed (while letting
the client paginate in the roomlist incrementally, to ensure instant responsivity, no matter how large your room list
is).
Simplified Sliding Sync is now implemented natively in Synapse as of 1.114, and so there is no longer any need to
run a Sliding Sync Proxy in order to use the API. In fact, the Sliding Sync Proxy is being deprecated and the old
Sliding Sync code will be removed from matrix-rust-sdk in the nearish future. Unfortunately we don’t have bandwidth
to maintain both the native implementation Synapse as well as the proxy shim - and the proxy inevitably has suffered
from a lot of limitations (e.g. having to do a full v2 initial sync for new logins, slowing them down to v2
performance - as well as duplicating storage between Synapse and the proxy). There’ll be a dedicated deprecation
blog post for the proxy and pre-simplified Sliding Sync shortly. Meanwhile, work is well underway for native Sliding
Sync support in conduit, conduwuit and
grapevine - conduwuit guesses “next few weeks” for native Simplified
Sync Support to land. Dendrite is unfortunately still not funded, so will happen on a best effort basis.
In terms of performance, native Simplified Sliding Sync in Synapse is spectacular - outperforming the proxy throughout,
and making the old sync v2 API look positively prehistoric. Gone are the days of waiting for your app to sync, even if
you’ve been offline for weeks/months; gone are the days of waiting minutes to login, and gone are the days of staring
at a spinner as your app launches. It really is a new era - and having been hyping it since the first demo at FOSDEM
2023, I promise we won’t bang on about sliding sync any more after this; it’s finally here and landed and we can move
on and enjoy it! For more details, see Ivan’s talk from The Matrix Conference:
Today's Matrix Live:
https://youtube.com/watch?v=kI2lSCVEunw
2. Next Generation Auth
Next Generation Auth is what we’re calling the migration to using industry standard OpenID Connect as the authentication
API in Matrix 2.0, moving on from the custom auth API that Matrix has historically used. We’re calling this Next Gen
Auth because we were seeing a lot of folks incorrectly assuming that adopting OpenID Connect for auth meant we would
somehow be encouraging 3rd party social login or single-sign-on - which is not the case.
Next Gen Auth is simply swapping out Matrix’s old custom auth APIs with equivalents which are defined by the OpenID
Foundation; after all, Matrix is a communication protocol, not an authentication protocol. In return, we get much more
mature and secure authentication APIs - and access to the whole OpenID Identity Provider ecosystem, including support
for Two Factor Auth and Multi-Factor Auth, hardware authentication tokens, passkeys, and device-based login flows
(aka QR Login). We also stop both Matrix clients and servers having to implement the sprawling legacy Matrix
authentication API surface - ensuring that users only ever hand their account password to their auth server, rather
than having to trust their clients to handle it securely, which in turn plays much nicer with password managers
(who only have to remember how to auth with your auth server, rather than a myriad different clients). It also lets
you share authentication between apps if you want; gives us access_token refresh (at last!) to avoid long-lived
access_tokens hanging around; and in future will also support OIDC scopes so you can limit the access particular
clients get to your account.
In short, Next Gen Auth is transformative, and the initial implementation at
matrix-authentication-service (MAS) is ready for admins
to deploy and use (and in future will be available embedded in Synapse too). Unfortunately it is not yet live on
matrix.org (given we have to migrate tens of millions of accounts, complete with all the social login complexities)
but we’re hoping to get it there in the coming months.
Probably one of the clearest immediate benefits of Next Gen Auth is the ability to do a full login including setting
up all your end-to-end-encryption simply by scanning a QR code on an existing client, courtesy of
MSC4108 and OAuth 2.0 Device Authorization Grants. In
other words, you don’t have to specify a server, or your username, or account password, or your recovery key - you just
scan a QR code and you’re in. The very latest Element X releases on iOS & Android have this implemented and enabled by
default, and so if you’re on a server which has deployed MAS, you can go to “link new device” in Element Web/Desktop to
show a QR code to instantly log in.
For more info on all things Next Gen Auth, see Quentin’s talk from The Matrix Conference:
Today's Matrix Live:
https://youtube.com/watch?v=wOW8keNafdE
3. Native Matrix Group VoIP/Video: MatrixRTC
Next we have MatrixRTC: end-to-end-encrypted group voice and video conferencing over Matrix. Historically, group VoIP in
Matrix has relied on third party conferencing systems (Jitsi, and before that FreeSWITCH) - providing no support for
Matrix’s end-to-end encryption, or indeed Matrix’s user identities, decentralisation or decentralised access control.
With MatrixRTC this changes: we now have a standard way to establish large-scale end-to-end-encrypted group video
calls via Matrix, leveraging all the benefits of Matrix’s end-to-end-encryption infrastructure, user identity, room
permissions, etc. It also supports different media stacks to actually handle the media conferencing - today, the main
implementation uses the LiveKit SFU, but there’s also an
experimental full-mesh WebRTC implementation.
Element Call has been the driving app behind MatrixRTC, and as of today is now enabled in the release versions of both
Element Web/Desktop and Element X to provide native MatrixRTC calling embedded in the apps: if you hit the video call
button you will now have the option to spin up a MatrixRTC call via Element Call rather than via Jitsi or Legacy 1:1
calling, and if the room is end-to-end-encrypted, all the conference will be too.
Meanwhile - MatrixRTC isn’t just Element Call: Famedly showed off experimental interop with FluffyChat at
FOSDEM back in
Feb, and Element showed off experimental interop with BigBlueButton in August.
Given more and more conferencing tools are converging on LiveKit as a best-in-class SFU, it’s an amazing opportunity to
use Matrix and MatrixRTC to power the end-to-end-encryption and decentralisation and get standardised voip/video interop
from the outset.
For more info, see Timo’s talk from The Matrix Conference:
Today's Matrix Live:
https://youtube.com/watch?v=OXPuYbfiXDQ
That said, there are a few caveats right now:
- We do not have interoperability between legacy Matrix 1:1 voice/video calling and MatrixRTC (and it’s not clear
if/when we will get to it) - but Matrix 2.0 clients like Element X exclusively use MatrixRTC for VoIP/video,
including for 1:1 calls. This is in order to only maintain one VoIP stack, and so that you get multidevice and
multiuser support for free, even in 1:1s. As a result, we’re in the process of figuring out how to warn legacy
callers that MatrixRTC-only clients won’t be able to answer their calls (e.g.
MSC4220) - this hasn’t shipped yet.
- iOS 18 broke CallKit + WebRTC, so Element X iOS has had to disable fancy OS-natively-integrated MatrixRTC calling;
and has a support issue open with Apple to try to solve this.
- We’ve had some fun teething issues thanks to the volume of signalling in MatrixRTC exposing some sync bugs - these
are almost solved, but probably mean MatrixRTC should still be considered beta for a few more days until the fixes
land.
4. Invisible Encryption
The final pillar of Matrix 2.0 is Invisible Encryption - making Matrix’s end-to-end encryption as seamless and
invisible as the centralised alternatives (Signal, WhatsApp, iMessage and friends). This does not mean reducing
security in any way - just the opposite, in fact. It means:
- Ensuring that Unable To Decrypt (UTD) bugs never happen. Huge amounts of work has gone into this over the course of
the year, especially via complement-crypto as a comprehensive
end-to-end-test suite for both matrix-rust-sdk and matrix-js-sdk based Matrix clients. We are finally at the point
where UTDs are so rare that most people simply never see them, and any reports get jumped on (and complement-crypto
tests get written) whenever they emerge. Anecdotally, I now get way more “waiting for message…” errors on WhatsApp
than I do on Matrix, these days! For more details on the Hunt For UTDs, see Kegan’s talk from The Matrix Conference:
Today's Matrix Live:
https://youtube.com/watch?v=FHzh2Y7BABQ
-
We are excluding non-cross-signed devices from Matrix (MSC4153).
The fact that Matrix ever supported the idea of users enabling encryption on a device without proving that they are
the valid owner by signing it (by verifying it with another device, or providing their recovery key/passphrase) is a
nasty hangover from back before we introduced cross-signing. Nowadays, the fact that non-cross-signed devices exist
acts to reduce security and complicate UX and implementations with big scary red warnings whenever an unverified
device joins a conversation. Instead, once MSC4153 lands, we’re going to simply exclude unverified devices
entirely - not encrypt to them, and not decrypt from them. We can then get rid of all the confusing warnings
associated with them.
-
We’re also solving the confusing “grey shield” warnings: “the authenticity of this message cannot be confirmed on this
device” and similar. These are avoidable warnings caused by message keys which can no longer be tracked back to the
original sender - e.g. if they’re restored from backup, or if the original sending device has been deleted. We’re
fixing these with authenticated backup (MSC4048) and
including device keys on Olm events (MSC4147)
respectively.
-
Finally, we’re moving to Trust On First Use (TOFU). This means that even if you didn’t explicitly verify another user,
you still get warned if their identity changes (unlike previously, when it was ignored). An initial implementation
just landed in matrix-rust-sdk a few weeks ago, and so
appropriate warnings at the application level should arrive shortly -
matching the equivalent Signal or WhatsApp “this user’s identity has changed” warnings, albeit not yet synced between
devices.
As you can probably tell, Invisible Encryption is something of a moving target; for instance, the scope could extend to
cover all scenarios where users might expect messages to be decryptable (e.g. Dehydrated Devices: the ability to
decrypt messages sent to you when you are not logged in anywhere - or Sharing Keys when new users join a room/space).
However, the 4 points above are the main ones, and they are in the process of landing right now.
The end result is already an immeasurable improvement in the reliability and robustness of Matrix’s encryption - and
once the remaining pieces land, the user experience of a Matrix client should be that the encryption is almost entirely
invisible to the user - unless something bad is happening; much like TLS.
Valere & Patrick’s talk on Invisible Crypto from The Matrix Conference would get embedded here, but unfortunately the
audio was mangled - we’ll rerecord it and publish it shortly. In the meantime, you can find their slides
here.
What’s next?
Well, first of all we’re going to stop announcing Matrix 2.0 - it’s finally here (modulo the spec release)!
There’s obviously a bit of follow-up still to be done though:
- Getting MAS live on matrix.org
- Gracefully handling the lack of interop between legacy Matrix calls and MatrixRTC
- Landing the various remaining components of Invisible Crypto
- Broadening the implementation base for these APIs, and rolling it out across the ecosystem
Beyond Matrix 2.0, though, there’s a large pile of other areas which need attention:
- Improving state resolution. We’re currently investigating a set of issues where the merge resolution algorithm has
misbehaved, and are planning a new room version to address it - watch this space for updates.
- Trust and Safety. Abuse is increasing concern, and moderation and trust & safety tooling work has ended up fragmented
and balkanised. The Governing Board is putting together a cross-ecosystem working group to try to address this -
again, watch this space for updates.
- All the business-as-usual MSCs which have stacked up - Custom Emoji, Extensible Profiles, Custom Presence, etc.
- It’d also be good to finally realise the full performance advantages of faster room joins…
And then, in the longer term - what might Matrix 3.0 bring? Honestly, at this point, it’s an open question. Could it
be landing major Trust & Safety changes in order to radically empower users to avoid and mitigate abuse? Could it be
switching to MLS (or Decentralised MLS) for encryption? Could it be the glorious return of P2P Matrix (if it was
funded)? Could it even be figuring out if/how to converge with (or layer on top of) MIMI or other federation
protocols? Answers on a postcard to #sct-office:matrix.org please!
Conclusion
If ever there was a time to exhort your friends to give Matrix another go - this is it.
Matrix 2.0 has been a long time coming, and we need to get the word out that the step change forwards has finally
arrived, and that apps built on Matrix 2.0 can seriously outperform the mainstream alternatives. Ideally, this could
be Matrix’s “Firefox moment” - when from the ashes of old open source code, a new generation appears which can punch
its weight against the proprietary incumbents.
Right now the only Matrix 2.0 client is Element X (and Element Web/Desktop if you enable the experimental Simplified
Sliding Sync implementation in labs on develop/nightly builds) - see element.io/blog for full
details - but we expect to see at least matrix-rust-sdk and matrix-js-sdk based clients using the new APIs as a matter
of course in the coming months, and then hopefully everyone else too.
So: if you run a Matrix server, please consider
deploying MAS to enable next-gen auth
(Sebastien Spaeth’s blog also has a good indie tutorial) -
and add Element Call to give MatrixRTC a try. Over the coming months we
should see more support in more Matrix distributions, until hopefully we will all be living in a Matrix 2.0 world!
Huge thanks are due to everyone who has helped design, build, and iterate on Matrix 2.0, and to everyone who has kept
the faith as we’ve put it all together. A special thanks also to BWI who helped fund much of Element’s work on this for
the benefit of Matrix as a whole.
Finally, if you want Matrix to prevail - please join the Foundation and support the project financially. We urgently
need organisational members, especially after the costs of running The Matrix Conference, and particularly if your
organisation is commercially dependent on Matrix, you simply must become a member. While it’s very flattering that
Matrix gets treated as a commons these days, without financial support the underlying Matrix project will die and your
project will fail. Whereas if everyone building on Matrix supported us, we would be moving way faster and with fewer
constraints - so please get involved at matrix.org/support and help.
Thanks for flying Matrix!
Matthew
Login
