Normal view

There are new articles available, click to refresh the page.
Before yesterdayMain stream

The Matrix Holiday Special 2024

25 December 2024 at 07:00

Hi all,

Once again we celebrate the end of another year with the traditional Matrix Holiday Special! (see also 2023, 2022, 2021, 2020, 2019, 2018, 2017, 2016 and 2015 just in case you missed them).

This year, it is an incredible relief to be able to sit down and write an update which is overwhelmingly positive - in stark contrast to the rather mixed bags of 2022 and 2023. This is not to say that things are perfect: most notably, The Matrix.org Foundation has not yet hit its funding goals, and urgently needs more organisations who depend on Matrix to join as members in order to be financially sustainable. However, in terms of progress of Matrix towards outperforming the centralised alternatives; growth of the ecosystem; the success of the first ever Matrix Conference; we couldn’t be happier - and hopefully the more Matrix matures, the more folks will want to join the Foundation to help fund it.

So, precisely why are we feeling so happy right now?

🔗Matrix 2.0

Matrix 2.0 is the project to ensure that Matrix can be used to build apps which outcompete the incumbent legacy mainstream communication apps. Since announcing the project at FOSDEM 2023, we’ve been hard at work iterating on:

  • Sliding Sync, providing instant sync, instant login and instant launch.
  • Next Generation Auth via OIDC, to support instant login by QR code and consistent secure auth no matter the client.
  • Native Multiparty VoIP via MatrixRTC, to provide consistent end-to-end-encrypted calling conferencing within Matrix using Matrix’s encryption and security model.
  • Invisible Cryptography, to ensure that encryption in Matrix is seamless and no longer confuses users with unable-to-decrypt errors, scary shields and warnings, or other avoidable UX fails.

All of these projects are big, and we’ve been taking the time to iterate and get things right rather than cut corners – the whole name of the game has been to take Matrix from 1.0 (it works) to 2.0 (it works fast and delightfully, and outperforms the others). However, in September at the Matrix Conference we got to the point of shipping working implementations of all of the Matrix 2.0 MSCs, with the expectation of using these implementations to prove the viability of the MSCs and so propose them for merging into the spec proper.

Sliding Sync ended up evolving into MSC4186: Simplified Sliding Sync, and is now natively integrated into Synapse (no more need to run a sliding sync proxy!) and deployed on matrix.org, and implemented in matrix-rust-sdk and matrix-js-sdk. MatrixRTC is MSC4143 and dependents and is also deployed on matrix.org and call.element.io. Invisible Cryptography is a mix of MSCs: MSC4161 (Crypto terminology for non-technical users), MSC4153 (Exclude non-cross-signed devices), MSC3834 (Opportunistic user key pinning (TOFU)), and is mostly now implemented in matrix-rust-sdk - and Unable To Decrypt problems have been radically reduced (see Kegan’s excellent Matrix Conference talk for details). Finally, Next Gen Auth is MSC3861 and is planned to be deployed on matrix.org via matrix-authentication-service in Feb 2025.

It’s been controversial to ship Matrix 2.0 implementations prior to the MSCs being fully finalised and merged, but given the MSCs are backwards compatible with Matrix 1.0, and there’s unquestionable benefit to the ecosystem in getting these step-changes in the hands of users ASAP, we believe the aggressive roll-out is justified. Meanwhile, now the implementations are out and post-launch teething issues have largely been resolved, the MSCs will progress forwards.

One of the things we somehow failed to provide when announcing the implementations at the Matrix Conference was a playground for folks to experiment with Matrix 2.0 themselves. There’s now one based on Element’s stack of Synapse + matrix-authentication-service + Element Web + Element Call available at element-docker-demo in case you want to do a quick docker compose up to see what all the fuss is about! Meanwhile, matrix.org should support all the new MSCs in February – which might even coincide with the MSCs being finalised, you never know!

Rather than going through Matrix 2.0 in detail again, best bet is to check out the launch talk from The Matrix Conference…

Today's Matrix Live: https://youtube.com/watch?v=ZiRYdqkzjDU

…and in terms of seeing a Matrix 2.0 client in action, the Element X launch talk shows what you can do with it!

Today's Matrix Live: https://youtube.com/watch?v=gHyHO3xPfQU

Honestly, it is insanely exciting to see Matrix having evolved from the “good enough for enthusiastic geeks” to the “wow, this feels better than Signal” phase that we’re entering now. Meanwhile, matrix-rust-sdk is tracking all the latest Matrix 2.0 work, so any client built on matrix-rust-sdk (Fractal, Element X, iamb, etc) can benefit from it immediately. There’s also some really exciting matrix-rust-sdk improvements on the near horizon in the form of the long-awaited persistent event cache, which will accelerate all event operations enormously by avoiding needless server requests, as well as providing full offline support.

🔗The Matrix Conference

Talking of The Matrix Conference - this was by far the highlight of the year; not just due to being an excellent excuse to get Matrix 2.0 implementations launched, but because it really showed the breadth and maturity of the wider Matrix ecosystem.

One of the most interesting dynamics was that by far the busiest track was the Public Sector talk track (sponsored by Element) – standing room only, with folks queuing outside or watching the livefeed, whether this was Gematik talking about Matrix powering communications for the German healthcare industry, SwissPost showing off their nationwide Matrix deployment for Switzerland, DINUM showing off Tchap for France, NATO explaining NI²CE (their Matrix messenger), Försäkringskassan showing off Matrix for Sweden with SAFOS, Tele2 showcasing Tele2 Samarbete (Matrix based collaboration from one of Sweden’s main telcos), FITKO explaining how to do Government-to-Citizen communication with Matrix in Germany, ZenDiS using Matrix for secure communication in the German sovereign workspace openDesk project, or IBM showing off their Matrix healthcare deployments.

This felt really surprising: not only are we in an era where Matrix appears to be completely dominating secure communication and collaboration in the public sector – but it’s not just GovTech folks interested, but the wider Matrix community too.

I think it’s fair to say that when we created Matrix, we didn’t entirely anticipate this super-strong interest from government deployments – although in retrospect it makes perfect sense, given that more than anyone, nations wish to control their own infrastructure and run it securely without being operationally dependent on centralised solutions run out of other countries. A particular eye-opener recently has been seeing US Senators Ron Wyden (D) and Eric Schmitt (R) campaigning for the US Government to deploy Matrix in a way similar to France, Germany, Sweden and others. If this comes to pass, then it will surely create a whole new level of Matrix momentum!

It’s worth noting that while many Matrix vendors like Element, Nordeck, Famedly, connect2x and others have ended up mainly focusing commercially on public sector business (given that’s empirically where the money is right now) – the goal for Matrix itself continues to be mainstream uptake.

Matrix’s goal has always been to be the missing communication layer of the web for everyone, providing a worthy modern open replacement to both centralised messaging silos as well as outdated communication networks like email and the PSTN. It would be a sore failure of Matrix’s potential if it “only” ended up being successful for public sector communication! As it happens, our FOSDEM 2025 mainstage talk was just accepted, and happens to be named “The Road To Mainstream Matrix.” So watch this space to find out in February how all the Matrix 2.0 work might support mainstream Matrix uptake in the long-run, and how we plan to ensure Matrix expands beyond GovTech!

🔗The Governing Board

Another transformative aspect of 2024 was the formation of The Matrix.org Foundation Governing Board. Over to Josh with the details…

The election of our first ever Governing Board this year has gone a long way to ensuring we can truly call Matrix a public good, as something that is not only shared under an open source license and developed in the open, but also openly governed by elected representatives from across the ecosystem.

It took forming the Spec Core Team and the Foundation, both critical milestones on a journey of openness and independence, to pave the way. And with the Governing Board, we have a greater diversity of perspectives and backgrounds looking after Matrix than ever before!

The Governing Board is in the process of establishing its norms and processes and just last week published the first Governing Board report. Soon it’ll have elected committee chairs and vice chairs, and it appears to be on track to introduce our first working groups – official bodies to work together on initiatives in support of Matrix – at FOSDEM. Working groups will be another massive step forward, as they enable us to harmonize work across the ecosystem, such as on Trust & Safety and community events.

One last note on this, I want to shout out Greg Sutcliffe and Kim Brose, our first duly elected Chair and Vice Chair of the Governing Board, who have been doing great work to keep things in motion.

🔗Growing Membership

A key part of building the Governing Board has been recruiting to our membership program, which brings together organizations, communities, and individuals who are invested in Matrix. Our members illustrate the breadth of the ecosystem, and many of them are funders who help sustain our mission.

The Foundation has gone from being completely subsidized by Element in 2022, to having nearly half of its annual budget covered by its 11 funding members.

Of course, only being able to sustain half our annual budget is not nearly good enough, and it means that we live hand-to-mouth, extending our financial runway a bit at a time. It’s a nail biter of a ride for the hardworking staff who labor under this uncertainty, but we savor every win and all the progress we’ve made.

I’d like to take this opportunity to thank our funding members, including Element, our Gold Members, Automattic and Futurewei Technologies, our Silver Members, ERCOM, Fairkom, Famedly, Fractal Networks, Gematik, IndieHosters, Verji Tech, and XWiki.

We look forward to welcoming two new funding members in the coming weeks!

Our community-side members also play an important role, and we’re grateful for their work and participation. This includes our Associate Members, Eclipse Foundation, GNOME Foundation, and KDE, and our Ecosystem Members: Cinny, Community Moderation Effort, Conduit, Draupnir, Elm SDK, FluffyChat, Fractal, Matrix Community Events, NeoChat, Nheko-Reborn, Polychat, Rory&::LibMatrix, Thunderbird, and Trixnity.

If you’d like to see Matrix continue its momentum and the Foundation to further its work in ensuring Matrix is an independently and collectively governed protocol, please join the Foundation today. We need your help!

Back to you, Matthew!

🔗Focus

In 2023, we went through the nightmarishly painful process of ruthlessly focusing the core team exclusively on stabilising and polishing the foundations of Matrix – shelving all our next-generation showcases and projects and instead focusing purely on refining and evolving today’s Matrix core use cases for chat and VoIP.

In 2024, I’m proud to say that we’ve kept that focus – and indeed improved on it (for instance, we’ve stepped back on DMA work for much of the year in order to focus instead on the Trust & Safety work which has gone into Matrix 1.11, 1.12, and 1.13). As a result, despite a smaller team, we’ve made huge progress with Matrix 2.0, and the results speak for themselves. Anecdotally, I now wince whenever I have to use another messaging system – not because of loyalty to Matrix, but because the experience is going to be worse: WhatsApp has more "Waiting for message, this may take a while" errors (aka Unable To Decrypts or UTDs) than Matrix does, takes longer to launch and sync and has no threads; iMessage’s multidevice support can literally take hours to sync up; Signal just feels clunky and my message history is fragmented all over the place. It feels so good to be in that place, at last.

Meanwhile, it seems that Element’s move to switch development of Synapse and other projects to AGPL may have been for the best – it’s helped concretely address the issue of lack of commercial support from downstream projects, and Element is now in a much better position to continue funding Synapse and other core Matrix work. It’s also reassuring to see that 3rd party contributions to Synapse are as active as ever, and all the post-AGPL work on Synapse such as native sliding sync shows Element’s commitment to improving Synapse. Finally, while Dendrite dev is currently slow, the project is not abandoned, and critical fixes should keep coming – and if/when funding is available P2P Matrix & Dendrite work should resume as before. It wouldn’t be the first time Dendrite has come back from stasis!

🔗The Future

Beyond locking down Matrix 2.0 in the spec and getting folks using it, there are two big new projects on the horizon: MLS and State Res v3.

MLS is Messaging Layer Security (RFC 9420), the IETF standard for group end-to-end-encryption, and we’ve been experimenting with it for years now, starting around 2019, to evaluate it for use in Matrix alongside or instead of our current Double Ratchet implementation (Olm/Megolm). The complication on MLS is that it requires a logically centralised function to coordinate changes to the membership of the MLS group – whereas Matrix is of course fully decentralised; there’s never a central coordination point for a given conversation. As a result, we’ve been through several iterations of how to decentralise MLS to make it play nice with Matrix – essentially letting each server maintain its own MLS group, and then defining merge operations to merge those groups together. You can see the historical work over at https://arewemlsyet.com.

However, the resulting dialect of MLS (DMLS) has quite different properties to vanilla RFC 9420 MLS – for instance, you have to keep around some historical key data in case it’s needed to recover from a network partition, which undermines forward secrecy to some extent. Also, by design, Matrix network partitions can last forever, which means that the existing formal verification work that has been done around MLS’s encryption may not apply, and would need to be redone for DMLS.

Meanwhile, we’ve been participating in MIMI (More Instant Messaging Interoperability), an IETF working group focused on building a new messaging protocol to pair with MLS’s encryption. A hard requirement for MIMI is to utilise MLS for E2EE, and we went through quite a journey to see if Matrix could be used for MIMI, and understand how Matrix could be used with pure MLS (e.g. by having a centralised Matrix dialect like Linearized Matrix). Right now, MIMI is heading off in its own direction, but we’re keeping an eye on it and haven’t given up on converging somehow with it in future. And if nothing else, the exercise taught us a lot about marrying up Matrix and MLS!

Over the last few months there has been more and more interest in using MLS in Matrix, and at The Matrix Conference we gave an update on the latest MLS thinking, following a workshop at the conference with Franziskus from Cryspen (local MLS formal-verification experts in Berlin). Specifically, the idea is that it might be possible to come up with a dialect of Matrix which used pure RFC 9420 MLS rather than DMLS, potentially using normal Matrix rather than Linearized Matrix… albeit with MLS group changes mediated by a single ‘hub’ server in the conversation. The good news is that Cryspen proposed a mechanism where in the event of a network partition, both sides of the partition could elect a new hub and then merge the groups back together if the partition healed (handling history-sharing as an out-of-band problem, similar to the problem of sharing E2EE history when you join a new room in Matrix today). This would then significantly reduce the disadvantages of rooms having to have a centralised hub, given if the hub broke you could seamlessly continue the conversation on a new one.

So, we’ve now had a chance to sketch this out as MSC4244 - RFC 9420 MLS for Matrix, with two dependent MSCs (MSC4245 - Immutable encryption, and MSC4246 - Sending to-device messages as/to a server) and it’s looking rather exciting. This is essentially the protocol that Travis & I would have proposed to MIMI had the WG not dismissed decentralisation and dismissed Matrix interop - showing how it’s possible to use MLS for cryptographic group membership of the devices in a conversation, while still using Matrix for the user membership and access control around the room (complete with decentralisation). Best of all, it should also provide a solution to the longstanding problem of “Homeserver Control of Room Membership” highlighted by Albrecht & co from RHUL in 2022, by using MLS to prove that room membership changes are initiated by clients rather than malicious servers.

Now, we’re deliberately releasing this as a fairly early draft from the Spec Core Team in order to try to ensure that MLS spec work is done in the open, and to give everyone interested the opportunity to collaborate openly and avoid fragmentation. In the end, the SCT has to sign off on MSCs which are merged into Matrix, and we are responsible for ensuring Matrix has a coherent and secure architecture at the protocol layer – and for something as critical as encryption, the SCT’s role in coordinating the work is doubly important. So: if you’re interested in this space, we’d explicitly welcome collaboration and feedback on these MSCs in order to get the best possible outcome for Matrix – working together in the open, as per the Foundation’s values of ‘collaboration rather than competition’, and ‘transparency rather than stealth’.

Meanwhile, the other big new project on the horizon is State Resolution v3. Old-timers may remember that when we launched Matrix 1.0, one of the big changes was the arrival of State Resolution v2 (MSC1442), which fixed various nasty issues in the original merge conflict resolution algorithm Matrix uses to keep servers in sync with each other. Now, State Res v2 has subsequently served us relatively well (especially relative to State Res v1), but there have still been a few situations where rooms have state reset unexpectedly – and we’re currently in the process of chasing them down and proposing some refinements to the algorithm. There’s nothing to see yet, although part of the work here has been to dust off TARDIS, our trusty Time Agnostic Room DAG Inspection Service, to help visualise different scenarios and compare different resolution algorithms. So watch this space for some very pretty explanations once v3 lands!

🔗Happy New Year!

Matrix feels like it entered a whole new era in 2024 – with the Foundation properly spreading its wings, hosting The Matrix Conference, operationalising the Governing Board, and Matrix uptake exploding across the public sector of 20+ countries. Funding continues to be an existential risk, but as Matrix continues to accelerate we’re hopeful that more organisations who depend on Matrix will lean in to support the Foundation and ensure Matrix continues to prosper.

Meanwhile, 2025 is shaping up to be really exciting. It feels like we’ve come out of the darkness of the last few years with a 2.0 which is better than we could have possibly hoped, and I can’t wait to see where it goes from here!

Thanks to everyone for supporting the project - especially if you are a member of the Foundation (and if not, please join here!). We hope you have a fantastic end of the year; see you on the other side, and thanks for flying Matrix :)

Matrix 2.0 Is Here!

29 October 2024 at 07:00

Hi all,

Since the outset of Matrix, our aim has always been to provide a protocol that lets you build open, decentralised, secure communication apps which outperform the mainstream centralised alternatives. It’s been a twisty journey - first focusing on making Matrix work at all (back in 2014), and then getting it out of beta with Matrix 1.0 in 2019, and now focusing on making Matrix fast, usable and mainstream-ready with Matrix 2.0.

Meanwhile, the pendulum of decentralisation continues to accelerate in our direction. Our friends at Bluesky have shown that it’s possible to build decentralised social apps which are mainstream friendly enough for Presidents to recommend them; Elon continues to destroy Twitter and showcase the importance of decentralisation to everyone, and even Meta is dabbling in decentralised social media (and decentralised communication!)

So, where does Matrix sit in all this? Well, in order to make the transition to mainstream, we’ve been beavering away to implement four main pillars in Matrix 2.0:

  1. Instant login, instant launch, and instant sync (aka Simplified Sliding Sync, MSC4186)
  2. Next Generation Auth (aka Native OIDC, MSC3861)
  3. Native Matrix Encrypted Multiparty VoIP/Video (aka MatrixRTC, MSC4143)
  4. Invisible Encryption (MSC4153 & friends).

Between these, we believe that Matrix can now be used to build apps which genuinely outperform the mainstream alternatives - and what’s more, these MSCs now all have implementations you can use today. The MSCs themselves are not all finalised and ready for merge (but are getting close), and when they pass FCP (Final Comment Period) to merge into the spec, we will formally bump the spec release to version 2.0.

We actually declared Matrix 2.0 as ready for action back at The Matrix Conference last month, and now that the videos have been published you can watch the launch right here:

Today's Matrix Live: https://youtube.com/watch?v=ZiRYdqkzjDU

Since the conference talk things have already moved on a bit, though, and we’ve landed a bunch of tweaks to address teething issues - and so here’s the current state of action:

🔗1. Simplified Sliding Sync

Simplified Sliding Sync is the final version of Sliding Sync - the API which provides instant login, launch & sync in Matrix 2.0. To say that the API has been through a lot of iterations is an understatement, but we’re finally there with MSC4186, which simplifies the original Sliding Sync API (MSC3575) by removing the concept of a server-determined room list ordering entirely, and instead lets the client sort the roomlist as needed (while letting the client paginate in the roomlist incrementally, to ensure instant responsivity, no matter how large your room list is).

Simplified Sliding Sync is now implemented natively in Synapse as of 1.114, and so there is no longer any need to run a Sliding Sync Proxy in order to use the API. In fact, the Sliding Sync Proxy is being deprecated and the old Sliding Sync code will be removed from matrix-rust-sdk in the nearish future. Unfortunately we don’t have bandwidth to maintain both the native implementation Synapse as well as the proxy shim - and the proxy inevitably has suffered from a lot of limitations (e.g. having to do a full v2 initial sync for new logins, slowing them down to v2 performance - as well as duplicating storage between Synapse and the proxy). There’ll be a dedicated deprecation blog post for the proxy and pre-simplified Sliding Sync shortly. Meanwhile, work is well underway for native Sliding Sync support in conduit, conduwuit and grapevine - conduwuit guesses “next few weeks” for native Simplified Sync Support to land. Dendrite is unfortunately still not funded, so will happen on a best effort basis.

In terms of performance, native Simplified Sliding Sync in Synapse is spectacular - outperforming the proxy throughout, and making the old sync v2 API look positively prehistoric. Gone are the days of waiting for your app to sync, even if you’ve been offline for weeks/months; gone are the days of waiting minutes to login, and gone are the days of staring at a spinner as your app launches. It really is a new era - and having been hyping it since the first demo at FOSDEM 2023, I promise we won’t bang on about sliding sync any more after this; it’s finally here and landed and we can move on and enjoy it! For more details, see Ivan’s talk from The Matrix Conference:

Today's Matrix Live: https://youtube.com/watch?v=kI2lSCVEunw

🔗2. Next Generation Auth

Next Generation Auth is what we’re calling the migration to using industry standard OpenID Connect as the authentication API in Matrix 2.0, moving on from the custom auth API that Matrix has historically used. We’re calling this Next Gen Auth because we were seeing a lot of folks incorrectly assuming that adopting OpenID Connect for auth meant we would somehow be encouraging 3rd party social login or single-sign-on - which is not the case.

Next Gen Auth is simply swapping out Matrix’s old custom auth APIs with equivalents which are defined by the OpenID Foundation; after all, Matrix is a communication protocol, not an authentication protocol. In return, we get much more mature and secure authentication APIs - and access to the whole OpenID Identity Provider ecosystem, including support for Two Factor Auth and Multi-Factor Auth, hardware authentication tokens, passkeys, and device-based login flows (aka QR Login). We also stop both Matrix clients and servers having to implement the sprawling legacy Matrix authentication API surface - ensuring that users only ever hand their account password to their auth server, rather than having to trust their clients to handle it securely, which in turn plays much nicer with password managers (who only have to remember how to auth with your auth server, rather than a myriad different clients). It also lets you share authentication between apps if you want; gives us access_token refresh (at last!) to avoid long-lived access_tokens hanging around; and in future will also support OIDC scopes so you can limit the access particular clients get to your account.

In short, Next Gen Auth is transformative, and the initial implementation at matrix-authentication-service (MAS) is ready for admins to deploy and use (and in future will be available embedded in Synapse too). Unfortunately it is not yet live on matrix.org (given we have to migrate tens of millions of accounts, complete with all the social login complexities) but we’re hoping to get it there in the coming months.

Probably one of the clearest immediate benefits of Next Gen Auth is the ability to do a full login including setting up all your end-to-end-encryption simply by scanning a QR code on an existing client, courtesy of MSC4108 and OAuth 2.0 Device Authorization Grants. In other words, you don’t have to specify a server, or your username, or account password, or your recovery key - you just scan a QR code and you’re in. The very latest Element X releases on iOS & Android have this implemented and enabled by default, and so if you’re on a server which has deployed MAS, you can go to “link new device” in Element Web/Desktop to show a QR code to instantly log in.

For more info on all things Next Gen Auth, see Quentin’s talk from The Matrix Conference:

Today's Matrix Live: https://youtube.com/watch?v=wOW8keNafdE

🔗3. Native Matrix Group VoIP/Video: MatrixRTC

Next we have MatrixRTC: end-to-end-encrypted group voice and video conferencing over Matrix. Historically, group VoIP in Matrix has relied on third party conferencing systems (Jitsi, and before that FreeSWITCH) - providing no support for Matrix’s end-to-end encryption, or indeed Matrix’s user identities, decentralisation or decentralised access control.

With MatrixRTC this changes: we now have a standard way to establish large-scale end-to-end-encrypted group video calls via Matrix, leveraging all the benefits of Matrix’s end-to-end-encryption infrastructure, user identity, room permissions, etc. It also supports different media stacks to actually handle the media conferencing - today, the main implementation uses the LiveKit SFU, but there’s also an experimental full-mesh WebRTC implementation.

Element Call has been the driving app behind MatrixRTC, and as of today is now enabled in the release versions of both Element Web/Desktop and Element X to provide native MatrixRTC calling embedded in the apps: if you hit the video call button you will now have the option to spin up a MatrixRTC call via Element Call rather than via Jitsi or Legacy 1:1 calling, and if the room is end-to-end-encrypted, all the conference will be too.

Meanwhile - MatrixRTC isn’t just Element Call: Famedly showed off experimental interop with FluffyChat at FOSDEM back in Feb, and Element showed off experimental interop with BigBlueButton in August. Given more and more conferencing tools are converging on LiveKit as a best-in-class SFU, it’s an amazing opportunity to use Matrix and MatrixRTC to power the end-to-end-encryption and decentralisation and get standardised voip/video interop from the outset.

For more info, see Timo’s talk from The Matrix Conference:

Today's Matrix Live: https://youtube.com/watch?v=OXPuYbfiXDQ

That said, there are a few caveats right now:

  • We do not have interoperability between legacy Matrix 1:1 voice/video calling and MatrixRTC (and it’s not clear if/when we will get to it) - but Matrix 2.0 clients like Element X exclusively use MatrixRTC for VoIP/video, including for 1:1 calls. This is in order to only maintain one VoIP stack, and so that you get multidevice and multiuser support for free, even in 1:1s. As a result, we’re in the process of figuring out how to warn legacy callers that MatrixRTC-only clients won’t be able to answer their calls (e.g. MSC4220) - this hasn’t shipped yet.
  • iOS 18 broke CallKit + WebRTC, so Element X iOS has had to disable fancy OS-natively-integrated MatrixRTC calling; and has a support issue open with Apple to try to solve this.
  • We’ve had some fun teething issues thanks to the volume of signalling in MatrixRTC exposing some sync bugs - these are almost solved, but probably mean MatrixRTC should still be considered beta for a few more days until the fixes land.

🔗4. Invisible Encryption

The final pillar of Matrix 2.0 is Invisible Encryption - making Matrix’s end-to-end encryption as seamless and invisible as the centralised alternatives (Signal, WhatsApp, iMessage and friends). This does not mean reducing security in any way - just the opposite, in fact. It means:

  1. Ensuring that Unable To Decrypt (UTD) bugs never happen. Huge amounts of work has gone into this over the course of the year, especially via complement-crypto as a comprehensive end-to-end-test suite for both matrix-rust-sdk and matrix-js-sdk based Matrix clients. We are finally at the point where UTDs are so rare that most people simply never see them, and any reports get jumped on (and complement-crypto tests get written) whenever they emerge. Anecdotally, I now get way more “waiting for message…” errors on WhatsApp than I do on Matrix, these days! For more details on the Hunt For UTDs, see Kegan’s talk from The Matrix Conference:
Today's Matrix Live: https://youtube.com/watch?v=FHzh2Y7BABQ
  1. We are excluding non-cross-signed devices from Matrix (MSC4153). The fact that Matrix ever supported the idea of users enabling encryption on a device without proving that they are the valid owner by signing it (by verifying it with another device, or providing their recovery key/passphrase) is a nasty hangover from back before we introduced cross-signing. Nowadays, the fact that non-cross-signed devices exist acts to reduce security and complicate UX and implementations with big scary red warnings whenever an unverified device joins a conversation. Instead, once MSC4153 lands, we’re going to simply exclude unverified devices entirely - not encrypt to them, and not decrypt from them. We can then get rid of all the confusing warnings associated with them.

  2. We’re also solving the confusing “grey shield” warnings: “the authenticity of this message cannot be confirmed on this device” and similar. These are avoidable warnings caused by message keys which can no longer be tracked back to the original sender - e.g. if they’re restored from backup, or if the original sending device has been deleted. We’re fixing these with authenticated backup (MSC4048) and including device keys on Olm events (MSC4147) respectively.

  3. Finally, we’re moving to Trust On First Use (TOFU). This means that even if you didn’t explicitly verify another user, you still get warned if their identity changes (unlike previously, when it was ignored). An initial implementation just landed in matrix-rust-sdk a few weeks ago, and so appropriate warnings at the application level should arrive shortly - matching the equivalent Signal or WhatsApp “this user’s identity has changed” warnings, albeit not yet synced between devices.

As you can probably tell, Invisible Encryption is something of a moving target; for instance, the scope could extend to cover all scenarios where users might expect messages to be decryptable (e.g. Dehydrated Devices: the ability to decrypt messages sent to you when you are not logged in anywhere - or Sharing Keys when new users join a room/space). However, the 4 points above are the main ones, and they are in the process of landing right now.

The end result is already an immeasurable improvement in the reliability and robustness of Matrix’s encryption - and once the remaining pieces land, the user experience of a Matrix client should be that the encryption is almost entirely invisible to the user - unless something bad is happening; much like TLS.

Valere & Patrick’s talk on Invisible Crypto from The Matrix Conference would get embedded here, but unfortunately the audio was mangled - we’ll rerecord it and publish it shortly. In the meantime, you can find their slides here.

🔗What’s next?

Well, first of all we’re going to stop announcing Matrix 2.0 - it’s finally here (modulo the spec release)!

There’s obviously a bit of follow-up still to be done though:

  • Getting MAS live on matrix.org
  • Gracefully handling the lack of interop between legacy Matrix calls and MatrixRTC
  • Landing the various remaining components of Invisible Crypto
  • Broadening the implementation base for these APIs, and rolling it out across the ecosystem

Beyond Matrix 2.0, though, there’s a large pile of other areas which need attention:

  • Improving state resolution. We’re currently investigating a set of issues where the merge resolution algorithm has misbehaved, and are planning a new room version to address it - watch this space for updates.
  • Trust and Safety. Abuse is increasing concern, and moderation and trust & safety tooling work has ended up fragmented and balkanised. The Governing Board is putting together a cross-ecosystem working group to try to address this - again, watch this space for updates.
  • All the business-as-usual MSCs which have stacked up - Custom Emoji, Extensible Profiles, Custom Presence, etc.
  • It’d also be good to finally realise the full performance advantages of faster room joins…

And then, in the longer term - what might Matrix 3.0 bring? Honestly, at this point, it’s an open question. Could it be landing major Trust & Safety changes in order to radically empower users to avoid and mitigate abuse? Could it be switching to MLS (or Decentralised MLS) for encryption? Could it be the glorious return of P2P Matrix (if it was funded)? Could it even be figuring out if/how to converge with (or layer on top of) MIMI or other federation protocols? Answers on a postcard to #sct-office:matrix.org please!

🔗Conclusion

If ever there was a time to exhort your friends to give Matrix another go - this is it.

Matrix 2.0 has been a long time coming, and we need to get the word out that the step change forwards has finally arrived, and that apps built on Matrix 2.0 can seriously outperform the mainstream alternatives. Ideally, this could be Matrix’s “Firefox moment” - when from the ashes of old open source code, a new generation appears which can punch its weight against the proprietary incumbents.

Right now the only Matrix 2.0 client is Element X (and Element Web/Desktop if you enable the experimental Simplified Sliding Sync implementation in labs on develop/nightly builds) - see element.io/blog for full details - but we expect to see at least matrix-rust-sdk and matrix-js-sdk based clients using the new APIs as a matter of course in the coming months, and then hopefully everyone else too.

So: if you run a Matrix server, please consider deploying MAS to enable next-gen auth (Sebastien Spaeth’s blog also has a good indie tutorial) - and add Element Call to give MatrixRTC a try. Over the coming months we should see more support in more Matrix distributions, until hopefully we will all be living in a Matrix 2.0 world!

Huge thanks are due to everyone who has helped design, build, and iterate on Matrix 2.0, and to everyone who has kept the faith as we’ve put it all together. A special thanks also to BWI who helped fund much of Element’s work on this for the benefit of Matrix as a whole.

Finally, if you want Matrix to prevail - please join the Foundation and support the project financially. We urgently need organisational members, especially after the costs of running The Matrix Conference, and particularly if your organisation is commercially dependent on Matrix, you simply must become a member. While it’s very flattering that Matrix gets treated as a commons these days, without financial support the underlying Matrix project will die and your project will fail. Whereas if everyone building on Matrix supported us, we would be moving way faster and with fewer constraints - so please get involved at matrix.org/support and help.

Thanks for flying Matrix!

Matthew

Update on Native Matrix interoperability with WhatsApp

16 September 2024 at 07:00

Hi all,

Back at FOSDEM in February we showed off how Matrix could be used for E2EE-preserving messaging interoperability as required by the Digital Markets Act messaging interoperability - and we announced that Element had been working with Meta on integrating with its DMA APIs in order to connect WhatsApp to Matrix. You can see the video here, and we also demoed interop working at the technical level to the European Commission a few days beforehand.

Subsequently WhatsApp launched its DMA portal on March 8th, and the proposed Reference Offer (i.e. the terms you have to accept as a Requesting Party in order to interoperate) was revealed. The Reference Offer for Facebook Messenger was launched on September 6th. At the time of the WhatsApp launch we flagged up some significant unresolved questions - the main points being that:

  1. WhatsApp would require their users to manually enable DMA in settings before they can receive any traffic from interconnecting service providers (e.g. Element) - meaning that WhatsApp users would not be reachable by default.

  2. WhatsApp would require the client IP of any interconnecting users, in order to apply ‘platform integrity’ anti-abuse / trust & safety controls.

  3. WhatsApp would not allow an interconnecting service to buffer messages serverside.

  4. WhatsApp would require each Matrix server provider to sign a separate agreement in order to interconnect - i.e. you can’t bridge other server’s users unless those servers have signed a contract with Meta.

Now, the good news is that we’ve subsequently been talking with WhatsApp to see if we could progress these points - and we’re happy to say that they’ve listened to us and we’ve made progress on the first 3 items:

  1. Meta recently shared an update on the messaging interoperability user experience and will allow all EU users to be reachable by interoperable services by default. It’ll also give people the option of how they want to manage their inbox as well as a range of features like read receipts, typing indicators and reactions.

  2. We’ve come up with a plan with WhatsApp to reduce the amount of matrix user data we share with WhatsApp. WhatsApp’s interop solution however, doesn’t yet support multi-device conversations or shared conversation history like normal Matrix, which means that normal Matrix server-side synchronised history won’t work for these conversations.

  3. In terms of not allowing open federation: this looks unlikely to change, given Meta needs to know who is responsible for the servers who connect to them, and ensure they agree to the terms of use as required by DMA.

During discussions, another point came up which we’d previously overlooked: section 7.5.1 of the current reference offer states: Partner User Location. Any Partner Users that Partner Enlists or provides access to the Interoperable Messaging Services must be located and remain in the EEA”. In other words, interop would only be available to Matrix users physically in the EEA, which is obviously against the Matrix Foundation’s manifesto to provide secure communication to everyone. Moreover, to demonstrate compliance the Matrix side would have to geolocate the client’s IP.

It turns out that this limitation to EU users ended up being the biggest obstacle for productising the native Matrix<>WhatsApp bridge, as it is unclear whether it’s financially viable for anyone (e.g. Element) to launch such a bridge if it only works for Matrix users physically within the EEA (not to mention the costs and privacy issues of geolocating Matrix users).

Now, on one hand, deploying Matrix as a mature standards-based protocol for WhatsApp interop with native E2EE feels like a worthy goal: indeed, it effectively gives DMA interoperators a stable standardised API with pre-existing SDKs to implement against, rather than having to implement against proprietary and potentially shifting vendor APIs. So overall it moves the needle towards the end goal of Matrix’ mission.

On the other hand, this all may be moot if the return on investment of building DMA interop with WhatsApp via Matrix is too far away for any company in the Matrix ecosystem to be able to afford the investment, and if there isn’t an appetite for anyone to fund it. Funding constraints on both the Foundation and the ecosystem today are such that this work will only happen if explicitly sponsored by an organisation who is willing to commit to fund it.

So: if you are an organisation with users in the EU who would like them to interoperate with EU WhatsApp users via Matrix, and have the funds to sponsor development of building out an official production-grade Matrix<->WhatsApp bridge, please get in touch with me.

Alternatively, if the geographic constraints are a showstopper for you, please let us know.

We’re assuming that there may be smaller messaging providers, or domain-specific messaging services who want to connect their end-users through to WA end-users, and may be happy to be constrained to EU geography. However, bridge developers need evidence and financial support to progress this. Meanwhile, if you are interested in the strategic importance of the Digital Markets Act, this is an opportunity to put your money where your mouth is.

Looking forward to hearing feedback!

thanks,

Matthew

❌
❌