Navigating the complex world of containerized testing environments can be challenging, especially when dealing with Docker-in-Docker (DinD). As a senior DevOps engineer and Docker Captain, I’ve seen firsthand the hurdles that teams face with DinD, and here I’ll share why Testcontainers Cloud is a transformative alternative that’s reshaping the way we handle container-based testing.

Understanding Docker-in-Docker

Docker-in-Docker allows you to run Docker within a Docker container. It’s like Inception for containers — a Docker daemon running inside a Docker container, capable of building and running other containers.

How Docker-in-Docker works

• Nested Docker daemons: In a typical Docker setup, the Docker daemon runs on the host machine, managing containers directly on the host’s operating system. With DinD, you start a Docker daemon inside a container. This inner Docker daemon operates independently, enabling the container to build and manage its own set of containers.
• Privileged mode and access to host resources: To run Docker inside a Docker container, the container needs elevated privileges. This is achieved by running the container in privileged mode using the --privileged flag:

docker run --privileged -d docker:dind

• The --privileged flag grants the container almost all the capabilities of the host machine, including access to device files and the ability to perform system administration tasks. Although this setup enables the inner Docker daemon to function, it poses significant security risks, as it can potentially allow the container to affect the host system adversely.

• Filesystem considerations: The inner Docker daemon stores images and containers within the file system of the DinD container, typically under /var/lib/docker. Because Docker uses advanced file system features like copy-on-write layers, running an inner Docker daemon within a containerized file system (which may itself use such features) can lead to complex interactions and potential conflicts.
• Cgroups and namespace isolation: Docker relies on Linux kernel features like cgroups and namespaces for resource isolation and management. When running Docker inside a container, these features must be correctly configured to allow nesting. This process can introduce additional complexity in ensuring that resource limits and isolation behave as expected.

Why teams use Docker-in-Docker

• Isolated build environments: DinD allows each continuous integration (CI) job to run in a clean, isolated Docker environment, ensuring that builds and tests are not affected by residual state from previous jobs or other jobs running concurrently.
• Consistency across environments: By encapsulating the Docker daemon within a container, teams can replicate the same Docker environment across different stages of the development pipeline, from local development to CI/CD systems.

Challenges with DinD

Although DinD provides certain benefits, it also introduces significant challenges, such as:

• Security risks: Running containers in privileged mode can expose the host system to security vulnerabilities, as the container gains extensive access to host resources.
• Stability issues: Nested containers can lead to storage driver conflicts and other instability issues, causing unpredictable build failures.
• Complex debugging: Troubleshooting issues in a nested Docker environment can be complicated, as it involves multiple layers of abstraction and isolation.

Real-world challenges

Although Docker-in-Docker might sound appealing, it often introduces more problems than it solves. Before diving into those challenges, let’s briefly discuss Testcontainers and its role in modern testing practices.

What is Testcontainers?

Testcontainers is a popular open source library designed to support integration testing by providing lightweight, disposable instances of common databases, web browsers, or any service that can run in a Docker container. It allows developers to write tests that interact with real instances of external resources, rather than relying on mocks or stubs.

Key features of Testcontainers

• Realistic testing environment: By using actual services in containers, tests are more reliable and closer to real-world scenarios.
• Isolation: Each test session, or even each test can run in a clean environment, reducing flakiness due to shared state.
• Easy cleanup: Containers are ephemeral and are automatically cleaned up after tests, preventing resource leaks.

Dependency on the Docker daemon

A core component of Testcontainers’ functionality lies in its interaction with the Docker daemon. Testcontainers orchestrates Docker resources by starting and stopping containers as needed for tests. This tight integration means that access to a Docker environment is essential wherever the tests are run.

The DinD challenge with Testcontainers in CI

When teams try to include Testcontainers-based integration testing in their CI/CD pipelines, they often face the challenge of providing Docker access within the CI environment. Because Testcontainers requires communication with the Docker daemon, many teams resort to using Docker-in-Docker to emulate a Docker environment inside the CI job.

However, this approach introduces significant challenges, especially when trying to scale Testcontainers usage across the organization.

Case study: The CI pipeline nightmare

We had a Jenkins CI pipeline that utilized Testcontainers for integration tests. To provide the necessary Docker environment, we implemented DinD. Initially, it seemed to work fine, but soon we encountered:

• Unstable builds: Random failures due to storage driver conflicts and issues with nested container layers. The nested Docker environment sometimes clashed with the host, causing unpredictable behavior.
• Security concerns: Running containers in privileged mode raised red flags during security audits. Because DinD requires privileged mode to function correctly, it posed significant security risks, potentially allowing containers to access the host system.
• Performance bottlenecks: Builds were slow, and resource consumption was high. The overhead of running Docker within Docker led to longer feedback loops, hindering developer productivity.
• Complex debugging: Troubleshooting nested containers became time-consuming. Logs and errors were difficult to trace through the multiple layers of containers, making issue resolution challenging.

We spent countless hours trying to patch these issues, but it felt like playing a game of whack-a-mole.

Why Testcontainers Cloud is a better choice

Testcontainers Cloud is a cloud-based service designed to simplify and enhance your container-based testing. By offloading container execution to the cloud, it provides a secure, scalable, and efficient environment for your integration tests.

How TestContainers Cloud addresses DinD’s shortcomings

Enhanced security

• No more privileged mode: Eliminates the need for running containers in privileged mode, reducing the attack surface.
• Isolation: Tests run in isolated cloud environments, minimizing risks to the host system.
• Compliance-friendly: Easier to pass security audits without exposing the Docker socket or granting elevated permissions.

Improved performance

• Scalability: Leverage cloud resources to run tests faster and handle higher loads.
• Resource efficiency: Offloading execution frees up local and CI/CD resources.

Simplified configuration

• Plug-and-play integration: Minimal changes are required to switch from local Docker to Testcontainers Cloud.
• No nested complexity: Avoid the intricacies and pitfalls of nested Docker daemons.

Better observability and debugging

• Detailed logs: Access comprehensive logs through the Testcontainers Cloud dashboard.
• Real-time monitoring: Monitor containers and resources in real time with enhanced visibility.

Getting started with Testcontainers Cloud

Let’s dive into how you can get the most out of Testcontainers Cloud.

Switching to Testcontainers Cloud allows you to run tests without needing a local Docker daemon:

• No local Docker required: Testcontainers Cloud handles container execution in the cloud.
• Consistent environment: Ensures that your tests run in the same environment across different machines.

Additionally, you can easily integrate Testcontainers Cloud into your CI pipeline to run the same tests without scaling your CI infrastructure.

Using Testcontainers Cloud with GitHub Actions

Here’s how you can set up Testcontainers Cloud in your GitHub Actions workflow.

1. Create a new service account

• Log in to Testcontainers Cloud dashboard.
• Navigate to Service Accounts:
  • Create a new service account dedicated to your CI environment.
• Generate an access token:
  • Copy the access token. Remember, you can only view it once, so store it securely.

2. Set the TC_CLOUD_TOKEN environment variable

• In GitHub Actions:
  • Go to your repository’s Settings > Secrets and variables > Actions.
  • Add a new Repository Secret named TC_CLOUD_TOKEN and paste the access token.

3. Add Testcontainers Cloud to your workflow

Update your GitHub Actions workflow (.github/workflows/ci.yml) to include the Testcontainers Cloud setup.

Example workflow:

name: CI Pipeline

on:
  push:
    branches: [ main ]
  pull_request:
    branches: [ main ]

jobs:
  build:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3

      # ... other preparation steps (dependencies, compilation, etc.) ...

      - name: Set up Java
        uses: actions/setup-java@v3
        with:
          distribution: 'temurin'
          java-version: '17'

      - name: Setup Testcontainers Cloud Client
        uses: atomicjar/testcontainers-cloud-setup-action@v1
        with:
          token: ${{ secrets.TC_CLOUD_TOKEN }}

      # ... steps to execute your tests ...
      - name: Run Tests
        run: ./mvnw test

Notes:

• The atomicjar/testcontainers-cloud-setup-action GitHub Action automates the installation and authentication of the Testcontainers Cloud Agent in your CI environment.
• Ensure that your TC_CLOUD_TOKEN is kept secure using GitHub’s encrypted secrets.

Clarifying the components: Testcontainers Cloud Agent and Testcontainers Cloud

To make everything clear:

• Testcontainers Cloud Agent (CLI in CI environments): In CI environments like GitHub Actions, you use the Testcontainers Cloud Agent (installed via the GitHub Action or command line) to connect your CI jobs to Testcontainers Cloud.
• Testcontainers Cloud: The cloud service that runs your containers, offloading execution from your CI environment.

In CI environments:

• Use the Testcontainers Cloud Agent (CLI) within your CI jobs.
• Authenticate using the TC_CLOUD_TOKEN.
• Tests executed in the CI environment will use Testcontainers Cloud.

Monitoring and debugging

Take advantage of the Testcontainers Cloud dashboard:

• Session logs: View logs for individual test sessions.
• Container details: Inspect container statuses and resource usage.
• Debugging: Access container logs and output for troubleshooting.

Why developers prefer Testcontainers Cloud over DinD

Real-world impact

After integrating Testcontainers Cloud, our team observed the following:

• Faster build times: Tests ran significantly faster due to optimized resource utilization.
• Reduced maintenance: Less time spent on debugging and fixing CI pipeline issues.
• Enhanced security: Eliminated the need for privileged mode, satisfying security audits.
• Better observability: Improved logging and monitoring capabilities.

Addressing common concerns

Security and compliance

• Data isolation: Each test runs in an isolated environment.
• Encrypted communication: Secure data transmission.
• Compliance: Meets industry-standard security practices.

Cost considerations

• Efficiency gains: Time saved on maintenance offsets the cost.
• Resource optimization: Reduces the need for expensive CI infrastructure.

Compatibility

• Multi-language support: Works with Java, Node.js, Python, Go, .NET, and more.
• Seamless integration: Minimal changes required to existing test code.

Conclusion

Switching to Testcontainers Cloud, with the help of the Testcontainers Cloud Agent, has been a game-changer for our team and many others in the industry. It addresses the key pain points associated with Docker-in-Docker and offers a secure, efficient, and developer-friendly alternative.

Key takeaways

• Security: Eliminates the need for privileged containers and Docker socket exposure.
• Performance: Accelerates test execution with scalable cloud resources.
• Simplicity: Simplifies configuration and reduces maintenance overhead.
• Observability: Enhances debugging with detailed logs and monitoring tools.

As someone who has navigated these challenges, I recommend trying Testcontainers Cloud. It’s time to move beyond the complexities of DinD and adopt a solution designed for modern development workflows.

Additional resources

• Testcontainers Cloud documentation:
  • Getting started
• Testcontainers library docs:
  • Java
  • Node.js
  • Python
• Best practices:
  • Testcontainers Best Practices
• Community support:
  • Join the Testcontainers Slack
  • GitHub discussions
  • Stack Overflow

This October has been tremendously packed with my duty as Seeed Studio’s Ranger. From OSHWA to Pycon APAC 2024, every event left a great spot on my mind. Here is the recap for my activity this October 2024.

1. October 3rd 2024, OSHWA 24 Hours Membership Drive Show & Tell

This live online session is initiated by OSHWA and runs 24 hours straight. My session is scheduled to run at 6 PM local time. After rushing to be at home before my schedule, I managed to get on time and share my open-source projects on Hackster. After sharing my open-source projects, I introduce the Seeed Studio Co-Create Gadget campaign to the audience.

2. October 17th 2024, Transforming IoT with Edge Intelligence Seminar

This seminar is initiated by the student organization on my campus and has two speakers. The other speaker is one of my ex student whom now work as a cloud engineer. It is the perfect opportunity to introduce Seeed Studio to my campus while also giving the students insight into the latest technology in AI, Machine Learning, IoT, and Edge Computing.

In the seminar, I mainly focus on the next step of IoT which is the inclusivity of Machine Learning in IoT solutions and how reComputer could be a small compact solution to implement it.

3. October 25th 2024, Pycon APAC 2024

This is my highlight of the month because the Python Indonesia community is becoming the host for this year’s annual Python Conference of the Asia Pacific area. In total, there are 80 Speakers from across Asia pacific with hundreds of participants.

At Pycon I managed to schedule a workshop session on 25th October 2024 titled “Building IoT and Camera Project with XIAO ESP32S3 and Micropython”.

In this workshop, I shared various things that the XIAO ESP32S3 can do with Micropython such as basic input and output, camera stream, displaying JSON weather data on the OLED screen, and interacting with the Grove Vision AI V2 sensor.

4. November 2nd 2024, Internet of Things Seminar at Universitas Banten Jaya

A friend from another campus has invited me to this seminar. The location of campus is in Serang, a neighboring city from where I live and it is a 2-hour drive from home.

The material that I shared in this seminar is the same as what I shared at my previous seminar on my campus. The organizer told me that they don’t have much insight regarding IoT on campus, so this seminar could be a huge step for advancing the knowledge on their campus.

The post Hacktober Recap, Workshop and Seminar for Open Source Community appeared first on Latest Open Tech From Seeed.

In nearly 1,000 days, the WordPress community has created 1,000 Block themes—coming together to use the full potential of the Site Editor and unleash new creative possibilities for everyone.

First introduced in WordPress 5.9, Block themes have steadily evolved, improving flexibility and functionality for themers, users, and agencies alike. Now, design tools allow customizing almost every detail. With style variations, users can change the overall look of their site in a few clicks. You can even use curation options to customize the editing process itself. But we’re not done! We can’t wait to keep pushing Block themes even further. Thank you to every early adopter who, by embracing early features with passion, helped shape the Block themes we love today with feedback and testing.

If you haven’t yet explored Block themes, check out some of the resources below to get inspired:

• Use one of the 1,000 Block themes available and see what you can create. For an easy way to explore, try WordPress Playground to quickly get started with a temporary site. 
• Read the Theme handbook to understand types of themes, how themes work, and how to get started building. 
• Use the Create Block Theme plugin to streamline Block theme creation with a dedicated tutorial on Learn WordPress that will help you get started.  
• Get inspired by watching and joining community members building themes together. This is a great way for folks new to Block themes to learn, hands-on. 

Let’s celebrate and share our contributions! Please comment on the Theme Team’s post dedicated to highlighting this milestone to share your favorite Block theme and thank those who have contributed along the way. 

Thank you to @kristastevens for editorial help, @beafialho for the featured image, and @kafleg for reviewing.

Cloud Expo Asia 2024 in Singapore drew thousands of cloud professionals and tech business leaders to explore and exchange the latest in cloud computing, security, GenAI, sustainability, DevOps, and more. At our Cloud Expo Asia booth, Docker showcased our latest innovations in AI integration, containerization, security best practices, and updated product offerings. Here are a few highlights from our experience at the event.

AI/ML and GenAI everywhere

AI/ML and GenAI were hot topics at Cloud Expo Asia. Docker CPO Giri Sreenivas’s talk on Transforming App Development: Docker’s Advanced Containerization and AI Integration highlighted that GenAI impacts software in two big ways — it accelerates product development and creates new types of products and experiences. He discussed how containers are an ideal tool for containerizing GenAI workflows in development, ensuring consistency across CI/CD pipelines and reproducibility across diverse platforms in production.

Sreenivas highlighted the Docker extension for GitHub Copilot as an example of how Docker helps empower development teams to focus on innovation — closing the gap from the first line of code to production. Sreenivas also gave a sneak peek into upcoming products designed to streamline GenAI development to illustrate Docker’s commitment to evolving solutions to meet emerging needs. 

Adopting security best practices and shifting left

Developer efficiency and security were also popular themes at the event. When Sreenivas mentioned in his talk that security vulnerabilities that cost dollars to fix early in development would cost hundreds of dollars later in production, members of the audience nodded in agreement.

Docker CTO Justin Cormack gave a keynote address titled “The Docker Effect: Driving Developer Efficiency and Innovation in a Hybrid World.” He discussed how implementing best practices and investing in the inner loop are crucial for today’s development teams. 

One best practice, for example, is shifting left and identifying problems as quickly as possible in the software development lifecycle. This approach improves efficiency and reduces costs by detecting and addressing software issues earlier before they become expensive problems.

Cormack also provided a few tips for meeting the security and control needs of modern enterprises with a layered approach. Start with key building blocks, he explained, such as trusted content, which provides dev teams with a good foundation to build securely from the start. 

At the Docker event booth, we demonstrated Docker Scout, which helps development teams identify, analyze, and remediate security vulnerabilities early in the dev process. Docker Business customers can take advantage of enterprise controls, letting admins, IT teams, and security teams continuously monitor and manage risk and compliance with confidence. 

New Docker innovations and updated plan

From students to C-level executives who visited our booth, everyone was eager to learn more about containers and Docker. People lined up to see an end-to-end demo of how the suite of Docker products, such as Docker Desktop, Docker Hub, Docker Build Cloud, and Docker Scout, work together seamlessly to enable development teams to work more efficiently. 

Attendees also had the opportunity to learn more about Docker’s updated plans, which makes accessing the full suite of Docker products and solutions easy, with options for individual developers, small teams, and large enterprises.

Thanks, Cloud Expo Asia!

We enjoyed our conversations with event attendees and appreciate everyone who helped make this such a successful event. Thank you to the organizers, speakers, sponsors, and the community for a productive, information-packed experience.

From accelerating app development, supporting best practices of shifting left, meeting the security and control needs of modern enterprises, and innovating with GenAI, Docker wants to be your trusted partner to navigate the challenges in modern app development. 

Explore our Docker updated plans to learn how Docker can empower your teams, or contact our sales team to discover how we can help you innovate with confidence.

Learn more

• Find out how we’re investing in innovation and customer relationships.
• Read about updated Docker plans: Simpler, More Value, Better Development and Productivity.
• Subscribe to the Docker Newsletter. 

At the heart of our community is our shared pledge to create a space that is harassment-free, welcoming, and inclusive for all. Our Community Code of Conduct already outlines a clear set of expectations, while also providing examples of unacceptable actions. Today, we are reinforcing our values by adding another element to our list of unacceptable behaviors: Publishing private messages without consent.

Why This Addition Matters

The relationships we build within our community often involve private discussions. These conversations may involve sensitive matters, personal experiences, or simply casual exchanges. Regardless of the content, every individual should feel confident that their private communications will remain private unless they grant explicit permission to share them.

Sharing private messages without consent is a breach of trust that can also lead to unintended harm, including emotional distress or misrepresentation. When members of our community feel they cannot trust others in their personal conversations, it undermines the collaborative spirit that is crucial to our collective success.

How This Change Protects the Community

By explicitly addressing the publication of private messages without consent, we are reinforcing an existing unacceptable behavior in our Community Code of Conduct: Other conduct which could reasonably be considered inappropriate in a professional setting. Sharing private communications without permission is a clear violation of professional integrity.

This new addition ensures that private messages receive the same level of protection as personal information and that sensitive communications shared in confidence will not be disclosed without prior consent. An important exception to this is when sharing private messages is necessary for reporting incidents or concerns to the Incident Response Team, as part of our commitment to maintaining a safe and respectful environment.

Ultimately, this change encourages honest, constructive engagement across all levels of participation.

Moving Forward Together

The strength of our community lies in the trust we place in one another. By clarifying and reinforcing our expectations, we are taking another step toward maintaining an inclusive, respectful, and safe environment for everyone.This new addition will take effect immediately, and violations will be handled in accordance with our existing enforcement guidelines. Together, we can ensure our community remains a place of collaboration, trust, and mutual respect.

In the midst of our legal battles with Silver Lake and WP Engine, I wanted to take a moment to highlight something positive.

Because of my friendships with the co-founders of Slack, Stewart Butterfield and Cal Henderson, WordPress.org has had a free version of the Pro version of Slack since they started in 2009. We switched from IRC to Slack, and it was like superpowers were unlocked for our team.

Over the past 10 years, Slack has been our secret weapon of productivity compared to many other open source projects. Its amazing collaboration features have allowed us to scale WordPress from running just a few blogs to now powering around 43% of all websites in the world, almost 10 times the runner-up in the market.

As we have scaled from very small to very large, Slack has scaled right alongside us, seemingly effortlessly. WordPress.org currently has 49,286 users on its Slack Business+ instance, which would cost at least $8.8M/yr if we were paying. (And we may need to go to their enterprise grid, to support e-discovery in the lawsuit attacks from WP Engine, which would cost even more.)

This incredible generosity was continued by the enlightened leadership of Marc Benioff at Salesforce when they bought Slack in 2020. However, it has not been widely known or recognized on our Five for the Future page, which only highlights self-reported contributor hours and doesn’t mention Salesforce at all.

This is a grave error, and we are correcting it today. Going forward:

• I would like every business in the world to see the amazing collaboration and productivity gains Slack has enabled for our community of tens of thousands of volunteers worldwide and consider adopting it for their own business.
• Salesforce will have a complimentary top sponsor slot at our flagship WordCamp events in the United States, Europe, and Asia, which attract thousands of people each.
• We will update our Five for the Future program to reflect contributions such as Salesforce’s going forward.

We just want to repeat: Thank you. We hope to deepen our partnership with Salesforce in the future.

This post was contributed by Jesús Espino, Principal Engineer at Mattermost.

In the ever-evolving software development landscape, ensuring robust and reliable plugin integration is no small feat. For Mattermost, relying solely on mocks for plugin testing became a limitation, leading to brittle tests and overlooked integration issues. Enter Testcontainers, an open source tool that provides isolated Docker environments, making complex integration testing not only feasible but efficient. 

In this blog post, we dive into how Mattermost has embraced Testcontainers to overhaul its testing strategy, achieving greater automation, improved accuracy, and seamless plugin integration with minimal overhead.

The previous approach

In the past, Mattermost relied heavily on mocks to test plugins. While this approach had its merits, it also had significant drawbacks. The tests were brittle, meaning they would often break when changes were made to the codebase. This made the tests challenging to develop and maintain, as developers had to constantly update the mocks to reflect the changes in the code.

Furthermore, the use of mocks meant that the integration aspect of testing was largely overlooked. The tests did not account for how the different components of the system interacted with each other, which could lead to unforeseen issues in the production environment. 

The previous approach additionally did not allow for proper integration testing in an automated way. The lack of automation made the testing process time-consuming and prone to human error. These challenges necessitated a shift in Mattermost’s testing strategy, leading to the adoption of Testcontainers for complex integration testing.

Mattermost’s approach to integration testing

Testcontainers for Go

Mattermost uses Testcontainers for Go to create an isolated testing environment for our plugins. This testing environment includes the Mattermost server, the PostgreSQL server, and, in certain cases, an API mock server. The plugin is then installed on the Mattermost server, and through regular API calls or end-to-end testing frameworks like Playwright, we perform the required testing.

We have created a specialized Testcontainers module for the Mattermost server. This module uses PostgreSQL as a dependency, ensuring that the testing environment closely mirrors the production environment. Our module allows the developer to install and configure any plugin you want in the Mattermost server easily.

To improve the system’s isolation, the Mattermost module includes a container for the server and a container for the PostgreSQL database, which are connected through an internal Docker network.

Additionally, the Mattermost module exposes utility functionality that allows direct access to the database, to the Mattermost API through the Go client, and some utility functions that enable admins to create users, channels, teams, and change the configuration, among other things. This functionality is invaluable for performing complex operations during testing, including API calls, users/teams/channel creation, configuration changes, or even SQL query execution. 

This approach provides a powerful set of tools with which to set up our tests and prepare everything for verifying the behavior that we expect. Combined with the disposable nature of the test container instances, this makes the system easy to understand while remaining isolated.

This comprehensive approach to testing ensures that all aspects of the Mattermost server and its plugins are thoroughly tested, thereby increasing their reliability and functionality. But, let’s see a code example of the usage.

We can start setting up our Mattermost environment with a plugin like this:

pluginConfig := map[string]any{}
options := []mmcontainer.MattermostCustomizeRequestOption{
  mmcontainer.WithPlugin("sample.tar.gz", "sample", pluginConfig),
}
mattermost, err := mmcontainer.RunContainer(context.Background(), options...)
defer mattermost.Terminate(context.Background()

Once your Mattermost instance is initialized, you can create a test like this:

func TestSample(t *testing.T) {
    client, err mattermost.GetClient()
    require.NoError(t, err)
    reqURL := client.URL + "/plugins/sample/sample-endpoint"
    resp, err := client.DoAPIRequest(context.Background(), http.MethodGet, reqURL, "", "")
    require.NoError(t, err, "cannot fetch url %s", reqURL)
    defer resp.Body.Close()
    bodyBytes, err := io.ReadAll(resp.Body)
    require.NoError(t, err)
    require.Equal(t, 200, resp.StatusCode)
    assert.Contains(t, string(bodyBytes), "sample-response") 
}

Here, you can decide when you tear down your Mattermost instance and recreate it. Once per test? Once per a set of tests? It is up to you and depends strictly on your needs and the nature of your tests.

Testcontainers for Node.js

In addition to using Testcontainers for Go, Mattermost leverages Testcontainers for Node.js to set up our testing environment. In case you’re unfamiliar, Testcontainers for Node.js is a Node.js library that provides similar functionality to Testcontainers for Go. Using Testcontainers for Node.js, we can set up our environment in the same way we did with Testcontainers for Go. This allows us to write Playwright tests using JavaScript and run them in the isolated Mattermost environment created by Testcontainers, enabling us to perform integration testing that interacts directly with the plugin user interface. The code is available on GitHub.  

This approach provides the same advantages as Testcontainers for Go, and it allows us to use a more interface-based testing tool — like Playwright in this case. Let me show a bit of code with the Node.js and Playwright implementation:

We start and stop the containers for each test:

test.beforeAll(async () => { mattermost = await RunContainer() })
test.afterAll(async () => { await mattermost.stop(); })

Then we can use our Mattermost instance like any other server running to run our Playwright tests:

test.describe('sample slash command', () => {
  test('try to run a sample slash command', async ({ page }) => {
    const url = mattermost.url()
    await login(page, url, "regularuser", "regularuser")
    await expect(page.getByLabel('town square public channel')).toBeVisible();
    await page.getByTestId('post_textbox').fill("/sample run")
    await page.getByTestId('SendMessageButton').click();
    await expect(page.getByText('Sample command result', { exact: true })).toBeVisible();
    await logout(page)
  });  
});

With these two approaches, we can create integration tests covering the API and the interface without having to mock or use any other synthetic environment. Also, we can test things in absolute isolation because we consciously decide whether we want to reuse the Testcontainers instances. We can also reach a high degree of isolation and thereby avoid the flakiness induced by contaminated environments when doing end-to-end testing.

Examples of usage

Currently, we are using this approach for two plugins.

1. Mattermost AI Copilot

This integration helps users in their daily tasks using AI large language models (LLMs), providing things like thread and meeting summarization and context-based interrogation.

This plugin has a rich interface, so we used the Testcontainers for Node and Playwright approach to ensure we could properly test the system through the interface. Also, this plugin needs to call the AI LLM through an API. To avoid that resource-heavy task, we use an API mock, another container that simulates any API.

This approach gives us confidence in the server-side code but in the interface side as well, because we can ensure that we aren’t breaking anything during the development.

2. Mattermost MS Teams plugin

This integration is designed to connect MS Teams and Mattermost in a seamless way, synchronizing messages between both platforms.

For this plugin, we mainly need to do API calls, so we used Testcontainers for Go and directly hit the API using a client written in Go. In this case, again, our plugin depends on a third-party service: the Microsoft Graph API from Microsoft. For that, we also use an API mock, enabling us to test the whole plugin without depending on the third-party service.

We still have some integration tests with the real Teams API using the same Testcontainers infrastructure to ensure that we are properly handling the Microsoft Graph calls.

Benefits of using Testcontainers libraries

Using Testcontainers for integration testing offers benefits, such as:

• Isolation: Each test runs in its own Docker container, which means that tests are completely isolated from each other. This approach prevents tests from interfering with one another and ensures that each test starts with a clean slate.
• Repeatability: Because the testing environment is set up automatically, the tests are highly repeatable. This means that developers can run the tests multiple times and get the same results, which increases the reliability of the tests.
• Ease of use: Testcontainers is easy to use, as it handles all the complexities of setting up and tearing down Docker containers. This allows developers to focus on writing tests rather than managing the testing environment.
  

Testing made easy with Testcontainers

Mattermost’s use of Testcontainers libraries for complex integration testing in their plugins is a testament to the power and versatility of Testcontainers.

By creating a well-isolated and repeatable testing environment, Mattermost ensures that our plugins are thoroughly tested and highly reliable.

Learn more

• Subscribe to the Docker Newsletter. 
• Visit the Testcontainers website.
• Get started with Testcontainers Cloud by creating a free account.
• Vote on what’s next! Check out our public roadmap.
• Have questions? The Docker community is here to help.
• New to Docker? Get started.

The world of technology is ever-changing, and young minds prove time and again that age really is just a number when it comes to mastery and innovation. One of the most interesting stories of youth combined with innovation belongs to a 9th-grade student named Akash Muthukumar, whose workshop on deploying TinyML using the XIAO ESP32S3 Sense stirred waves in both the world of machine learning and embedded systems.

A Passion for Technology

Early in his childhood, Akash’s journey in the world of technology begins. As a child, Akash was fascinated by gadgets and how they work. Middle school saw Akash deep-diving into programming, robotics, and machine learning while playing with platforms like Arduino and TensorFlow Lite. And here came his curiosity and drive to learn about TinyML-a nascent field where Akash deployed machine learning models on microcontrollers and embedded systems.

Why TinyML?

TinyML, short for Tiny Machine Learning, is a revolution within the field of artificial intelligence; it’s extending the power of machine learning into the smallest and most power-efficient kinds of devices-microcontrollers. These are just the kind of devices now being used everywhere these days in things like IoTs, where there is every need to perform intelligently locally-for instance, speech recognition, anomaly detection, and gesture recognition-without being remotely dependent on the cloud for end-to-end processing.

For Akash, the coolness factor about TinyML was its ability to take already ‘smart’ devices to the next level. The deployability of machine learning models on tiny devices opened up a world of possibilities, creating innovative projects such as real-time object detection and predictive maintenance systems.

Workshop on TinyML and XIAO ESP32S3 Sense

Akash’s workshop focused on TinyML deployment on the XIAO ESP32S3, a powerful Seeed Studio microcontroller for edge AI applications. The Xiao ESP32S3 is compact, powerful, and affordable, thus ideal for students, hobbyists, and developers interested in exploring TinyML.

Akash took participants through the whole process, from training a model to deploying it on the microcontroller. Here is a breakdown of what Akash covered:

1. Intro to TinyML
Akash introduced the concepts of TinyML – what it is, why it is needed, how it works, and how it differs from normal machine learning. He noted that edge AI gets more relevant every day, and TinyML fared well in resource-constrained applications.

2. Introduction to XIAO ESP32S3
Then Akash presented the Xiao ESP32S3 Board: its features, specifications, and why it was a great platform for TinyML. Further, he presented the onboard Wi-Fi and Bluetooth capabilities, the low-power consumption, and compatibility with various sensors.

3. Building a Machine Learning Model
Akash then walked them through building a machine-learning model on Edge Impulse, one of the most popular platforms for TinyML model development. Next, train your model on any simple dataset like a gesture or keyword recognition dataset.

4. Deployment of Model on XIAO ESP32S3
Deployment Process: The heart of the workshop was the deployment process. First, Akash showed how one could convert a trained model to a deployable format on the Xiao ESP32S3 using TensorFlow Lite for Microcontrollers; then, he uploaded the model onto the board and ran inferences directly on the device.

5. Real Time Demonstration
The workshop concluded with a very exciting live demo: Akash showed how, in real-time, Xiao ESP32S3 was able to recognize hand gestures or detect certain sounds using the deployed TinyML model. This left the participants aghast and proved that even the most minute devices could do complex tasks using TinyML.

Empowering Next-Generation Innovators

Akash’s workshop was not limited to teaching some particular technology, but to inspire others. Being a 9th grader, he proved that none should be barred due to age factors from working on advanced fields like TinyML and can always contribute something meaningful. He made the workshop quite interactive, explaining each complex thing in an easy manner such that all participants, irrespective of age and skills, enjoyed it.

Akash has become a young leader in the tech community through his passion for teaching and deep knowledge of TinyML and embedded systems. This workshop reminded us that the future of technology rests in the hands of such young innovators who push beyond the edge of what’s possible.

Looking Ahead

And that is just where Akash Muthukumar gets started. With interests in TinyML, embedded systems, and machine learning, he definitely is going to keep making his presence known in the tech world. And as he does so, deeper into it all, it’s a dead giveaway that Akash is not only learning from the world but teaching too.

Akash’s workshop on deploying TinyML using the Xiao ESP32S3 is another good example of how the young mind has embraced technology and is showing the way. The world of TinyML is big, and with innovators like Akash at the helm, the future looks bright.

It is a story that inspires both novice and experienced developers alike to understand that with curiosity and passion in their hearts, commitments can help achieve great things even at a tender age!

The post Akash Muthukumar: A 9th Grader Leading the Future of TinyML with XIAO ESP32S3 appeared first on Latest Open Tech From Seeed.

I’ve heard from WP Engine customers that they are frustrated that WP Engine hasn’t been able to make updates, plugin directory, theme directory, and Openverse work on their sites. It saddens me that they’ve been negatively impacted by Silver Lake‘s commercial decisions.

On WP Engine’s homepage, they promise “Unmatched performance, automated updates, and bulletproof security ensure your sites thrive.”

WP Engine was well aware that we could remove access when they chose to ignore our efforts to resolve our differences and enter into a commercial licensing agreement. Heather Brunner, Lee Wittlinger, and their Board chose to take this risk. WPE was also aware that they were placing this risk directly on WPE customers. You could assume that WPE has a workaround ready, or they were simply reckless in supporting their customers. Silver Lake and WP Engine put their customers at risk, not me.

We have lifted the blocks of their servers from accessing ours, until October 1, UTC 00:00. Hopefully this helps them spin up their mirrors of all of WordPress.org’s resources that they were using for free while not paying, and making legal threats against us.

Hi everyone!

We are launching Nextcloud Hub 9 in Berlin from the Nextcloud Conference, our yearly community contributor and user event. Being here always reminds me of why we started Nextcloud in the first place: putting you back in control over your data.

More and more of our digital lives are controlled by a shrinking number of big tech firms and their CEO’s. We believe there is a better way.

So we designed Nextcloud with decentralisation in mind. You can run it at home, in your business, your government agency or your local sports club. You decide who has access, and when the services of a big tech firm fall over, you can keep working.

But life isn’t about being alone on an island – it is about connecting people! Nextcloud Hub 9 is perfect for that, too. Our Federation features allows Nextcloud servers to talk to each other, bringing our tens of millions of users together from across the globe.

Of course, this release brings much more than federation. We help the public sector automate its processes so they can help you better; a chat mode in our AI Assistant; we introduce a brand new whiteboard to support your brainstorms and meeting notes; and there is a new more compact design with a fresh new background!

As always, we look forward to your feedback! Nextcloud is people, as much as technology, and if you want you can come say hi to us at the dozens of events we visit every year. Check out our event list and meet some fellow Nextclouders!

Greetings,
Jos & the Nextcloud team

Decentralize your life!

Nextcloud Hub has grown into a leading open-source platform for collaboration, thanks to our tireless efforts to excel in privacy while bringing new cutting-edge features to the table with each release. Today, we launch Nextcloud Hub 9, a new iteration of our platform that pioneers automation tools, empowers teams, and drives digital transformation.

Sounds ambitious? We call it Tuesday!

A big design redo

To make Nextcloud more tidy and efficient while adding all the new features, we introduced countless design improvements around the entire platform and its applications.

Auto-magic with Nextcloud Flow

Our new Windmill-based app, part of Nextcloud Flow automation features, is an enterprise-focused process automation engine that is deeply integrated in Nextcloud Hub. It makes it possible to automate processes and flows: taking data from a form, parsing a PDF in an email, or processing a spreadsheet import, it kicks off actions like processing a payment through a third party provider, storing results in a table, notifying a user to approve the state and then generating and emailing a PDF.

A new creative medium

With the new Nextcloud Whiteboard, get creative together: even organize a drawing class in Nextcloud Talk! Now you have a limitless canvas to sketch, draw and plan collaboratively anywhere, even inside Nextcloud Talk calls.

Centralized? Let’s move on!

It is time to stop depending on big tech firms that control every aspect of our lives, and federation is the key! It means decentralising while letting separate systems interact with each other – exactly how email works. With Nextcloud Hub 9 we bring federation from files sharing and now to chats and video calls!

Your favorite cloud with hip new tools

We improved Nextcloud Hub with numerous new features throughout: Nextcloud Assistant’s Chat UI and support for hundreds of new languages for AI translations, a new wizard to request files in Nextcloud Files, AI-powered follow-up reminders and phishing detection in Nextcloud Mail, augmented reality calls with Nextcloud Talk, an easy way to fill in forms in Nextcloud Office, and so much more.

Collective innovation behind Nextcloud

Nextcloud is rooted in community, standing on the foundation of collective innovation. On behalf of the Nextcloud team, we want to thank everyone involved in coding, testing, sharing, providing feedback, supporting, and nurturing Nextcloud Hub, giving us a reason to celebrate this new release together.

Design that gives you more

We believe that software should be accessible to everyone and everywhere, easy to use and always ready to work the way you expect it to. As new features and apps join Nextcloud Hub, we want to make sure we stay true to these principles.

If you have a feeling that something is different this time around, you are right! With more and more features being added every release, we want to prevent a sense of overwhelm in Nextcloud Hub’s interface. That’s why we gave our interface a make-over, letting the whole platform evolve while remaining highly accessible and user friendly.

More compact

First, we made everything more compact by downsizing various elements: input fields, buttons, links. This change lets us make use of the space more efficiently compared to our old interface. The new design is more cohesive and user-friendly, and we trust you’ll notice the difference once you experience it.

More tidy

Using rounded corners for various elements like buttons and fields, we achieved a more balanced aesthetic that maintains a relaxed visual experience without being overly stylized. It feels familiar across modern apps and devices, and it also allows for more efficient use of visual space.

And with a fresh new dynamic background

In Nextcloud Hub, you can also use any background you wish. The interface color scheme adjusts to match your background’s palette, giving your platform a cohesive and polished appearance — even on your mobile devices!

We are celebrating the new look of Nextcloud Hub with a new background style available in your Appearance settings. The new version supports dynamic backgrounds that adapt to your light or dark interface theme preferences. This background is the first to support themes in Nextcloud Hub 9, with more dynamic background designs coming soon!

Even more design improvements

Nextcloud Hub 9 brings multiple other feature- and app-specific improvements across various applications: Nextcloud Files, Nextcloud Calendar, our mobile and desktop clients, and more. We are going to discuss them in the dedicated sections of this blog in more detail.

Now let’s move to new features, starting with one particular app that enables a whole new dimension of functionality in your Nextcloud Hub.

Welcome Nextcloud Flow

For most of us, it is easy to imagine learning, shopping, and banking from your mobile phone no matter where you are. For some, even accessing a government service entirely from the comfort of their home is a reality. For others — hopefully a future.

Nextcloud offers an extraordinary platform to work in, fully digitally. Our apps let you effortlessly handle, share and work on documents with Nextcloud Files and Nextcloud Office or track and process structured data with Nextcloud Tables. Nextcloud Talk and Nextcloud Groupware help with planning and communication, and dozens of other apps help with tasks like processing forms, tracking tasks and more.

All this is built on an open standards based platform. The Open Collaboration Services API supported by Nextcloud makes it easy to connect with other systems, sending or receiving data and building new apps.

The introduction of our new Windmill-based application brings digitalization to a new level. It allows automation of processes, taking data from within and outside of Nextcloud, tying in with its communication and planning tools and thus acting as a conductor leading an orchestra of processes!

How to Flow?

Nextcloud Flow combines features that help Nextcloud automate a variety of processes in your organisation. It consists of several components, from our Open Collaboration Services API (OCS API) and our structured data management application Nextcloud Tables to our existing workflow tool to our brand new business process automation application based on Windmill.

Windmill integration in Nextcloud Hub automates repetitive tasks by integrating your Nextcloud Hub apps and related assets into cohesive workflows. For instance, vacation requests, payment confirmations, or approvals for accessing and sharing data. The app provides a graphic interface to design, manage, and monitor workflows, making it easier to automate processes and improve efficiency.

The automation, build with the open source team at Windmill, uses everything that makes Nextcloud such a powerful platform: file handling, chat, notifications, structured data in Nextcloud Tables, Nextcloud Office and much more. Dozens of API’s have been developed, all as part of the standardized Open Collaboration Services API, making it easy to connect Nextcloud to other systems.

Understanding the flows 

Windmill identifies events in Nextcloud Hub and triggers custom ‘flows’ created by you to retrieve and exchange data and perform sequences of actions across various apps and assets. Each flow is composed of ‘bricks,’ or individual actions that you program using a script.

The new workflow automation app is very flexible. It can run consecutive flows, or execute multiple flows in parallel, trigger an approval request for an employee, interact with external services like payment systems, and much more.

Digitalize processes in the public sector

Windmill integration in Nextcloud Hub has been designed to help the public sector comply with legislation like The Online Access Act (Onlinezugangsgesetz, OZG), a law in Germany that mandates citizens’ ability to interact with government services online, or with the EMBAG legislation in Switzerland that requires the public sector to use and develop open source solutions.

You can learn more about the challenges the public sector faces and how Windmill integration in Nextcloud Hub can help in our earlier blog post.

“The Online Access Act obliges the public sector to provide all administrative services digitally going forward. Open source software should be prioritised here. We are therefore delighted that with ‘Open Collaboration Services’, a 100% open source solution has been created for the first time with which specialised procedures can be implemented quickly and easily. Open source enables the simple sharing of specialised procedures on Open CoDE, which saves costs and strengthens digital sovereignty.”

Markus Richter

State Secretary at the Federal Ministry of the Interior and Community and Federal Government Commissioner for Information Technology

Decentralization through federation

Federated features allow users of self-hosted platforms like Nextcloud Hub to communicate, share files, and collaborate across instances while storing their data locally. Federation is a foundational tool for digital sovereignty, enabling the creation of decentralized yet interconnected ecosystems.

Prior to the Nextcloud Hub 9 release, federated features supported in our platform included file sharing and collaboration, as well as federated chatting in Nextcloud Talk. Today, we are excited to announce that federated video calls also become available to our users!

Introducing federated calls

Users can join group chats on another Nextcloud server with their Federated Cloud ID. While in federated chat, they can use many of the core features like user mentions, markdown formatting, or polls.

With Nextcloud Hub 9, we add federated calls to our growing array of decentralized features. Now you can organize audio and video meetings among users of different Nextcloud Hubs — to meet with colleagues from a different government agency, or maybe a partner who is using Nextcloud too!

Joining a call on another server feels as if you are still in your own Nextcloud Hub – using your own account and a preferred interface, without a need for creating guest accounts or public links.

OCM, a foundation for federation features in Nextcloud

Federated Cloud Sharing API in Nexctloud has been invented by Nextcloud founders Frank Karlitschek and Bjoern Schießle. It is a collaboratively developed standard that is a part of Open Cloud Mesh (OCM), an initiative of the GÉANT Association, a European collaboration on e-infrastructure and services for research and education.

Nextcloud Whiteboard

With Nextcloud Hub 9 , we are introducing a new collaborative app: Nextcloud Whiteboard. It gives you a limitless canvas and a range of tools to sketch, note, plan, or brainstorm together with your team.

Nextcloud Whiteboard works seamlessly with Nextcloud Talk. For example, you can brainstorm project structures and draw mind maps together in real-time while in a call. Or use it in a live workshop to illustrate concepts, create diagrams, and interactively engage participants.

Nextcloud Whiteboard is very intuitive and easy to use, ready to become your daily creative collaboration tool right away. It supports hand drawing with multiple colors and stroke types, shapes, images, arrows and other objects. Wide ranges of shapes like rectangles, circles, diamonds, arrows, lines and more, together with arrow-binding and labeled arrows, let you be flexible and creative without the need for an entire design suite.

Combine it with other media: draw on design mock-ups, building plans, schemes and other visuals by importing images of your own into the canvas.

Once ready, you can share your whiteboard and present it using the laser pointer feature, or invite users to collaborate with you, even from outside Nextcloud. Want to save or share your drawings as files? You can easily export them in PNG or SVG or copy to your clipboard.

Performance and scalability

We never stop working on Nextcloud Hub’s performance, realizing that it benefits not only end users and admins running our systems, but also the planet we all share. That’s why no matter how much more powerful we keep making Nextcloud, we allow it to use less resources with every update, considerably lowering its environmental impact. And of course, allowing it to run on more diverse hardware without loss in performance too.

Faster, more lightweight apps

One of the steps towards better performance in Hub 9 has been a switch to Vite, a tool to bundle the JavaScript. This reduced the size of our apps significantly, making some up to 5x smaller and thus load much faster fir everyone.

We also made numerous other improvements that make Nextcloud Hub more snappy overall:

• Skip the time needed to set up a full filesystem for accessing public link-share DAV endpoints
• Delay getting (sub)admin status until needed
• Avoid queries during file share aggregation in Deck
• More lazy app bootstrap

High Performance back-end

Since 2021, Nextcloud Files runs on the High Performance back-end written in Rust that reduced its network traffic consumption by up to 90%. It also made the app swifter for users, for example by enabling instant notifications.

With Nextcloud Hub 9, we introduced support for High Performance back-end to Nextcloud Text. It greatly reduces the server load and renders edits much faster with accelerated synchronization during collaborative sessions.

Server-side performance improvements

We also worked on the server-side performance, making improvements that reduce the load on your infrastructure:

• Text: Reduce polling interval for read-only users
• Nextcloud Assistant’s Context Chat: improved queuing, with smarter memory handling
• Faster updates: avoiding duplicate repair steps, moving more repair/check steps out of update and to cron
• Command to get information about database changes needed for an update

Database Sharding

In our last release, we introduced the foundation for Database Sharding, enabling the ability to split the Nextcloud database into several smaller databases and distribute the load across multiple servers. With Nextcloud Hub 9, we’ve continued to develop this feature, making it applicable for most use cases.

Read-After-Write behavior

Previously, Nextcloud required synchronous replication, limiting database clusters to a maximum of four nodes. Now, we added support for an unlimited number of Read nodes using asynchronous replication. Combined with database sharding, Nextcloud’s scalability has significantly advanced, allowing a single cluster to support up to 10 times more users.

AI and Nextcloud Assistant

Chat with Nextcloud Assistant, translate between hundreds of languages, work with Analytics and enjoy accelerated performance.

AI has become an integral part of Nextcloud Hub: Many core applications like Nextcloud Mail and Nextcloud Talk have their own built-in AI tools, while Nextcloud Assistant is getting more capable both feature-wise and in the back-end. At the same time, we integrate Ethical AI principles into our platform, enabling new AI features with a focus on privacy, transparency, and user control.

Chat with Nextcloud Assistant

Nextcloud Assistant becomes even more convenient with a chat interface, where you can give prompts and ask questions in a dialog format. It will remember your conversations, making interactions more natural and smooth. Chat about your day, work on the next press release, or proofread your social media posts — locally on your server and without any data leaks!

Translate with Nextcloud Assistant

Translate app is now available directly from Nextcloud Assistant menu. Translation options are now almost limitless, as we added support for hundreds of languages that app can detect, read and translate to. The app can be deployed as an ExApp on a separate server, and now supports CPU and GPU, increasing performance and efficiency significantly.

Access Analytics with Nextcloud Assistant

Analytics, an application for processing and managing analytics data developed by our community, added supprt for Nextcloud Assistant’s Context Chat. This means that you can now ask it about the data from Analytics, and Context Chat will give you an answer if the app has relevant figures.

New API and a performance boost

We developed a new API that significantly improves the performance and scalability of the AI in Nextcloud, while also enabling the creation of new AI features. For example, it allows Assistant’s responses in the chat to be nearly instant, provided your back-end is sufficiently performant.

Nextcloud Files

Nextcloud Hub 9 makes file navigation easier, introduces the File Request feature, and keeps you up-to-date on your shares.

Request files

Need to collect files in a secure way from one or more users? Meet File Request, a feature that lets you request files with configurable security settings and tracking features.

To get started, click Create file request from the “+” menu of Nextcloud Files. Set a title and a description of your request and pick a drop folder. Set an expiration date and a password for your share, if you need an extra security layer.

When ready, you can simply share a link anywhere you prefer or add email addresses of recipients to share them right away via email. Every recipient will be asked to enter a name before dropping a file, to keep things on track.

Pasted a link in the wrong place? We’ve got you! To keep your requests under control, you can browse and manage active requests in the right sidebar and close them as needed.

We also added a push notification option to help you keep track of your files and folders. Now you can get notified if a user uploads a file you requested or downloads one you shared with them.

Filter files and folders

When you keep a lot of files in your system, navigating your library can become cumbersome, and time-consuming too. If you add just two more files daily, that’s already 60 more per month! To help you find what you need faster, we introduce file filters.

You can filter your files by file type, modification date and people. Fine-tune your browsing by combining different filters and filter types, and success is guaranteed.

Browse folders with tree view

Navigating complex folder paths is now easier with the folder tree view. The left sidebar in the new Nextcloud Files offers a classic tree-style display of your folders, making it simple to browse intricate project folders.

More improvements in Nextcloud Files

• Ability to keep the folder structure when uploading
• Group folders ACL: visual feedback when ACL update fails
• Small Group folder improvements
• Ability to enforce windows-compatible file names on the server
• Server-side DAV pagination
• Account management: sorting by last login
• Possibility to upload a complete folder from a “+” menu
• Delegate “Super-Admin” for user management
• Allow to specify the token of a public share / share renaming
• Improved grid view UI in Files

Nextcloud Office

PDF templates and forms, 3D transitions in slides, better interoperability and security, and more in the new version of Nextcloud Office.

PDF templates and form filling API

A new feature in Nextcloud Office for this release is the ability to handle document templates with forms. When a template document has a form, Nextcloud will provide a nice, easy-to-use interface for the user to fill in details. Then, the document will be created and opened, and the user can check if everything was added correctly and if they want to make any changes.

The form system is very powerful, yet easy to use. For example, you can fill in time-sheets, complete feedback forms, and do much more. And it’s not only limited to text: you can even create charts based on data from your documents. Organisations that use a lot of forms can use this to make the life of their team members a lot easier!

Developers can use the new document API in Nextcloud Office to create multiple documents from a central database or template, or collate information from multiple documents.

It is possible to streamline automated document creation by linking a database to a template using scripts. With templating, you can automate many interactions like reporting, proposal generation, client on boarding, or even production of marketing collateral.

Of course, the new Office API is available in the new Nextcloud Flow, which can automate a variety of things, from extracting data from documents to filling them in.

Presentations: 3D transitions and quicker loading

The presentation editor now supports fancy 3D transitions between slides that make your slideshows more dynamic and catchy. These use WebGL, putting the GPU in your laptop or desktop to work for buttery smooth animations.

You can access transitions via the transitions widget located in the editor’s sidebar. And the widget itself has been revamped too!

Presentations also became much faster all around, with swifter document opening and slide loading, and overall smoother scrolling and rendering of the elements.

Better security and interoperability

Rootless containers in Nextcloud Office now operate without requiring special privileges from the user, thanks to Linux Namespaces. This optional feature enhances security by isolating container processes more effectively.

Our Office suite in Nextcloud Hub 9 features improved security with several new features. The newly introduced Wholesome Encryption for ODF files offers enhanced performance, better tamper resistance, metadata hiding, and increased resistance to brute-forcing with the Argon2id key derivation function.

Nextcloud Office also became more interoperable. Work with metric-compatible Microsoft fonts has been improved together with compliance with ISO/IEC 29500 standard. New Excel-compatible functions were added like FILTER, SORT, and SORTBY, along with a smart justify feature.

Nextcloud Tables

Our no-code platform for data management gets new export and import features, optimized performance and more in Hub 9!

New column type: Users and groups

Assign a colleague or a team to a task in a table by adding their clickable name with a new column type Users and groups.

Export table schema

Export the table schema and import it as a new table to quickly create tables with a similar structure and populate them from scratch. This handy templating feature helps you quickly move tables from testing to production when you build environments based on Nextcloud Tables. And, of course, you can exchange these between organizations: Each city needs to handle things like a dog tax or registering a new citizen!

More in Nextcloud Office, Nextcloud Tables, and Nextcloud Collectives:

• New functions in spreadsheet editor: FILTER, SORT, SORTBY, UNIQUE, SEQUENCE, RANDARRAY, XLOOKUP, XMATCH
• Support of importing and exporting OOXML pivot tables (cell) format definitions
• “Copy Cells”, or “Fill Series” options when autofilling by dragging from a corner of cells in Calc
• Correct usage of ordinal suffixes 1st, 2nd, 3rd and so on
• Right click to insert and delete comments in Calc
• Formula wizard interface improvements
• PPTX files with heavy use of custom shapes now open faster
• Pressing Enter in an empty list item ends the list
• Transitions widget located in the sidebar has been revamped (it now uses iconView)
• Admin warnings to display if you have mis-configurations
• Firefox performance optimization and reduced garbage-collection giving snappier browser experience closer to that of Chrome / WebKit
• Continued accessibility improvement and widget annotations

Nextcloud Talk

Improved moderation features, offline browsing on mobile, and Apple Vision Pro support in one of the most secure apps for online meetings.

Keep your chats healthy

Inside public conversations or during online events, it’s important to have a firm way to moderate the conversation, uphold your community guidelines and deal with potential spam. Nextcloud Talk features many moderation tools that help you maintain a safe and healthy environment like role management in chats, time tracking of who’s speaking, voice muting, and more.

Remove participants

With Nextcloud Hub 9 we introduce the ability to ban users who show disruptive behavior, publish spam links or abuse other participants.

Browse conversations even offline

Users of Nextcloud Talk on Android devices can now browse past conversation lists and chat messages offline, to quickly get back to important information even without needing an internet connection.

Don’t miss calls: add a second speaker!

Away from your laptop? Not an excuse to miss a call! Now you can choose a secondary device, like your Bluetooth headphones or a kitchen speaker to notify you of incoming calls. To activate the feature, head to notification settings and select a second output device.

Summon Smart Picker easily

You can now summon the Smart Picker in Nextcloud Talk via the attachment menu to add GIFs and videos, generate images with AI, link conversations and Deck cards, and even send map locations. For quick access to Smart Picker, you can type a “/” shortcut in the message box as usual.

Use Nextcloud Talk with Apple Vision Pro

When we say you can have your meetings anywhere, we mean it! Now you can join calls even in a virtual environment using our Nextcloud Talk app for your Apple Vision Pro headset

Nextcloud Groupware

Discover multiple security, usability, and navigation improvements across Nextcloud Groupware apps.

Nextcloud Mail

Smart follow-up reminders

The Nextcloud Assistant can already summarise email threads and generate meeting agendas from the your email content. With Nextcloud Hub 9, the Assistant can also check emails to see if the sender expects a reply: If it determines an email needs a follow-up, it will place it in a Follow up section to make sure you don’t forget!

Advanced search filters

Filter your inbox based on various conditions, such as subject line, sender, or tags with new email filtering mechanism. Unlike most other mail clients, Nextcloud Mail installs these filters as sieve scripts on the server, meaning the filters work regardless of which client you’re using. No need to set up the filters on your desktop and also your laptop, and then be annoyed they are not applied when you check mail on your phone!

Phishing detection

Some modern phishing emails are becoming exceedingly sophisticated, and therefore dangerous! And sometimes you’re moving quickly through your day, without an eye on potential dangers. In this case, Nextcloud keeps you alert about risky links and phishing or impersonation attempts by showing you a warning message.

End-to-end encryption with Mailvelope

Want to encrypt your mail using PGP? We’ve worked with the team behind Mailvelope to make sure the plugin works reliably with Nextcloud Mail, and also introduced a simplified domain registration feature for your convenience.

Nextcloud Calendar

More informative event window

The new sidebar in Nextcloud Calendar groups events, showing if they are your personal calendars, shared, which app they come from, and groups all hidden calendars together.

Availability display for rooms and resources

Before creating a new event, you can check the availability of Talk conversations and resources. Easier planning of all online meetings with less risk of anything going not-quite-as-planned.

Setting calendar as free or busy

Now you can decide if Nextcloud Hub shows your status as free or busy during events in your calendar. For example, you can set a calendar as transparent so it never changes your status to Busy.

Upcoming events in Nextcloud Talk conversations

Now every Nextcloud Talk conversation shows upcoming events happening within, reminding you of future meetings and making room availability more transparent.

Nextcloud Contacts

Delete or rename groups

Easily delete or rename groups with actions from the group menu, saving some of your precious time.

Set out-of-office replacements

When setting a Nextcloud-wide out-of-office status, you can assign another user or colleague to be your replacement during your absence. Their name will be visible in other apps like Nextcloud Talk, Nextcloud Mail, Nextcloud Calendar, and other locations throughout Nextcloud Hub.

Other improvements in Nextcloud Groupware:

• Mail signatures and encryption with S/Mime
• Mail: Add OCS API endpoint to send emails with automation
• Mail: print out email
• Mail: fix misplaced action menus
• Mail: fire workflow events for incoming messages
• Mail: API to read mails
• Mail: simplified domain registration for Mailvelope
• Mail: Fileviewer application
• Allow enforcing strict email format
• Contacts: migrate from Webpack to Vue
• Contacts: migrate from Webpack to Vite for faster loading
• Contacts: birthday and biography support for the system address book
• Calendar: set start day of the week
• Calendar: improve sidebar display of calendars
• Calendar: auto refresh
• CalDAV: invitations for shared calendars
• CalDAV: fix unclear event update email for recurring events
• DAV: hide CalDAV settings if no calendar-dependent apps are enabled
• Calendar/CalDAV: delta sync for subscriptions and adaptive sync rates
• CalDAV: send invitation emails via Nextcloud Mail

Even more security in Nextcloud Hub 9

Nextcloud Hub 9 lets you strengthen your security and administrative controls even further with several new fine-grained security tools and mechanics:

• Ability to set up a PIN code for passwordless authentication to additionally secure your account should anybody obtain your security key.
• Ability to restrict admin actions to internal IP addresses, disabling admin settings for users outside of trusted IP ranges.
• More flexible admin rights delegation: option to enable user management rights for users without granting admin permissions
• Ability to enable E2EE and SSE in Nextcloud Hub together
• Enhanced metadata extraction capabilities including PDF files
• Support for Azure Information Protection file tags

Nextcloud Clients

Refined design, optimized synchronization and more improvements in Nextcloud apps for desktop and mobile.

More native UI

We revised the design of Nextcloud clients for Windows and iOS to make them look more native in many ways, including transparency, buttons, spacing, and more.

New release channels for desktop app

New release channels are now available for the desktop client. Use the Daily channel to
stay ahead with the latest features and improvements in the making, or Enterprise channel with updates for enterprise customers.

Two-way sync for Android

With two-way sync enabled for a folder, files added or updated on device automatically sync to your cloud, and the other way round. Currently, only internal folders created in Nextcloud are supported.

Other improvements in Nextcloud clients:

• Improved handling of multi-monitor setups on macOS
• Desktop client: Multiple Proxy Settings.
• Login Flow v2 on Android and iOS.
• Ability to enforce Windows-compatible file names on the server.
• Improve notifications of file changes from notify_push server app.

Nextcloud ecosystem

New integrations join Nextcloud Hub, while our App Store receives a donation mechanism and gets more informative.

New in integrations

LLama 3

A third iteration of the popular open-source LLM with significant improvements in data quality and scalability.

XWiki

Open-source enterprise wiki and knowledge management platform. Integration? Partnership?

Enterprise support for new apps

Dovecot

Full-service email platform by OpenExchange

Stalwart Mail Server

Modern, high-performance email SMTP and MTA solution

Mailvelope

PGP for end-to-end email encryption.

All these open source technologies are available with our community edition, but we also partnered with the projects and companies behind them to offer an enterprise solution for our customers.

Nextcloud App Store

Donations

We want to give app creators an opportunity to spend more time on their apps for Nextcloud. So, we introduced donation links in our app store, allowing users to provide a tip to the developer of their favorite app. We support links to PayPal, Stripe and other solutions. Nextcloud keeps no commission — 100% of the funds goes to the developers. You can read more about donations in our documentation.

Enterprise apps

While donations are great, we believe the best approach to monetization for open source are enterprise subscription services when possible. So we offer app and integration developers the chance to work with us to offer an enterprise subscription from our app store – making it easy for prospective customers to request a quote.

Improved search and more

We also made a number of UI improvements, including a better search and the ability for app developers to signal needs about their app or notify about their update policy.

Take Nextcloud Hub 9 for a spin!

If you’re still with us, it means we’ve successfully brought you to the most important part of this journey.

Nextcloud is not a craft of marketing, it is a craft of engineering with true passion for privacy, transparency and humanity. That’s why we keep bringing the results of our work to you for free. Click below and start using the latest and greatest of Nextcloud Hub today!

Get Nextcloud Hub 9

Download and install Nextcloud Hub 9 here!

Get Nextcloud Hub

You can download Nextcloud Hub 9 from our installation page. Among available options, you can use a zip file or the all-in-one image for fresh installation.

Over the coming weeks, we will make it available to our home users via updater using the usual staged roll-out process. You can switch to the Beta channel temporarily if you want to try the new Hub right away. Don’t forget to switch back to stable after you’ve updated to Nextcloud Hub 9!

Note: Nextcloud caches the results coming from our updater server, so it can take some time for the new version to show up and you likely have to refresh the page, wait a bit, then try again.

How to update: AIO users

For new Nextcloud All-in-One (AIO) users, you can install Hub 9 directly by checking the “Install Nextcloud 30” box.

We have a dedicated setup guide for AIO users. Follow the guidelines to easily set up Hub 9.

The Nextcloud Enterprise version of Hub 9 will become available soon, once we successfully run the additional testing and complete certification.

Want to help, but aren’t super technical?

Give us a review!

Sourceforge

Review us

Alternative.to

Review us

Gartner Peer Insights

Review us

The post Be connected with Nextcloud Hub 9 appeared first on Nextcloud.

Any WP Engine customers having trouble with their sites should contact WP Engine support and ask them to fix it.

WP Engine needs a trademark license, they don’t have one. I won’t bore you with the story of how WP Engine broke thousands of customer sites yesterday in their haphazard attempt to block our attempts to inform the wider WordPress community regarding their disabling and locking down a WordPress core feature in order to extract profit.

What I will tell you is that, pending their legal claims and litigation against WordPress.org, WP Engine no longer has free access to WordPress.org’s resources.

WP Engine wants to control your WordPress experience, they need to run their own user login system, update servers, plugin directory, theme directory, pattern directory, block directory, translations, photo directory, job board, meetups, conferences, bug tracker, forums, Slack, Ping-o-matic, and showcase. Their servers can no longer access our servers for free.

The reason WordPress sites don’t get hacked as much anymore is we work with hosts to block vulnerabilities at the network layer, WP Engine will need to replicate that security research on their own.

Why should WordPress.org provide these services to WP Engine for free, given their attacks on us?

WP Engine is free to offer their hacked up, bastardized simulacra of WordPress’s GPL code to their customers, and they can experience WordPress as WP Engine envisions it, with them getting all of the profits and providing all of the services.

If you want to experience WordPress, use any other host in the world besides WP Engine. WP Engine is not WordPress.

It has to be said and repeated: WP Engine is not WordPress. My own mother was confused and thought WP Engine was an official thing. Their branding, marketing, advertising, and entire promise to customers is that they’re giving you WordPress, but they’re not. And they’re profiting off of the confusion. WP Engine needs a trademark license to continue their business.

I spoke yesterday at WordCamp about how Lee Wittlinger at Silver Lake, a private equity firm with $102B assets under management, can hollow out an open source community. (To summarize, they do about half a billion in revenue on top of WordPress and contribute back 40 hours a week, Automattic is a similar size and contributes back 3,915 hours a week.) Today, I would like to offer a specific, technical example of how they break the trust and sanctity of our software’s promise to users to save themselves money so they can extract more profits from you.

WordPress is a content management system, and the content is sacred. Every change you make to every page, every post, is tracked in a revision system, just like the Wikipedia. This means if you make a mistake, you can always undo it. It also means if you’re trying to figure out why something is on a page, you can see precisely the history and edits that led to it. These revisions are stored in our database.

This is very important, it’s at the core of the user promise of protecting your data, and it’s why WordPress is architected and designed to never lose anything.

WP Engine turns this off. They disable revisions because it costs them more money to store the history of the changes in the database, and they don’t want to spend that to protect your content. It strikes to the very heart of what WordPress does, and they shatter it, the integrity of your content. If you make a mistake, you have no way to get your content back, breaking the core promise of what WordPress does, which is manage and protect your content.

Here is a screenshot of their support page saying they disable this across their 1.5 million WordPress installs.

They say it’s slowing down your site, but what they mean is they want to avoid paying to store that data. We tested revisions on all of the recommended hosts on WordPress.org, and none disabled revisions by default. Why is WP Engine the only one that does? They are strip-mining the WordPress ecosystem, giving our users a crappier experience so they can make more money.

What WP Engine gives you is not WordPress, it’s something that they’ve chopped up, hacked, butchered to look like WordPress, but actually they’re giving you a cheap knock-off and charging you more for it.

This is one of the many reasons they are a cancer to WordPress, and it’s important to remember that unchecked, cancer will spread. WP Engine is setting a poor standard that others may look at and think is ok to replicate. We must set a higher standard to ensure WordPress is here for the next 100 years.

If you are a customer of “WordPress Engine,” you should contact their support immediately to at least get the 3 revisions they allow turned on so you don’t accidentally lose something important. Ideally, they should go to unlimited. Remember that you, the customer, hold the power; they are nothing without the money you give them. And as you vote with your dollars, consider literally any other WordPress host as WP Engine is the only one we’ve found that completely disables revisions by default.