Raspberry Pi OS comes with Python pre-installed, and you need to use its virtual environments to install packages. The latest issue of The MagPi, out today, features this handy tutorial, penned by our documentation lead Nate Contino, to get you started.

Raspberry Pi OS comes with Python 3 pre-installed. Interfering with the system Python installation can cause problems for your operating system. When you install third-party Python libraries, always use the correct package-management tools.

On Linux, you can install python dependencies in two ways:

• use apt to install pre-configured system packages
• use pip to install libraries using Python’s dependency manager in a virtual environment

Install Python packages using apt

Packages installed via apt are packaged specifically for Raspberry Pi OS. These packages usually come pre-compiled, so they install faster. Because apt manages dependencies for all packages, installing with this method includes all of the sub-dependencies needed to run the package. And apt ensures that you don’t break other packages if you uninstall.

For instance, to install the Python 3 library that supports the Raspberry Pi Build HAT, run the following command:

$ sudo apt install python3-build-hat

To find Python packages distributed with apt, use apt search. In most cases, Python packages use the prefix python- or python3-: for instance, you can find the numpy package under the name python3-numpy.

Install Python libraries using pip

In older versions of Raspberry Pi OS, you could install libraries directly into the system version of Python using pip. Since Raspberry Pi OS Bookworm, users cannot install libraries directly into the system version of Python.

Instead, install libraries into a virtual environment (venv). To install a library at the system level for all users, install it with apt.

Attempting to install a Python package system-wide outputs an error similar to the following:

$ pip install buildhat
error: externally-managed-environment

× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
    python3-xyz, where xyz is the package you are trying to
    install.
    
    If you wish to install a non-Debian-packaged Python package,
    create a virtual environment using python3 -m venv path/to/venv.
    Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
    sure you have python3-full installed.
    
    For more information visit http://rptl.io/venv

note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.

Python users have long dealt with conflicts between OS package managers like apt and Python-specific package management tools like pip. These conflicts include both Python-level API incompatibilities and conflicts over file ownership.

Starting in Raspberry Pi OS Bookworm, packages installed via pip must be installed into a Python virtual environment (venv). A virtual environment is a container where you can safely install third-party modules so they won’t interfere with your system Python.

Use pip with virtual environments

To use a virtual environment, create a container to store the environment. There are several ways you can do this depending on how you want to work with Python:

per-project environments

Many users create separate virtual environments for each Python project. Locate the virtual environment in the root folder of each project, typically with a shared name like env. Run the following command from the root folder of each project to create a virtual environment configuration folder:

$ python -m venv env

Before you work on a project, run the following command from the root of the project to start using the virtual environment:

$ source env/bin/activate

You should then see a prompt similar to the following:

(env) $

When you finish working on a project, run the following command from any directory to leave the virtual environment:

$ deactivate

per-user environments

Instead of creating a virtual environment for each of your Python projects, you can create a single virtual environment for your user account. Activate that virtual environment before running any of your Python code. This approach can be more convenient for workflows that share many libraries across projects.

When creating a virtual environment for multiple projects across an entire user account, consider locating the virtual environment configuration files in your home directory. Store your configuration in a folder whose name begins with a period to hide the folder by default, preventing it from cluttering your home folder.

Use the following command to create a virtual environment in a hidden folder in the current user’s home directory:

$ python -m venv ~/.env

Run the following command from any directory to start using the virtual environment:

$ source ~/.env/bin/activate

You should then see a prompt similar to the following:

(.env) $

To leave the virtual environment, run the following command from any directory:

$ deactivate

Create a virtual environment

Run the following command to create a virtual environment configuration folder, replacing <env-name> with the name you would like to use for the virtual environment (e.g. env):

$ python -m venv <env-name>

Enter a virtual environment

Then, execute the bin/activate script in the virtual environment configuration folder to enter the virtual environment:

$ source <env-name>/bin/activate

You should then see a prompt similar to the following:

(<env-name>) $

The (<env-name>) command prompt prefix indicates that the current terminal session is in a virtual environment named <env-name>.

To check that you’re in a virtual environment, use pip list to view the list of installed packages:

(<env-name>) $ pip list
Package    Version
---------- -------
pip        23.0.1
setuptools 66.1.1

The list should be much shorter than the list of packages installed in your system Python. You can now safely install packages with pip. Any packages you install with pip while in a virtual environment only install to that virtual environment. In a virtual environment, the python or python3 commands automatically use the virtual environment’s version of Python and installed packages instead of the system Python.

Exit a virtual environment

To leave a virtual environment, run the following command:

(<env-name>) $ deactivate

Use the Thonny editor

We recommend Thonny for editing Python code on the Raspberry Pi.

By default, Thonny uses the system Python. However, you can switch to using a Python virtual environment by clicking on the interpreter menu in the bottom right of the Thonny window. Select a configured environment or configure a new virtual environment with Configure interpreter.

The MagPi #148 out NOW!

You can grab the new issue right now from Tesco, Sainsbury’s, Asda, WHSmith, and other newsagents, including the Raspberry Pi Store in Cambridge. It’s also available at our online store, which ships around the world. You can also get it via our app on Android or iOS.

You can also subscribe to the print version of The MagPi. Not only do we deliver it globally, but people who sign up to the six- or twelve-month print subscription get a FREE Raspberry Pi Pico W!

The post Using Python with virtual environments | The MagPi #148 appeared first on Raspberry Pi.

labwc – a new Wayland compositor

Today we are releasing a new version of Raspberry Pi OS. This version includes a significant change, albeit one that we hope most people won’t even notice. So we thought we’d better tell you about it to make sure you do…

First, a brief history lesson. Linux desktops, like their Unix predecessors, have for many years used the X Window system. This is the underlying technology which displays the desktop, handles windows, moves the mouse, and all that other stuff that you don’t really think about because it (usually) just works. X is prehistoric in computing terms, serving us well since the early 80s. But after 40 years, cracks are beginning to show in the design of X.

As a result, many Linux distributions are moving to a new windowing technology called Wayland. Wayland has many advantages over X, particularly performance. Under X, two separate applications help draw a window:

• the display server creates windows on the screen and gives applications a place to draw their content
• the window manager positions windows relative to each other and decorates windows with title bars and frames.

Wayland combines these two functions into a single application called the compositor. Applications running on a Wayland system only need to talk to one thing, instead of two, to display a window. As you might imagine, this is a much more efficient way to draw application windows.

Wayland also provides a security advantage. Under X, all applications communicated back and forth with the display server; consequently, any application could observe any other application. Wayland isolates applications at the compositor level, so applications cannot observe each other.

We first started thinking about Wayland at Raspberry Pi around ten years ago; at that time, it was nowhere near ready to use. Over the last few years, we have taken cautious steps towards Wayland. When we released Bullseye back in 2021, we switched to a new X window manager, mutter, which could also be used as a Wayland compositor. We included the option to switch it to Wayland mode to see how it worked.

With the release of Bookworm in 2023, we replaced mutter with a new dedicated Wayland compositor called wayfire and made Wayland the default mode of operation for Raspberry Pi 4 and 5, while continuing to run X on lower-powered models. We spent a lot of time optimising wayfire for Raspberry Pi hardware, but it still didn’t run well enough on older Pis, so we couldn’t switch to it everywhere.

All of this was a learning experience – we learned more about Wayland, how it interacted with our hardware, and what we needed to do to get the best out of it. As we continued to work with wayfire, we realised it was developing in a direction that would make it less compatible with our hardware. At this point, we knew it wasn’t the best choice to provide a good Wayland experience for Raspberry Pis. So we started looking at alternatives.

This search eventually led us to a compositor called labwc. Our initial experiments were encouraging: we were able to use it in Raspberry Pi OS after only a few hours of work. Closer investigation revealed labwc to be a much better fit for the Raspberry Pi graphics hardware than wayfire. We contacted the developers and found that their future direction very much aligned with our own.

labwc is built on top of a system called wlroots, a set of libraries which provide the basic functionality of a Wayland system. wlroots has been developed closely alongside the Wayland protocol. Using wlroots, anyone who wants to write a Wayland compositor doesn’t need to reinvent the wheel; we can take advantage of the experience of those who designed Wayland, since they know it best.

So we made the decision to switch. For most of this year, we have been working on porting labwc to the Raspberry Pi Desktop. This has very much been a collaborative process with the developers of both labwc and wlroots: both have helped us immensely with their support as we contribute features and optimisations needed for our desktop.

After much optimisation for our hardware, we have reached the point where labwc desktops run just as fast as X on older Raspberry Pi models. Today, we make the switch with our latest desktop image: Raspberry Pi Desktop now runs Wayland by default across all models.

When you update an existing installation of Bookworm, you will see a prompt asking to switch to labwc the next time you reboot:

We recommend that most people switch to labwc.

Existing Pi 4 or 5 Bookworm installations running wayfire shouldn’t change in any noticeable way, besides the loss of a couple of animations which we haven’t yet implemented in labwc. Because we will no longer support wayfire with updates on Raspberry Pi OS, it’s best to adopt labwc as soon as possible.

Older Pis that currently use X should also switch to labwc. To ensure backwards compatibility with older applications, labwc includes a library called Xwayland, which provides a virtual X implementation running on top of Wayland. labwc provides this virtual implementation automatically for any application that isn’t compatible with Wayland. With Xwayland, you can continue to use older applications that you rely on while benefiting from the latest security and performance updates.

As with any software update, we cannot possibly test all possible configurations and applications. If you switch to labwc and experience an issue, you can always switch back to X. To do this, open a terminal window and type:

sudo raspi-config 

This launches the command-line Raspberry Pi Configuration application. Use the arrow keys to select “6 Advanced Options” and hit ‘enter’ to open the menu. Select “A6 Wayland” and choose “W1 X11 Openbox window manager with X11 backend”. Hit ‘escape’ to exit the application; when you restart your device, your desktop should restart with X.

We don’t expect this to be necessary for many people, but the option is there, just in case! Of course, if you prefer to stick with wayfire or X for any reason, the upgrade prompt offers you the option to do so – this is not a compulsory upgrade, just one that we recommend.

Improved touch screen support

While labwc is the biggest change to the OS in this release, it’s not the only one. We have also significantly improved support for using the Desktop with a touch screen. Specifically, Raspberry Pi Desktop now automatically shows and hides the virtual keyboard, and supports right-click and double-click equivalents for touch displays.

This change comes as a result of integrating the Squeekboard virtual keyboard. When the system detects a touch display, the virtual keyboard automatically displays at the bottom of the screen whenever it is possible to enter text. The keyboard also automatically hides when no text entry is possible.

This auto show and hide should work with most applications, but it isn’t supported by everything. For applications which do not support it, you can instead use the keyboard icon at the right end of the taskbar to manually toggle the keyboard on and off.

If you don’t want to use the virtual keyboard with a touch screen, or you want to use it without a touch screen and click on it with the mouse, you can turn it on or off in the Display tab of Raspberry Pi Configuration. The new virtual keyboard only works with labwc; it’s not compatible with wayfire or X.

In addition to the virtual keyboard, we added long press detection on touch screens to generate the equivalent of a right-click with a mouse. You can use this to launch context-sensitive menus anywhere in the taskbar and the file manager.

We also added double-tap detection on touch screens to generate a double-click. While this previously worked on X, it didn’t work in wayfire. Double-tap to double-click is now supported in labwc.

Better Raspberry Pi Connect integration

We’ve had a lot of very positive feedback about Raspberry Pi Connect, our remote access software that allows you to control your Raspberry Pi from any computer anywhere in the world. This release integrates Connect into the Desktop.

By default, you will now see the Connect icon in the taskbar at all times. Previously, this indicated that Connect was running. Now, the icon indicates that Connect is installed and ready to use, but is not necessarily running. Hovering the mouse over the icon brings up a tooltip displaying the current status.

You can now enable or disable Connect directly from the menu which pops up when the icon is clicked. Previously, this was an option in Raspberry Pi Configuration, but that option has been removed. Now, all the options to control Connect live in the icon menu.

If you don’t plan to use Connect, you can uninstall it from Recommended Software, or you can remove the icon from the taskbar by right-clicking the taskbar and choosing “Add / Remove Plugins…”.

Other things

This release includes some other small changes worth mentioning:

• We rewrote the panel application for the taskbar at the top of the screen. In the previous version, even if you removed a plugin from the panel, it remained in memory. Now, when you remove a plugin, the panel never loads it into memory at all. Rather than all the individual plugins being part of a single application, each plugin is now a separate library. The panel only loads the libraries for the plugins that you choose to display on your screen. This won’t make much difference to many people, but can save you a bit of RAM if you remove several plugins. This also makes it easier to develop new plugins, both for us and third parties.

• We introduced a new Screen Configuration tool, raindrop. This works exactly the same as the old version, arandr, and even looks similar. Under the hood, we rewrote the old application in C to improve support for labwc and touch screens. Because the new tool is native, performance should be snappier! Going forward, we’ll only maintain the new native version.

How to get it

The new release is available today in apt, Raspberry Pi Imager, or as a download from the software page on raspberrypi.com.

sudo apt install labwc

To update an existing Raspberry Pi OS Bookworm install to this release, run the following commands:

sudo apt update

sudo apt full-upgrade

When you next reboot, you will see the prompt described above which offers the switch to labwc.

To switch to the new Screen Configuration tool, run the following commands:

sudo apt purge arandr

sudo apt install raindrop

The new on-screen keyboard can either be installed from Recommended Software – it’s called Squeekboard – or from the command line with:

sudo apt install squeekboard wfplug-squeek

We hope you like the new desktop experience. Or perhaps more accurately, we hope you won’t notice much difference! As always, your comments are very welcome below.

The post A new release of Raspberry Pi OS appeared first on Raspberry Pi.